You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@iceberg.apache.org by gi...@apache.org on 2023/05/01 00:33:40 UTC
[iceberg-docs] branch asf-site updated: deploy: 864d2c65c4d6ca00070da13648315e4c731d4993
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/iceberg-docs.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 31531d85 deploy: 864d2c65c4d6ca00070da13648315e4c731d4993
31531d85 is described below
commit 31531d85fad117a411d12ac35b608a5ae2e65c7a
Author: jackye1995 <ja...@users.noreply.github.com>
AuthorDate: Mon May 1 00:33:36 2023 +0000
deploy: 864d2c65c4d6ca00070da13648315e4c731d4993
---
docs/1.2.1/api/index.html | 6 ++-
docs/1.2.1/aws/index.html | 6 ++-
docs/1.2.1/branching/index.html | 49 ++++++++++++++++++++
docs/1.2.1/configuration/index.html | 6 ++-
docs/1.2.1/custom-catalog/index.html | 6 ++-
docs/1.2.1/dell/index.html | 6 ++-
docs/1.2.1/delta-lake-migration/index.html | 54 ++++++++++++++++++++++
docs/1.2.1/evolution/index.html | 6 ++-
docs/1.2.1/flink-actions/index.html | 6 ++-
docs/1.2.1/flink-configuration/index.html | 8 ++--
docs/1.2.1/flink-connector/index.html | 6 ++-
docs/1.2.1/flink-ddl/index.html | 6 ++-
docs/1.2.1/flink-queries/index.html | 44 ++++++++++++++++--
docs/1.2.1/flink-writes/index.html | 13 ++++--
docs/1.2.1/flink/index.html | 13 ++++--
docs/1.2.1/getting-started/index.html | 6 ++-
.../{performance => hive-migration}/index.html | 20 ++++++--
docs/1.2.1/hive/index.html | 6 ++-
docs/1.2.1/index.html | 6 ++-
docs/1.2.1/index.xml | 12 +++--
docs/1.2.1/java-api-quickstart/index.html | 6 ++-
docs/1.2.1/jdbc/index.html | 6 ++-
docs/1.2.1/maintenance/index.html | 6 ++-
docs/1.2.1/nessie/index.html | 6 ++-
docs/1.2.1/partitioning/index.html | 6 ++-
docs/1.2.1/performance/index.html | 6 ++-
docs/1.2.1/reliability/index.html | 6 ++-
docs/1.2.1/schemas/index.html | 6 ++-
docs/1.2.1/sitemap.xml | 2 +-
docs/1.2.1/spark-configuration/index.html | 6 ++-
docs/1.2.1/spark-ddl/index.html | 30 ++++++++++--
docs/1.2.1/spark-procedures/index.html | 6 ++-
docs/1.2.1/spark-queries/index.html | 6 ++-
docs/1.2.1/spark-structured-streaming/index.html | 6 ++-
docs/1.2.1/spark-writes/index.html | 34 ++++++++++++--
docs/1.2.1/{schemas => table-migration}/index.html | 14 ++++--
docs/1.2.1/tables/branching/index.html | 1 +
37 files changed, 357 insertions(+), 81 deletions(-)
diff --git a/docs/1.2.1/api/index.html b/docs/1.2.1/api/index.html
index 2e37f0b4..31d47225 100644
--- a/docs/1.2.1/api/index.html
+++ b/docs/1.2.1/api/index.html
@@ -5,7 +5,7 @@
<span class=icon-bar></span></button>
<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/1.2.1//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.2.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg.a [...]
<i class="fa fa-chevron-right"></i>
-<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul class=sub-menu><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li></ul></div><li><a class="chevron-toggle collapsed" data-toggle=collaps [...]
+<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li></ul></div><li [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Spark class=collapse><ul class=sub-menu><li><a href=../spark-ddl/>DDL</a></li><li><a href=../getting-started/>Getting Started</a></li><li><a href=../spark-procedures/>Procedures</a></li><li><a href=../spark-queries/>Queries</a></li><li><a href=../spark-structured-streaming/>Structured Streaming</a></li><li><a href=../spark-writes/>Writes</a></li></ul></div><li><a class="chevron-toggle collapsed" data-toggle=collapse data-parent=full href [...]
<i class="fa fa-chevron-right"></i>
@@ -13,7 +13,9 @@
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Integrations class=collapse><ul class=sub-menu><li><a href=../aws/>AWS</a></li><li><a href=../dell/>Dell</a></li><li><a href=../jdbc/>JDBC</a></li><li><a href=../nessie/>Nessie</a></li></ul></div><li><a class=chevron-toggle data-toggle=collapse data-parent=full href=#API><span>API</span>
<i class="fa fa-chevron-right"></i>
-<i class="fa fa-chevron-down"></i></a></li><div id=API class="collapse in"><ul class=sub-menu><li><a href=../java-api-quickstart/>Java Quickstart</a></li><li><a id=active href=../api/>Java API</a></li><li><a href=../custom-catalog/>Java Custom Catalog</a></li></ul></div><li><a href=https://iceberg.apache.org/docs/1.2.1/../../javadoc/latest><span>Javadoc</span></a></li><li><a target=_blank href=https://py.iceberg.apache.org/><span>PyIceberg</span></a></li></div></div><div id=content class [...]
+<i class="fa fa-chevron-down"></i></a></li><div id=API class="collapse in"><ul class=sub-menu><li><a href=../java-api-quickstart/>Java Quickstart</a></li><li><a id=active href=../api/>Java API</a></li><li><a href=../custom-catalog/>Java Custom Catalog</a></li></ul></div><li><a href=https://iceberg.apache.org/docs/1.2.1/../../javadoc/latest><span>Javadoc</span></a></li><li><a target=_blank href=https://py.iceberg.apache.org/><span>PyIceberg</span></a></li><li><a class="chevron-toggle coll [...]
+<i class="fa fa-chevron-right"></i>
+<i class="fa fa-chevron-down"></i></a></li><div id=Migration class=collapse><ul class=sub-menu><li><a href=../table-migration/>Overview</a></li><li><a href=../hive-migration/>Hive Migration</a></li><li><a href=../delta-lake-migration/>Delta Lake Migration</a></li></ul></div></div></div><div id=content class=markdown-body><div class=margin-for-toc><h1 id=iceberg-java-api>Iceberg Java API</h1><h2 id=tables>Tables</h2><p>The main purpose of the Iceberg API is to manage table metadata, like [...]
</span></span></code></pre></div><p>To configure a scan, call <code>filter</code> and <code>select</code> on the <code>TableScan</code> to get a new <code>TableScan</code> with those changes.</p><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-java data-lang=java><span style=display:flex><span>TableScan filteredScan <span style=color:#f92672>=</span> scan<span style=color:#f92672>.</span><span [...]
</span></span></code></pre></div><p>Calls to configuration methods create a new <code>TableScan</code> so that each <code>TableScan</code> is immutable and won’t change unexpectedly if shared across threads.</p><p>When a scan is configured, <code>planFiles</code>, <code>planTasks</code>, and <code>schema</code> are used to return files, tasks, and the read projection.</p><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4; [...]
</span></span><span style=display:flex><span> <span style=color:#f92672>.</span><span style=color:#a6e22e>filter</span><span style=color:#f92672>(</span>Expressions<span style=color:#f92672>.</span><span style=color:#a6e22e>equal</span><span style=color:#f92672>(</span><span style=color:#e6db74>"id"</span><span style=color:#f92672>,</span> <span style=color:#ae81ff>5</span><span style=color:#f92672>))</span>
diff --git a/docs/1.2.1/aws/index.html b/docs/1.2.1/aws/index.html
index 78800295..a1a66118 100644
--- a/docs/1.2.1/aws/index.html
+++ b/docs/1.2.1/aws/index.html
@@ -5,7 +5,7 @@
<span class=icon-bar></span></button>
<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/1.2.1//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.2.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg.a [...]
<i class="fa fa-chevron-right"></i>
-<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul class=sub-menu><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li></ul></div><li><a class="chevron-toggle collapsed" data-toggle=collaps [...]
+<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li></ul></div><li [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Spark class=collapse><ul class=sub-menu><li><a href=../spark-ddl/>DDL</a></li><li><a href=../getting-started/>Getting Started</a></li><li><a href=../spark-procedures/>Procedures</a></li><li><a href=../spark-queries/>Queries</a></li><li><a href=../spark-structured-streaming/>Structured Streaming</a></li><li><a href=../spark-writes/>Writes</a></li></ul></div><li><a class="chevron-toggle collapsed" data-toggle=collapse data-parent=full href [...]
<i class="fa fa-chevron-right"></i>
@@ -13,7 +13,9 @@
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Integrations class="collapse in"><ul class=sub-menu><li><a id=active href=../aws/>AWS</a></li><li><a href=../dell/>Dell</a></li><li><a href=../jdbc/>JDBC</a></li><li><a href=../nessie/>Nessie</a></li></ul></div><li><a class="chevron-toggle collapsed" data-toggle=collapse data-parent=full href=#API><span>API</span>
<i class="fa fa-chevron-right"></i>
-<i class="fa fa-chevron-down"></i></a></li><div id=API class=collapse><ul class=sub-menu><li><a href=../java-api-quickstart/>Java Quickstart</a></li><li><a href=../api/>Java API</a></li><li><a href=../custom-catalog/>Java Custom Catalog</a></li></ul></div><li><a href=https://iceberg.apache.org/docs/1.2.1/../../javadoc/latest><span>Javadoc</span></a></li><li><a target=_blank href=https://py.iceberg.apache.org/><span>PyIceberg</span></a></li></div></div><div id=content class=markdown-body> [...]
+<i class="fa fa-chevron-down"></i></a></li><div id=API class=collapse><ul class=sub-menu><li><a href=../java-api-quickstart/>Java Quickstart</a></li><li><a href=../api/>Java API</a></li><li><a href=../custom-catalog/>Java Custom Catalog</a></li></ul></div><li><a href=https://iceberg.apache.org/docs/1.2.1/../../javadoc/latest><span>Javadoc</span></a></li><li><a target=_blank href=https://py.iceberg.apache.org/><span>PyIceberg</span></a></li><li><a class="chevron-toggle collapsed" data-tog [...]
+<i class="fa fa-chevron-right"></i>
+<i class="fa fa-chevron-down"></i></a></li><div id=Migration class=collapse><ul class=sub-menu><li><a href=../table-migration/>Overview</a></li><li><a href=../hive-migration/>Hive Migration</a></li><li><a href=../delta-lake-migration/>Delta Lake Migration</a></li></ul></div></div></div><div id=content class=markdown-body><div class=margin-for-toc><h1 id=iceberg-aws-integrations>Iceberg AWS Integrations</h1><p>Iceberg provides integration with different AWS services through the <code>iceb [...]
This section describes how to use Iceberg with AWS.</p><h2 id=enabling-aws-integration>Enabling AWS Integration</h2><p>The <code>iceberg-aws</code> module is bundled with Spark and Flink engine runtimes for all versions from <code>0.11.0</code> onwards.
However, the AWS clients are not bundled so that you can use the same client version as your application.
You will need to provide the AWS v2 SDK because that is what Iceberg depends on.
diff --git a/docs/1.2.1/branching/index.html b/docs/1.2.1/branching/index.html
new file mode 100644
index 00000000..b8c1a0b2
--- /dev/null
+++ b/docs/1.2.1/branching/index.html
@@ -0,0 +1,49 @@
+<!doctype html><html><head><meta charset=utf-8><meta http-equiv=x-ua-compatible content="IE=edge"><meta name=viewport content="width=device-width,initial-scale=1"><meta name=description content><meta name=author content><title>Branching and Tagging</title><link href=../css/bootstrap.css rel=stylesheet><link href=../css/markdown.css rel=stylesheet><link href=../css/katex.min.css rel=stylesheet><link href=../css/iceberg-theme.css rel=stylesheet><link href=../font-awesome-4.7.0/css/font-awe [...]
+<span class=sr-only>Toggle navigation</span>
+<span class=icon-bar></span>
+<span class=icon-bar></span>
+<span class=icon-bar></span></button>
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/1.2.1//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.2.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg.a [...]
+<i class="fa fa-chevron-right"></i>
+<i class="fa fa-chevron-down"></i></a></li><div id=Tables class="collapse in"><ul class=sub-menu><li><a id=active href=../branching/>Branching and Tagging</a></li><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li [...]
+<i class="fa fa-chevron-right"></i>
+<i class="fa fa-chevron-down"></i></a></li><div id=Spark class=collapse><ul class=sub-menu><li><a href=../spark-ddl/>DDL</a></li><li><a href=../getting-started/>Getting Started</a></li><li><a href=../spark-procedures/>Procedures</a></li><li><a href=../spark-queries/>Queries</a></li><li><a href=../spark-structured-streaming/>Structured Streaming</a></li><li><a href=../spark-writes/>Writes</a></li></ul></div><li><a class="chevron-toggle collapsed" data-toggle=collapse data-parent=full href [...]
+<i class="fa fa-chevron-right"></i>
+<i class="fa fa-chevron-down"></i></a></li><div id=Flink class=collapse><ul class=sub-menu><li><a href=../flink/>Flink Getting Started</a></li><li><a href=../flink-connector/>Flink Connector</a></li><li><a href=../flink-ddl/>Flink DDL</a></li><li><a href=../flink-queries/>Flink Queries</a></li><li><a href=../flink-writes/>Flink Writes</a></li><li><a href=../flink-actions/>Flink Actions</a></li><li><a href=../flink-configuration/>Flink Configuration</a></li></ul></div><li><a href=../hive/ [...]
+<i class="fa fa-chevron-right"></i>
+<i class="fa fa-chevron-down"></i></a></li><div id=Integrations class=collapse><ul class=sub-menu><li><a href=../aws/>AWS</a></li><li><a href=../dell/>Dell</a></li><li><a href=../jdbc/>JDBC</a></li><li><a href=../nessie/>Nessie</a></li></ul></div><li><a class="chevron-toggle collapsed" data-toggle=collapse data-parent=full href=#API><span>API</span>
+<i class="fa fa-chevron-right"></i>
+<i class="fa fa-chevron-down"></i></a></li><div id=API class=collapse><ul class=sub-menu><li><a href=../java-api-quickstart/>Java Quickstart</a></li><li><a href=../api/>Java API</a></li><li><a href=../custom-catalog/>Java Custom Catalog</a></li></ul></div><li><a href=https://iceberg.apache.org/docs/1.2.1/../../javadoc/latest><span>Javadoc</span></a></li><li><a target=_blank href=https://py.iceberg.apache.org/><span>PyIceberg</span></a></li><li><a class="chevron-toggle collapsed" data-tog [...]
+<i class="fa fa-chevron-right"></i>
+<i class="fa fa-chevron-down"></i></a></li><div id=Migration class=collapse><ul class=sub-menu><li><a href=../table-migration/>Overview</a></li><li><a href=../hive-migration/>Hive Migration</a></li><li><a href=../delta-lake-migration/>Delta Lake Migration</a></li></ul></div></div></div><div id=content class=markdown-body><div class=margin-for-toc><h1 id=branching-and-tagging>Branching and Tagging</h1><h2 id=overview>Overview</h2><p>Iceberg table metadata maintains a log of snapshots whic [...]
+Snapshots are fundamental in Iceberg as they are the basis for reader isolation and time travel queries.
+For controlling metadata size and storage costs, Iceberg provides snapshot lifecycle management procedures such as <a href=../../spark/spark-procedures/#expire-snapshots><code>expire_snapshots</code></a> for removing unused snapshots and no longer neccessary data files based on table snapshot retention properties.</p><p><strong>For more sophisticated snapshot lifecycle management, Iceberg supports branches and tags which are named references to snapshots with their own independent lifecy [...]
+Branches are independent lineages of snapshots and point to the head of the lineage.
+Branches and tags have a maximum reference age property which control when the reference to the snapshot itself should be expired.
+Branches have retention properties which define the minimum number of snapshots to retain on a branch as well as the maximum age of individual snapshots to retain on the branch.
+These properties are used when the expireSnapshots procedure is run.
+For details on the algorithm for expireSnapshots, refer to the <a href=../../../spec#snapshot-retention-policy>spec</a>.</p><h2 id=use-cases>Use Cases</h2><p>Branching and tagging can be used for handling GDPR requirements and retaining important historical snapshots for auditing.
+Branches can also be used as part of data engineering workflows, for enabling experimental branches for testing and validating new jobs.
+See below for some examples of how branching and tagging can facilitate these use cases.</p><h3 id=historical-tags>Historical Tags</h3><p>Tags can be used for retaining important historical snapshots for auditing purposes.</p><p><img src=../img/historical-snapshot-tag.png alt="Historical Tags"></p><p>The above diagram demonstrates retaininig important historical snapshot with the following retention policy, defined
+via Spark SQL.</p><ol><li>Retain 1 snapshot per week for 1 month. This can be achieved by tagging the weekly snapshot and setting the tag retention to be a month.
+snapshots will be kept, and the branch reference itself will be retained for 1 week.</li></ol><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-sql data-lang=sql><span style=display:flex><span><span style=color:#75715e>-- Create a tag for the first end of week snapshot. Retain the snapshot for a week
+</span></span></span><span style=display:flex><span><span style=color:#75715e></span><span style=color:#66d9ef>ALTER</span> <span style=color:#66d9ef>TABLE</span> prod.db.<span style=color:#66d9ef>table</span> <span style=color:#66d9ef>CREATE</span> TAG <span style=color:#e6db74>'EOW-01'</span> <span style=color:#66d9ef>AS</span> <span style=color:#66d9ef>OF</span> <span style=color:#66d9ef>VERSION</span> <span style=color:#ae81ff>7</span> RETAIN <span style=color:#ae81ff>7</span> DAYS
+</span></span></code></pre></div><ol start=2><li>Retain 1 snapshot per month for 6 months. This can be achieved by tagging the monthly snapshot and setting the tag retention to be 6 months.</li></ol><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-sql data-lang=sql><span style=display:flex><span><span style=color:#75715e>-- Create a tag for the first end of month snapshot. Retain the snapshot f [...]
+</span></span></span><span style=display:flex><span><span style=color:#75715e></span><span style=color:#66d9ef>ALTER</span> <span style=color:#66d9ef>TABLE</span> prod.db.<span style=color:#66d9ef>table</span> <span style=color:#66d9ef>CREATE</span> TAG <span style=color:#e6db74>'EOM-01'</span> <span style=color:#66d9ef>AS</span> <span style=color:#66d9ef>OF</span> <span style=color:#66d9ef>VERSION</span> <span style=color:#ae81ff>30</span> RETAIN <span style=color:#ae81ff>180</s [...]
+</span></span></code></pre></div><ol start=3><li>Retain 1 snapshot per year forever. This can be achieved by tagging the annual snapshot. The default retention for branches and tags is forever.</li></ol><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-sql data-lang=sql><span style=display:flex><span><span style=color:#75715e>-- Create a tag for the end of the year and retain it forever.
+</span></span></span><span style=display:flex><span><span style=color:#75715e></span><span style=color:#66d9ef>ALTER</span> <span style=color:#66d9ef>TABLE</span> prod.db.<span style=color:#66d9ef>table</span> <span style=color:#66d9ef>CREATE</span> TAG <span style=color:#e6db74>'EOY-2023'</span> <span style=color:#66d9ef>AS</span> <span style=color:#66d9ef>OF</span> <span style=color:#66d9ef>VERSION</span> <span style=color:#ae81ff>365</span>
+</span></span></code></pre></div><ol start=4><li>Create a temporary “test-branch” which is retained for 7 days and the latest 2 snapshots on the branch are retained.</li></ol><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-sql data-lang=sql><span style=display:flex><span><span style=color:#75715e>-- Create a branch "test-branch" which will be retained for 7 days along with [...]
+</span></span></span><span style=display:flex><span><span style=color:#75715e></span><span style=color:#66d9ef>ALTER</span> <span style=color:#66d9ef>TABLE</span> prod.db.<span style=color:#66d9ef>table</span> <span style=color:#66d9ef>CREATE</span> BRANCH test<span style=color:#f92672>-</span>branch RETAIN <span style=color:#ae81ff>7</span> DAYS <span style=color:#66d9ef>WITH</span> RETENTION <span style=color:#ae81ff>2</span> SNAPSHOTS
+</span></span></code></pre></div><h3 id=audit-branch>Audit Branch</h3><p><img src=../img/audit-branch.png alt="Audit Branch"></p><p>The above diagram shows an example of using an audit branch for validating a write workflow.</p><ol><li>First ensure <code>write.wap.enabled</code> is set.</li></ol><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-sql data-lang=sql><span style=display:flex><span><s [...]
+</span></span><span style=display:flex><span> <span style=color:#e6db74>'write.wap.enabled''true'</span>
+</span></span><span style=display:flex><span>)
+</span></span></code></pre></div><ol start=2><li>Create <code>audit-branch</code> starting from snapshot 3, which will be written to and retained for 1 week.</li></ol><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-sql data-lang=sql><span style=display:flex><span><span style=color:#66d9ef>ALTER</span> <span style=color:#66d9ef>TABLE</span> db.<span style=color:#66d9ef>table</span> <span style= [...]
+</span></span></code></pre></div><ol start=3><li>Writes are performed on a separate <code>audit-branch</code> independent from the main table history.</li></ol><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-sql data-lang=sql><span style=display:flex><span><span style=color:#75715e>-- WAP Branch write
+</span></span></span><span style=display:flex><span><span style=color:#75715e></span><span style=color:#66d9ef>SET</span> spark.wap.branch <span style=color:#f92672>=</span> <span style=color:#e6db74>'audit-branch'</span>
+</span></span><span style=display:flex><span><span style=color:#66d9ef>INSERT</span> <span style=color:#66d9ef>INTO</span> prod.db.<span style=color:#66d9ef>table</span> <span style=color:#66d9ef>VALUES</span> (<span style=color:#ae81ff>3</span>, <span style=color:#e6db74>'c'</span>)
+</span></span></code></pre></div><ol start=4><li>A validation workflow can validate (e.g. data quality) the state of <code>audit-branch</code>.</li><li>After validation, the main branch can be <code>fastForward</code> to the head of <code>audit-branch</code> to update the main table state.</li></ol><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-java data-lang=java><span style=display:flex><sp [...]
+</span></span></code></pre></div><ol start=6><li>The branch reference will be removed when <code>expireSnapshots</code> is run 1 week later.</li></ol><h2 id=usage>Usage</h2><p>Creating, querying and writing to branches and tags are supported in the Iceberg Java library, and in Spark and Flink engine integrations.</p><ul><li><a href=../../java-api-quickstart/#branching-and-tagging>Iceberg Java Library</a></li><li><a href=../spark-ddl/#branching-and-tagging-ddl>Spark DDLs</a></li><li><a hr [...]
+<script src=https://iceberg.apache.org/docs/1.2.1//js/jquery.easing.min.js></script>
+<script type=text/javascript src=https://iceberg.apache.org/docs/1.2.1//js/search.js></script>
+<script src=https://iceberg.apache.org/docs/1.2.1//js/bootstrap.min.js></script>
+<script src=https://iceberg.apache.org/docs/1.2.1//js/iceberg-theme.js></script></html>
\ No newline at end of file
diff --git a/docs/1.2.1/configuration/index.html b/docs/1.2.1/configuration/index.html
index 397533ce..2c32e95d 100644
--- a/docs/1.2.1/configuration/index.html
+++ b/docs/1.2.1/configuration/index.html
@@ -5,7 +5,7 @@
<span class=icon-bar></span></button>
<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/1.2.1//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.2.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg.a [...]
<i class="fa fa-chevron-right"></i>
-<i class="fa fa-chevron-down"></i></a></li><div id=Tables class="collapse in"><ul class=sub-menu><li><a id=active href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li></ul></div><li><a class="chevron-toggle collapsed" data [...]
+<i class="fa fa-chevron-down"></i></a></li><div id=Tables class="collapse in"><ul class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a id=active href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Spark class=collapse><ul class=sub-menu><li><a href=../spark-ddl/>DDL</a></li><li><a href=../getting-started/>Getting Started</a></li><li><a href=../spark-procedures/>Procedures</a></li><li><a href=../spark-queries/>Queries</a></li><li><a href=../spark-structured-streaming/>Structured Streaming</a></li><li><a href=../spark-writes/>Writes</a></li></ul></div><li><a class="chevron-toggle collapsed" data-toggle=collapse data-parent=full href [...]
<i class="fa fa-chevron-right"></i>
@@ -13,7 +13,9 @@
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Integrations class=collapse><ul class=sub-menu><li><a href=../aws/>AWS</a></li><li><a href=../dell/>Dell</a></li><li><a href=../jdbc/>JDBC</a></li><li><a href=../nessie/>Nessie</a></li></ul></div><li><a class="chevron-toggle collapsed" data-toggle=collapse data-parent=full href=#API><span>API</span>
<i class="fa fa-chevron-right"></i>
-<i class="fa fa-chevron-down"></i></a></li><div id=API class=collapse><ul class=sub-menu><li><a href=../java-api-quickstart/>Java Quickstart</a></li><li><a href=../api/>Java API</a></li><li><a href=../custom-catalog/>Java Custom Catalog</a></li></ul></div><li><a href=https://iceberg.apache.org/docs/1.2.1/../../javadoc/latest><span>Javadoc</span></a></li><li><a target=_blank href=https://py.iceberg.apache.org/><span>PyIceberg</span></a></li></div></div><div id=content class=markdown-body> [...]
+<i class="fa fa-chevron-down"></i></a></li><div id=API class=collapse><ul class=sub-menu><li><a href=../java-api-quickstart/>Java Quickstart</a></li><li><a href=../api/>Java API</a></li><li><a href=../custom-catalog/>Java Custom Catalog</a></li></ul></div><li><a href=https://iceberg.apache.org/docs/1.2.1/../../javadoc/latest><span>Javadoc</span></a></li><li><a target=_blank href=https://py.iceberg.apache.org/><span>PyIceberg</span></a></li><li><a class="chevron-toggle collapsed" data-tog [...]
+<i class="fa fa-chevron-right"></i>
+<i class="fa fa-chevron-down"></i></a></li><div id=Migration class=collapse><ul class=sub-menu><li><a href=../table-migration/>Overview</a></li><li><a href=../hive-migration/>Hive Migration</a></li><li><a href=../delta-lake-migration/>Delta Lake Migration</a></li></ul></div></div></div><div id=content class=markdown-body><div class=margin-for-toc><h1 id=configuration>Configuration</h1><h2 id=table-properties>Table properties</h2><p>Iceberg tables support table properties to configure tab [...]
The value of these properties are not persisted as a part of the table metadata.</p><table><thead><tr><th>Property</th><th>Default</th><th>Description</th></tr></thead><tbody><tr><td>format-version</td><td>1</td><td>Table’s format version (can be 1 or 2) as defined in the <a href=../../../spec/#format-versioning>Spec</a>.</td></tr></tbody></table><h3 id=compatibility-flags>Compatibility flags</h3><table><thead><tr><th>Property</th><th>Default</th><th>Description</th></tr></thead><t [...]
Any other custom catalog can access the properties by implementing <code>Catalog.initialize(catalogName, catalogProperties)</code>.
The properties can be manually constructed or passed in from a compute engine like Spark or Flink.
diff --git a/docs/1.2.1/custom-catalog/index.html b/docs/1.2.1/custom-catalog/index.html
index 532f5169..42910da0 100644
--- a/docs/1.2.1/custom-catalog/index.html
+++ b/docs/1.2.1/custom-catalog/index.html
@@ -5,7 +5,7 @@
<span class=icon-bar></span></button>
<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/1.2.1//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.2.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg.a [...]
<i class="fa fa-chevron-right"></i>
-<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul class=sub-menu><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li></ul></div><li><a class="chevron-toggle collapsed" data-toggle=collaps [...]
+<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li></ul></div><li [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Spark class=collapse><ul class=sub-menu><li><a href=../spark-ddl/>DDL</a></li><li><a href=../getting-started/>Getting Started</a></li><li><a href=../spark-procedures/>Procedures</a></li><li><a href=../spark-queries/>Queries</a></li><li><a href=../spark-structured-streaming/>Structured Streaming</a></li><li><a href=../spark-writes/>Writes</a></li></ul></div><li><a class="chevron-toggle collapsed" data-toggle=collapse data-parent=full href [...]
<i class="fa fa-chevron-right"></i>
@@ -13,7 +13,9 @@
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Integrations class=collapse><ul class=sub-menu><li><a href=../aws/>AWS</a></li><li><a href=../dell/>Dell</a></li><li><a href=../jdbc/>JDBC</a></li><li><a href=../nessie/>Nessie</a></li></ul></div><li><a class=chevron-toggle data-toggle=collapse data-parent=full href=#API><span>API</span>
<i class="fa fa-chevron-right"></i>
-<i class="fa fa-chevron-down"></i></a></li><div id=API class="collapse in"><ul class=sub-menu><li><a href=../java-api-quickstart/>Java Quickstart</a></li><li><a href=../api/>Java API</a></li><li><a id=active href=../custom-catalog/>Java Custom Catalog</a></li></ul></div><li><a href=https://iceberg.apache.org/docs/1.2.1/../../javadoc/latest><span>Javadoc</span></a></li><li><a target=_blank href=https://py.iceberg.apache.org/><span>PyIceberg</span></a></li></div></div><div id=content class [...]
+<i class="fa fa-chevron-down"></i></a></li><div id=API class="collapse in"><ul class=sub-menu><li><a href=../java-api-quickstart/>Java Quickstart</a></li><li><a href=../api/>Java API</a></li><li><a id=active href=../custom-catalog/>Java Custom Catalog</a></li></ul></div><li><a href=https://iceberg.apache.org/docs/1.2.1/../../javadoc/latest><span>Javadoc</span></a></li><li><a target=_blank href=https://py.iceberg.apache.org/><span>PyIceberg</span></a></li><li><a class="chevron-toggle coll [...]
+<i class="fa fa-chevron-right"></i>
+<i class="fa fa-chevron-down"></i></a></li><div id=Migration class=collapse><ul class=sub-menu><li><a href=../table-migration/>Overview</a></li><li><a href=../hive-migration/>Hive Migration</a></li><li><a href=../delta-lake-migration/>Delta Lake Migration</a></li></ul></div></div></div><div id=content class=markdown-body><div class=margin-for-toc><h1 id=custom-catalog>Custom Catalog</h1><p>It’s possible to read an iceberg table either from an hdfs path or from a hive table. It&rsqu [...]
</span></span><span style=display:flex><span> <span style=color:#66d9ef>private</span> String dbName<span style=color:#f92672>;</span>
</span></span><span style=display:flex><span> <span style=color:#66d9ef>private</span> String tableName<span style=color:#f92672>;</span>
</span></span><span style=display:flex><span> <span style=color:#66d9ef>private</span> Configuration conf<span style=color:#f92672>;</span>
diff --git a/docs/1.2.1/dell/index.html b/docs/1.2.1/dell/index.html
index 3ba1ce87..bb1be796 100644
--- a/docs/1.2.1/dell/index.html
+++ b/docs/1.2.1/dell/index.html
@@ -5,7 +5,7 @@
<span class=icon-bar></span></button>
<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/1.2.1//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.2.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg.a [...]
<i class="fa fa-chevron-right"></i>
-<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul class=sub-menu><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li></ul></div><li><a class="chevron-toggle collapsed" data-toggle=collaps [...]
+<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li></ul></div><li [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Spark class=collapse><ul class=sub-menu><li><a href=../spark-ddl/>DDL</a></li><li><a href=../getting-started/>Getting Started</a></li><li><a href=../spark-procedures/>Procedures</a></li><li><a href=../spark-queries/>Queries</a></li><li><a href=../spark-structured-streaming/>Structured Streaming</a></li><li><a href=../spark-writes/>Writes</a></li></ul></div><li><a class="chevron-toggle collapsed" data-toggle=collapse data-parent=full href [...]
<i class="fa fa-chevron-right"></i>
@@ -13,7 +13,9 @@
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Integrations class="collapse in"><ul class=sub-menu><li><a href=../aws/>AWS</a></li><li><a id=active href=../dell/>Dell</a></li><li><a href=../jdbc/>JDBC</a></li><li><a href=../nessie/>Nessie</a></li></ul></div><li><a class="chevron-toggle collapsed" data-toggle=collapse data-parent=full href=#API><span>API</span>
<i class="fa fa-chevron-right"></i>
-<i class="fa fa-chevron-down"></i></a></li><div id=API class=collapse><ul class=sub-menu><li><a href=../java-api-quickstart/>Java Quickstart</a></li><li><a href=../api/>Java API</a></li><li><a href=../custom-catalog/>Java Custom Catalog</a></li></ul></div><li><a href=https://iceberg.apache.org/docs/1.2.1/../../javadoc/latest><span>Javadoc</span></a></li><li><a target=_blank href=https://py.iceberg.apache.org/><span>PyIceberg</span></a></li></div></div><div id=content class=markdown-body> [...]
+<i class="fa fa-chevron-down"></i></a></li><div id=API class=collapse><ul class=sub-menu><li><a href=../java-api-quickstart/>Java Quickstart</a></li><li><a href=../api/>Java API</a></li><li><a href=../custom-catalog/>Java Custom Catalog</a></li></ul></div><li><a href=https://iceberg.apache.org/docs/1.2.1/../../javadoc/latest><span>Javadoc</span></a></li><li><a target=_blank href=https://py.iceberg.apache.org/><span>PyIceberg</span></a></li><li><a class="chevron-toggle collapsed" data-tog [...]
+<i class="fa fa-chevron-right"></i>
+<i class="fa fa-chevron-down"></i></a></li><div id=Migration class=collapse><ul class=sub-menu><li><a href=../table-migration/>Overview</a></li><li><a href=../hive-migration/>Hive Migration</a></li><li><a href=../delta-lake-migration/>Delta Lake Migration</a></li></ul></div></div></div><div id=content class=markdown-body><div class=margin-for-toc><h1 id=iceberg-dell-integration>Iceberg Dell Integration</h1><h2 id=dell-ecs-integration>Dell ECS Integration</h2><p>Iceberg can be used with D [...]
</span></span><span style=display:flex><span>SPARK_VERSION<span style=color:#f92672>=</span>3.2_2.12
</span></span><span style=display:flex><span>ECS_CLIENT_VERSION<span style=color:#f92672>=</span>3.3.2
</span></span><span style=display:flex><span>
diff --git a/docs/1.2.1/delta-lake-migration/index.html b/docs/1.2.1/delta-lake-migration/index.html
new file mode 100644
index 00000000..87ab0e45
--- /dev/null
+++ b/docs/1.2.1/delta-lake-migration/index.html
@@ -0,0 +1,54 @@
+<!doctype html><html><head><meta charset=utf-8><meta http-equiv=x-ua-compatible content="IE=edge"><meta name=viewport content="width=device-width,initial-scale=1"><meta name=description content><meta name=author content><title>Delta Lake Migration</title><link href=../css/bootstrap.css rel=stylesheet><link href=../css/markdown.css rel=stylesheet><link href=../css/katex.min.css rel=stylesheet><link href=../css/iceberg-theme.css rel=stylesheet><link href=../font-awesome-4.7.0/css/font-awes [...]
+<span class=sr-only>Toggle navigation</span>
+<span class=icon-bar></span>
+<span class=icon-bar></span>
+<span class=icon-bar></span></button>
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/1.2.1//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.2.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg.a [...]
+<i class="fa fa-chevron-right"></i>
+<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li></ul></div><li [...]
+<i class="fa fa-chevron-right"></i>
+<i class="fa fa-chevron-down"></i></a></li><div id=Spark class=collapse><ul class=sub-menu><li><a href=../spark-ddl/>DDL</a></li><li><a href=../getting-started/>Getting Started</a></li><li><a href=../spark-procedures/>Procedures</a></li><li><a href=../spark-queries/>Queries</a></li><li><a href=../spark-structured-streaming/>Structured Streaming</a></li><li><a href=../spark-writes/>Writes</a></li></ul></div><li><a class="chevron-toggle collapsed" data-toggle=collapse data-parent=full href [...]
+<i class="fa fa-chevron-right"></i>
+<i class="fa fa-chevron-down"></i></a></li><div id=Flink class=collapse><ul class=sub-menu><li><a href=../flink/>Flink Getting Started</a></li><li><a href=../flink-connector/>Flink Connector</a></li><li><a href=../flink-ddl/>Flink DDL</a></li><li><a href=../flink-queries/>Flink Queries</a></li><li><a href=../flink-writes/>Flink Writes</a></li><li><a href=../flink-actions/>Flink Actions</a></li><li><a href=../flink-configuration/>Flink Configuration</a></li></ul></div><li><a href=../hive/ [...]
+<i class="fa fa-chevron-right"></i>
+<i class="fa fa-chevron-down"></i></a></li><div id=Integrations class=collapse><ul class=sub-menu><li><a href=../aws/>AWS</a></li><li><a href=../dell/>Dell</a></li><li><a href=../jdbc/>JDBC</a></li><li><a href=../nessie/>Nessie</a></li></ul></div><li><a class="chevron-toggle collapsed" data-toggle=collapse data-parent=full href=#API><span>API</span>
+<i class="fa fa-chevron-right"></i>
+<i class="fa fa-chevron-down"></i></a></li><div id=API class=collapse><ul class=sub-menu><li><a href=../java-api-quickstart/>Java Quickstart</a></li><li><a href=../api/>Java API</a></li><li><a href=../custom-catalog/>Java Custom Catalog</a></li></ul></div><li><a href=https://iceberg.apache.org/docs/1.2.1/../../javadoc/latest><span>Javadoc</span></a></li><li><a target=_blank href=https://py.iceberg.apache.org/><span>PyIceberg</span></a></li><li><a class=chevron-toggle data-toggle=collapse [...]
+<i class="fa fa-chevron-right"></i>
+<i class="fa fa-chevron-down"></i></a></li><div id=Migration class="collapse in"><ul class=sub-menu><li><a href=../table-migration/>Overview</a></li><li><a href=../hive-migration/>Hive Migration</a></li><li><a id=active href=../delta-lake-migration/>Delta Lake Migration</a></li></ul></div></div></div><div id=content class=markdown-body><div class=margin-for-toc><h1 id=delta-lake-table-migration>Delta Lake Table Migration</h1><p>Delta Lake is a table format that supports Parquet file form [...]
+it is common to migrate all snapshots to maintain the history of the data.</p><p>Currently, Iceberg supports the Snapshot Table action for migrating from Delta Lake to Iceberg tables.
+Since Delta Lake tables maintain transactions, all available transactions will be committed to the new Iceberg table as transactions in order.
+For Delta Lake tables, any additional data files added after the initial migration will be included in their corresponding transactions and subsequently added to the new Iceberg table using the Add Transaction action.
+The Add Transaction action, a variant of the Add File action, is still under development.</p><h2 id=enabling-migration-from-delta-lake-to-iceberg>Enabling Migration from Delta Lake to Iceberg</h2><p>The <code>iceberg-delta-lake</code> module is not bundled with Spark and Flink engine runtimes. To enable migration from delta lake features, the minimum required dependencies are:</p><ul><li><a href=https://repo1.maven.org/maven2/org/apache/iceberg/iceberg-delta-lake/1.2.1/iceberg-delta-lake [...]
+The supported actions are:</p><ul><li><code>snapshotDeltaLakeTable</code>: snapshot an existing Delta Lake table to an Iceberg table</li></ul><h3 id=default-implementation>Default Implementation</h3><p>The <code>iceberg-delta-lake</code> module also provides a default implementation of the interface which can be accessed by</p><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-java data-lang=java [...]
+</span></span></code></pre></div><h2 id=snapshot-delta-lake-table-to-iceberg>Snapshot Delta Lake Table to Iceberg</h2><p>The action <code>snapshotDeltaLakeTable</code> reads the Delta Lake table’s transactions and converts them to a new Iceberg table with the same schema and partitioning in one iceberg transaction.
+The original Delta Lake table remains unchanged.</p><p>The newly created table can be changed or written to without affecting the source table, but the snapshot uses the original table’s data files.
+Existing data files are added to the Iceberg table’s metadata and can be read using a name-to-id mapping created from the original table schema.</p><p>When inserts or overwrites run on the snapshot, new files are placed in the snapshot table’s location. The location is default to be the same as that
+of the source Delta Lake Table. Users can also specify a different location for the snapshot table.</p><div class=info>Because tables created by <code>snapshotDeltaLakeTable</code> are not the sole owners of their data files, they are prohibited from
+actions like <code>expire_snapshots</code> which would physically delete data files. Iceberg deletes, which only effect metadata,
+are still allowed. In addition, any operations which affect the original data files will disrupt the Snapshot’s
+integrity. DELETE statements executed against the original Delta Lake table will remove original data files and the
+<code>snapshotDeltaLakeTable</code> table will no longer be able to access them.</div><h4 id=usage>Usage</h4><table><thead><tr><th>Required Input</th><th>Configured By</th><th>Description</th></tr></thead><tbody><tr><td>Source Table Location</td><td>Argument <a href=https://iceberg.apache.org/javadoc/latest/org/apache/iceberg/delta/DeltaLakeToIcebergMigrationActionsProvider.html#snapshotDeltaLakeTable(java.lang.String)><code>sourceTableLocation</code></a></td><td>The location of the sour [...]
+</span></span><span style=display:flex><span><span style=color:#f92672>import</span> org.apache.iceberg.catalog.Catalog<span style=color:#f92672>;</span>
+</span></span><span style=display:flex><span><span style=color:#f92672>import</span> org.apache.hadoop.conf.Configuration<span style=color:#f92672>;</span>
+</span></span><span style=display:flex><span><span style=color:#f92672>import</span> org.apache.iceberg.delta.DeltaLakeToIcebergMigrationActionsProvider<span style=color:#f92672>;</span>
+</span></span><span style=display:flex><span>
+</span></span><span style=display:flex><span>String sourceDeltaLakeTableLocation <span style=color:#f92672>=</span> <span style=color:#e6db74>"s3://my-bucket/delta-table"</span><span style=color:#f92672>;</span>
+</span></span><span style=display:flex><span>String destTableLocation <span style=color:#f92672>=</span> <span style=color:#e6db74>"s3://my-bucket/iceberg-table"</span><span style=color:#f92672>;</span>
+</span></span><span style=display:flex><span>TableIdentifier destTableIdentifier <span style=color:#f92672>=</span> TableIdentifier<span style=color:#f92672>.</span><span style=color:#a6e22e>of</span><span style=color:#f92672>(</span><span style=color:#e6db74>"my_db"</span><span style=color:#f92672>,</span> <span style=color:#e6db74>"my_table"</span><span style=color:#f92672>);</span>
+</span></span><span style=display:flex><span>Catalog icebergCatalog <span style=color:#f92672>=</span> <span style=color:#f92672>...;</span> <span style=color:#75715e>// Iceberg Catalog fetched from engines like Spark or created via CatalogUtil.loadCatalog
+</span></span></span><span style=display:flex><span><span style=color:#75715e></span>Configuration hadoopConf <span style=color:#f92672>=</span> <span style=color:#f92672>...;</span> <span style=color:#75715e>// Hadoop Configuration fetched from engines like Spark and have proper file system configuration to access the Delta Lake table.
+</span></span></span><span style=display:flex><span><span style=color:#75715e></span>
+</span></span><span style=display:flex><span>DeltaLakeToIcebergMigrationActionsProvider<span style=color:#f92672>.</span><span style=color:#a6e22e>defaultActions</span><span style=color:#f92672>()</span>
+</span></span><span style=display:flex><span> <span style=color:#f92672>.</span><span style=color:#a6e22e>snapshotDeltaLakeTable</span><span style=color:#f92672>(</span>sourceDeltaLakeTableLocation<span style=color:#f92672>)</span>
+</span></span><span style=display:flex><span> <span style=color:#f92672>.</span><span style=color:#a6e22e>as</span><span style=color:#f92672>(</span>destTableIdentifier<span style=color:#f92672>)</span>
+</span></span><span style=display:flex><span> <span style=color:#f92672>.</span><span style=color:#a6e22e>icebergCatalog</span><span style=color:#f92672>(</span>icebergCatalog<span style=color:#f92672>)</span>
+</span></span><span style=display:flex><span> <span style=color:#f92672>.</span><span style=color:#a6e22e>tableLocation</span><span style=color:#f92672>(</span>destTableLocation<span style=color:#f92672>)</span>
+</span></span><span style=display:flex><span> <span style=color:#f92672>.</span><span style=color:#a6e22e>deltaLakeConfiguration</span><span style=color:#f92672>(</span>hadoopConf<span style=color:#f92672>)</span>
+</span></span><span style=display:flex><span> <span style=color:#f92672>.</span><span style=color:#a6e22e>tableProperty</span><span style=color:#f92672>(</span><span style=color:#e6db74>"my_property"</span><span style=color:#f92672>,</span> <span style=color:#e6db74>"my_value"</span><span style=color:#f92672>)</span>
+</span></span><span style=display:flex><span> <span style=color:#f92672>.</span><span style=color:#a6e22e>execute</span><span style=color:#f92672>();</span>
+</span></span></code></pre></div></div><div id=toc class=markdown-body><div id=full><nav id=TableOfContents><ul><li><a href=#enabling-migration-from-delta-lake-to-iceberg>Enabling Migration from Delta Lake to Iceberg</a><ul><li><a href=#compatibilities>Compatibilities</a></li><li><a href=#api>API</a></li><li><a href=#default-implementation>Default Implementation</a></li></ul></li><li><a href=#snapshot-delta-lake-table-to-iceberg>Snapshot Delta Lake Table to Iceberg</a><ul><li></li></ul>< [...]
+<script src=https://iceberg.apache.org/docs/1.2.1//js/jquery.easing.min.js></script>
+<script type=text/javascript src=https://iceberg.apache.org/docs/1.2.1//js/search.js></script>
+<script src=https://iceberg.apache.org/docs/1.2.1//js/bootstrap.min.js></script>
+<script src=https://iceberg.apache.org/docs/1.2.1//js/iceberg-theme.js></script></html>
\ No newline at end of file
diff --git a/docs/1.2.1/evolution/index.html b/docs/1.2.1/evolution/index.html
index 11264acf..1cb52caa 100644
--- a/docs/1.2.1/evolution/index.html
+++ b/docs/1.2.1/evolution/index.html
@@ -5,7 +5,7 @@
<span class=icon-bar></span></button>
<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/1.2.1//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.2.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg.a [...]
<i class="fa fa-chevron-right"></i>
-<i class="fa fa-chevron-down"></i></a></li><div id=Tables class="collapse in"><ul class=sub-menu><li><a href=../configuration/>Configuration</a></li><li><a id=active href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li></ul></div><li><a class="chevron-toggle collapsed" data [...]
+<i class="fa fa-chevron-down"></i></a></li><div id=Tables class="collapse in"><ul class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a href=../configuration/>Configuration</a></li><li><a id=active href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Spark class=collapse><ul class=sub-menu><li><a href=../spark-ddl/>DDL</a></li><li><a href=../getting-started/>Getting Started</a></li><li><a href=../spark-procedures/>Procedures</a></li><li><a href=../spark-queries/>Queries</a></li><li><a href=../spark-structured-streaming/>Structured Streaming</a></li><li><a href=../spark-writes/>Writes</a></li></ul></div><li><a class="chevron-toggle collapsed" data-toggle=collapse data-parent=full href [...]
<i class="fa fa-chevron-right"></i>
@@ -13,7 +13,9 @@
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Integrations class=collapse><ul class=sub-menu><li><a href=../aws/>AWS</a></li><li><a href=../dell/>Dell</a></li><li><a href=../jdbc/>JDBC</a></li><li><a href=../nessie/>Nessie</a></li></ul></div><li><a class="chevron-toggle collapsed" data-toggle=collapse data-parent=full href=#API><span>API</span>
<i class="fa fa-chevron-right"></i>
-<i class="fa fa-chevron-down"></i></a></li><div id=API class=collapse><ul class=sub-menu><li><a href=../java-api-quickstart/>Java Quickstart</a></li><li><a href=../api/>Java API</a></li><li><a href=../custom-catalog/>Java Custom Catalog</a></li></ul></div><li><a href=https://iceberg.apache.org/docs/1.2.1/../../javadoc/latest><span>Javadoc</span></a></li><li><a target=_blank href=https://py.iceberg.apache.org/><span>PyIceberg</span></a></li></div></div><div id=content class=markdown-body> [...]
+<i class="fa fa-chevron-down"></i></a></li><div id=API class=collapse><ul class=sub-menu><li><a href=../java-api-quickstart/>Java Quickstart</a></li><li><a href=../api/>Java API</a></li><li><a href=../custom-catalog/>Java Custom Catalog</a></li></ul></div><li><a href=https://iceberg.apache.org/docs/1.2.1/../../javadoc/latest><span>Javadoc</span></a></li><li><a target=_blank href=https://py.iceberg.apache.org/><span>PyIceberg</span></a></li><li><a class="chevron-toggle collapsed" data-tog [...]
+<i class="fa fa-chevron-right"></i>
+<i class="fa fa-chevron-down"></i></a></li><div id=Migration class=collapse><ul class=sub-menu><li><a href=../table-migration/>Overview</a></li><li><a href=../hive-migration/>Hive Migration</a></li><li><a href=../delta-lake-migration/>Delta Lake Migration</a></li></ul></div></div></div><div id=content class=markdown-body><div class=margin-for-toc><h1 id=evolution>Evolution</h1><p>Iceberg supports <strong>in-place table evolution</strong>. You can <a href=#schema-evolution>evolve a table [...]
<em>The data for 2008 is partitioned by month. Starting from 2009 the table is updated so that the data is instead partitioned by day. Both partitioning layouts are able to coexist in the same table.</em></p><p>Iceberg uses <a href=../partitioning>hidden partitioning</a>, so you don’t <em>need</em> to write queries for a specific partition layout to be fast. Instead, you can write queries that select the data you need, and Iceberg automatically prunes out files that don’t con [...]
For example, the following code could be used to update the partition spec to add a new partition field that places <code>id</code> column values into 8 buckets and remove an existing partition field <code>category</code>:</p><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-java data-lang=java><span style=display:flex><span>Table sampleTable <span style=color:#f92672>=</span> <span style=color: [...]
</span></span><span style=display:flex><span>sampleTable<span style=color:#f92672>.</span><span style=color:#a6e22e>updateSpec</span><span style=color:#f92672>()</span>
diff --git a/docs/1.2.1/flink-actions/index.html b/docs/1.2.1/flink-actions/index.html
index 77ac1850..204722a1 100644
--- a/docs/1.2.1/flink-actions/index.html
+++ b/docs/1.2.1/flink-actions/index.html
@@ -5,7 +5,7 @@
<span class=icon-bar></span></button>
<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/1.2.1//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.2.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg.a [...]
<i class="fa fa-chevron-right"></i>
-<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul class=sub-menu><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li></ul></div><li><a class="chevron-toggle collapsed" data-toggle=collaps [...]
+<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li></ul></div><li [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Spark class=collapse><ul class=sub-menu><li><a href=../spark-ddl/>DDL</a></li><li><a href=../getting-started/>Getting Started</a></li><li><a href=../spark-procedures/>Procedures</a></li><li><a href=../spark-queries/>Queries</a></li><li><a href=../spark-structured-streaming/>Structured Streaming</a></li><li><a href=../spark-writes/>Writes</a></li></ul></div><li><a class=chevron-toggle data-toggle=collapse data-parent=full href=#Flink><spa [...]
<i class="fa fa-chevron-right"></i>
@@ -13,7 +13,9 @@
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Integrations class=collapse><ul class=sub-menu><li><a href=../aws/>AWS</a></li><li><a href=../dell/>Dell</a></li><li><a href=../jdbc/>JDBC</a></li><li><a href=../nessie/>Nessie</a></li></ul></div><li><a class="chevron-toggle collapsed" data-toggle=collapse data-parent=full href=#API><span>API</span>
<i class="fa fa-chevron-right"></i>
-<i class="fa fa-chevron-down"></i></a></li><div id=API class=collapse><ul class=sub-menu><li><a href=../java-api-quickstart/>Java Quickstart</a></li><li><a href=../api/>Java API</a></li><li><a href=../custom-catalog/>Java Custom Catalog</a></li></ul></div><li><a href=https://iceberg.apache.org/docs/1.2.1/../../javadoc/latest><span>Javadoc</span></a></li><li><a target=_blank href=https://py.iceberg.apache.org/><span>PyIceberg</span></a></li></div></div><div id=content class=markdown-body> [...]
+<i class="fa fa-chevron-down"></i></a></li><div id=API class=collapse><ul class=sub-menu><li><a href=../java-api-quickstart/>Java Quickstart</a></li><li><a href=../api/>Java API</a></li><li><a href=../custom-catalog/>Java Custom Catalog</a></li></ul></div><li><a href=https://iceberg.apache.org/docs/1.2.1/../../javadoc/latest><span>Javadoc</span></a></li><li><a target=_blank href=https://py.iceberg.apache.org/><span>PyIceberg</span></a></li><li><a class="chevron-toggle collapsed" data-tog [...]
+<i class="fa fa-chevron-right"></i>
+<i class="fa fa-chevron-down"></i></a></li><div id=Migration class=collapse><ul class=sub-menu><li><a href=../table-migration/>Overview</a></li><li><a href=../hive-migration/>Hive Migration</a></li><li><a href=../delta-lake-migration/>Delta Lake Migration</a></li></ul></div></div></div><div id=content class=markdown-body><div class=margin-for-toc><h2 id=rewrite-files-action>Rewrite files action.</h2><p>Iceberg provides API to rewrite small files into large files by submitting Flink batch [...]
</span></span><span style=display:flex><span>
</span></span><span style=display:flex><span>TableLoader tableLoader <span style=color:#f92672>=</span> TableLoader<span style=color:#f92672>.</span><span style=color:#a6e22e>fromHadoopTable</span><span style=color:#f92672>(</span><span style=color:#e6db74>"hdfs://nn:8020/warehouse/path"</span><span style=color:#f92672>);</span>
</span></span><span style=display:flex><span>Table table <span style=color:#f92672>=</span> tableLoader<span style=color:#f92672>.</span><span style=color:#a6e22e>loadTable</span><span style=color:#f92672>();</span>
diff --git a/docs/1.2.1/flink-configuration/index.html b/docs/1.2.1/flink-configuration/index.html
index 8148b9d5..e4611138 100644
--- a/docs/1.2.1/flink-configuration/index.html
+++ b/docs/1.2.1/flink-configuration/index.html
@@ -5,7 +5,7 @@
<span class=icon-bar></span></button>
<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/1.2.1//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.2.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg.a [...]
<i class="fa fa-chevron-right"></i>
-<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul class=sub-menu><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li></ul></div><li><a class="chevron-toggle collapsed" data-toggle=collaps [...]
+<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li></ul></div><li [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Spark class=collapse><ul class=sub-menu><li><a href=../spark-ddl/>DDL</a></li><li><a href=../getting-started/>Getting Started</a></li><li><a href=../spark-procedures/>Procedures</a></li><li><a href=../spark-queries/>Queries</a></li><li><a href=../spark-structured-streaming/>Structured Streaming</a></li><li><a href=../spark-writes/>Writes</a></li></ul></div><li><a class=chevron-toggle data-toggle=collapse data-parent=full href=#Flink><spa [...]
<i class="fa fa-chevron-right"></i>
@@ -13,7 +13,9 @@
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Integrations class=collapse><ul class=sub-menu><li><a href=../aws/>AWS</a></li><li><a href=../dell/>Dell</a></li><li><a href=../jdbc/>JDBC</a></li><li><a href=../nessie/>Nessie</a></li></ul></div><li><a class="chevron-toggle collapsed" data-toggle=collapse data-parent=full href=#API><span>API</span>
<i class="fa fa-chevron-right"></i>
-<i class="fa fa-chevron-down"></i></a></li><div id=API class=collapse><ul class=sub-menu><li><a href=../java-api-quickstart/>Java Quickstart</a></li><li><a href=../api/>Java API</a></li><li><a href=../custom-catalog/>Java Custom Catalog</a></li></ul></div><li><a href=https://iceberg.apache.org/docs/1.2.1/../../javadoc/latest><span>Javadoc</span></a></li><li><a target=_blank href=https://py.iceberg.apache.org/><span>PyIceberg</span></a></li></div></div><div id=content class=markdown-body> [...]
+<i class="fa fa-chevron-down"></i></a></li><div id=API class=collapse><ul class=sub-menu><li><a href=../java-api-quickstart/>Java Quickstart</a></li><li><a href=../api/>Java API</a></li><li><a href=../custom-catalog/>Java Custom Catalog</a></li></ul></div><li><a href=https://iceberg.apache.org/docs/1.2.1/../../javadoc/latest><span>Javadoc</span></a></li><li><a target=_blank href=https://py.iceberg.apache.org/><span>PyIceberg</span></a></li><li><a class="chevron-toggle collapsed" data-tog [...]
+<i class="fa fa-chevron-right"></i>
+<i class="fa fa-chevron-down"></i></a></li><div id=Migration class=collapse><ul class=sub-menu><li><a href=../table-migration/>Overview</a></li><li><a href=../hive-migration/>Hive Migration</a></li><li><a href=../delta-lake-migration/>Delta Lake Migration</a></li></ul></div></div></div><div id=content class=markdown-body><div class=margin-for-toc><h1 id=flink-configuration>Flink Configuration</h1><h2 id=catalog-configuration>Catalog Configuration</h2><p>A catalog is created and named by [...]
<code><config_key></code>=<code><config_value></code> with catalog implementation config):</p><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-sql data-lang=sql><span style=display:flex><span><span style=color:#66d9ef>CREATE</span> <span style=color:#66d9ef>CATALOG</span> <span style=color:#f92672><</span><span style=color:#66d9ef>catalog_name</span><span style=color:#f92672>></span [...]
</span></span><span style=display:flex><span> <span style=color:#e6db74>'type'</span><span style=color:#f92672>=</span><span style=color:#e6db74>'iceberg'</span>,
</span></span><span style=display:flex><span> <span style=color:#f92672>`<</span>config_key<span style=color:#f92672>>`=`<</span>config_value<span style=color:#f92672>>`</span>
@@ -32,7 +34,7 @@
.getConfiguration()
.set(FlinkReadOptions.SPLIT_FILE_OPEN_COST_OPTION, 1000L);
...
-</code></pre><p><code>Read option</code> has the highest priority, followed by <code>Flink configuration</code> and then <code>Table property</code>.</p><table><thead><tr><th>Read option</th><th>Flink configuration</th><th>Table property</th><th>Default</th><th>Description</th></tr></thead><tbody><tr><td>snapshot-id</td><td>N/A</td><td>N/A</td><td>null</td><td>For time travel in batch mode. Read data from the specified snapshot-id.</td></tr><tr><td>case-sensitive</td><td>connector.iceber [...]
+</code></pre><p><code>Read option</code> has the highest priority, followed by <code>Flink configuration</code> and then <code>Table property</code>.</p><table><thead><tr><th>Read option</th><th>Flink configuration</th><th>Table property</th><th>Default</th><th>Description</th></tr></thead><tbody><tr><td>snapshot-id</td><td>N/A</td><td>N/A</td><td>null</td><td>For time travel in batch mode. Read data from the specified snapshot-id.</td></tr><tr><td>case-sensitive</td><td>connector.iceber [...]
.table(table)
.tableLoader(tableLoader)
.set("write-format", "orc")
diff --git a/docs/1.2.1/flink-connector/index.html b/docs/1.2.1/flink-connector/index.html
index 5948572e..2798ba31 100644
--- a/docs/1.2.1/flink-connector/index.html
+++ b/docs/1.2.1/flink-connector/index.html
@@ -5,7 +5,7 @@
<span class=icon-bar></span></button>
<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/1.2.1//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.2.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg.a [...]
<i class="fa fa-chevron-right"></i>
-<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul class=sub-menu><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li></ul></div><li><a class="chevron-toggle collapsed" data-toggle=collaps [...]
+<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li></ul></div><li [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Spark class=collapse><ul class=sub-menu><li><a href=../spark-ddl/>DDL</a></li><li><a href=../getting-started/>Getting Started</a></li><li><a href=../spark-procedures/>Procedures</a></li><li><a href=../spark-queries/>Queries</a></li><li><a href=../spark-structured-streaming/>Structured Streaming</a></li><li><a href=../spark-writes/>Writes</a></li></ul></div><li><a class=chevron-toggle data-toggle=collapse data-parent=full href=#Flink><spa [...]
<i class="fa fa-chevron-right"></i>
@@ -13,7 +13,9 @@
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Integrations class=collapse><ul class=sub-menu><li><a href=../aws/>AWS</a></li><li><a href=../dell/>Dell</a></li><li><a href=../jdbc/>JDBC</a></li><li><a href=../nessie/>Nessie</a></li></ul></div><li><a class="chevron-toggle collapsed" data-toggle=collapse data-parent=full href=#API><span>API</span>
<i class="fa fa-chevron-right"></i>
-<i class="fa fa-chevron-down"></i></a></li><div id=API class=collapse><ul class=sub-menu><li><a href=../java-api-quickstart/>Java Quickstart</a></li><li><a href=../api/>Java API</a></li><li><a href=../custom-catalog/>Java Custom Catalog</a></li></ul></div><li><a href=https://iceberg.apache.org/docs/1.2.1/../../javadoc/latest><span>Javadoc</span></a></li><li><a target=_blank href=https://py.iceberg.apache.org/><span>PyIceberg</span></a></li></div></div><div id=content class=markdown-body> [...]
+<i class="fa fa-chevron-down"></i></a></li><div id=API class=collapse><ul class=sub-menu><li><a href=../java-api-quickstart/>Java Quickstart</a></li><li><a href=../api/>Java API</a></li><li><a href=../custom-catalog/>Java Custom Catalog</a></li></ul></div><li><a href=https://iceberg.apache.org/docs/1.2.1/../../javadoc/latest><span>Javadoc</span></a></li><li><a target=_blank href=https://py.iceberg.apache.org/><span>PyIceberg</span></a></li><li><a class="chevron-toggle collapsed" data-tog [...]
+<i class="fa fa-chevron-right"></i>
+<i class="fa fa-chevron-down"></i></a></li><div id=Migration class=collapse><ul class=sub-menu><li><a href=../table-migration/>Overview</a></li><li><a href=../hive-migration/>Hive Migration</a></li><li><a href=../delta-lake-migration/>Delta Lake Migration</a></li></ul></div></div></div><div id=content class=markdown-body><div class=margin-for-toc><h1 id=flink-connector>Flink Connector</h1><p>Apache Flink supports creating Iceberg table directly without creating the explicit Flink catalog [...]
which is just mapping to the underlying iceberg table instead of maintaining iceberg table directly in current Flink catalog.</p><p>To create the table in Flink SQL by using SQL syntax <code>CREATE TABLE test (..) WITH ('connector'='iceberg', ...)</code>, Flink iceberg connector provides the following table properties:</p><ul><li><code>connector</code>: Use the constant <code>iceberg</code>.</li><li><code>catalog-name</code>: User-specified catalog name. It’s required because the c [...]
</span></span><span style=display:flex><span> id BIGINT,
</span></span><span style=display:flex><span> <span style=color:#66d9ef>data</span> STRING
diff --git a/docs/1.2.1/flink-ddl/index.html b/docs/1.2.1/flink-ddl/index.html
index 9eaf006e..0b4ba29a 100644
--- a/docs/1.2.1/flink-ddl/index.html
+++ b/docs/1.2.1/flink-ddl/index.html
@@ -5,7 +5,7 @@
<span class=icon-bar></span></button>
<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/1.2.1//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.2.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg.a [...]
<i class="fa fa-chevron-right"></i>
-<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul class=sub-menu><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li></ul></div><li><a class="chevron-toggle collapsed" data-toggle=collaps [...]
+<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li></ul></div><li [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Spark class=collapse><ul class=sub-menu><li><a href=../spark-ddl/>DDL</a></li><li><a href=../getting-started/>Getting Started</a></li><li><a href=../spark-procedures/>Procedures</a></li><li><a href=../spark-queries/>Queries</a></li><li><a href=../spark-structured-streaming/>Structured Streaming</a></li><li><a href=../spark-writes/>Writes</a></li></ul></div><li><a class=chevron-toggle data-toggle=collapse data-parent=full href=#Flink><spa [...]
<i class="fa fa-chevron-right"></i>
@@ -13,7 +13,9 @@
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Integrations class=collapse><ul class=sub-menu><li><a href=../aws/>AWS</a></li><li><a href=../dell/>Dell</a></li><li><a href=../jdbc/>JDBC</a></li><li><a href=../nessie/>Nessie</a></li></ul></div><li><a class="chevron-toggle collapsed" data-toggle=collapse data-parent=full href=#API><span>API</span>
<i class="fa fa-chevron-right"></i>
-<i class="fa fa-chevron-down"></i></a></li><div id=API class=collapse><ul class=sub-menu><li><a href=../java-api-quickstart/>Java Quickstart</a></li><li><a href=../api/>Java API</a></li><li><a href=../custom-catalog/>Java Custom Catalog</a></li></ul></div><li><a href=https://iceberg.apache.org/docs/1.2.1/../../javadoc/latest><span>Javadoc</span></a></li><li><a target=_blank href=https://py.iceberg.apache.org/><span>PyIceberg</span></a></li></div></div><div id=content class=markdown-body> [...]
+<i class="fa fa-chevron-down"></i></a></li><div id=API class=collapse><ul class=sub-menu><li><a href=../java-api-quickstart/>Java Quickstart</a></li><li><a href=../api/>Java API</a></li><li><a href=../custom-catalog/>Java Custom Catalog</a></li></ul></div><li><a href=https://iceberg.apache.org/docs/1.2.1/../../javadoc/latest><span>Javadoc</span></a></li><li><a target=_blank href=https://py.iceberg.apache.org/><span>PyIceberg</span></a></li><li><a class="chevron-toggle collapsed" data-tog [...]
+<i class="fa fa-chevron-right"></i>
+<i class="fa fa-chevron-down"></i></a></li><div id=Migration class=collapse><ul class=sub-menu><li><a href=../table-migration/>Overview</a></li><li><a href=../hive-migration/>Hive Migration</a></li><li><a href=../delta-lake-migration/>Delta Lake Migration</a></li></ul></div></div></div><div id=content class=markdown-body><div class=margin-for-toc><h2 id=ddl-commands>DDL commands</h2><h3 id=create-catalog><code>CREATE Catalog</code></h3><h4 id=hive-catalog>Hive catalog</h4><p>This creates [...]
</span></span><span style=display:flex><span> <span style=color:#e6db74>'type'</span><span style=color:#f92672>=</span><span style=color:#e6db74>'iceberg'</span>,
</span></span><span style=display:flex><span> <span style=color:#e6db74>'catalog-type'</span><span style=color:#f92672>=</span><span style=color:#e6db74>'hive'</span>,
</span></span><span style=display:flex><span> <span style=color:#e6db74>'uri'</span><span style=color:#f92672>=</span><span style=color:#e6db74>'thrift://localhost:9083'</span>,
diff --git a/docs/1.2.1/flink-queries/index.html b/docs/1.2.1/flink-queries/index.html
index e61faf68..647354e1 100644
--- a/docs/1.2.1/flink-queries/index.html
+++ b/docs/1.2.1/flink-queries/index.html
@@ -5,7 +5,7 @@
<span class=icon-bar></span></button>
<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/1.2.1//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.2.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg.a [...]
<i class="fa fa-chevron-right"></i>
-<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul class=sub-menu><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li></ul></div><li><a class="chevron-toggle collapsed" data-toggle=collaps [...]
+<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li></ul></div><li [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Spark class=collapse><ul class=sub-menu><li><a href=../spark-ddl/>DDL</a></li><li><a href=../getting-started/>Getting Started</a></li><li><a href=../spark-procedures/>Procedures</a></li><li><a href=../spark-queries/>Queries</a></li><li><a href=../spark-structured-streaming/>Structured Streaming</a></li><li><a href=../spark-writes/>Writes</a></li></ul></div><li><a class=chevron-toggle data-toggle=collapse data-parent=full href=#Flink><spa [...]
<i class="fa fa-chevron-right"></i>
@@ -13,7 +13,9 @@
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Integrations class=collapse><ul class=sub-menu><li><a href=../aws/>AWS</a></li><li><a href=../dell/>Dell</a></li><li><a href=../jdbc/>JDBC</a></li><li><a href=../nessie/>Nessie</a></li></ul></div><li><a class="chevron-toggle collapsed" data-toggle=collapse data-parent=full href=#API><span>API</span>
<i class="fa fa-chevron-right"></i>
-<i class="fa fa-chevron-down"></i></a></li><div id=API class=collapse><ul class=sub-menu><li><a href=../java-api-quickstart/>Java Quickstart</a></li><li><a href=../api/>Java API</a></li><li><a href=../custom-catalog/>Java Custom Catalog</a></li></ul></div><li><a href=https://iceberg.apache.org/docs/1.2.1/../../javadoc/latest><span>Javadoc</span></a></li><li><a target=_blank href=https://py.iceberg.apache.org/><span>PyIceberg</span></a></li></div></div><div id=content class=markdown-body> [...]
+<i class="fa fa-chevron-down"></i></a></li><div id=API class=collapse><ul class=sub-menu><li><a href=../java-api-quickstart/>Java Quickstart</a></li><li><a href=../api/>Java API</a></li><li><a href=../custom-catalog/>Java Custom Catalog</a></li></ul></div><li><a href=https://iceberg.apache.org/docs/1.2.1/../../javadoc/latest><span>Javadoc</span></a></li><li><a target=_blank href=https://py.iceberg.apache.org/><span>PyIceberg</span></a></li><li><a class="chevron-toggle collapsed" data-tog [...]
+<i class="fa fa-chevron-right"></i>
+<i class="fa fa-chevron-down"></i></a></li><div id=Migration class=collapse><ul class=sub-menu><li><a href=../table-migration/>Overview</a></li><li><a href=../hive-migration/>Hive Migration</a></li><li><a href=../delta-lake-migration/>Delta Lake Migration</a></li></ul></div></div></div><div id=content class=markdown-body><div class=margin-for-toc><h1 id=flink-queries>Flink Queries</h1><p>Iceberg support streaming and batch read With <a href=https://flink.apache.org/>Apache Flink</a>&rsqu [...]
</span></span></span><span style=display:flex><span><span style=color:#75715e></span><span style=color:#66d9ef>SET</span> execution.runtime<span style=color:#f92672>-</span><span style=color:#66d9ef>mode</span> <span style=color:#f92672>=</span> streaming;
</span></span><span style=display:flex><span>
</span></span><span style=display:flex><span><span style=color:#75715e>-- Execute the flink job in batch mode for current session context
@@ -34,6 +36,15 @@
</span></span></span><span style=display:flex><span><span style=color:#75715e></span><span style=color:#66d9ef>SELECT</span> <span style=color:#f92672>*</span> <span style=color:#66d9ef>FROM</span> sample <span style=color:#75715e>/*+ OPTIONS('streaming'='true', 'monitor-interval'='1s', 'start-snapshot-id'='3821550127947089987')*/</span> ;
</span></span></code></pre></div><p>There are some options that could be set in Flink SQL hint options for streaming job, see <a href=#Read-options>read options</a> for details.</p><h3 id=flip-27-source-for-sql>FLIP-27 source for SQL</h3><p>Here are the SQL settings for the <a href=https://cwiki.apache.org/confluence/display/FLINK/FLIP-27%3A+Refactor+Source+Interface>FLIP-27</a> source. All other SQL settings and options documented above are applicable to the FLIP-27 source.</p><div clas [...]
</span></span></span><span style=display:flex><span><span style=color:#75715e></span><span style=color:#66d9ef>SET</span> <span style=color:#66d9ef>table</span>.<span style=color:#66d9ef>exec</span>.iceberg.use<span style=color:#f92672>-</span>flip27<span style=color:#f92672>-</span><span style=color:#66d9ef>source</span> <span style=color:#f92672>=</span> <span style=color:#66d9ef>true</span>;
+</span></span></code></pre></div><h3 id=reading-branches-and-tags-with-sql>Reading branches and tags with SQL</h3><p>Branch and tags can be read via SQL by specifying options. For more details
+refer to <a href=../flink-configuration/#read-options>Flink Configuration</a></p><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-sql data-lang=sql><span style=display:flex><span><span style=color:#75715e>--- Read from branch b1
+</span></span></span><span style=display:flex><span><span style=color:#75715e></span><span style=color:#66d9ef>SELECT</span> <span style=color:#f92672>*</span> <span style=color:#66d9ef>FROM</span> <span style=color:#66d9ef>table</span> <span style=color:#75715e>/*+ OPTIONS('branch'='b1') */</span> ;
+</span></span><span style=display:flex><span>
+</span></span><span style=display:flex><span><span style=color:#75715e>--- Read from tag t1
+</span></span></span><span style=display:flex><span><span style=color:#75715e></span><span style=color:#66d9ef>SELECT</span> <span style=color:#f92672>*</span> <span style=color:#66d9ef>FROM</span> <span style=color:#66d9ef>table</span> <span style=color:#75715e>/*+ OPTIONS('tag'='t1') */</span>;
+</span></span><span style=display:flex><span>
+</span></span><span style=display:flex><span><span style=color:#75715e>--- Incremental scan from tag t1 to tag t2
+</span></span></span><span style=display:flex><span><span style=color:#75715e></span><span style=color:#66d9ef>SELECT</span> <span style=color:#f92672>*</span> <span style=color:#66d9ef>FROM</span> <span style=color:#66d9ef>table</span> <span style=color:#75715e>/*+ OPTIONS('streaming'='true', 'monitor-interval'='1s', 'start-tag'='t1', 'end-tag'='t2') */</span>;
</span></span></code></pre></div><h2 id=reading-with-datastream>Reading with DataStream</h2><p>Iceberg support streaming or batch read in Java API now.</p><h3 id=batch-read>Batch Read</h3><p>This example will read all records from iceberg table and then print to the stdout console in flink batch job:</p><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-java data-lang=java><span style=display:fle [...]
</span></span><span style=display:flex><span>TableLoader tableLoader <span style=color:#f92672>=</span> TableLoader<span style=color:#f92672>.</span><span style=color:#a6e22e>fromHadoopTable</span><span style=color:#f92672>(</span><span style=color:#e6db74>"hdfs://nn:8020/warehouse/path"</span><span style=color:#f92672>);</span>
</span></span><span style=display:flex><span>DataStream<span style=color:#f92672><</span>RowData<span style=color:#f92672>></span> batch <span style=color:#f92672>=</span> FlinkSource<span style=color:#f92672>.</span><span style=color:#a6e22e>forRowData</span><span style=color:#f92672>()</span>
@@ -109,7 +120,32 @@ CDC read is not supported yet.</p><div class=highlight><pre tabindex=0 style=col
</span></span><span style=display:flex><span><span style=color:#75715e>// Submit and execute this streaming read job.
</span></span></span><span style=display:flex><span><span style=color:#75715e></span>env<span style=color:#f92672>.</span><span style=color:#a6e22e>execute</span><span style=color:#f92672>(</span><span style=color:#e6db74>"Test Iceberg Streaming Read"</span><span style=color:#f92672>);</span>
</span></span></code></pre></div><p>There are other options that could be set by Java API, please see the
-<a href=../../../javadoc/1.2.0/org/apache/iceberg/flink/source/IcebergSource.html>IcebergSource#Builder</a>.</p><h3 id=read-as-avro-genericrecord>Read as Avro GenericRecord</h3><p>FLIP-27 Iceberg source provides <code>AvroGenericRecordReaderFunction</code> that converts
+<a href=../../../javadoc/1.2.0/org/apache/iceberg/flink/source/IcebergSource.html>IcebergSource#Builder</a>.</p><h3 id=reading-branches-and-tags-with-datastream>Reading branches and tags with DataStream</h3><p>Branches and tags can also be read via the DataStream API</p><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-java data-lang=java><span style=display:flex><span>StreamExecutionEnvironment [...]
+</span></span><span style=display:flex><span>TableLoader tableLoader <span style=color:#f92672>=</span> TableLoader<span style=color:#f92672>.</span><span style=color:#a6e22e>fromHadoopTable</span><span style=color:#f92672>(</span><span style=color:#e6db74>"hdfs://nn:8020/warehouse/path"</span><span style=color:#f92672>);</span>
+</span></span><span style=display:flex><span><span style=color:#75715e>// Read from branch
+</span></span></span><span style=display:flex><span><span style=color:#75715e></span>DataStream<span style=color:#f92672><</span>RowData<span style=color:#f92672>></span> batch <span style=color:#f92672>=</span> FlinkSource<span style=color:#f92672>.</span><span style=color:#a6e22e>forRowData</span><span style=color:#f92672>()</span>
+</span></span><span style=display:flex><span> <span style=color:#f92672>.</span><span style=color:#a6e22e>env</span><span style=color:#f92672>(</span>env<span style=color:#f92672>)</span>
+</span></span><span style=display:flex><span> <span style=color:#f92672>.</span><span style=color:#a6e22e>tableLoader</span><span style=color:#f92672>(</span>tableLoader<span style=color:#f92672>)</span>
+</span></span><span style=display:flex><span> <span style=color:#f92672>.</span><span style=color:#a6e22e>branch</span><span style=color:#f92672>(</span><span style=color:#e6db74>"test-branch"</span><span style=color:#f92672>)</span>
+</span></span><span style=display:flex><span> <span style=color:#f92672>.</span><span style=color:#a6e22e>streaming</span><span style=color:#f92672>(</span><span style=color:#66d9ef>false</span><span style=color:#f92672>)</span>
+</span></span><span style=display:flex><span> <span style=color:#f92672>.</span><span style=color:#a6e22e>build</span><span style=color:#f92672>();</span>
+</span></span><span style=display:flex><span>
+</span></span><span style=display:flex><span><span style=color:#75715e>// Read from tag
+</span></span></span><span style=display:flex><span><span style=color:#75715e></span>DataStream<span style=color:#f92672><</span>RowData<span style=color:#f92672>></span> batch <span style=color:#f92672>=</span> FlinkSource<span style=color:#f92672>.</span><span style=color:#a6e22e>forRowData</span><span style=color:#f92672>()</span>
+</span></span><span style=display:flex><span> <span style=color:#f92672>.</span><span style=color:#a6e22e>env</span><span style=color:#f92672>(</span>env<span style=color:#f92672>)</span>
+</span></span><span style=display:flex><span> <span style=color:#f92672>.</span><span style=color:#a6e22e>tableLoader</span><span style=color:#f92672>(</span>tableLoader<span style=color:#f92672>)</span>
+</span></span><span style=display:flex><span> <span style=color:#f92672>.</span><span style=color:#a6e22e>tag</span><span style=color:#f92672>(</span><span style=color:#e6db74>"test-tag"</span><span style=color:#f92672>)</span>
+</span></span><span style=display:flex><span> <span style=color:#f92672>.</span><span style=color:#a6e22e>streaming</span><span style=color:#f92672>(</span><span style=color:#66d9ef>false</span><span style=color:#f92672>)</span>
+</span></span><span style=display:flex><span> <span style=color:#f92672>.</span><span style=color:#a6e22e>build</span><span style=color:#f92672>();</span>
+</span></span><span style=display:flex><span>
+</span></span><span style=display:flex><span><span style=color:#75715e>// Streaming read from start-tag
+</span></span></span><span style=display:flex><span><span style=color:#75715e></span>DataStream<span style=color:#f92672><</span>RowData<span style=color:#f92672>></span> batch <span style=color:#f92672>=</span> FlinkSource<span style=color:#f92672>.</span><span style=color:#a6e22e>forRowData</span><span style=color:#f92672>()</span>
+</span></span><span style=display:flex><span> <span style=color:#f92672>.</span><span style=color:#a6e22e>env</span><span style=color:#f92672>(</span>env<span style=color:#f92672>)</span>
+</span></span><span style=display:flex><span> <span style=color:#f92672>.</span><span style=color:#a6e22e>tableLoader</span><span style=color:#f92672>(</span>tableLoader<span style=color:#f92672>)</span>
+</span></span><span style=display:flex><span> <span style=color:#f92672>.</span><span style=color:#a6e22e>streaming</span><span style=color:#f92672>(</span><span style=color:#66d9ef>true</span><span style=color:#f92672>)</span>
+</span></span><span style=display:flex><span> <span style=color:#f92672>.</span><span style=color:#a6e22e>startTag</span><span style=color:#f92672>(</span><span style=color:#e6db74>"test-tag"</span><span style=color:#f92672>)</span>
+</span></span><span style=display:flex><span> <span style=color:#f92672>.</span><span style=color:#a6e22e>build</span><span style=color:#f92672>();</span>
+</span></span></code></pre></div><h3 id=read-as-avro-genericrecord>Read as Avro GenericRecord</h3><p>FLIP-27 Iceberg source provides <code>AvroGenericRecordReaderFunction</code> that converts
Flink <code>RowData</code> Avro <code>GenericRecord</code>. You can use the convert to read from
Iceberg table as Avro GenericRecord DataStream.</p><p>Please make sure <code>flink-avro</code> jar is included in the classpath.
Also <code>iceberg-flink-runtime</code> shaded bundle jar can’t be used
@@ -169,7 +205,7 @@ For unpartitioned tables, the partitions table will contain only the record_coun
</span></span></code></pre></div><table><thead><tr><th>content</th><th>file_path</th><th>file_format</th><th>partition</th><th>record_count</th><th>file_size_in_bytes</th><th>column_sizes</th><th>value_counts</th><th>null_value_counts</th><th>nan_value_counts</th><th>lower_bounds</th><th>upper_bounds</th><th>key_metadata</th><th>split_offsets</th><th>equality_ids</th><th>sort_order_id</th></tr></thead><tbody><tr><td>0</td><td>s3://…/dt=20210102/00000-0-756e2512-49ae-45bb-aae3-c0ca47 [...]
</span></span></code></pre></div><table><thead><tr><th>path</th><th>length</th><th>partition_spec_id</th><th>added_snapshot_id</th><th>added_data_files_count</th><th>existing_data_files_count</th><th>deleted_data_files_count</th><th>partition_summaries</th></tr></thead><tbody><tr><td>s3://…/metadata/a85f78c5-3222-4b37-b7e4-faf944425d48-m0.avro</td><td>6376</td><td>0</td><td>6272782676904868561</td><td>2</td><td>0</td><td>0</td><td>[{false, false, 20210101, 20210101}]</td></tr></tbod [...]
This usually occurs when reading from V1 table, where <code>contains_nan</code> is not populated.</li></ol><h3 id=references>References</h3><p>To show a table’s known snapshot references:</p><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-sql data-lang=sql><span style=display:flex><span><span style=color:#66d9ef>SELECT</span> <span style=color:#f92672>*</span> <span style=color:#66d9ef>F [...]
-</span></span></code></pre></div><table><thead><tr><th>name</th><th>type</th><th>snapshot_id</th><th>max_reference_age_in_ms</th><th>min_snapshots_to_keep</th><th>max_snapshot_age_in_ms</th></tr></thead><tbody><tr><td>main</td><td>BRANCH</td><td>4686954189838128572</td><td>10</td><td>20</td><td>30</td></tr><tr><td>testTag</td><td>TAG</td><td>4686954189838128572</td><td>10</td><td>null</td><td>null</td></tr></tbody></table></div><div id=toc class=markdown-body><div id=full><nav id=TableOf [...]
+</span></span></code></pre></div><table><thead><tr><th>name</th><th>type</th><th>snapshot_id</th><th>max_reference_age_in_ms</th><th>min_snapshots_to_keep</th><th>max_snapshot_age_in_ms</th></tr></thead><tbody><tr><td>main</td><td>BRANCH</td><td>4686954189838128572</td><td>10</td><td>20</td><td>30</td></tr><tr><td>testTag</td><td>TAG</td><td>4686954189838128572</td><td>10</td><td>null</td><td>null</td></tr></tbody></table></div><div id=toc class=markdown-body><div id=full><nav id=TableOf [...]
<script src=https://iceberg.apache.org/docs/1.2.1//js/jquery.easing.min.js></script>
<script type=text/javascript src=https://iceberg.apache.org/docs/1.2.1//js/search.js></script>
<script src=https://iceberg.apache.org/docs/1.2.1//js/bootstrap.min.js></script>
diff --git a/docs/1.2.1/flink-writes/index.html b/docs/1.2.1/flink-writes/index.html
index 4a77e0ea..fbdb067e 100644
--- a/docs/1.2.1/flink-writes/index.html
+++ b/docs/1.2.1/flink-writes/index.html
@@ -5,7 +5,7 @@
<span class=icon-bar></span></button>
<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/1.2.1//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.2.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg.a [...]
<i class="fa fa-chevron-right"></i>
-<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul class=sub-menu><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li></ul></div><li><a class="chevron-toggle collapsed" data-toggle=collaps [...]
+<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li></ul></div><li [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Spark class=collapse><ul class=sub-menu><li><a href=../spark-ddl/>DDL</a></li><li><a href=../getting-started/>Getting Started</a></li><li><a href=../spark-procedures/>Procedures</a></li><li><a href=../spark-queries/>Queries</a></li><li><a href=../spark-structured-streaming/>Structured Streaming</a></li><li><a href=../spark-writes/>Writes</a></li></ul></div><li><a class=chevron-toggle data-toggle=collapse data-parent=full href=#Flink><spa [...]
<i class="fa fa-chevron-right"></i>
@@ -13,7 +13,9 @@
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Integrations class=collapse><ul class=sub-menu><li><a href=../aws/>AWS</a></li><li><a href=../dell/>Dell</a></li><li><a href=../jdbc/>JDBC</a></li><li><a href=../nessie/>Nessie</a></li></ul></div><li><a class="chevron-toggle collapsed" data-toggle=collapse data-parent=full href=#API><span>API</span>
<i class="fa fa-chevron-right"></i>
-<i class="fa fa-chevron-down"></i></a></li><div id=API class=collapse><ul class=sub-menu><li><a href=../java-api-quickstart/>Java Quickstart</a></li><li><a href=../api/>Java API</a></li><li><a href=../custom-catalog/>Java Custom Catalog</a></li></ul></div><li><a href=https://iceberg.apache.org/docs/1.2.1/../../javadoc/latest><span>Javadoc</span></a></li><li><a target=_blank href=https://py.iceberg.apache.org/><span>PyIceberg</span></a></li></div></div><div id=content class=markdown-body> [...]
+<i class="fa fa-chevron-down"></i></a></li><div id=API class=collapse><ul class=sub-menu><li><a href=../java-api-quickstart/>Java Quickstart</a></li><li><a href=../api/>Java API</a></li><li><a href=../custom-catalog/>Java Custom Catalog</a></li></ul></div><li><a href=https://iceberg.apache.org/docs/1.2.1/../../javadoc/latest><span>Javadoc</span></a></li><li><a target=_blank href=https://py.iceberg.apache.org/><span>PyIceberg</span></a></li><li><a class="chevron-toggle collapsed" data-tog [...]
+<i class="fa fa-chevron-right"></i>
+<i class="fa fa-chevron-down"></i></a></li><div id=Migration class=collapse><ul class=sub-menu><li><a href=../table-migration/>Overview</a></li><li><a href=../hive-migration/>Hive Migration</a></li><li><a href=../delta-lake-migration/>Delta Lake Migration</a></li></ul></div></div></div><div id=content class=markdown-body><div class=margin-for-toc><h1 id=flink-writes>Flink Writes</h1><p>Iceberg support batch and streaming writes With <a href=https://flink.apache.org/>Apache Flink</a>&rsqu [...]
</span></span><span style=display:flex><span><span style=color:#66d9ef>INSERT</span> <span style=color:#66d9ef>INTO</span> <span style=color:#f92672>`</span>hive_catalog<span style=color:#f92672>`</span>.<span style=color:#f92672>`</span><span style=color:#66d9ef>default</span><span style=color:#f92672>`</span>.<span style=color:#f92672>`</span>sample<span style=color:#f92672>`</span> <span style=color:#66d9ef>SELECT</span> id, <span style=color:#66d9ef>data</span> <span style=color:#66d [...]
</span></span></code></pre></div><h3 id=insert-overwrite><code>INSERT OVERWRITE</code></h3><p>To replace data in the table with the result of a query, use <code>INSERT OVERWRITE</code> in batch job (flink streaming job does not support <code>INSERT OVERWRITE</code>). Overwrites are atomic operations for Iceberg tables.</p><p>Partitions that have rows produced by the SELECT query will be replaced, for example:</p><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#2 [...]
</span></span></code></pre></div><p>Iceberg also support overwriting given partitions by the <code>select</code> values:</p><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-sql data-lang=sql><span style=display:flex><span><span style=color:#66d9ef>INSERT</span> OVERWRITE <span style=color:#f92672>`</span>hive_catalog<span style=color:#f92672>`</span>.<span style=color:#f92672>`</span><span styl [...]
@@ -87,6 +89,11 @@ Please use non-shaded <code>iceberg-flink</code> jar instead.</p><div class=high
</span></span><span style=display:flex><span> <span style=color:#f92672>.</span><span style=color:#a6e22e>table</span><span style=color:#f92672>(</span>table<span style=color:#f92672>)</span>
</span></span><span style=display:flex><span> <span style=color:#f92672>.</span><span style=color:#a6e22e>tableLoader</span><span style=color:#f92672>(</span>tableLoader<span style=color:#f92672>)</span>
</span></span><span style=display:flex><span> <span style=color:#f92672>.</span><span style=color:#a6e22e>append</span><span style=color:#f92672>();</span>
+</span></span></code></pre></div><h3 id=branch-writes>Branch Writes</h3><p>Writing to branches in Iceberg tables is also supported via the <code>toBranch</code> API in <code>FlinkSink</code>
+For more information on branches please refer to <a href=../../tables/branching>branches</a>.</p><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-java data-lang=java><span style=display:flex><span>FlinkSink<span style=color:#f92672>.</span><span style=color:#a6e22e>forRowData</span><span style=color:#f92672>(</span>input<span style=color:#f92672>)</span>
+</span></span><span style=display:flex><span> <span style=color:#f92672>.</span><span style=color:#a6e22e>tableLoader</span><span style=color:#f92672>(</span>tableLoader<span style=color:#f92672>)</span>
+</span></span><span style=display:flex><span> <span style=color:#f92672>.</span><span style=color:#a6e22e>toBranch</span><span style=color:#f92672>(</span><span style=color:#e6db74>"audit-branch"</span><span style=color:#f92672>)</span>
+</span></span><span style=display:flex><span> <span style=color:#f92672>.</span><span style=color:#a6e22e>append</span><span style=color:#f92672>();</span>
</span></span></code></pre></div><h3 id=metrics>Metrics</h3><p>The following Flink metrics are provided by the Flink Iceberg sink.</p><p>Parallel writer metrics are added under the sub group of <code>IcebergStreamWriter</code>.
They should have the following key-value tags.</p><ul><li>table: full table name (like iceberg.my_db.my_table)</li><li>subtask_index: writer subtask index starting from 0</li></ul><table><thead><tr><th>Metric name</th><th>Metric type</th><th>Description</th></tr></thead><tbody><tr><td>lastFlushDurationMs</td><td>Gague</td><td>The duration (in milli) that writer subtasks take to flush and upload the files during checkpoint.</td></tr><tr><td>flushedDataFiles</td><td>Counter</td><td>Number [...]
They should have the following key-value tags.</p><ul><li>table: full table name (like iceberg.my_db.my_table)</li></ul><table><thead><tr><th>Metric name</th><th>Metric type</th><th>Description</th></tr></thead><tbody><tr><td>lastCheckpointDurationMs</td><td>Gague</td><td>The duration (in milli) that the committer operator checkpoints its state.</td></tr><tr><td>lastCommitDurationMs</td><td>Gague</td><td>The duration (in milli) that the Iceberg table commit takes.</td></tr><tr><td>commit [...]
@@ -99,7 +106,7 @@ As a result, there won’t be any Iceberg commits attempted.</li></ul><p>If
</span></span><span style=display:flex><span> <span style=color:#f92672>.</span><span style=color:#a6e22e>set</span><span style=color:#f92672>(</span>FlinkWriteOptions<span style=color:#f92672>.</span><span style=color:#a6e22e>OVERWRITE_MODE</span><span style=color:#f92672>,</span> <span style=color:#e6db74>"true"</span><span style=color:#f92672>);</span>
</span></span></code></pre></div><p>For Flink SQL, write options can be passed in via SQL hints like this:</p><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-sql data-lang=sql><span style=display:flex><span><span style=color:#66d9ef>INSERT</span> <span style=color:#66d9ef>INTO</span> tableName <span style=color:#75715e>/*+ OPTIONS('upsert-enabled'='true') */</span>
</span></span><span style=display:flex><span>...
-</span></span></code></pre></div><p>Check out all the options here: <a href=../flink-configuration#write-options>write-options</a></p></div><div id=toc class=markdown-body><div id=full><nav id=TableOfContents><ul><li><a href=#writing-with-sql>Writing with SQL</a><ul><li><a href=#insert-into><code>INSERT INTO</code></a></li><li><a href=#insert-overwrite><code>INSERT OVERWRITE</code></a></li><li><a href=#upsert><code>UPSERT</code></a></li></ul></li><li><a href=#writing-with-datastream>Writ [...]
+</span></span></code></pre></div><p>Check out all the options here: <a href=../flink-configuration#write-options>write-options</a></p></div><div id=toc class=markdown-body><div id=full><nav id=TableOfContents><ul><li><a href=#writing-with-sql>Writing with SQL</a><ul><li><a href=#insert-into><code>INSERT INTO</code></a></li><li><a href=#insert-overwrite><code>INSERT OVERWRITE</code></a></li><li><a href=#upsert><code>UPSERT</code></a></li></ul></li><li><a href=#writing-with-datastream>Writ [...]
<script src=https://iceberg.apache.org/docs/1.2.1//js/jquery.easing.min.js></script>
<script type=text/javascript src=https://iceberg.apache.org/docs/1.2.1//js/search.js></script>
<script src=https://iceberg.apache.org/docs/1.2.1//js/bootstrap.min.js></script>
diff --git a/docs/1.2.1/flink/index.html b/docs/1.2.1/flink/index.html
index 5d4bf93e..dda6d63e 100644
--- a/docs/1.2.1/flink/index.html
+++ b/docs/1.2.1/flink/index.html
@@ -5,7 +5,7 @@
<span class=icon-bar></span></button>
<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/1.2.1//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.2.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg.a [...]
<i class="fa fa-chevron-right"></i>
-<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul class=sub-menu><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li></ul></div><li><a class="chevron-toggle collapsed" data-toggle=collaps [...]
+<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li></ul></div><li [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Spark class=collapse><ul class=sub-menu><li><a href=../spark-ddl/>DDL</a></li><li><a href=../getting-started/>Getting Started</a></li><li><a href=../spark-procedures/>Procedures</a></li><li><a href=../spark-queries/>Queries</a></li><li><a href=../spark-structured-streaming/>Structured Streaming</a></li><li><a href=../spark-writes/>Writes</a></li></ul></div><li><a class=chevron-toggle data-toggle=collapse data-parent=full href=#Flink><spa [...]
<i class="fa fa-chevron-right"></i>
@@ -13,7 +13,9 @@
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Integrations class=collapse><ul class=sub-menu><li><a href=../aws/>AWS</a></li><li><a href=../dell/>Dell</a></li><li><a href=../jdbc/>JDBC</a></li><li><a href=../nessie/>Nessie</a></li></ul></div><li><a class="chevron-toggle collapsed" data-toggle=collapse data-parent=full href=#API><span>API</span>
<i class="fa fa-chevron-right"></i>
-<i class="fa fa-chevron-down"></i></a></li><div id=API class=collapse><ul class=sub-menu><li><a href=../java-api-quickstart/>Java Quickstart</a></li><li><a href=../api/>Java API</a></li><li><a href=../custom-catalog/>Java Custom Catalog</a></li></ul></div><li><a href=https://iceberg.apache.org/docs/1.2.1/../../javadoc/latest><span>Javadoc</span></a></li><li><a target=_blank href=https://py.iceberg.apache.org/><span>PyIceberg</span></a></li></div></div><div id=content class=markdown-body> [...]
+<i class="fa fa-chevron-down"></i></a></li><div id=API class=collapse><ul class=sub-menu><li><a href=../java-api-quickstart/>Java Quickstart</a></li><li><a href=../api/>Java API</a></li><li><a href=../custom-catalog/>Java Custom Catalog</a></li></ul></div><li><a href=https://iceberg.apache.org/docs/1.2.1/../../javadoc/latest><span>Javadoc</span></a></li><li><a target=_blank href=https://py.iceberg.apache.org/><span>PyIceberg</span></a></li><li><a class="chevron-toggle collapsed" data-tog [...]
+<i class="fa fa-chevron-right"></i>
+<i class="fa fa-chevron-down"></i></a></li><div id=Migration class=collapse><ul class=sub-menu><li><a href=../table-migration/>Overview</a></li><li><a href=../hive-migration/>Hive Migration</a></li><li><a href=../delta-lake-migration/>Delta Lake Migration</a></li></ul></div></div></div><div id=content class=markdown-body><div class=margin-for-toc><h1 id=flink>Flink</h1><p>Apache Iceberg supports both <a href=https://flink.apache.org/>Apache Flink</a>’s DataStream API and Table API. [...]
</span></span><span style=display:flex><span>SCALA_VERSION<span style=color:#f92672>=</span>2.12
</span></span><span style=display:flex><span>APACHE_FLINK_URL<span style=color:#f92672>=</span>https://archive.apache.org/dist/flink/
</span></span><span style=display:flex><span>wget <span style=color:#e6db74>${</span>APACHE_FLINK_URL<span style=color:#e6db74>}</span>/flink-<span style=color:#e6db74>${</span>FLINK_VERSION<span style=color:#e6db74>}</span>/flink-<span style=color:#e6db74>${</span>FLINK_VERSION<span style=color:#e6db74>}</span>-bin-scala_<span style=color:#e6db74>${</span>SCALA_VERSION<span style=color:#e6db74>}</span>.tgz
@@ -114,6 +116,11 @@
</span></span><span style=display:flex><span> <span style=color:#f92672>.</span><span style=color:#a6e22e>append</span><span style=color:#f92672>();</span>
</span></span><span style=display:flex><span>
</span></span><span style=display:flex><span>env<span style=color:#f92672>.</span><span style=color:#a6e22e>execute</span><span style=color:#f92672>(</span><span style=color:#e6db74>"Test Iceberg DataStream"</span><span style=color:#f92672>);</span>
+</span></span></code></pre></div><h3 id=branch-writes>Branch Writes</h3><p>Writing to branches in Iceberg tables is also supported via the <code>toBranch</code> API in <code>FlinkSink</code>
+For more information on branches please refer to <a href=../../tables/branching>branches</a>.</p><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-java data-lang=java><span style=display:flex><span>FlinkSink<span style=color:#f92672>.</span><span style=color:#a6e22e>forRowData</span><span style=color:#f92672>(</span>input<span style=color:#f92672>)</span>
+</span></span><span style=display:flex><span> <span style=color:#f92672>.</span><span style=color:#a6e22e>tableLoader</span><span style=color:#f92672>(</span>tableLoader<span style=color:#f92672>)</span>
+</span></span><span style=display:flex><span> <span style=color:#f92672>.</span><span style=color:#a6e22e>toBranch</span><span style=color:#f92672>(</span><span style=color:#e6db74>"audit-branch"</span><span style=color:#f92672>)</span>
+</span></span><span style=display:flex><span> <span style=color:#f92672>.</span><span style=color:#a6e22e>append</span><span style=color:#f92672>();</span>
</span></span></code></pre></div><h2 id=reading>Reading</h2><p>Submit a Flink <strong>batch</strong> job using the following sentences:</p><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-sql data-lang=sql><span style=display:flex><span><span style=color:#75715e>-- Execute the flink job in batch mode for current session context
</span></span></span><span style=display:flex><span><span style=color:#75715e></span><span style=color:#66d9ef>SET</span> execution.runtime<span style=color:#f92672>-</span><span style=color:#66d9ef>mode</span> <span style=color:#f92672>=</span> batch;
</span></span><span style=display:flex><span><span style=color:#66d9ef>SELECT</span> <span style=color:#f92672>*</span> <span style=color:#66d9ef>FROM</span> <span style=color:#f92672>`</span>hive_catalog<span style=color:#f92672>`</span>.<span style=color:#f92672>`</span><span style=color:#66d9ef>default</span><span style=color:#f92672>`</span>.<span style=color:#f92672>`</span>sample<span style=color:#f92672>`</span>;
@@ -134,7 +141,7 @@
.tableLoader(tableLoader)
.streaming(false)
.build();
-</code></pre><h2 id=type-conversion>Type conversion</h2><p>Iceberg’s integration for Flink automatically converts between Flink and Iceberg types. When writing to a table with types that are not supported by Flink, like UUID, Iceberg will accept and convert values from the Flink type.</p><h3 id=flink-to-iceberg>Flink to Iceberg</h3><p>Flink types are converted to Iceberg types according to the following table:</p><table><thead><tr><th>Flink</th><th>Iceberg</th><th>Notes</th></tr></ [...]
+</code></pre><h2 id=type-conversion>Type conversion</h2><p>Iceberg’s integration for Flink automatically converts between Flink and Iceberg types. When writing to a table with types that are not supported by Flink, like UUID, Iceberg will accept and convert values from the Flink type.</p><h3 id=flink-to-iceberg>Flink to Iceberg</h3><p>Flink types are converted to Iceberg types according to the following table:</p><table><thead><tr><th>Flink</th><th>Iceberg</th><th>Notes</th></tr></ [...]
<script src=https://iceberg.apache.org/docs/1.2.1//js/jquery.easing.min.js></script>
<script type=text/javascript src=https://iceberg.apache.org/docs/1.2.1//js/search.js></script>
<script src=https://iceberg.apache.org/docs/1.2.1//js/bootstrap.min.js></script>
diff --git a/docs/1.2.1/getting-started/index.html b/docs/1.2.1/getting-started/index.html
index 10d634cf..8c5f753e 100644
--- a/docs/1.2.1/getting-started/index.html
+++ b/docs/1.2.1/getting-started/index.html
@@ -5,7 +5,7 @@
<span class=icon-bar></span></button>
<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/1.2.1//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.2.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg.a [...]
<i class="fa fa-chevron-right"></i>
-<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul class=sub-menu><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li></ul></div><li><a class=chevron-toggle data-toggle=collapse data-paren [...]
+<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li></ul></div><li [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Spark class="collapse in"><ul class=sub-menu><li><a href=../spark-ddl/>DDL</a></li><li><a id=active href=../getting-started/>Getting Started</a></li><li><a href=../spark-procedures/>Procedures</a></li><li><a href=../spark-queries/>Queries</a></li><li><a href=../spark-structured-streaming/>Structured Streaming</a></li><li><a href=../spark-writes/>Writes</a></li></ul></div><li><a class="chevron-toggle collapsed" data-toggle=collapse data-p [...]
<i class="fa fa-chevron-right"></i>
@@ -13,7 +13,9 @@
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Integrations class=collapse><ul class=sub-menu><li><a href=../aws/>AWS</a></li><li><a href=../dell/>Dell</a></li><li><a href=../jdbc/>JDBC</a></li><li><a href=../nessie/>Nessie</a></li></ul></div><li><a class="chevron-toggle collapsed" data-toggle=collapse data-parent=full href=#API><span>API</span>
<i class="fa fa-chevron-right"></i>
-<i class="fa fa-chevron-down"></i></a></li><div id=API class=collapse><ul class=sub-menu><li><a href=../java-api-quickstart/>Java Quickstart</a></li><li><a href=../api/>Java API</a></li><li><a href=../custom-catalog/>Java Custom Catalog</a></li></ul></div><li><a href=https://iceberg.apache.org/docs/1.2.1/../../javadoc/latest><span>Javadoc</span></a></li><li><a target=_blank href=https://py.iceberg.apache.org/><span>PyIceberg</span></a></li></div></div><div id=content class=markdown-body> [...]
+<i class="fa fa-chevron-down"></i></a></li><div id=API class=collapse><ul class=sub-menu><li><a href=../java-api-quickstart/>Java Quickstart</a></li><li><a href=../api/>Java API</a></li><li><a href=../custom-catalog/>Java Custom Catalog</a></li></ul></div><li><a href=https://iceberg.apache.org/docs/1.2.1/../../javadoc/latest><span>Javadoc</span></a></li><li><a target=_blank href=https://py.iceberg.apache.org/><span>PyIceberg</span></a></li><li><a class="chevron-toggle collapsed" data-tog [...]
+<i class="fa fa-chevron-right"></i>
+<i class="fa fa-chevron-down"></i></a></li><div id=Migration class=collapse><ul class=sub-menu><li><a href=../table-migration/>Overview</a></li><li><a href=../hive-migration/>Hive Migration</a></li><li><a href=../delta-lake-migration/>Delta Lake Migration</a></li></ul></div></div></div><div id=content class=markdown-body><div class=margin-for-toc><h1 id=getting-started>Getting Started</h1><p>The latest version of Iceberg is <a href=../../../releases>1.2.0</a>.</p><p>Spark is currently th [...]
We recommend you to get started with Spark to understand Iceberg concepts and features with examples.
You can also view documentations of using Iceberg with other compute engine under the <a href=https://iceberg.apache.org/multi-engine-support>Multi-Engine Support</a> page.</p><h2 id=using-iceberg-in-spark-3>Using Iceberg in Spark 3</h2><p>To use Iceberg in a Spark shell, use the <code>--packages</code> option:</p><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-sh data-lang=sh><span style=disp [...]
</span></span></code></pre></div><div class=info>If you want to include Iceberg in your Spark installation, add the <a href="https://search.maven.org/remotecontent?filepath=org/apache/iceberg/iceberg-spark-runtime-3.2_2.12/1.2.0/iceberg-spark-runtime-3.2_2.12-1.2.0.jar"><code>iceberg-spark-runtime-3.2_2.12</code> Jar</a> to Spark’s <code>jars</code> folder.</div><h3 id=adding-catalogs>Adding catalogs</h3><p>Iceberg comes with <a href=../spark-configuration#catalogs>catalogs</a> tha [...]
diff --git a/docs/1.2.1/performance/index.html b/docs/1.2.1/hive-migration/index.html
similarity index 60%
copy from docs/1.2.1/performance/index.html
copy to docs/1.2.1/hive-migration/index.html
index 64ee8013..37d38ca8 100644
--- a/docs/1.2.1/performance/index.html
+++ b/docs/1.2.1/hive-migration/index.html
@@ -1,11 +1,11 @@
-<!doctype html><html><head><meta charset=utf-8><meta http-equiv=x-ua-compatible content="IE=edge"><meta name=viewport content="width=device-width,initial-scale=1"><meta name=description content><meta name=author content><title>Performance</title><link href=../css/bootstrap.css rel=stylesheet><link href=../css/markdown.css rel=stylesheet><link href=../css/katex.min.css rel=stylesheet><link href=../css/iceberg-theme.css rel=stylesheet><link href=../font-awesome-4.7.0/css/font-awesome.min.c [...]
+<!doctype html><html><head><meta charset=utf-8><meta http-equiv=x-ua-compatible content="IE=edge"><meta name=viewport content="width=device-width,initial-scale=1"><meta name=description content><meta name=author content><title>Hive Migration</title><link href=../css/bootstrap.css rel=stylesheet><link href=../css/markdown.css rel=stylesheet><link href=../css/katex.min.css rel=stylesheet><link href=../css/iceberg-theme.css rel=stylesheet><link href=../font-awesome-4.7.0/css/font-awesome.mi [...]
<span class=sr-only>Toggle navigation</span>
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/1.2.1//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.2.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg.a [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/1.2.1//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.2.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg.a [...]
<i class="fa fa-chevron-right"></i>
-<i class="fa fa-chevron-down"></i></a></li><div id=Tables class="collapse in"><ul class=sub-menu><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a id=active href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li></ul></div><li><a class="chevron-toggle collapsed" data [...]
+<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li></ul></div><li [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Spark class=collapse><ul class=sub-menu><li><a href=../spark-ddl/>DDL</a></li><li><a href=../getting-started/>Getting Started</a></li><li><a href=../spark-procedures/>Procedures</a></li><li><a href=../spark-queries/>Queries</a></li><li><a href=../spark-structured-streaming/>Structured Streaming</a></li><li><a href=../spark-writes/>Writes</a></li></ul></div><li><a class="chevron-toggle collapsed" data-toggle=collapse data-parent=full href [...]
<i class="fa fa-chevron-right"></i>
@@ -13,7 +13,19 @@
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Integrations class=collapse><ul class=sub-menu><li><a href=../aws/>AWS</a></li><li><a href=../dell/>Dell</a></li><li><a href=../jdbc/>JDBC</a></li><li><a href=../nessie/>Nessie</a></li></ul></div><li><a class="chevron-toggle collapsed" data-toggle=collapse data-parent=full href=#API><span>API</span>
<i class="fa fa-chevron-right"></i>
-<i class="fa fa-chevron-down"></i></a></li><div id=API class=collapse><ul class=sub-menu><li><a href=../java-api-quickstart/>Java Quickstart</a></li><li><a href=../api/>Java API</a></li><li><a href=../custom-catalog/>Java Custom Catalog</a></li></ul></div><li><a href=https://iceberg.apache.org/docs/1.2.1/../../javadoc/latest><span>Javadoc</span></a></li><li><a target=_blank href=https://py.iceberg.apache.org/><span>PyIceberg</span></a></li></div></div><div id=content class=markdown-body> [...]
+<i class="fa fa-chevron-down"></i></a></li><div id=API class=collapse><ul class=sub-menu><li><a href=../java-api-quickstart/>Java Quickstart</a></li><li><a href=../api/>Java API</a></li><li><a href=../custom-catalog/>Java Custom Catalog</a></li></ul></div><li><a href=https://iceberg.apache.org/docs/1.2.1/../../javadoc/latest><span>Javadoc</span></a></li><li><a target=_blank href=https://py.iceberg.apache.org/><span>PyIceberg</span></a></li><li><a class=chevron-toggle data-toggle=collapse [...]
+<i class="fa fa-chevron-right"></i>
+<i class="fa fa-chevron-down"></i></a></li><div id=Migration class="collapse in"><ul class=sub-menu><li><a href=../table-migration/>Overview</a></li><li><a id=active href=../hive-migration/>Hive Migration</a></li><li><a href=../delta-lake-migration/>Delta Lake Migration</a></li></ul></div></div></div><div id=content class=markdown-body><div class=margin-for-toc><h1 id=hive-table-migration>Hive Table Migration</h1><p>Apache Hive supports ORC, Parquet, and Avro file formats that could be m [...]
+When migrating data to an Iceberg table, which provides versioning and transactional updates, only the most recent data files need to be migrated.</p><p>Iceberg supports all three migration actions: Snapshot Table, Migrate Table, and Add Files for migrating from Hive tables to Iceberg tables. Since Hive tables do not maintain snapshots,
+the migration process essentially involves creating a new Iceberg table with the existing schema and committing all data files across all partitions to the new Iceberg table.
+After the initial migration, any new data files are added to the new Iceberg table using the Add Files action.</p><h2 id=enabling-migration-from-hive-to-iceberg>Enabling Migration from Hive to Iceberg</h2><p>The Hive table migration actions are supported by the Spark Integration module via Spark Procedures.
+The procedures are bundled in the Spark runtime jar, which is available in the <a href=https://iceberg.apache.org/releases/#downloads>Iceberg Release Downloads</a>.</p><h2 id=snapshot-hive-table-to-iceberg>Snapshot Hive Table to Iceberg</h2><p>To snapshot a Hive table, users can run the following Spark SQL:</p><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-sql data-lang=sql><span style=displa [...]
+</span></span></code></pre></div><p>See <a href=../spark-procedures/#snapshot>Spark Procedure: snapshot</a> for more details.</p><h2 id=migrate-hive-table-to-iceberg>Migrate Hive Table To Iceberg</h2><p>To migrate a Hive table to Iceberg, users can run the following Spark SQL:</p><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-sql data-lang=sql><span style=display:flex><span><span style=color: [...]
+</span></span></code></pre></div><p>See <a href=../spark-procedures/#migrate>Spark Procedure: migrate</a> for more details.</p><h2 id=add-files-from-hive-table-to-iceberg>Add Files From Hive Table to Iceberg</h2><p>To add data files from a Hive table to a given Iceberg table, users can run the following Spark SQL:</p><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-sql data-lang=sql><span style [...]
+</span></span><span style=display:flex><span><span style=color:#66d9ef>table</span> <span style=color:#f92672>=></span> <span style=color:#e6db74>'db.tbl'</span>,
+</span></span><span style=display:flex><span>source_table <span style=color:#f92672>=></span> <span style=color:#e6db74>'db.src_tbl'</span>
+</span></span><span style=display:flex><span>)
+</span></span></code></pre></div><p>See <a href=../spark-procedures/#add_files>Spark Procedure: add_files</a> for more details.</p></div><div id=toc class=markdown-body><div id=full><nav id=TableOfContents><ul><li><a href=#enabling-migration-from-hive-to-iceberg>Enabling Migration from Hive to Iceberg</a></li><li><a href=#snapshot-hive-table-to-iceberg>Snapshot Hive Table to Iceberg</a></li><li><a href=#migrate-hive-table-to-iceberg>Migrate Hive Table To Iceberg</a></li><li><a href=#add- [...]
<script src=https://iceberg.apache.org/docs/1.2.1//js/jquery.easing.min.js></script>
<script type=text/javascript src=https://iceberg.apache.org/docs/1.2.1//js/search.js></script>
<script src=https://iceberg.apache.org/docs/1.2.1//js/bootstrap.min.js></script>
diff --git a/docs/1.2.1/hive/index.html b/docs/1.2.1/hive/index.html
index c41f0af2..8398db35 100644
--- a/docs/1.2.1/hive/index.html
+++ b/docs/1.2.1/hive/index.html
@@ -5,7 +5,7 @@
<span class=icon-bar></span></button>
<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/1.2.1//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.2.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg.a [...]
<i class="fa fa-chevron-right"></i>
-<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul class=sub-menu><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li></ul></div><li><a class="chevron-toggle collapsed" data-toggle=collaps [...]
+<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li></ul></div><li [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Spark class=collapse><ul class=sub-menu><li><a href=../spark-ddl/>DDL</a></li><li><a href=../getting-started/>Getting Started</a></li><li><a href=../spark-procedures/>Procedures</a></li><li><a href=../spark-queries/>Queries</a></li><li><a href=../spark-structured-streaming/>Structured Streaming</a></li><li><a href=../spark-writes/>Writes</a></li></ul></div><li><a class="chevron-toggle collapsed" data-toggle=collapse data-parent=full href [...]
<i class="fa fa-chevron-right"></i>
@@ -13,7 +13,9 @@
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Integrations class=collapse><ul class=sub-menu><li><a href=../aws/>AWS</a></li><li><a href=../dell/>Dell</a></li><li><a href=../jdbc/>JDBC</a></li><li><a href=../nessie/>Nessie</a></li></ul></div><li><a class="chevron-toggle collapsed" data-toggle=collapse data-parent=full href=#API><span>API</span>
<i class="fa fa-chevron-right"></i>
-<i class="fa fa-chevron-down"></i></a></li><div id=API class=collapse><ul class=sub-menu><li><a href=../java-api-quickstart/>Java Quickstart</a></li><li><a href=../api/>Java API</a></li><li><a href=../custom-catalog/>Java Custom Catalog</a></li></ul></div><li><a href=https://iceberg.apache.org/docs/1.2.1/../../javadoc/latest><span>Javadoc</span></a></li><li><a target=_blank href=https://py.iceberg.apache.org/><span>PyIceberg</span></a></li></div></div><div id=content class=markdown-body> [...]
+<i class="fa fa-chevron-down"></i></a></li><div id=API class=collapse><ul class=sub-menu><li><a href=../java-api-quickstart/>Java Quickstart</a></li><li><a href=../api/>Java API</a></li><li><a href=../custom-catalog/>Java Custom Catalog</a></li></ul></div><li><a href=https://iceberg.apache.org/docs/1.2.1/../../javadoc/latest><span>Javadoc</span></a></li><li><a target=_blank href=https://py.iceberg.apache.org/><span>PyIceberg</span></a></li><li><a class="chevron-toggle collapsed" data-tog [...]
+<i class="fa fa-chevron-right"></i>
+<i class="fa fa-chevron-down"></i></a></li><div id=Migration class=collapse><ul class=sub-menu><li><a href=../table-migration/>Overview</a></li><li><a href=../hive-migration/>Hive Migration</a></li><li><a href=../delta-lake-migration/>Delta Lake Migration</a></li></ul></div></div></div><div id=content class=markdown-body><div class=margin-for-toc><h1 id=hive>Hive</h1><p>Iceberg supports reading and writing Iceberg tables through <a href=https://hive.apache.org>Hive</a> by using
a <a href=https://cwiki.apache.org/confluence/display/Hive/StorageHandlers>StorageHandler</a>.</p><h2 id=feature-support>Feature support</h2><p>Iceberg compatibility with Hive 2.x and Hive 3.1.2/3 supports the following features:</p><ul><li>Creating a table</li><li>Dropping a table</li><li>Reading a table</li><li>Inserting into a table (INSERT INTO)</li></ul><div class=warning>DML operations work only with MapReduce execution engine.</div><p>With Hive version 4.0.0-alpha-2 and above,
the Iceberg integration when using HiveCatalog supports the following additional features:</p><ul><li>Altering a table with expiring snapshots.</li><li>Create a table like an existing table (CTLT table)</li><li>Support adding parquet compression type via Table properties <a href=https://spark.apache.org/docs/2.4.3/sql-data-sources-parquet.html#configuration>Compression types</a></li><li>Altering a table metadata location</li><li>Supporting table rollback</li><li>Honours sort orders on ex [...]
the Iceberg integration when using HiveCatalog supports the following additional features:</p><ul><li>Creating an Iceberg identity-partitioned table</li><li>Creating an Iceberg table with any partition spec, including the various transforms supported by Iceberg</li><li>Creating a table from an existing table (CTAS table)</li><li>Altering a table while keeping Iceberg and Hive schemas in sync</li><li>Altering the partition schema (updating columns)</li><li>Altering the partition schema by [...]
diff --git a/docs/1.2.1/index.html b/docs/1.2.1/index.html
index 6eb47835..cd7c7518 100644
--- a/docs/1.2.1/index.html
+++ b/docs/1.2.1/index.html
@@ -5,7 +5,7 @@
<span class=icon-bar></span></button>
<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/1.2.1//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.2.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg.a [...]
<i class="fa fa-chevron-right"></i>
-<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul class=sub-menu><li><a href=./configuration/>Configuration</a></li><li><a href=./evolution/>Evolution</a></li><li><a href=./maintenance/>Maintenance</a></li><li><a href=./partitioning/>Partitioning</a></li><li><a href=./performance/>Performance</a></li><li><a href=./reliability/>Reliability</a></li><li><a href=./schemas/>Schemas</a></li></ul></div><li><a class="chevron-toggle collapsed" data-toggle=collapse data- [...]
+<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul class=sub-menu><li><a href=./branching/>Branching and Tagging</a></li><li><a href=./configuration/>Configuration</a></li><li><a href=./evolution/>Evolution</a></li><li><a href=./maintenance/>Maintenance</a></li><li><a href=./partitioning/>Partitioning</a></li><li><a href=./performance/>Performance</a></li><li><a href=./reliability/>Reliability</a></li><li><a href=./schemas/>Schemas</a></li></ul></div><li><a clas [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Spark class=collapse><ul class=sub-menu><li><a href=./spark-ddl/>DDL</a></li><li><a href=./getting-started/>Getting Started</a></li><li><a href=./spark-procedures/>Procedures</a></li><li><a href=./spark-queries/>Queries</a></li><li><a href=./spark-structured-streaming/>Structured Streaming</a></li><li><a href=./spark-writes/>Writes</a></li></ul></div><li><a class="chevron-toggle collapsed" data-toggle=collapse data-parent=full href=#Flin [...]
<i class="fa fa-chevron-right"></i>
@@ -13,7 +13,9 @@
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Integrations class=collapse><ul class=sub-menu><li><a href=./aws/>AWS</a></li><li><a href=./dell/>Dell</a></li><li><a href=./jdbc/>JDBC</a></li><li><a href=./nessie/>Nessie</a></li></ul></div><li><a class="chevron-toggle collapsed" data-toggle=collapse data-parent=full href=#API><span>API</span>
<i class="fa fa-chevron-right"></i>
-<i class="fa fa-chevron-down"></i></a></li><div id=API class=collapse><ul class=sub-menu><li><a href=./java-api-quickstart/>Java Quickstart</a></li><li><a href=./api/>Java API</a></li><li><a href=./custom-catalog/>Java Custom Catalog</a></li></ul></div><li><a href=https://iceberg.apache.org/docs/1.2.1/../../javadoc/latest><span>Javadoc</span></a></li><li><a target=_blank href=https://py.iceberg.apache.org/><span>PyIceberg</span></a></li></div></div><div id=content class=markdown-body><di [...]
+<i class="fa fa-chevron-down"></i></a></li><div id=API class=collapse><ul class=sub-menu><li><a href=./java-api-quickstart/>Java Quickstart</a></li><li><a href=./api/>Java API</a></li><li><a href=./custom-catalog/>Java Custom Catalog</a></li></ul></div><li><a href=https://iceberg.apache.org/docs/1.2.1/../../javadoc/latest><span>Javadoc</span></a></li><li><a target=_blank href=https://py.iceberg.apache.org/><span>PyIceberg</span></a></li><li><a class="chevron-toggle collapsed" data-toggle [...]
+<i class="fa fa-chevron-right"></i>
+<i class="fa fa-chevron-down"></i></a></li><div id=Migration class=collapse><ul class=sub-menu><li><a href=./table-migration/>Overview</a></li><li><a href=./hive-migration/>Hive Migration</a></li><li><a href=./delta-lake-migration/>Delta Lake Migration</a></li></ul></div></div></div><div id=content class=markdown-body><div class=margin-for-toc><h1 id=documentation>Documentation</h1><p><strong>Apache Iceberg is an open table format for huge analytic datasets.</strong> Iceberg adds tables [...]
<script src=https://iceberg.apache.org/docs/1.2.1//js/jquery.easing.min.js></script>
<script type=text/javascript src=https://iceberg.apache.org/docs/1.2.1//js/search.js></script>
<script src=https://iceberg.apache.org/docs/1.2.1//js/bootstrap.min.js></script>
diff --git a/docs/1.2.1/index.xml b/docs/1.2.1/index.xml
index 0904e651..3167ed0d 100644
--- a/docs/1.2.1/index.xml
+++ b/docs/1.2.1/index.xml
@@ -4,7 +4,7 @@ Using Iceberg in Spark 3 To use Iceberg in a Spark shell, use the --packages opt
Feature support Iceberg compatibility with Hive 2.x and Hive 3.1.2/3 supports the following features:
Creating a table Dropping a table Reading a table Inserting into a table (INSERT INTO) DML operations work only with MapReduce execution engine. With Hive version 4.0.0-alpha-2 and above, the Iceberg integration when using HiveCatalog supports the following additional features:
Altering a table with expiring snapshots.</description></item><item><title>AWS</title><link>https://iceberg.apache.org/docs/1.2.1/aws/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><guid>https://iceberg.apache.org/docs/1.2.1/aws/</guid><description>Iceberg AWS Integrations Iceberg provides integration with different AWS services through the iceberg-aws module. This section describes how to use Iceberg with AWS.
-Enabling AWS Integration The iceberg-aws module is bundled with Spark and Flink engine runtimes for all versions from 0.11.0 onwards. However, the AWS clients are not bundled so that you can use the same client version as your application. You will need to provide the AWS v2 SDK because that is what Iceberg depends on.</description></item><item><title>Configuration</title><link>https://iceberg.apache.org/docs/1.2.1/configuration/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><g [...]
+Enabling AWS Integration The iceberg-aws module is bundled with Spark and Flink engine runtimes for all versions from 0.11.0 onwards. However, the AWS clients are not bundled so that you can use the same client version as your application. You will need to provide the AWS v2 SDK because that is what Iceberg depends on.</description></item><item><title>Branching and Tagging</title><link>https://iceberg.apache.org/docs/1.2.1/branching/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDat [...]
Read properties Property Default Description read.split.target-size 134217728 (128 MB) Target size when combining data input splits read.split.metadata-target-size 33554432 (32 MB) Target size when combining metadata input splits read.split.planning-lookback 10 Number of bins to consider when combining input splits read.split.open-file-cost 4194304 (4 MB) The estimated cost to open a file, used as a minimum weight when combining splits.</description></item><item><title>Configuration</tit [...]
This creates an Iceberg catalog named hive_prod that loads tables from a Hive metastore:
spark.sql.catalog.hive_prod = org.apache.iceberg.spark.SparkCatalog spark.sql.catalog.hive_prod.type = hive spark.sql.catalog.hive_prod.uri = thrift://metastore-host:port # omit uri to use the same URI as Spark: hive.metastore.uris in hive-site.xml Iceberg also supports a directory-based catalog in HDFS that can be configured using type=hadoop:</description></item><item><title>DDL</title><link>https://iceberg.apache.org/docs/1.2.1/spark-ddl/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000 [...]
@@ -12,7 +12,8 @@ Iceberg uses Apache Spark&rsquo;s DataSourceV2 API for data source and catal
Spark 2.4 can&rsquo;t create Iceberg tables with DDL, instead use Spark 3 or the Iceberg API. CREATE TABLE Spark 3 can create tables in any Iceberg catalog with the clause USING iceberg:</description></item><item><title>Dell</title><link>https://iceberg.apache.org/docs/1.2.1/dell/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><guid>https://iceberg.apache.org/docs/1.2.1/dell/</guid><description>Iceberg Dell Integration Dell ECS Integration Iceberg can be used with Dell&r [...]
See Dell ECS for more information on Dell ECS.
Parameters When using Dell ECS with Iceberg, these configuration parameters are required:
-Name Description ecs.s3.endpoint ECS S3 service endpoint ecs.s3.access-key-id ECS Username ecs.s3.secret-access-key S3 Secret Key warehouse The location of data and metadata The warehouse should use the following formats:</description></item><item><title>Evolution</title><link>https://iceberg.apache.org/docs/1.2.1/evolution/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><guid>https://iceberg.apache.org/docs/1.2.1/evolution/</guid><description>Evolution Iceberg supports in-place [...]
+Name Description ecs.s3.endpoint ECS S3 service endpoint ecs.s3.access-key-id ECS Username ecs.s3.secret-access-key S3 Secret Key warehouse The location of data and metadata The warehouse should use the following formats:</description></item><item><title>Delta Lake Migration</title><link>https://iceberg.apache.org/docs/1.2.1/delta-lake-migration/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><guid>https://iceberg.apache.org/docs/1.2.1/delta-lake-migration/</guid><description>De [...]
+Currently, Iceberg supports the Snapshot Table action for migrating from Delta Lake to Iceberg tables. Since Delta Lake tables maintain transactions, all available transactions will be committed to the new Iceberg table as transactions in order.</description></item><item><title>Evolution</title><link>https://iceberg.apache.org/docs/1.2.1/evolution/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><guid>https://iceberg.apache.org/docs/1.2.1/evolution/</guid><description>Evolution I [...]
For example, Hive table partitioning cannot change so moving from a daily partition layout to an hourly partition layout requires a new table. And because queries are dependent on partitions, queries must be rewritten for the new table.</description></item><item><title>Flink Actions</title><link>https://iceberg.apache.org/docs/1.2.1/flink-actions/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><guid>https://iceberg.apache.org/docs/1.2.1/flink-actions/</guid><description>Rewrite [...]
import org.apache.iceberg.flink.actions.Actions; TableLoader tableLoader = TableLoader.fromHadoopTable(&#34;hdfs://nn:8020/warehouse/path&#34;); Table table = tableLoader.loadTable(); RewriteDataFilesActionResult result = Actions.forTable(table) .rewriteDataFiles() .execute(); For more details of the rewrite files action, please refer to RewriteDataFilesAction</description></item><item><title>Flink Configuration</title><link>https://iceberg.apache.org/docs/1.2.1/flink-configurati [...]
CREATE CATALOG &lt;catalog_name&gt; WITH ( &#39;type&#39;=&#39;iceberg&#39;, `&lt;config_key&gt;`=`&lt;config_value&gt;` ); The following properties can be set globally and are not limited to a specific catalog implementation:
@@ -25,7 +26,8 @@ Reading with SQL Iceberg support both streaming and batch read in Flink. Execute
-- Execute the flink job in streaming mode for current session context SET execution.runtime-mode = streaming; -- Execute the flink job in batch mode for current session context SET execution.</description></item><item><title>Flink Writes</title><link>https://iceberg.apache.org/docs/1.2.1/flink-writes/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><guid>https://iceberg.apache.org/docs/1.2.1/flink-writes/</guid><description>Flink Writes Iceberg support batch and streaming writes [...]
Writing with SQL Iceberg support both INSERT INTO and INSERT OVERWRITE.
INSERT INTO To append new data to a table with a Flink streaming job, use INSERT INTO:
-INSERT INTO `hive_catalog`.`default`.`sample` VALUES (1, &#39;a&#39;); INSERT INTO `hive_catalog`.`default`.`sample` SELECT id, data from other_kafka_table; INSERT OVERWRITE To replace data in the table with the result of a query, use INSERT OVERWRITE in batch job (flink streaming job does not support INSERT OVERWRITE).</description></item><item><title>Java API</title><link>https://iceberg.apache.org/docs/1.2.1/api/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><guid>ht [...]
+INSERT INTO `hive_catalog`.`default`.`sample` VALUES (1, &#39;a&#39;); INSERT INTO `hive_catalog`.`default`.`sample` SELECT id, data from other_kafka_table; INSERT OVERWRITE To replace data in the table with the result of a query, use INSERT OVERWRITE in batch job (flink streaming job does not support INSERT OVERWRITE).</description></item><item><title>Hive Migration</title><link>https://iceberg.apache.org/docs/1.2.1/hive-migration/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000< [...]
+Iceberg supports all three migration actions: Snapshot Table, Migrate Table, and Add Files for migrating from Hive tables to Iceberg tables. Since Hive tables do not maintain snapshots, the migration process essentially involves creating a new Iceberg table with the existing schema and committing all data files across all partitions to the new Iceberg table.</description></item><item><title>Java API</title><link>https://iceberg.apache.org/docs/1.2.1/api/</link><pubDate>Mon, 01 Jan 0001 0 [...]
Table metadata and operations are accessed through the Table interface. This interface will return table information.
Table metadata The Table interface provides access to the table metadata:
schema returns the current table schema spec returns the current table partition spec properties returns a map of key-value properties currentSnapshot returns the current table snapshot snapshots returns all valid snapshots for the table snapshot(id) returns a specific snapshot by ID location returns the table&rsquo;s base location Tables also provide refresh to update the table to the latest version, and expose helpers:</description></item><item><title>Java Custom Catalog</title><li [...]
@@ -38,7 +40,9 @@ Configurations Because each database and database service provider might require
Property Default Description uri the JDBC connection string jdbc.</description></item><item><title>Maintenance</title><link>https://iceberg.apache.org/docs/1.2.1/maintenance/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><guid>https://iceberg.apache.org/docs/1.2.1/maintenance/</guid><description>Maintenance Maintenance operations require the Table instance. Please refer Java API quickstart page to refer how to load an existing table. Recommended Maintenance Expire Snapshots Eac [...]
Snapshots accumulate until they are expired by the expireSnapshots operation. Regularly expiring snapshots is recommended to delete data files that are no longer needed, and to keep the size of table metadata small.</description></item><item><title>Nessie</title><link>https://iceberg.apache.org/docs/1.2.1/nessie/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><guid>https://iceberg.apache.org/docs/1.2.1/nessie/</guid><description>Iceberg Nessie Integration Iceberg provides integr [...]
multi-table transactions git-like operations (eg branches, tags, commits) hive-like metastore capabilities See Project Nessie for more information on Nessie. Nessie requires a server to run, see Getting Started to start a Nessie server.
-Enabling Nessie Catalog The iceberg-nessie module is bundled with Spark and Flink runtimes for all versions from 0.</description></item><item><title>Partitioning</title><link>https://iceberg.apache.org/docs/1.2.1/partitioning/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><guid>https://iceberg.apache.org/docs/1.2.1/partitioning/</guid><description>Partitioning What is partitioning? Partitioning is a way to make queries faster by grouping similar rows together when writing.
+Enabling Nessie Catalog The iceberg-nessie module is bundled with Spark and Flink runtimes for all versions from 0.</description></item><item><title>Overview</title><link>https://iceberg.apache.org/docs/1.2.1/table-migration/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><guid>https://iceberg.apache.org/docs/1.2.1/table-migration/</guid><description>Table Migration Apache Iceberg supports converting existing tables in other formats to Iceberg tables. This section introduces the [...]
+Migration Approaches There are two methods for executing table migration: full data migration and in-place metadata migration.
+Full data migration involves copying all data files from the source table to the new Iceberg table. This method makes the new table fully isolated from the source table, but is slower and doubles the space.</description></item><item><title>Partitioning</title><link>https://iceberg.apache.org/docs/1.2.1/partitioning/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><guid>https://iceberg.apache.org/docs/1.2.1/partitioning/</guid><description>Partitioning What is partitioning? Partit [...]
For example, queries for log entries from a logs table would usually include a time range, like this query for logs between 10 and 12 AM:
SELECT level, message FROM logs WHERE event_time BETWEEN &#39;2018-12-01 10:00:00&#39; AND &#39;2018-12-01 12:00:00&#39; Configuring the logs table to partition by the date of event_time will group log events into files with the same event date.</description></item><item><title>Performance</title><link>https://iceberg.apache.org/docs/1.2.1/performance/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><guid>https://iceberg.apache.org/docs/1.2.1/performance/</guid><d [...]
Planning in an Iceberg table fits on a single node because Iceberg&rsquo;s metadata can be used to prune metadata files that aren&rsquo;t needed, in addition to filtering data files that don&rsquo;t contain matching data.</description></item><item><title>Procedures</title><link>https://iceberg.apache.org/docs/1.2.1/spark-procedures/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><guid>https://iceberg.apache.org/docs/1.2.1/spark-procedures/</guid><description>Spark Pr [...]
diff --git a/docs/1.2.1/java-api-quickstart/index.html b/docs/1.2.1/java-api-quickstart/index.html
index c12e5c5c..7fd0c0d9 100644
--- a/docs/1.2.1/java-api-quickstart/index.html
+++ b/docs/1.2.1/java-api-quickstart/index.html
@@ -5,7 +5,7 @@
<span class=icon-bar></span></button>
<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/1.2.1//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.2.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg.a [...]
<i class="fa fa-chevron-right"></i>
-<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul class=sub-menu><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li></ul></div><li><a class="chevron-toggle collapsed" data-toggle=collaps [...]
+<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li></ul></div><li [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Spark class=collapse><ul class=sub-menu><li><a href=../spark-ddl/>DDL</a></li><li><a href=../getting-started/>Getting Started</a></li><li><a href=../spark-procedures/>Procedures</a></li><li><a href=../spark-queries/>Queries</a></li><li><a href=../spark-structured-streaming/>Structured Streaming</a></li><li><a href=../spark-writes/>Writes</a></li></ul></div><li><a class="chevron-toggle collapsed" data-toggle=collapse data-parent=full href [...]
<i class="fa fa-chevron-right"></i>
@@ -13,7 +13,9 @@
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Integrations class=collapse><ul class=sub-menu><li><a href=../aws/>AWS</a></li><li><a href=../dell/>Dell</a></li><li><a href=../jdbc/>JDBC</a></li><li><a href=../nessie/>Nessie</a></li></ul></div><li><a class=chevron-toggle data-toggle=collapse data-parent=full href=#API><span>API</span>
<i class="fa fa-chevron-right"></i>
-<i class="fa fa-chevron-down"></i></a></li><div id=API class="collapse in"><ul class=sub-menu><li><a id=active href=../java-api-quickstart/>Java Quickstart</a></li><li><a href=../api/>Java API</a></li><li><a href=../custom-catalog/>Java Custom Catalog</a></li></ul></div><li><a href=https://iceberg.apache.org/docs/1.2.1/../../javadoc/latest><span>Javadoc</span></a></li><li><a target=_blank href=https://py.iceberg.apache.org/><span>PyIceberg</span></a></li></div></div><div id=content class [...]
+<i class="fa fa-chevron-down"></i></a></li><div id=API class="collapse in"><ul class=sub-menu><li><a id=active href=../java-api-quickstart/>Java Quickstart</a></li><li><a href=../api/>Java API</a></li><li><a href=../custom-catalog/>Java Custom Catalog</a></li></ul></div><li><a href=https://iceberg.apache.org/docs/1.2.1/../../javadoc/latest><span>Javadoc</span></a></li><li><a target=_blank href=https://py.iceberg.apache.org/><span>PyIceberg</span></a></li><li><a class="chevron-toggle coll [...]
+<i class="fa fa-chevron-right"></i>
+<i class="fa fa-chevron-down"></i></a></li><div id=Migration class=collapse><ul class=sub-menu><li><a href=../table-migration/>Overview</a></li><li><a href=../hive-migration/>Hive Migration</a></li><li><a href=../delta-lake-migration/>Delta Lake Migration</a></li></ul></div></div></div><div id=content class=markdown-body><div class=margin-for-toc><h1 id=java-api-quickstart>Java API Quickstart</h1><h2 id=create-a-table>Create a table</h2><p>Tables are created using either a <a href=../../ [...]
You can initialize a Hive catalog with a name and some properties.
(see: <a href=../configuration/#catalog-properties>Catalog properties</a>)</p><p><strong>Note:</strong> Currently, <code>setConf</code> is always required for hive catalogs, but this will change in the future.</p><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-java data-lang=java><span style=display:flex><span><span style=color:#f92672>import</span> org.apache.iceberg.hive.HiveCatalog<span sty [...]
</span></span><span style=display:flex><span>
diff --git a/docs/1.2.1/jdbc/index.html b/docs/1.2.1/jdbc/index.html
index 9b07af03..cd9f293b 100644
--- a/docs/1.2.1/jdbc/index.html
+++ b/docs/1.2.1/jdbc/index.html
@@ -5,7 +5,7 @@
<span class=icon-bar></span></button>
<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/1.2.1//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.2.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg.a [...]
<i class="fa fa-chevron-right"></i>
-<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul class=sub-menu><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li></ul></div><li><a class="chevron-toggle collapsed" data-toggle=collaps [...]
+<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li></ul></div><li [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Spark class=collapse><ul class=sub-menu><li><a href=../spark-ddl/>DDL</a></li><li><a href=../getting-started/>Getting Started</a></li><li><a href=../spark-procedures/>Procedures</a></li><li><a href=../spark-queries/>Queries</a></li><li><a href=../spark-structured-streaming/>Structured Streaming</a></li><li><a href=../spark-writes/>Writes</a></li></ul></div><li><a class="chevron-toggle collapsed" data-toggle=collapse data-parent=full href [...]
<i class="fa fa-chevron-right"></i>
@@ -13,7 +13,9 @@
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Integrations class="collapse in"><ul class=sub-menu><li><a href=../aws/>AWS</a></li><li><a href=../dell/>Dell</a></li><li><a id=active href=../jdbc/>JDBC</a></li><li><a href=../nessie/>Nessie</a></li></ul></div><li><a class="chevron-toggle collapsed" data-toggle=collapse data-parent=full href=#API><span>API</span>
<i class="fa fa-chevron-right"></i>
-<i class="fa fa-chevron-down"></i></a></li><div id=API class=collapse><ul class=sub-menu><li><a href=../java-api-quickstart/>Java Quickstart</a></li><li><a href=../api/>Java API</a></li><li><a href=../custom-catalog/>Java Custom Catalog</a></li></ul></div><li><a href=https://iceberg.apache.org/docs/1.2.1/../../javadoc/latest><span>Javadoc</span></a></li><li><a target=_blank href=https://py.iceberg.apache.org/><span>PyIceberg</span></a></li></div></div><div id=content class=markdown-body> [...]
+<i class="fa fa-chevron-down"></i></a></li><div id=API class=collapse><ul class=sub-menu><li><a href=../java-api-quickstart/>Java Quickstart</a></li><li><a href=../api/>Java API</a></li><li><a href=../custom-catalog/>Java Custom Catalog</a></li></ul></div><li><a href=https://iceberg.apache.org/docs/1.2.1/../../javadoc/latest><span>Javadoc</span></a></li><li><a target=_blank href=https://py.iceberg.apache.org/><span>PyIceberg</span></a></li><li><a class="chevron-toggle collapsed" data-tog [...]
+<i class="fa fa-chevron-right"></i>
+<i class="fa fa-chevron-down"></i></a></li><div id=Migration class=collapse><ul class=sub-menu><li><a href=../table-migration/>Overview</a></li><li><a href=../hive-migration/>Hive Migration</a></li><li><a href=../delta-lake-migration/>Delta Lake Migration</a></li></ul></div></div></div><div id=content class=markdown-body><div class=margin-for-toc><h1 id=iceberg-jdbc-integration>Iceberg JDBC Integration</h1><h2 id=jdbc-catalog>JDBC Catalog</h2><p>Iceberg supports using a table in a relati [...]
The database that JDBC connects to must support atomic transaction to allow the JDBC catalog implementation to
properly support atomic Iceberg table commits and read serializable isolation.</p><h3 id=configurations>Configurations</h3><p>Because each database and database service provider might require different configurations,
the JDBC catalog allows arbitrary configurations through:</p><table><thead><tr><th>Property</th><th>Default</th><th>Description</th></tr></thead><tbody><tr><td>uri</td><td></td><td>the JDBC connection string</td></tr><tr><td>jdbc.<property_key></td><td></td><td>any key value pairs to configure the JDBC connection</td></tr></tbody></table><h3 id=examples>Examples</h3><h4 id=spark>Spark</h4><p>You can start a Spark session with a MySQL JDBC connection using the following configurations: [...]
diff --git a/docs/1.2.1/maintenance/index.html b/docs/1.2.1/maintenance/index.html
index 67248aed..a395c8fc 100644
--- a/docs/1.2.1/maintenance/index.html
+++ b/docs/1.2.1/maintenance/index.html
@@ -5,7 +5,7 @@
<span class=icon-bar></span></button>
<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/1.2.1//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.2.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg.a [...]
<i class="fa fa-chevron-right"></i>
-<i class="fa fa-chevron-down"></i></a></li><div id=Tables class="collapse in"><ul class=sub-menu><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a id=active href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li></ul></div><li><a class="chevron-toggle collapsed" data [...]
+<i class="fa fa-chevron-down"></i></a></li><div id=Tables class="collapse in"><ul class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a id=active href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Spark class=collapse><ul class=sub-menu><li><a href=../spark-ddl/>DDL</a></li><li><a href=../getting-started/>Getting Started</a></li><li><a href=../spark-procedures/>Procedures</a></li><li><a href=../spark-queries/>Queries</a></li><li><a href=../spark-structured-streaming/>Structured Streaming</a></li><li><a href=../spark-writes/>Writes</a></li></ul></div><li><a class="chevron-toggle collapsed" data-toggle=collapse data-parent=full href [...]
<i class="fa fa-chevron-right"></i>
@@ -13,7 +13,9 @@
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Integrations class=collapse><ul class=sub-menu><li><a href=../aws/>AWS</a></li><li><a href=../dell/>Dell</a></li><li><a href=../jdbc/>JDBC</a></li><li><a href=../nessie/>Nessie</a></li></ul></div><li><a class="chevron-toggle collapsed" data-toggle=collapse data-parent=full href=#API><span>API</span>
<i class="fa fa-chevron-right"></i>
-<i class="fa fa-chevron-down"></i></a></li><div id=API class=collapse><ul class=sub-menu><li><a href=../java-api-quickstart/>Java Quickstart</a></li><li><a href=../api/>Java API</a></li><li><a href=../custom-catalog/>Java Custom Catalog</a></li></ul></div><li><a href=https://iceberg.apache.org/docs/1.2.1/../../javadoc/latest><span>Javadoc</span></a></li><li><a target=_blank href=https://py.iceberg.apache.org/><span>PyIceberg</span></a></li></div></div><div id=content class=markdown-body> [...]
+<i class="fa fa-chevron-down"></i></a></li><div id=API class=collapse><ul class=sub-menu><li><a href=../java-api-quickstart/>Java Quickstart</a></li><li><a href=../api/>Java API</a></li><li><a href=../custom-catalog/>Java Custom Catalog</a></li></ul></div><li><a href=https://iceberg.apache.org/docs/1.2.1/../../javadoc/latest><span>Javadoc</span></a></li><li><a target=_blank href=https://py.iceberg.apache.org/><span>PyIceberg</span></a></li><li><a class="chevron-toggle collapsed" data-tog [...]
+<i class="fa fa-chevron-right"></i>
+<i class="fa fa-chevron-down"></i></a></li><div id=Migration class=collapse><ul class=sub-menu><li><a href=../table-migration/>Overview</a></li><li><a href=../hive-migration/>Hive Migration</a></li><li><a href=../delta-lake-migration/>Delta Lake Migration</a></li></ul></div></div></div><div id=content class=markdown-body><div class=margin-for-toc><h1 id=maintenance>Maintenance</h1><div class=info>Maintenance operations require the <code>Table</code> instance. Please refer <a href=../java [...]
</span></span><span style=display:flex><span><span style=color:#66d9ef>long</span> tsToExpire <span style=color:#f92672>=</span> System<span style=color:#f92672>.</span><span style=color:#a6e22e>currentTimeMillis</span><span style=color:#f92672>()</span> <span style=color:#f92672>-</span> <span style=color:#f92672>(</span><span style=color:#ae81ff>1000</span> <span style=color:#f92672>*</span> <span style=color:#ae81ff>60</span> <span style=color:#f92672>*</span> <span style=color:#ae81f [...]
</span></span></span><span style=display:flex><span><span style=color:#75715e></span>table<span style=color:#f92672>.</span><span style=color:#a6e22e>expireSnapshots</span><span style=color:#f92672>()</span>
</span></span><span style=display:flex><span> <span style=color:#f92672>.</span><span style=color:#a6e22e>expireOlderThan</span><span style=color:#f92672>(</span>tsToExpire<span style=color:#f92672>)</span>
diff --git a/docs/1.2.1/nessie/index.html b/docs/1.2.1/nessie/index.html
index 096c2e0e..bf2450f5 100644
--- a/docs/1.2.1/nessie/index.html
+++ b/docs/1.2.1/nessie/index.html
@@ -5,7 +5,7 @@
<span class=icon-bar></span></button>
<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/1.2.1//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.2.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg.a [...]
<i class="fa fa-chevron-right"></i>
-<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul class=sub-menu><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li></ul></div><li><a class="chevron-toggle collapsed" data-toggle=collaps [...]
+<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li></ul></div><li [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Spark class=collapse><ul class=sub-menu><li><a href=../spark-ddl/>DDL</a></li><li><a href=../getting-started/>Getting Started</a></li><li><a href=../spark-procedures/>Procedures</a></li><li><a href=../spark-queries/>Queries</a></li><li><a href=../spark-structured-streaming/>Structured Streaming</a></li><li><a href=../spark-writes/>Writes</a></li></ul></div><li><a class="chevron-toggle collapsed" data-toggle=collapse data-parent=full href [...]
<i class="fa fa-chevron-right"></i>
@@ -13,7 +13,9 @@
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Integrations class="collapse in"><ul class=sub-menu><li><a href=../aws/>AWS</a></li><li><a href=../dell/>Dell</a></li><li><a href=../jdbc/>JDBC</a></li><li><a id=active href=../nessie/>Nessie</a></li></ul></div><li><a class="chevron-toggle collapsed" data-toggle=collapse data-parent=full href=#API><span>API</span>
<i class="fa fa-chevron-right"></i>
-<i class="fa fa-chevron-down"></i></a></li><div id=API class=collapse><ul class=sub-menu><li><a href=../java-api-quickstart/>Java Quickstart</a></li><li><a href=../api/>Java API</a></li><li><a href=../custom-catalog/>Java Custom Catalog</a></li></ul></div><li><a href=https://iceberg.apache.org/docs/1.2.1/../../javadoc/latest><span>Javadoc</span></a></li><li><a target=_blank href=https://py.iceberg.apache.org/><span>PyIceberg</span></a></li></div></div><div id=content class=markdown-body> [...]
+<i class="fa fa-chevron-down"></i></a></li><div id=API class=collapse><ul class=sub-menu><li><a href=../java-api-quickstart/>Java Quickstart</a></li><li><a href=../api/>Java API</a></li><li><a href=../custom-catalog/>Java Custom Catalog</a></li></ul></div><li><a href=https://iceberg.apache.org/docs/1.2.1/../../javadoc/latest><span>Javadoc</span></a></li><li><a target=_blank href=https://py.iceberg.apache.org/><span>PyIceberg</span></a></li><li><a class="chevron-toggle collapsed" data-tog [...]
+<i class="fa fa-chevron-right"></i>
+<i class="fa fa-chevron-down"></i></a></li><div id=Migration class=collapse><ul class=sub-menu><li><a href=../table-migration/>Overview</a></li><li><a href=../hive-migration/>Hive Migration</a></li><li><a href=../delta-lake-migration/>Delta Lake Migration</a></li></ul></div></div></div><div id=content class=markdown-body><div class=margin-for-toc><h1 id=iceberg-nessie-integration>Iceberg Nessie Integration</h1><p>Iceberg provides integration with Nessie through the <code>iceberg-nessie</ [...]
This section describes how to use Iceberg with Nessie. Nessie provides several key features on top of Iceberg:</p><ul><li>multi-table transactions</li><li>git-like operations (eg branches, tags, commits)</li><li>hive-like metastore capabilities</li></ul><p>See <a href=https://projectnessie.org>Project Nessie</a> for more information on Nessie. Nessie requires a server to run, see
<a href=https://projectnessie.org/try/>Getting Started</a> to start a Nessie server.</p><h2 id=enabling-nessie-catalog>Enabling Nessie Catalog</h2><p>The <code>iceberg-nessie</code> module is bundled with Spark and Flink runtimes for all versions from <code>0.11.0</code>. To get started
with Nessie (with spark-3.3) and Iceberg simply add the Iceberg runtime to your process. Eg: <code>spark-sql --packages org.apache.iceberg:iceberg-spark-runtime-3.3_2.12:1.2.0</code>.</p><h2 id=spark-sql-extensions>Spark SQL Extensions</h2><p>From Spark 3.1 and above, Nessie SQL extensions can be used to manage the Nessie repo as shown below.
diff --git a/docs/1.2.1/partitioning/index.html b/docs/1.2.1/partitioning/index.html
index db910562..e460dbc6 100644
--- a/docs/1.2.1/partitioning/index.html
+++ b/docs/1.2.1/partitioning/index.html
@@ -5,7 +5,7 @@
<span class=icon-bar></span></button>
<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/1.2.1//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.2.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg.a [...]
<i class="fa fa-chevron-right"></i>
-<i class="fa fa-chevron-down"></i></a></li><div id=Tables class="collapse in"><ul class=sub-menu><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a id=active href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li></ul></div><li><a class="chevron-toggle collapsed" data [...]
+<i class="fa fa-chevron-down"></i></a></li><div id=Tables class="collapse in"><ul class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a id=active href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Spark class=collapse><ul class=sub-menu><li><a href=../spark-ddl/>DDL</a></li><li><a href=../getting-started/>Getting Started</a></li><li><a href=../spark-procedures/>Procedures</a></li><li><a href=../spark-queries/>Queries</a></li><li><a href=../spark-structured-streaming/>Structured Streaming</a></li><li><a href=../spark-writes/>Writes</a></li></ul></div><li><a class="chevron-toggle collapsed" data-toggle=collapse data-parent=full href [...]
<i class="fa fa-chevron-right"></i>
@@ -13,7 +13,9 @@
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Integrations class=collapse><ul class=sub-menu><li><a href=../aws/>AWS</a></li><li><a href=../dell/>Dell</a></li><li><a href=../jdbc/>JDBC</a></li><li><a href=../nessie/>Nessie</a></li></ul></div><li><a class="chevron-toggle collapsed" data-toggle=collapse data-parent=full href=#API><span>API</span>
<i class="fa fa-chevron-right"></i>
-<i class="fa fa-chevron-down"></i></a></li><div id=API class=collapse><ul class=sub-menu><li><a href=../java-api-quickstart/>Java Quickstart</a></li><li><a href=../api/>Java API</a></li><li><a href=../custom-catalog/>Java Custom Catalog</a></li></ul></div><li><a href=https://iceberg.apache.org/docs/1.2.1/../../javadoc/latest><span>Javadoc</span></a></li><li><a target=_blank href=https://py.iceberg.apache.org/><span>PyIceberg</span></a></li></div></div><div id=content class=markdown-body> [...]
+<i class="fa fa-chevron-down"></i></a></li><div id=API class=collapse><ul class=sub-menu><li><a href=../java-api-quickstart/>Java Quickstart</a></li><li><a href=../api/>Java API</a></li><li><a href=../custom-catalog/>Java Custom Catalog</a></li></ul></div><li><a href=https://iceberg.apache.org/docs/1.2.1/../../javadoc/latest><span>Javadoc</span></a></li><li><a target=_blank href=https://py.iceberg.apache.org/><span>PyIceberg</span></a></li><li><a class="chevron-toggle collapsed" data-tog [...]
+<i class="fa fa-chevron-right"></i>
+<i class="fa fa-chevron-down"></i></a></li><div id=Migration class=collapse><ul class=sub-menu><li><a href=../table-migration/>Overview</a></li><li><a href=../hive-migration/>Hive Migration</a></li><li><a href=../delta-lake-migration/>Delta Lake Migration</a></li></ul></div></div></div><div id=content class=markdown-body><div class=margin-for-toc><h1 id=partitioning>Partitioning</h1><h2 id=what-is-partitioning>What is partitioning?</h2><p>Partitioning is a way to make queries faster by g [...]
</span></span><span style=display:flex><span><span style=color:#66d9ef>WHERE</span> event_time <span style=color:#66d9ef>BETWEEN</span> <span style=color:#e6db74>'2018-12-01 10:00:00'</span> <span style=color:#66d9ef>AND</span> <span style=color:#e6db74>'2018-12-01 12:00:00'</span>
</span></span></code></pre></div><p>Configuring the <code>logs</code> table to partition by the date of <code>event_time</code> will group log events into files with the same event date. Iceberg keeps track of that date and will use it to skip files for other dates that don’t have useful data.</p><p>Iceberg can partition timestamps by year, month, day, and hour granularity. It can also use a categorical column, like <code>level</code> in this logs example, to store rows together an [...]
</span></span><span style=display:flex><span> <span style=color:#66d9ef>SELECT</span> <span style=color:#66d9ef>level</span>, message, event_time, format_time(event_time, <span style=color:#e6db74>'YYYY-MM-dd'</span>)
diff --git a/docs/1.2.1/performance/index.html b/docs/1.2.1/performance/index.html
index 64ee8013..f984376b 100644
--- a/docs/1.2.1/performance/index.html
+++ b/docs/1.2.1/performance/index.html
@@ -5,7 +5,7 @@
<span class=icon-bar></span></button>
<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/1.2.1//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.2.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg.a [...]
<i class="fa fa-chevron-right"></i>
-<i class="fa fa-chevron-down"></i></a></li><div id=Tables class="collapse in"><ul class=sub-menu><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a id=active href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li></ul></div><li><a class="chevron-toggle collapsed" data [...]
+<i class="fa fa-chevron-down"></i></a></li><div id=Tables class="collapse in"><ul class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a id=active href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Spark class=collapse><ul class=sub-menu><li><a href=../spark-ddl/>DDL</a></li><li><a href=../getting-started/>Getting Started</a></li><li><a href=../spark-procedures/>Procedures</a></li><li><a href=../spark-queries/>Queries</a></li><li><a href=../spark-structured-streaming/>Structured Streaming</a></li><li><a href=../spark-writes/>Writes</a></li></ul></div><li><a class="chevron-toggle collapsed" data-toggle=collapse data-parent=full href [...]
<i class="fa fa-chevron-right"></i>
@@ -13,7 +13,9 @@
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Integrations class=collapse><ul class=sub-menu><li><a href=../aws/>AWS</a></li><li><a href=../dell/>Dell</a></li><li><a href=../jdbc/>JDBC</a></li><li><a href=../nessie/>Nessie</a></li></ul></div><li><a class="chevron-toggle collapsed" data-toggle=collapse data-parent=full href=#API><span>API</span>
<i class="fa fa-chevron-right"></i>
-<i class="fa fa-chevron-down"></i></a></li><div id=API class=collapse><ul class=sub-menu><li><a href=../java-api-quickstart/>Java Quickstart</a></li><li><a href=../api/>Java API</a></li><li><a href=../custom-catalog/>Java Custom Catalog</a></li></ul></div><li><a href=https://iceberg.apache.org/docs/1.2.1/../../javadoc/latest><span>Javadoc</span></a></li><li><a target=_blank href=https://py.iceberg.apache.org/><span>PyIceberg</span></a></li></div></div><div id=content class=markdown-body> [...]
+<i class="fa fa-chevron-down"></i></a></li><div id=API class=collapse><ul class=sub-menu><li><a href=../java-api-quickstart/>Java Quickstart</a></li><li><a href=../api/>Java API</a></li><li><a href=../custom-catalog/>Java Custom Catalog</a></li></ul></div><li><a href=https://iceberg.apache.org/docs/1.2.1/../../javadoc/latest><span>Javadoc</span></a></li><li><a target=_blank href=https://py.iceberg.apache.org/><span>PyIceberg</span></a></li><li><a class="chevron-toggle collapsed" data-tog [...]
+<i class="fa fa-chevron-right"></i>
+<i class="fa fa-chevron-down"></i></a></li><div id=Migration class=collapse><ul class=sub-menu><li><a href=../table-migration/>Overview</a></li><li><a href=../hive-migration/>Hive Migration</a></li><li><a href=../delta-lake-migration/>Delta Lake Migration</a></li></ul></div></div></div><div id=content class=markdown-body><div class=margin-for-toc><h1 id=performance>Performance</h1><ul><li>Iceberg is designed for huge tables and is used in production where a <em>single table</em> can cont [...]
<script src=https://iceberg.apache.org/docs/1.2.1//js/jquery.easing.min.js></script>
<script type=text/javascript src=https://iceberg.apache.org/docs/1.2.1//js/search.js></script>
<script src=https://iceberg.apache.org/docs/1.2.1//js/bootstrap.min.js></script>
diff --git a/docs/1.2.1/reliability/index.html b/docs/1.2.1/reliability/index.html
index e3a27242..8bc45c52 100644
--- a/docs/1.2.1/reliability/index.html
+++ b/docs/1.2.1/reliability/index.html
@@ -5,7 +5,7 @@
<span class=icon-bar></span></button>
<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/1.2.1//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.2.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg.a [...]
<i class="fa fa-chevron-right"></i>
-<i class="fa fa-chevron-down"></i></a></li><div id=Tables class="collapse in"><ul class=sub-menu><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a id=active href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li></ul></div><li><a class="chevron-toggle collapsed" data [...]
+<i class="fa fa-chevron-down"></i></a></li><div id=Tables class="collapse in"><ul class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a id=active href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Spark class=collapse><ul class=sub-menu><li><a href=../spark-ddl/>DDL</a></li><li><a href=../getting-started/>Getting Started</a></li><li><a href=../spark-procedures/>Procedures</a></li><li><a href=../spark-queries/>Queries</a></li><li><a href=../spark-structured-streaming/>Structured Streaming</a></li><li><a href=../spark-writes/>Writes</a></li></ul></div><li><a class="chevron-toggle collapsed" data-toggle=collapse data-parent=full href [...]
<i class="fa fa-chevron-right"></i>
@@ -13,7 +13,9 @@
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Integrations class=collapse><ul class=sub-menu><li><a href=../aws/>AWS</a></li><li><a href=../dell/>Dell</a></li><li><a href=../jdbc/>JDBC</a></li><li><a href=../nessie/>Nessie</a></li></ul></div><li><a class="chevron-toggle collapsed" data-toggle=collapse data-parent=full href=#API><span>API</span>
<i class="fa fa-chevron-right"></i>
-<i class="fa fa-chevron-down"></i></a></li><div id=API class=collapse><ul class=sub-menu><li><a href=../java-api-quickstart/>Java Quickstart</a></li><li><a href=../api/>Java API</a></li><li><a href=../custom-catalog/>Java Custom Catalog</a></li></ul></div><li><a href=https://iceberg.apache.org/docs/1.2.1/../../javadoc/latest><span>Javadoc</span></a></li><li><a target=_blank href=https://py.iceberg.apache.org/><span>PyIceberg</span></a></li></div></div><div id=content class=markdown-body> [...]
+<i class="fa fa-chevron-down"></i></a></li><div id=API class=collapse><ul class=sub-menu><li><a href=../java-api-quickstart/>Java Quickstart</a></li><li><a href=../api/>Java API</a></li><li><a href=../custom-catalog/>Java Custom Catalog</a></li></ul></div><li><a href=https://iceberg.apache.org/docs/1.2.1/../../javadoc/latest><span>Javadoc</span></a></li><li><a target=_blank href=https://py.iceberg.apache.org/><span>PyIceberg</span></a></li><li><a class="chevron-toggle collapsed" data-tog [...]
+<i class="fa fa-chevron-right"></i>
+<i class="fa fa-chevron-down"></i></a></li><div id=Migration class=collapse><ul class=sub-menu><li><a href=../table-migration/>Overview</a></li><li><a href=../hive-migration/>Hive Migration</a></li><li><a href=../delta-lake-migration/>Delta Lake Migration</a></li></ul></div></div></div><div id=content class=markdown-body><div class=margin-for-toc><h1 id=reliability>Reliability</h1><p>Iceberg was designed to solve correctness problems that affect Hive tables running in S3.</p><p>Hive tabl [...]
<script src=https://iceberg.apache.org/docs/1.2.1//js/jquery.easing.min.js></script>
<script type=text/javascript src=https://iceberg.apache.org/docs/1.2.1//js/search.js></script>
<script src=https://iceberg.apache.org/docs/1.2.1//js/bootstrap.min.js></script>
diff --git a/docs/1.2.1/schemas/index.html b/docs/1.2.1/schemas/index.html
index ad1f0e48..4fa1ceee 100644
--- a/docs/1.2.1/schemas/index.html
+++ b/docs/1.2.1/schemas/index.html
@@ -5,7 +5,7 @@
<span class=icon-bar></span></button>
<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/1.2.1//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.2.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg.a [...]
<i class="fa fa-chevron-right"></i>
-<i class="fa fa-chevron-down"></i></a></li><div id=Tables class="collapse in"><ul class=sub-menu><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a id=active href=../schemas/>Schemas</a></li></ul></div><li><a class="chevron-toggle collapsed" data [...]
+<i class="fa fa-chevron-down"></i></a></li><div id=Tables class="collapse in"><ul class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a id=active href=../schemas/>Schemas</a></li [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Spark class=collapse><ul class=sub-menu><li><a href=../spark-ddl/>DDL</a></li><li><a href=../getting-started/>Getting Started</a></li><li><a href=../spark-procedures/>Procedures</a></li><li><a href=../spark-queries/>Queries</a></li><li><a href=../spark-structured-streaming/>Structured Streaming</a></li><li><a href=../spark-writes/>Writes</a></li></ul></div><li><a class="chevron-toggle collapsed" data-toggle=collapse data-parent=full href [...]
<i class="fa fa-chevron-right"></i>
@@ -13,7 +13,9 @@
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Integrations class=collapse><ul class=sub-menu><li><a href=../aws/>AWS</a></li><li><a href=../dell/>Dell</a></li><li><a href=../jdbc/>JDBC</a></li><li><a href=../nessie/>Nessie</a></li></ul></div><li><a class="chevron-toggle collapsed" data-toggle=collapse data-parent=full href=#API><span>API</span>
<i class="fa fa-chevron-right"></i>
-<i class="fa fa-chevron-down"></i></a></li><div id=API class=collapse><ul class=sub-menu><li><a href=../java-api-quickstart/>Java Quickstart</a></li><li><a href=../api/>Java API</a></li><li><a href=../custom-catalog/>Java Custom Catalog</a></li></ul></div><li><a href=https://iceberg.apache.org/docs/1.2.1/../../javadoc/latest><span>Javadoc</span></a></li><li><a target=_blank href=https://py.iceberg.apache.org/><span>PyIceberg</span></a></li></div></div><div id=content class=markdown-body> [...]
+<i class="fa fa-chevron-down"></i></a></li><div id=API class=collapse><ul class=sub-menu><li><a href=../java-api-quickstart/>Java Quickstart</a></li><li><a href=../api/>Java API</a></li><li><a href=../custom-catalog/>Java Custom Catalog</a></li></ul></div><li><a href=https://iceberg.apache.org/docs/1.2.1/../../javadoc/latest><span>Javadoc</span></a></li><li><a target=_blank href=https://py.iceberg.apache.org/><span>PyIceberg</span></a></li><li><a class="chevron-toggle collapsed" data-tog [...]
+<i class="fa fa-chevron-right"></i>
+<i class="fa fa-chevron-down"></i></a></li><div id=Migration class=collapse><ul class=sub-menu><li><a href=../table-migration/>Overview</a></li><li><a href=../hive-migration/>Hive Migration</a></li><li><a href=../delta-lake-migration/>Delta Lake Migration</a></li></ul></div></div></div><div id=content class=markdown-body><div class=margin-for-toc><h1 id=schemas>Schemas</h1><p>Iceberg tables support the following types:</p><table><thead><tr><th>Type</th><th>Description</th><th>Notes</th>< [...]
<script src=https://iceberg.apache.org/docs/1.2.1//js/jquery.easing.min.js></script>
<script type=text/javascript src=https://iceberg.apache.org/docs/1.2.1//js/search.js></script>
<script src=https://iceberg.apache.org/docs/1.2.1//js/bootstrap.min.js></script>
diff --git a/docs/1.2.1/sitemap.xml b/docs/1.2.1/sitemap.xml
index 39ee65fc..f24acd0a 100644
--- a/docs/1.2.1/sitemap.xml
+++ b/docs/1.2.1/sitemap.xml
@@ -1 +1 @@
-<?xml version="1.0" encoding="utf-8" standalone="yes"?><urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:xhtml="http://www.w3.org/1999/xhtml"><url><loc>https://iceberg.apache.org/docs/1.2.1/getting-started/</loc></url><url><loc>https://iceberg.apache.org/docs/1.2.1/hive/</loc></url><url><loc>https://iceberg.apache.org/docs/1.2.1/aws/</loc></url><url><loc>https://iceberg.apache.org/docs/1.2.1/categories/</loc></url><url><loc>https://iceberg.apache.org/docs/1.2.1/configurat [...]
\ No newline at end of file
+<?xml version="1.0" encoding="utf-8" standalone="yes"?><urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:xhtml="http://www.w3.org/1999/xhtml"><url><loc>https://iceberg.apache.org/docs/1.2.1/getting-started/</loc></url><url><loc>https://iceberg.apache.org/docs/1.2.1/hive/</loc></url><url><loc>https://iceberg.apache.org/docs/1.2.1/aws/</loc></url><url><loc>https://iceberg.apache.org/docs/1.2.1/branching/</loc></url><url><loc>https://iceberg.apache.org/docs/1.2.1/categories/ [...]
\ No newline at end of file
diff --git a/docs/1.2.1/spark-configuration/index.html b/docs/1.2.1/spark-configuration/index.html
index 718e93c2..0a86372c 100644
--- a/docs/1.2.1/spark-configuration/index.html
+++ b/docs/1.2.1/spark-configuration/index.html
@@ -5,7 +5,7 @@
<span class=icon-bar></span></button>
<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/1.2.1//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.2.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg.a [...]
<i class="fa fa-chevron-right"></i>
-<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul class=sub-menu><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li></ul></div><li><a class="chevron-toggle collapsed" data-toggle=collaps [...]
+<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li></ul></div><li [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Spark class=collapse><ul class=sub-menu><li><a href=../spark-ddl/>DDL</a></li><li><a href=../getting-started/>Getting Started</a></li><li><a href=../spark-procedures/>Procedures</a></li><li><a href=../spark-queries/>Queries</a></li><li><a href=../spark-structured-streaming/>Structured Streaming</a></li><li><a href=../spark-writes/>Writes</a></li></ul></div><li><a class="chevron-toggle collapsed" data-toggle=collapse data-parent=full href [...]
<i class="fa fa-chevron-right"></i>
@@ -13,7 +13,9 @@
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Integrations class=collapse><ul class=sub-menu><li><a href=../aws/>AWS</a></li><li><a href=../dell/>Dell</a></li><li><a href=../jdbc/>JDBC</a></li><li><a href=../nessie/>Nessie</a></li></ul></div><li><a class="chevron-toggle collapsed" data-toggle=collapse data-parent=full href=#API><span>API</span>
<i class="fa fa-chevron-right"></i>
-<i class="fa fa-chevron-down"></i></a></li><div id=API class=collapse><ul class=sub-menu><li><a href=../java-api-quickstart/>Java Quickstart</a></li><li><a href=../api/>Java API</a></li><li><a href=../custom-catalog/>Java Custom Catalog</a></li></ul></div><li><a href=https://iceberg.apache.org/docs/1.2.1/../../javadoc/latest><span>Javadoc</span></a></li><li><a target=_blank href=https://py.iceberg.apache.org/><span>PyIceberg</span></a></li></div></div><div id=content class=markdown-body> [...]
+<i class="fa fa-chevron-down"></i></a></li><div id=API class=collapse><ul class=sub-menu><li><a href=../java-api-quickstart/>Java Quickstart</a></li><li><a href=../api/>Java API</a></li><li><a href=../custom-catalog/>Java Custom Catalog</a></li></ul></div><li><a href=https://iceberg.apache.org/docs/1.2.1/../../javadoc/latest><span>Javadoc</span></a></li><li><a target=_blank href=https://py.iceberg.apache.org/><span>PyIceberg</span></a></li><li><a class="chevron-toggle collapsed" data-tog [...]
+<i class="fa fa-chevron-right"></i>
+<i class="fa fa-chevron-down"></i></a></li><div id=Migration class=collapse><ul class=sub-menu><li><a href=../table-migration/>Overview</a></li><li><a href=../hive-migration/>Hive Migration</a></li><li><a href=../delta-lake-migration/>Delta Lake Migration</a></li></ul></div></div></div><div id=content class=markdown-body><div class=margin-for-toc><h1 id=spark-configuration>Spark Configuration</h1><h2 id=catalogs>Catalogs</h2><p>Spark adds an API to plug in table catalogs that are used to [...]
</span></span><span style=display:flex><span>spark.sql.catalog.hive_prod.type = hive
</span></span><span style=display:flex><span>spark.sql.catalog.hive_prod.uri = thrift://metastore-host:port
</span></span><span style=display:flex><span># omit uri to use the same URI as Spark: hive.metastore.uris in hive-site.xml
diff --git a/docs/1.2.1/spark-ddl/index.html b/docs/1.2.1/spark-ddl/index.html
index 5bfe1312..fe7378ce 100644
--- a/docs/1.2.1/spark-ddl/index.html
+++ b/docs/1.2.1/spark-ddl/index.html
@@ -5,7 +5,7 @@
<span class=icon-bar></span></button>
<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/1.2.1//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.2.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg.a [...]
<i class="fa fa-chevron-right"></i>
-<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul class=sub-menu><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li></ul></div><li><a class=chevron-toggle data-toggle=collapse data-paren [...]
+<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li></ul></div><li [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Spark class="collapse in"><ul class=sub-menu><li><a id=active href=../spark-ddl/>DDL</a></li><li><a href=../getting-started/>Getting Started</a></li><li><a href=../spark-procedures/>Procedures</a></li><li><a href=../spark-queries/>Queries</a></li><li><a href=../spark-structured-streaming/>Structured Streaming</a></li><li><a href=../spark-writes/>Writes</a></li></ul></div><li><a class="chevron-toggle collapsed" data-toggle=collapse data-p [...]
<i class="fa fa-chevron-right"></i>
@@ -13,7 +13,9 @@
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Integrations class=collapse><ul class=sub-menu><li><a href=../aws/>AWS</a></li><li><a href=../dell/>Dell</a></li><li><a href=../jdbc/>JDBC</a></li><li><a href=../nessie/>Nessie</a></li></ul></div><li><a class="chevron-toggle collapsed" data-toggle=collapse data-parent=full href=#API><span>API</span>
<i class="fa fa-chevron-right"></i>
-<i class="fa fa-chevron-down"></i></a></li><div id=API class=collapse><ul class=sub-menu><li><a href=../java-api-quickstart/>Java Quickstart</a></li><li><a href=../api/>Java API</a></li><li><a href=../custom-catalog/>Java Custom Catalog</a></li></ul></div><li><a href=https://iceberg.apache.org/docs/1.2.1/../../javadoc/latest><span>Javadoc</span></a></li><li><a target=_blank href=https://py.iceberg.apache.org/><span>PyIceberg</span></a></li></div></div><div id=content class=markdown-body> [...]
+<i class="fa fa-chevron-down"></i></a></li><div id=API class=collapse><ul class=sub-menu><li><a href=../java-api-quickstart/>Java Quickstart</a></li><li><a href=../api/>Java API</a></li><li><a href=../custom-catalog/>Java Custom Catalog</a></li></ul></div><li><a href=https://iceberg.apache.org/docs/1.2.1/../../javadoc/latest><span>Javadoc</span></a></li><li><a target=_blank href=https://py.iceberg.apache.org/><span>PyIceberg</span></a></li><li><a class="chevron-toggle collapsed" data-tog [...]
+<i class="fa fa-chevron-right"></i>
+<i class="fa fa-chevron-down"></i></a></li><div id=Migration class=collapse><ul class=sub-menu><li><a href=../table-migration/>Overview</a></li><li><a href=../hive-migration/>Hive Migration</a></li><li><a href=../delta-lake-migration/>Delta Lake Migration</a></li></ul></div></div></div><div id=content class=markdown-body><div class=margin-for-toc><h1 id=spark-ddl>Spark DDL</h1><p>To use Iceberg in Spark, first configure <a href=../spark-configuration>Spark catalogs</a>.</p><p>Iceberg use [...]
</span></span><span style=display:flex><span> id bigint <span style=color:#66d9ef>COMMENT</span> <span style=color:#e6db74>'unique id'</span>,
</span></span><span style=display:flex><span> <span style=color:#66d9ef>data</span> string)
</span></span><span style=display:flex><span><span style=color:#66d9ef>USING</span> iceberg
@@ -135,7 +137,29 @@ For example, if you partition by days and move to partitioning by hours, overwri
</span></span><span style=display:flex><span><span style=color:#75715e>-- single column
</span></span></span><span style=display:flex><span><span style=color:#75715e></span><span style=color:#66d9ef>ALTER</span> <span style=color:#66d9ef>TABLE</span> prod.db.sample <span style=color:#66d9ef>DROP</span> IDENTIFIER FIELDS id, <span style=color:#66d9ef>data</span>
</span></span><span style=display:flex><span><span style=color:#75715e>-- multiple columns
-</span></span></span></code></pre></div><p>Note that although the identifier is removed, the column will still exist in the table schema.</p></div><div id=toc class=markdown-body><div id=full><nav id=TableOfContents><ul><li><a href=#create-table><code>CREATE TABLE</code></a><ul><li><a href=#partitioned-by><code>PARTITIONED BY</code></a></li></ul></li><li><a href=#create-table--as-select><code>CREATE TABLE ... AS SELECT</code></a></li><li><a href=#replace-table--as-select><code>REPLACE TA [...]
+</span></span></span></code></pre></div><p>Note that although the identifier is removed, the column will still exist in the table schema.</p><h3 id=branching-and-tagging-ddl>Branching and Tagging DDL</h3><h4 id=alter-table--create-branch><code>ALTER TABLE ... CREATE BRANCH</code></h4><p>Branches can be created via the <code>CREATE BRANCH</code> statement, which includes
+the snapshot to create the branch at and an optional retention clause.</p><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-sql data-lang=sql><span style=display:flex><span><span style=color:#75715e>-- CREATE audit-branch at snapshot 1234 with default retention.
+</span></span></span><span style=display:flex><span><span style=color:#75715e></span><span style=color:#66d9ef>ALTER</span> <span style=color:#66d9ef>TABLE</span> prod.db.sample <span style=color:#66d9ef>CREATE</span> BRANCH audit<span style=color:#f92672>-</span>branch
+</span></span><span style=display:flex><span><span style=color:#66d9ef>AS</span> <span style=color:#66d9ef>OF</span> <span style=color:#66d9ef>VERSION</span> <span style=color:#ae81ff>1234</span>
+</span></span><span style=display:flex><span>
+</span></span><span style=display:flex><span><span style=color:#75715e>-- CREATE audit-branch at snapshot 1234, retain audit-branch for 31 days, and retain the latest 31 days. The latest 3 snapshot snapshots, and 2 days worth of snapshots
+</span></span></span><span style=display:flex><span><span style=color:#75715e></span><span style=color:#66d9ef>ALTER</span> <span style=color:#66d9ef>TABLE</span> prod.db.sample <span style=color:#66d9ef>CREATE</span> BRANCH audit<span style=color:#f92672>-</span>branch
+</span></span><span style=display:flex><span><span style=color:#66d9ef>AS</span> <span style=color:#66d9ef>OF</span> <span style=color:#66d9ef>VERSION</span> <span style=color:#ae81ff>1234</span> RETAIN <span style=color:#ae81ff>30</span> DAYS
+</span></span><span style=display:flex><span><span style=color:#66d9ef>WITH</span> RETENTION <span style=color:#ae81ff>3</span> SNAPSHOTS <span style=color:#ae81ff>2</span> DAYS
+</span></span></code></pre></div><h4 id=alter-table--create-tag><code>ALTER TABLE ... CREATE TAG</code></h4><p>Tags can be created via the <code>CREATE TAG</code> statement, which includes
+the snapshot to create the branch at and an optional retention clause.</p><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-sql data-lang=sql><span style=display:flex><span><span style=color:#75715e>-- CREATE historical-tag at snapshot 1234 with default retention.
+</span></span></span><span style=display:flex><span><span style=color:#75715e></span><span style=color:#66d9ef>ALTER</span> <span style=color:#66d9ef>TABLE</span> prod.db.sample <span style=color:#66d9ef>CREATE</span> TAG historical<span style=color:#f92672>-</span>tag <span style=color:#66d9ef>AS</span> <span style=color:#66d9ef>OF</span> <span style=color:#66d9ef>VERSION</span> <span style=color:#ae81ff>1234</span>
+</span></span><span style=display:flex><span>
+</span></span><span style=display:flex><span><span style=color:#75715e>-- CREATE historical-tag at snapshot 1234 and retain it for 1 year.
+</span></span></span><span style=display:flex><span><span style=color:#75715e></span><span style=color:#66d9ef>ALTER</span> <span style=color:#66d9ef>TABLE</span> prod.db.sample <span style=color:#66d9ef>CREATE</span> TAG historical<span style=color:#f92672>-</span>tag
+</span></span><span style=display:flex><span><span style=color:#66d9ef>AS</span> <span style=color:#66d9ef>OF</span> <span style=color:#66d9ef>VERSION</span> <span style=color:#ae81ff>1234</span> RETAIN <span style=color:#ae81ff>365</span> DAYS
+</span></span></code></pre></div><h3 id=alter-table--replace-branch><code>ALTER TABLE ... REPLACE BRANCH</code></h3><p>The snapshot which a branch references can be updated via
+the <code>REPLACE BRANCH</code> sql. Retention can also be updated in this statement.</p><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-sql data-lang=sql><span style=display:flex><span><span style=color:#75715e>-- REPLACE audit-branch to reference snapshot 4567 and update the retention to 60 days
+</span></span></span><span style=display:flex><span><span style=color:#75715e></span><span style=color:#66d9ef>ALTER</span> <span style=color:#66d9ef>TABLE</span> prod.db.sample <span style=color:#66d9ef>REPLACE</span> BRANCH audit<span style=color:#f92672>-</span>branch
+</span></span><span style=display:flex><span><span style=color:#66d9ef>AS</span> <span style=color:#66d9ef>OF</span> <span style=color:#66d9ef>VERSION</span> <span style=color:#ae81ff>4567</span> RETAIN <span style=color:#ae81ff>60</span> DAYS
+</span></span></code></pre></div><h4 id=alter-table--drop-branch><code>ALTER TABLE ... DROP BRANCH</code></h4><p>Branches can be removed via the <code>DROP BRANCH</code> sql</p><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-sql data-lang=sql><span style=display:flex><span><span style=color:#66d9ef>ALTER</span> <span style=color:#66d9ef>TABLE</span> prod.db.sample <span style=color:#66d9ef>DRO [...]
+</span></span></code></pre></div><h4 id=alter-table--drop-tag><code>ALTER TABLE ... DROP TAG</code></h4><p>Tags can be removed via the <code>DROP TAG</code> sql</p><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-sql data-lang=sql><span style=display:flex><span><span style=color:#66d9ef>ALTER</span> <span style=color:#66d9ef>TABLE</span> prod.db.sample <span style=color:#66d9ef>DROP</span> TAG [...]
+</span></span></code></pre></div></div><div id=toc class=markdown-body><div id=full><nav id=TableOfContents><ul><li><a href=#create-table><code>CREATE TABLE</code></a><ul><li><a href=#partitioned-by><code>PARTITIONED BY</code></a></li></ul></li><li><a href=#create-table--as-select><code>CREATE TABLE ... AS SELECT</code></a></li><li><a href=#replace-table--as-select><code>REPLACE TABLE ... AS SELECT</code></a></li><li><a href=#drop-table><code>DROP TABLE</code></a><ul><li><a href=#drop-ta [...]
<script src=https://iceberg.apache.org/docs/1.2.1//js/jquery.easing.min.js></script>
<script type=text/javascript src=https://iceberg.apache.org/docs/1.2.1//js/search.js></script>
<script src=https://iceberg.apache.org/docs/1.2.1//js/bootstrap.min.js></script>
diff --git a/docs/1.2.1/spark-procedures/index.html b/docs/1.2.1/spark-procedures/index.html
index b7028fc1..b8177ac0 100644
--- a/docs/1.2.1/spark-procedures/index.html
+++ b/docs/1.2.1/spark-procedures/index.html
@@ -5,7 +5,7 @@
<span class=icon-bar></span></button>
<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/1.2.1//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.2.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg.a [...]
<i class="fa fa-chevron-right"></i>
-<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul class=sub-menu><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li></ul></div><li><a class=chevron-toggle data-toggle=collapse data-paren [...]
+<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li></ul></div><li [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Spark class="collapse in"><ul class=sub-menu><li><a href=../spark-ddl/>DDL</a></li><li><a href=../getting-started/>Getting Started</a></li><li><a id=active href=../spark-procedures/>Procedures</a></li><li><a href=../spark-queries/>Queries</a></li><li><a href=../spark-structured-streaming/>Structured Streaming</a></li><li><a href=../spark-writes/>Writes</a></li></ul></div><li><a class="chevron-toggle collapsed" data-toggle=collapse data-p [...]
<i class="fa fa-chevron-right"></i>
@@ -13,7 +13,9 @@
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Integrations class=collapse><ul class=sub-menu><li><a href=../aws/>AWS</a></li><li><a href=../dell/>Dell</a></li><li><a href=../jdbc/>JDBC</a></li><li><a href=../nessie/>Nessie</a></li></ul></div><li><a class="chevron-toggle collapsed" data-toggle=collapse data-parent=full href=#API><span>API</span>
<i class="fa fa-chevron-right"></i>
-<i class="fa fa-chevron-down"></i></a></li><div id=API class=collapse><ul class=sub-menu><li><a href=../java-api-quickstart/>Java Quickstart</a></li><li><a href=../api/>Java API</a></li><li><a href=../custom-catalog/>Java Custom Catalog</a></li></ul></div><li><a href=https://iceberg.apache.org/docs/1.2.1/../../javadoc/latest><span>Javadoc</span></a></li><li><a target=_blank href=https://py.iceberg.apache.org/><span>PyIceberg</span></a></li></div></div><div id=content class=markdown-body> [...]
+<i class="fa fa-chevron-down"></i></a></li><div id=API class=collapse><ul class=sub-menu><li><a href=../java-api-quickstart/>Java Quickstart</a></li><li><a href=../api/>Java API</a></li><li><a href=../custom-catalog/>Java Custom Catalog</a></li></ul></div><li><a href=https://iceberg.apache.org/docs/1.2.1/../../javadoc/latest><span>Javadoc</span></a></li><li><a target=_blank href=https://py.iceberg.apache.org/><span>PyIceberg</span></a></li><li><a class="chevron-toggle collapsed" data-tog [...]
+<i class="fa fa-chevron-right"></i>
+<i class="fa fa-chevron-down"></i></a></li><div id=Migration class=collapse><ul class=sub-menu><li><a href=../table-migration/>Overview</a></li><li><a href=../hive-migration/>Hive Migration</a></li><li><a href=../delta-lake-migration/>Delta Lake Migration</a></li></ul></div></div></div><div id=content class=markdown-body><div class=margin-for-toc><h1 id=spark-procedures>Spark Procedures</h1><p>To use Iceberg in Spark, first configure <a href=../spark-configuration>Spark catalogs</a>. Sto [...]
</span></span></code></pre></div><h3 id=positional-arguments>Positional arguments</h3><p>When passing arguments by position, only the ending arguments may be omitted if they are optional.</p><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-sql data-lang=sql><span style=display:flex><span><span style=color:#66d9ef>CALL</span> <span style=color:#66d9ef>catalog_name</span>.<span style=color:#66d9e [...]
</span></span></code></pre></div><h2 id=snapshot-management>Snapshot management</h2><h3 id=rollback_to_snapshot><code>rollback_to_snapshot</code></h3><p>Roll back a table to a specific snapshot ID.</p><p>To roll back to a specific time, use <a href=#rollback_to_timestamp><code>rollback_to_timestamp</code></a>.</p><div class=info>This procedure invalidates all cached Spark plans that reference the affected table.</div><h4 id=usage-1>Usage</h4><table><thead><tr><th>Argument Name</th><th>Re [...]
</span></span></code></pre></div><h3 id=rollback_to_timestamp><code>rollback_to_timestamp</code></h3><p>Roll back a table to the snapshot that was current at some time.</p><div class=info>This procedure invalidates all cached Spark plans that reference the affected table.</div><h4 id=usage-2>Usage</h4><table><thead><tr><th>Argument Name</th><th>Required?</th><th>Type</th><th>Description</th></tr></thead><tbody><tr><td><code>table</code></td><td>✔️</td><td>string</td><td>Name of the table [...]
diff --git a/docs/1.2.1/spark-queries/index.html b/docs/1.2.1/spark-queries/index.html
index 19773f85..f30d4a87 100644
--- a/docs/1.2.1/spark-queries/index.html
+++ b/docs/1.2.1/spark-queries/index.html
@@ -5,7 +5,7 @@
<span class=icon-bar></span></button>
<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/1.2.1//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.2.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg.a [...]
<i class="fa fa-chevron-right"></i>
-<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul class=sub-menu><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li></ul></div><li><a class=chevron-toggle data-toggle=collapse data-paren [...]
+<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li></ul></div><li [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Spark class="collapse in"><ul class=sub-menu><li><a href=../spark-ddl/>DDL</a></li><li><a href=../getting-started/>Getting Started</a></li><li><a href=../spark-procedures/>Procedures</a></li><li><a id=active href=../spark-queries/>Queries</a></li><li><a href=../spark-structured-streaming/>Structured Streaming</a></li><li><a href=../spark-writes/>Writes</a></li></ul></div><li><a class="chevron-toggle collapsed" data-toggle=collapse data-p [...]
<i class="fa fa-chevron-right"></i>
@@ -13,7 +13,9 @@
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Integrations class=collapse><ul class=sub-menu><li><a href=../aws/>AWS</a></li><li><a href=../dell/>Dell</a></li><li><a href=../jdbc/>JDBC</a></li><li><a href=../nessie/>Nessie</a></li></ul></div><li><a class="chevron-toggle collapsed" data-toggle=collapse data-parent=full href=#API><span>API</span>
<i class="fa fa-chevron-right"></i>
-<i class="fa fa-chevron-down"></i></a></li><div id=API class=collapse><ul class=sub-menu><li><a href=../java-api-quickstart/>Java Quickstart</a></li><li><a href=../api/>Java API</a></li><li><a href=../custom-catalog/>Java Custom Catalog</a></li></ul></div><li><a href=https://iceberg.apache.org/docs/1.2.1/../../javadoc/latest><span>Javadoc</span></a></li><li><a target=_blank href=https://py.iceberg.apache.org/><span>PyIceberg</span></a></li></div></div><div id=content class=markdown-body> [...]
+<i class="fa fa-chevron-down"></i></a></li><div id=API class=collapse><ul class=sub-menu><li><a href=../java-api-quickstart/>Java Quickstart</a></li><li><a href=../api/>Java API</a></li><li><a href=../custom-catalog/>Java Custom Catalog</a></li></ul></div><li><a href=https://iceberg.apache.org/docs/1.2.1/../../javadoc/latest><span>Javadoc</span></a></li><li><a target=_blank href=https://py.iceberg.apache.org/><span>PyIceberg</span></a></li><li><a class="chevron-toggle collapsed" data-tog [...]
+<i class="fa fa-chevron-right"></i>
+<i class="fa fa-chevron-down"></i></a></li><div id=Migration class=collapse><ul class=sub-menu><li><a href=../table-migration/>Overview</a></li><li><a href=../hive-migration/>Hive Migration</a></li><li><a href=../delta-lake-migration/>Delta Lake Migration</a></li></ul></div></div></div><div id=content class=markdown-body><div class=margin-for-toc><h1 id=spark-queries>Spark Queries</h1><p>To use Iceberg in Spark, first configure <a href=../spark-configuration>Spark catalogs</a>.</p><p>Ice [...]
</span></span></span></code></pre></div><p>Metadata tables, like <code>history</code> and <code>snapshots</code>, can use the Iceberg table name as a namespace.</p><p>For example, to read from the <code>files</code> metadata table for <code>prod.db.table</code>:</p><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-sql data-lang=sql><span style=display:flex><span><span style=color:#66d9ef>SELECT< [...]
</span></span></code></pre></div><table><thead><tr><th>content</th><th>file_path</th><th>file_format</th><th>spec_id</th><th>partition</th><th>record_count</th><th>file_size_in_bytes</th><th>column_sizes</th><th>value_counts</th><th>null_value_counts</th><th>nan_value_counts</th><th>lower_bounds</th><th>upper_bounds</th><th>key_metadata</th><th>split_offsets</th><th>equality_ids</th><th>sort_order_id</th></tr></thead><tbody><tr><td>0</td><td>s3:/…/table/data/00000-3-8d6d60e8-d427-48 [...]
</span></span></code></pre></div><h3 id=catalogs-with-dataframereader>Catalogs with DataFrameReader</h3><p>Iceberg 0.11.0 adds multi-catalog support to <code>DataFrameReader</code> in both Spark 3 and 2.4.</p><p>Paths and table names can be loaded with Spark’s <code>DataFrameReader</code> interface. How tables are loaded depends on how
diff --git a/docs/1.2.1/spark-structured-streaming/index.html b/docs/1.2.1/spark-structured-streaming/index.html
index 1bc933d6..db6a5ac4 100644
--- a/docs/1.2.1/spark-structured-streaming/index.html
+++ b/docs/1.2.1/spark-structured-streaming/index.html
@@ -5,7 +5,7 @@
<span class=icon-bar></span></button>
<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/1.2.1//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.2.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg.a [...]
<i class="fa fa-chevron-right"></i>
-<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul class=sub-menu><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li></ul></div><li><a class=chevron-toggle data-toggle=collapse data-paren [...]
+<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li></ul></div><li [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Spark class="collapse in"><ul class=sub-menu><li><a href=../spark-ddl/>DDL</a></li><li><a href=../getting-started/>Getting Started</a></li><li><a href=../spark-procedures/>Procedures</a></li><li><a href=../spark-queries/>Queries</a></li><li><a id=active href=../spark-structured-streaming/>Structured Streaming</a></li><li><a href=../spark-writes/>Writes</a></li></ul></div><li><a class="chevron-toggle collapsed" data-toggle=collapse data-p [...]
<i class="fa fa-chevron-right"></i>
@@ -13,7 +13,9 @@
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Integrations class=collapse><ul class=sub-menu><li><a href=../aws/>AWS</a></li><li><a href=../dell/>Dell</a></li><li><a href=../jdbc/>JDBC</a></li><li><a href=../nessie/>Nessie</a></li></ul></div><li><a class="chevron-toggle collapsed" data-toggle=collapse data-parent=full href=#API><span>API</span>
<i class="fa fa-chevron-right"></i>
-<i class="fa fa-chevron-down"></i></a></li><div id=API class=collapse><ul class=sub-menu><li><a href=../java-api-quickstart/>Java Quickstart</a></li><li><a href=../api/>Java API</a></li><li><a href=../custom-catalog/>Java Custom Catalog</a></li></ul></div><li><a href=https://iceberg.apache.org/docs/1.2.1/../../javadoc/latest><span>Javadoc</span></a></li><li><a target=_blank href=https://py.iceberg.apache.org/><span>PyIceberg</span></a></li></div></div><div id=content class=markdown-body> [...]
+<i class="fa fa-chevron-down"></i></a></li><div id=API class=collapse><ul class=sub-menu><li><a href=../java-api-quickstart/>Java Quickstart</a></li><li><a href=../api/>Java API</a></li><li><a href=../custom-catalog/>Java Custom Catalog</a></li></ul></div><li><a href=https://iceberg.apache.org/docs/1.2.1/../../javadoc/latest><span>Javadoc</span></a></li><li><a target=_blank href=https://py.iceberg.apache.org/><span>PyIceberg</span></a></li><li><a class="chevron-toggle collapsed" data-tog [...]
+<i class="fa fa-chevron-right"></i>
+<i class="fa fa-chevron-down"></i></a></li><div id=Migration class=collapse><ul class=sub-menu><li><a href=../table-migration/>Overview</a></li><li><a href=../hive-migration/>Hive Migration</a></li><li><a href=../delta-lake-migration/>Delta Lake Migration</a></li></ul></div></div></div><div id=content class=markdown-body><div class=margin-for-toc><h1 id=spark-structured-streaming>Spark Structured Streaming</h1><p>Iceberg uses Apache Spark’s DataSourceV2 API for data source and cata [...]
with different levels of support in Spark versions.</p><p>As of Spark 3, DataFrame reads and writes are supported.</p><table><thead><tr><th>Feature support</th><th>Spark 3</th><th>Spark 2.4</th><th>Notes</th></tr></thead><tbody><tr><td><a href=#streaming-writes>DataFrame write</a></td><td>✔</td><td>✔</td><td></td></tr></tbody></table><h2 id=streaming-reads>Streaming Reads</h2><p>Iceberg supports processing incremental data in spark structured streaming jobs which starts from a historical [...]
</span></span><span style=display:flex><span> <span style=color:#f92672>.</span>format<span style=color:#f92672>(</span><span style=color:#e6db74>"iceberg"</span><span style=color:#f92672>)</span>
</span></span><span style=display:flex><span> <span style=color:#f92672>.</span>option<span style=color:#f92672>(</span><span style=color:#e6db74>"stream-from-timestamp"</span><span style=color:#f92672>,</span> <span style=color:#a6e22e>Long</span><span style=color:#f92672>.</span>toString<span style=color:#f92672>(</span>streamStartTimestamp<span style=color:#f92672>))</span>
diff --git a/docs/1.2.1/spark-writes/index.html b/docs/1.2.1/spark-writes/index.html
index 988596b6..7f642127 100644
--- a/docs/1.2.1/spark-writes/index.html
+++ b/docs/1.2.1/spark-writes/index.html
@@ -5,7 +5,7 @@
<span class=icon-bar></span></button>
<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/1.2.1//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.2.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg.a [...]
<i class="fa fa-chevron-right"></i>
-<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul class=sub-menu><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li></ul></div><li><a class=chevron-toggle data-toggle=collapse data-paren [...]
+<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li></ul></div><li [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Spark class="collapse in"><ul class=sub-menu><li><a href=../spark-ddl/>DDL</a></li><li><a href=../getting-started/>Getting Started</a></li><li><a href=../spark-procedures/>Procedures</a></li><li><a href=../spark-queries/>Queries</a></li><li><a href=../spark-structured-streaming/>Structured Streaming</a></li><li><a id=active href=../spark-writes/>Writes</a></li></ul></div><li><a class="chevron-toggle collapsed" data-toggle=collapse data-p [...]
<i class="fa fa-chevron-right"></i>
@@ -13,7 +13,9 @@
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Integrations class=collapse><ul class=sub-menu><li><a href=../aws/>AWS</a></li><li><a href=../dell/>Dell</a></li><li><a href=../jdbc/>JDBC</a></li><li><a href=../nessie/>Nessie</a></li></ul></div><li><a class="chevron-toggle collapsed" data-toggle=collapse data-parent=full href=#API><span>API</span>
<i class="fa fa-chevron-right"></i>
-<i class="fa fa-chevron-down"></i></a></li><div id=API class=collapse><ul class=sub-menu><li><a href=../java-api-quickstart/>Java Quickstart</a></li><li><a href=../api/>Java API</a></li><li><a href=../custom-catalog/>Java Custom Catalog</a></li></ul></div><li><a href=https://iceberg.apache.org/docs/1.2.1/../../javadoc/latest><span>Javadoc</span></a></li><li><a target=_blank href=https://py.iceberg.apache.org/><span>PyIceberg</span></a></li></div></div><div id=content class=markdown-body> [...]
+<i class="fa fa-chevron-down"></i></a></li><div id=API class=collapse><ul class=sub-menu><li><a href=../java-api-quickstart/>Java Quickstart</a></li><li><a href=../api/>Java API</a></li><li><a href=../custom-catalog/>Java Custom Catalog</a></li></ul></div><li><a href=https://iceberg.apache.org/docs/1.2.1/../../javadoc/latest><span>Javadoc</span></a></li><li><a target=_blank href=https://py.iceberg.apache.org/><span>PyIceberg</span></a></li><li><a class="chevron-toggle collapsed" data-tog [...]
+<i class="fa fa-chevron-right"></i>
+<i class="fa fa-chevron-down"></i></a></li><div id=Migration class=collapse><ul class=sub-menu><li><a href=../table-migration/>Overview</a></li><li><a href=../hive-migration/>Hive Migration</a></li><li><a href=../delta-lake-migration/>Delta Lake Migration</a></li></ul></div></div></div><div id=content class=markdown-body><div class=margin-for-toc><h1 id=spark-writes>Spark Writes</h1><p>To use Iceberg in Spark, first configure <a href=../spark-configuration>Spark catalogs</a>.</p><p>Some [...]
</span></span></code></pre></div><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-sql data-lang=sql><span style=display:flex><span><span style=color:#66d9ef>INSERT</span> <span style=color:#66d9ef>INTO</span> prod.db.<span style=color:#66d9ef>table</span> <span style=color:#66d9ef>SELECT</span> ...
</span></span></code></pre></div><h3 id=merge-into><code>MERGE INTO</code></h3><p>Spark 3 added support for <code>MERGE INTO</code> queries that can express row-level updates.</p><p>Iceberg supports <code>MERGE INTO</code> by rewriting data files that contain rows that need to be updated in an <code>overwrite</code> commit.</p><p><strong><code>MERGE INTO</code> is recommended instead of <code>INSERT OVERWRITE</code></strong> because Iceberg can replace only the affected data files, and b [...]
</span></span></span><span style=display:flex><span><span style=color:#75715e></span><span style=color:#66d9ef>USING</span> (<span style=color:#66d9ef>SELECT</span> ...) s <span style=color:#75715e>-- the source updates
@@ -61,7 +63,31 @@
</span></span><span style=display:flex><span><span style=color:#66d9ef>UPDATE</span> prod.db.orders <span style=color:#66d9ef>AS</span> t1
</span></span><span style=display:flex><span><span style=color:#66d9ef>SET</span> order_status <span style=color:#f92672>=</span> <span style=color:#e6db74>'returned'</span>
</span></span><span style=display:flex><span><span style=color:#66d9ef>WHERE</span> <span style=color:#66d9ef>EXISTS</span> (<span style=color:#66d9ef>SELECT</span> oid <span style=color:#66d9ef>FROM</span> prod.db.returned_orders <span style=color:#66d9ef>WHERE</span> t1.oid <span style=color:#f92672>=</span> oid)
-</span></span></code></pre></div><p>For more complex row-level updates based on incoming data, see the section on <code>MERGE INTO</code>.</p><h2 id=writing-with-dataframes>Writing with DataFrames</h2><p>Spark 3 introduced the new <code>DataFrameWriterV2</code> API for writing to tables using data frames. The v2 API is recommended for several reasons:</p><ul><li>CTAS, RTAS, and overwrite by filter are supported</li><li>All operations consistently write columns to a table by name</li><li> [...]
+</span></span></code></pre></div><p>For more complex row-level updates based on incoming data, see the section on <code>MERGE INTO</code>.</p><h2 id=writing-to-branches>Writing to Branches</h2><p>Branch writes can be performed via SQL by providing a branch identifier, <code>branch_yourBranch</code> in the operation.
+Branch writes can also be performed as part of a write-audit-publish (WAP) workflow by specifying the <code>spark.wap.branch</code> config.
+Note WAP branch and branch identifier cannot both be specified.
+Also, the branch must exist before performing the write.
+The operation does <strong>not</strong> create the branch if it does not exist.
+For more information on branches please refer to <a href=../../tables/branching>branches</a></p><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-sql data-lang=sql><span style=display:flex><span><span style=color:#75715e>-- INSERT (1,' a') (2, 'b') into the audit branch.
+</span></span></span><span style=display:flex><span><span style=color:#75715e></span><span style=color:#66d9ef>INSERT</span> <span style=color:#66d9ef>INTO</span> prod.db.<span style=color:#66d9ef>table</span>.branch_audit <span style=color:#66d9ef>VALUES</span> (<span style=color:#ae81ff>1</span>, <span style=color:#e6db74>'a'</span>), (<span style=color:#ae81ff>2</span>, <span style=color:#e6db74>'b'</span>);
+</span></span><span style=display:flex><span>
+</span></span><span style=display:flex><span><span style=color:#75715e>-- MERGE INTO audit branch
+</span></span></span><span style=display:flex><span><span style=color:#75715e></span>MERGE <span style=color:#66d9ef>INTO</span> prod.db.<span style=color:#66d9ef>table</span>.branch_audit t
+</span></span><span style=display:flex><span><span style=color:#66d9ef>USING</span> (<span style=color:#66d9ef>SELECT</span> ...) s
+</span></span><span style=display:flex><span><span style=color:#66d9ef>ON</span> t.id <span style=color:#f92672>=</span> s.id
+</span></span><span style=display:flex><span><span style=color:#66d9ef>WHEN</span> ...
+</span></span><span style=display:flex><span>
+</span></span><span style=display:flex><span><span style=color:#75715e>-- UPDATE audit branch
+</span></span></span><span style=display:flex><span><span style=color:#75715e></span><span style=color:#66d9ef>UPDATE</span> prod.db.<span style=color:#66d9ef>table</span>.branch_audit <span style=color:#66d9ef>AS</span> t1
+</span></span><span style=display:flex><span><span style=color:#66d9ef>SET</span> val <span style=color:#f92672>=</span> <span style=color:#e6db74>'c'</span>
+</span></span><span style=display:flex><span>
+</span></span><span style=display:flex><span><span style=color:#75715e>-- DELETE FROM audit branch
+</span></span></span><span style=display:flex><span><span style=color:#75715e></span><span style=color:#66d9ef>DELETE</span> <span style=color:#66d9ef>FROM</span> prod.dbl.<span style=color:#66d9ef>table</span>.branch_audit <span style=color:#66d9ef>WHERE</span> id <span style=color:#f92672>=</span> <span style=color:#ae81ff>2</span>;
+</span></span><span style=display:flex><span>
+</span></span><span style=display:flex><span><span style=color:#75715e>-- WAP Branch write
+</span></span></span><span style=display:flex><span><span style=color:#75715e></span><span style=color:#66d9ef>SET</span> spark.wap.branch <span style=color:#f92672>=</span> audit<span style=color:#f92672>-</span>branch
+</span></span><span style=display:flex><span><span style=color:#66d9ef>INSERT</span> <span style=color:#66d9ef>INTO</span> prod.db.<span style=color:#66d9ef>table</span> <span style=color:#66d9ef>VALUES</span> (<span style=color:#ae81ff>3</span>, <span style=color:#e6db74>'c'</span>);
+</span></span></code></pre></div><h2 id=writing-with-dataframes>Writing with DataFrames</h2><p>Spark 3 introduced the new <code>DataFrameWriterV2</code> API for writing to tables using data frames. The v2 API is recommended for several reasons:</p><ul><li>CTAS, RTAS, and overwrite by filter are supported</li><li>All operations consistently write columns to a table by name</li><li>Hidden partition expressions are supported in <code>partitionedBy</code></li><li>Overwrite behavior is explic [...]
Using <code>format("iceberg")</code> loads an isolated table reference that will not automatically refresh tables used by queries.</div><h3 id=appending-data>Appending data</h3><p>To append a dataframe to an Iceberg table, use <code>append</code>:</p><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-scala data-lang=scala><span style=display:flex><span><span style=color:#66d9ef>val</span> data<sp [...]
</span></span><span style=display:flex><span>data<span style=color:#f92672>.</span>writeTo<span style=color:#f92672>(</span><span style=color:#e6db74>"prod.db.table"</span><span style=color:#f92672>).</span>append<span style=color:#f92672>()</span>
</span></span></code></pre></div><h4 id=spark-24>Spark 2.4</h4><p>In Spark 2.4, use the v1 API with <code>append</code> mode and <code>iceberg</code> format:</p><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-scala data-lang=scala><span style=display:flex><span>data<span style=color:#f92672>.</span>write
@@ -123,7 +149,7 @@ which can be used in query.</div><p>Here we just registered the bucket function
</span></span><span style=display:flex><span> <span style=color:#f92672>.</span>writeTo<span style=color:#f92672>(</span><span style=color:#e6db74>"prod.db.sample"</span><span style=color:#f92672>)</span>
</span></span><span style=display:flex><span> <span style=color:#f92672>.</span>append<span style=color:#f92672>()</span>
</span></span></code></pre></div><h2 id=type-compatibility>Type compatibility</h2><p>Spark and Iceberg support different set of types. Iceberg does the type conversion automatically, but not for all combinations,
-so you may want to understand the type conversion in Iceberg in prior to design the types of columns in your tables.</p><h3 id=spark-type-to-iceberg-type>Spark type to Iceberg type</h3><p>This type conversion table describes how Spark types are converted to the Iceberg types. The conversion applies on both creating Iceberg table and writing to Iceberg table via Spark.</p><table><thead><tr><th>Spark</th><th>Iceberg</th><th>Notes</th></tr></thead><tbody><tr><td>boolean</td><td>boolean</td> [...]
+so you may want to understand the type conversion in Iceberg in prior to design the types of columns in your tables.</p><h3 id=spark-type-to-iceberg-type>Spark type to Iceberg type</h3><p>This type conversion table describes how Spark types are converted to the Iceberg types. The conversion applies on both creating Iceberg table and writing to Iceberg table via Spark.</p><table><thead><tr><th>Spark</th><th>Iceberg</th><th>Notes</th></tr></thead><tbody><tr><td>boolean</td><td>boolean</td> [...]
<script src=https://iceberg.apache.org/docs/1.2.1//js/jquery.easing.min.js></script>
<script type=text/javascript src=https://iceberg.apache.org/docs/1.2.1//js/search.js></script>
<script src=https://iceberg.apache.org/docs/1.2.1//js/bootstrap.min.js></script>
diff --git a/docs/1.2.1/schemas/index.html b/docs/1.2.1/table-migration/index.html
similarity index 56%
copy from docs/1.2.1/schemas/index.html
copy to docs/1.2.1/table-migration/index.html
index ad1f0e48..b4281980 100644
--- a/docs/1.2.1/schemas/index.html
+++ b/docs/1.2.1/table-migration/index.html
@@ -1,11 +1,11 @@
-<!doctype html><html><head><meta charset=utf-8><meta http-equiv=x-ua-compatible content="IE=edge"><meta name=viewport content="width=device-width,initial-scale=1"><meta name=description content><meta name=author content><title>Schemas</title><link href=../css/bootstrap.css rel=stylesheet><link href=../css/markdown.css rel=stylesheet><link href=../css/katex.min.css rel=stylesheet><link href=../css/iceberg-theme.css rel=stylesheet><link href=../font-awesome-4.7.0/css/font-awesome.min.css r [...]
+<!doctype html><html><head><meta charset=utf-8><meta http-equiv=x-ua-compatible content="IE=edge"><meta name=viewport content="width=device-width,initial-scale=1"><meta name=description content><meta name=author content><title>Overview</title><link href=../css/bootstrap.css rel=stylesheet><link href=../css/markdown.css rel=stylesheet><link href=../css/katex.min.css rel=stylesheet><link href=../css/iceberg-theme.css rel=stylesheet><link href=../font-awesome-4.7.0/css/font-awesome.min.css [...]
<span class=sr-only>Toggle navigation</span>
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/1.2.1//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.2.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg.a [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/1.2.1//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.2.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg.a [...]
<i class="fa fa-chevron-right"></i>
-<i class="fa fa-chevron-down"></i></a></li><div id=Tables class="collapse in"><ul class=sub-menu><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a id=active href=../schemas/>Schemas</a></li></ul></div><li><a class="chevron-toggle collapsed" data [...]
+<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li></ul></div><li [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Spark class=collapse><ul class=sub-menu><li><a href=../spark-ddl/>DDL</a></li><li><a href=../getting-started/>Getting Started</a></li><li><a href=../spark-procedures/>Procedures</a></li><li><a href=../spark-queries/>Queries</a></li><li><a href=../spark-structured-streaming/>Structured Streaming</a></li><li><a href=../spark-writes/>Writes</a></li></ul></div><li><a class="chevron-toggle collapsed" data-toggle=collapse data-parent=full href [...]
<i class="fa fa-chevron-right"></i>
@@ -13,7 +13,13 @@
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Integrations class=collapse><ul class=sub-menu><li><a href=../aws/>AWS</a></li><li><a href=../dell/>Dell</a></li><li><a href=../jdbc/>JDBC</a></li><li><a href=../nessie/>Nessie</a></li></ul></div><li><a class="chevron-toggle collapsed" data-toggle=collapse data-parent=full href=#API><span>API</span>
<i class="fa fa-chevron-right"></i>
-<i class="fa fa-chevron-down"></i></a></li><div id=API class=collapse><ul class=sub-menu><li><a href=../java-api-quickstart/>Java Quickstart</a></li><li><a href=../api/>Java API</a></li><li><a href=../custom-catalog/>Java Custom Catalog</a></li></ul></div><li><a href=https://iceberg.apache.org/docs/1.2.1/../../javadoc/latest><span>Javadoc</span></a></li><li><a target=_blank href=https://py.iceberg.apache.org/><span>PyIceberg</span></a></li></div></div><div id=content class=markdown-body> [...]
+<i class="fa fa-chevron-down"></i></a></li><div id=API class=collapse><ul class=sub-menu><li><a href=../java-api-quickstart/>Java Quickstart</a></li><li><a href=../api/>Java API</a></li><li><a href=../custom-catalog/>Java Custom Catalog</a></li></ul></div><li><a href=https://iceberg.apache.org/docs/1.2.1/../../javadoc/latest><span>Javadoc</span></a></li><li><a target=_blank href=https://py.iceberg.apache.org/><span>PyIceberg</span></a></li><li><a class=chevron-toggle data-toggle=collapse [...]
+<i class="fa fa-chevron-right"></i>
+<i class="fa fa-chevron-down"></i></a></li><div id=Migration class="collapse in"><ul class=sub-menu><li><a id=active href=../table-migration/>Overview</a></li><li><a href=../hive-migration/>Hive Migration</a></li><li><a href=../delta-lake-migration/>Delta Lake Migration</a></li></ul></div></div></div><div id=content class=markdown-body><div class=margin-for-toc><h1 id=table-migration>Table Migration</h1><p>Apache Iceberg supports converting existing tables in other formats to Iceberg tab [...]
+In practice, users can use operations like <a href=../spark-ddl/#create-table--as-select>Create-Table-As-Select</a>, <a href=../spark-writes/#insert-into>INSERT</a>, and Change-Data-Capture pipelines to perform such migration.</p><p>In-place metadata migration preserves the existing data files while incorporating Iceberg metadata on top of them.
+This method is not only faster but also eliminates the need for data duplication. However, the new table and the source table are not fully isolated. In other words, if any processes vacuum data files from the source table, the new table will also be affected.</p><p>In this doc, we will describe more about in-place metadata migration.</p><p><img src=../../../img/iceberg-in-place-metadata-migration.png alt="In-Place Metadata Migration"></p><p>Apache Iceberg supports the in-place metadata [...]
+Consequently, Migrate Table requires all modifications working on the source table to be stopped before the action is performed.</p><p>Stop all writers interacting with the source table. Readers that also support Iceberg may continue reading.</p><p><img src=../../../img/iceberg-migrateaction-step1.png alt="Migrate Table Step 1"></p><ul><li>Create a new Iceberg table with the same identifier and metadata (schema, partition spec, etc.) as the source table. Rename the source table for a bac [...]
+In practice, these files can be new data files in Hive tables or new snapshots (versions) of Delta Lake tables. The Add Files action is essential for incorporating these files into the Iceberg table.</p><h1 id=migrating-from-different-table-formats>Migrating From Different Table Formats</h1><ul><li><a href=../hive-migration>From Hive to Iceberg</a></li><li><a href=../delta-lake-migration>From Delta Lake to Iceberg</a></li></ul></div><div id=toc class=markdown-body><div id=full><nav id=Ta [...]
<script src=https://iceberg.apache.org/docs/1.2.1//js/jquery.easing.min.js></script>
<script type=text/javascript src=https://iceberg.apache.org/docs/1.2.1//js/search.js></script>
<script src=https://iceberg.apache.org/docs/1.2.1//js/bootstrap.min.js></script>
diff --git a/docs/1.2.1/tables/branching/index.html b/docs/1.2.1/tables/branching/index.html
new file mode 100644
index 00000000..f430c47b
--- /dev/null
+++ b/docs/1.2.1/tables/branching/index.html
@@ -0,0 +1 @@
+<!doctype html><html lang=en-us><head><title>https://iceberg.apache.org/docs/1.2.1/branching/</title><link rel=canonical href=https://iceberg.apache.org/docs/1.2.1/branching/><meta name=robots content="noindex"><meta charset=utf-8><meta http-equiv=refresh content="0; url=https://iceberg.apache.org/docs/1.2.1/branching/"></head></html>
\ No newline at end of file