You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@iceberg.apache.org by gi...@apache.org on 2023/10/06 16:57:15 UTC
[iceberg-docs] branch asf-site updated: deploy: 7724c28425ab68fc11b68b655c558226aca8cb20
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/iceberg-docs.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 1fab7c67 deploy: 7724c28425ab68fc11b68b655c558226aca8cb20
1fab7c67 is described below
commit 1fab7c67e0578c0ec059523e3f4b6925dc0ddffc
Author: aokolnychyi <ao...@users.noreply.github.com>
AuthorDate: Fri Oct 6 16:57:11 2023 +0000
deploy: 7724c28425ab68fc11b68b655c558226aca8cb20
---
about/about/index.html | 2 +-
benchmarks/index.html | 2 +-
blogs/index.html | 2 +-
common/index.xml | 4 ++--
community/index.html | 2 +-
concepts/catalog/index.html | 2 +-
contribute/index.html | 2 +-
gcm-stream-spec/index.html | 2 +-
getting-started/index.html | 20 +-------------------
hive-quickstart/index.html | 2 +-
how-to-release/index.html | 18 +++++++++---------
index.html | 2 +-
index.xml | 4 ++--
landingpagesearch.json | 2 +-
multi-engine-support/index.html | 6 +++---
puffin-spec/index.html | 2 +-
releases/index.html | 13 +++++++------
roadmap/index.html | 2 +-
security/index.html | 2 +-
services/data-compaction/index.html | 2 +-
services/expressive-sql/index.html | 2 +-
services/hidden-partitioning/index.html | 2 +-
services/schema-evolution/index.html | 2 +-
services/time-travel/index.html | 2 +-
spark-quickstart/index.html | 12 ++++++------
spec/index.html | 4 ++--
talks/index.html | 2 +-
terms/index.html | 2 +-
trademarks/index.html | 2 +-
vendors/index.html | 2 +-
view-spec/index.html | 2 +-
31 files changed, 55 insertions(+), 72 deletions(-)
diff --git a/about/about/index.html b/about/about/index.html
index b77d55fc..533c7c50 100644
--- a/about/about/index.html
+++ b/about/about/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.1</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.4.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
<script src=https://iceberg.apache.org//js/jquery.easing.min.js></script>
<script type=text/javascript src=https://iceberg.apache.org//js/search.js></script>
<script src=https://iceberg.apache.org//js/bootstrap.min.js></script>
diff --git a/benchmarks/index.html b/benchmarks/index.html
index 53b978e4..2e49d009 100644
--- a/benchmarks/index.html
+++ b/benchmarks/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.1</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.4.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
Also note that JMH benchmarks run within the same JVM as the system-under-test, so results might vary between runs.</p><h2 id=running-benchmarks-on-github>Running Benchmarks on GitHub</h2><p>It is possible to run one or more Benchmarks via the <strong>JMH Benchmarks</strong> GH action on your own fork of the Iceberg repo. This GH action takes the following inputs:</p><ul><li>The repository name where those benchmarks should be run against, such as <code>apache/iceberg</code> or <code>< [...]
It is likely that there is variability in results across different runs, therefore the benchmark results shouldn’t be used to form assumptions around production choices.</p><h2 id=running-benchmarks-locally>Running Benchmarks locally</h2><p>Below are the existing benchmarks shown with the actual commands on how to run them locally.</p><h3 id=icebergsourcenestedlistparquetdatawritebenchmark>IcebergSourceNestedListParquetDataWriteBenchmark</h3><p>A benchmark that evaluates the perfor [...]
column used in the filter predicate. The performance is compared to the built-in file source in Spark. To run this benchmark for either spark-2 or spark-3:</p><p><code>./gradlew :iceberg-spark:iceberg-spark[2|3]:jmh -PjmhIncludeRegex=IcebergSourceFlatParquetDataFilterBenchmark -PjmhOutputPath=benchmark/iceberg-source-flat-parquet-data-filter-benchmark-result.txt</code></p><h3 id=icebergsourcenestedparquetdatafilterbenchmark>IcebergSourceNestedParquetDataFilterBenchmark</h3><p>A benchmark [...]
diff --git a/blogs/index.html b/blogs/index.html
index e4c57e21..9383092a 100644
--- a/blogs/index.html
+++ b/blogs/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.1</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.4.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
<script src=https://iceberg.apache.org//js/jquery.easing.min.js></script>
<script type=text/javascript src=https://iceberg.apache.org//js/search.js></script>
<script src=https://iceberg.apache.org//js/bootstrap.min.js></script>
diff --git a/common/index.xml b/common/index.xml
index 51192431..c6ccfa4b 100644
--- a/common/index.xml
+++ b/common/index.xml
@@ -1,7 +1,7 @@
<?xml version="1.0" encoding="utf-8" standalone="yes"?><rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom"><channel><title>Commons on Apache Iceberg</title><link>https://iceberg.apache.org/common/</link><description>Recent content in Commons on Apache Iceberg</description><generator>Hugo -- gohugo.io</generator><language>en-us</language><atom:link href="https://iceberg.apache.org/common/index.xml" rel="self" type="application/rss+xml"/><item><title>Hive and Iceberg Quickstart</ti [...]
Docker Images Creating a Table Writing Data to a Table Reading Data from a Table Next Steps Docker Images The fastest way to get started is to use Apache Hive images which provides a SQL-like interface to create and query Iceberg tables from your laptop.</description></item><item><title>Spark and Iceberg Quickstart</title><link>https://iceberg.apache.org/spark-quickstart/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><guid>https://iceberg.apache.org/spark-quickstart/</guid><des [...]
-Docker-Compose Creating a table Writing Data to a Table Reading Data from a Table Adding A Catalog Next Steps Docker-Compose The fastest way to get started is to use a docker-compose file that uses the tabulario/spark-iceberg image which contains a local Spark cluster with a configured Iceberg catalog.</description></item><item><title>Releases</title><link>https://iceberg.apache.org/releases/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><guid>https://iceberg.apache.org/release [...]
-1.3.1 source tar.gz &ndash; signature &ndash; sha512 1.3.1 Spark 3.4_2.12 runtime Jar &ndash; 3.4_2.13 1.3.1 Spark 3.3_2.12 runtime Jar &ndash; 3.3_2.13 1.3.1 Spark 3.2_2.12 runtime Jar &ndash; 3.2_2.13 1.3.1 Spark 3.1 runtime Jar 1.3.1 Flink 1.17 runtime Jar 1.3.1 Flink 1.16 runtime Jar 1.3.1 Flink 1.15 runtime Jar 1.3.1 Hive runtime Jar To use Iceberg in Spark or Flink, download the runtime JAR for your engine version and add it to the jars folder of your installati [...]
+Docker-Compose Creating a table Writing Data to a Table Reading Data from a Table Adding A Catalog Next Steps Docker-Compose The fastest way to get started is to use a docker-compose file that uses the tabulario/spark-iceberg image which contains a local Spark cluster with a configured Iceberg catalog.</description></item><item><title>Releases</title><link>https://iceberg.apache.org/releases/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><guid>https://iceberg.apache.org/release [...]
+1.4.0 source tar.gz &ndash; signature &ndash; sha512 1.4.0 Spark 3.5_2.12 runtime Jar &ndash; 3.5_2.13 1.4.0 Spark 3.4_2.12 runtime Jar &ndash; 3.4_2.13 1.4.0 Spark 3.3_2.12 runtime Jar &ndash; 3.3_2.13 1.4.0 Spark 3.2_2.12 runtime Jar &ndash; 3.2_2.13 1.4.0 Flink 1.17 runtime Jar 1.4.0 Flink 1.16 runtime Jar 1.4.0 Flink 1.15 runtime Jar 1.4.0 Hive runtime Jar 1.4.0 aws-bundle Jar 1.4.0 gcp-bundle Jar 1.4.0 azure-bundle Jar To use Iceberg in Spark or Flink, downlo [...]
Running Benchmarks on GitHub It is possible to run one or more Benchmarks via the JMH Benchmarks GH action on your own fork of the Iceberg repo.</description></item><item><title>Blogs</title><link>https://iceberg.apache.org/blogs/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><guid>https://iceberg.apache.org/blogs/</guid><description>Iceberg Blogs Here is a list of company blogs that talk about Iceberg. The blogs are ordered from most recent to oldest.
From Hive Tables to Iceberg Tables: Hassle-Free Date: July 14th, 2023, Company: Cloudera
Authors: Srinivas Rishindra Pothireddi
diff --git a/community/index.html b/community/index.html
index 3580d1d8..a4f17332 100644
--- a/community/index.html
+++ b/community/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.1</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.4.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
and all who engage with the community are expected to be respectful, open, come with the best interests of the community in mind,
and abide by the Apache Foundation <a href=https://www.apache.org/foundation/policies/conduct.html>Code of Conduct</a>.</p><h3 id=participants-with-corporate-interests>Participants with Corporate Interests</h3><p>A wide range of corporate entities have interests that overlap in both features and frameworks related to Iceberg and while we
encourage engagement and contributions, the community is not a venue for marketing, solicitation, or recruitment.</p><p>Any vendor who wants to participate in the Apache Iceberg community Slack workspace should create a dedicated vendor channel
diff --git a/concepts/catalog/index.html b/concepts/catalog/index.html
index 34f594f6..e0a0e6f0 100644
--- a/concepts/catalog/index.html
+++ b/concepts/catalog/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.1</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.4.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
it easier to make custom catalogs compatible with engines like Athena or Starburst without requiring the inclusion of a Jar into the classpath.</p></div><div id=toc class=markdown-body><div id=full><nav id=TableOfContents><ul><li><a href=#overview>Overview</a></li><li><a href=#catalog-implementations>Catalog Implementations</a></li><li><a href=#decoupling-using-the-rest-catalog>Decoupling Using the REST Catalog</a></li></ul></nav></div></div></div></div></section></body><script src=https [...]
<script src=https://iceberg.apache.org//js/jquery.easing.min.js></script>
<script type=text/javascript src=https://iceberg.apache.org//js/search.js></script>
diff --git a/contribute/index.html b/contribute/index.html
index f5597f64..6271ac4d 100644
--- a/contribute/index.html
+++ b/contribute/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.1</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.4.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
these are hard rules and they’re meant as a collection of helpful suggestions to make contributing as seamless of an
experience as possible.</p><p>If you are thinking of contributing but first would like to discuss the change you wish to make, we welcome you to
head over to the <a href=https://iceberg.apache.org/community/>Community</a> page on the official Iceberg documentation site
diff --git a/gcm-stream-spec/index.html b/gcm-stream-spec/index.html
index c6d66446..01d08e80 100644
--- a/gcm-stream-spec/index.html
+++ b/gcm-stream-spec/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.1</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.4.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
</code></pre><p>where</p><ul><li><code>Magic</code> is four bytes 0x41, 0x47, 0x53, 0x31 (“AGS1”, short for: AES GCM Stream, version 1)</li><li><code>BlockLength</code> is four bytes (little endian) integer keeping the length of the equal-size split blocks before encryption. The length is specified in bytes.</li><li><code>CipherBlockᵢ</code> is the i-th enciphered block in the file, with the structure defined below.</li></ul><h3 id=cipher-block-structure>Cipher Block structur [...]
<script src=https://iceberg.apache.org//js/jquery.easing.min.js></script>
<script type=text/javascript src=https://iceberg.apache.org//js/search.js></script>
diff --git a/getting-started/index.html b/getting-started/index.html
index b3c6e0d8..52cc0335 100644
--- a/getting-started/index.html
+++ b/getting-started/index.html
@@ -1,19 +1 @@
-<!--
- - Licensed to the Apache Software Foundation (ASF) under one or more
- - contributor license agreements. See the NOTICE file distributed with
- - this work for additional information regarding copyright ownership.
- - The ASF licenses this file to You under the Apache License, Version 2.0
- - (the "License"); you may not use this file except in compliance with
- - the License. You may obtain a copy of the License at
- -
- - http://www.apache.org/licenses/LICENSE-2.0
- -
- - Unless required by applicable law or agreed to in writing, software
- - distributed under the License is distributed on an "AS IS" BASIS,
- - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- - See the License for the specific language governing permissions and
- - limitations under the License.
- -->
-<head>
- <meta http-equiv="Refresh" content="0; url='/docs/latest/getting-started'" />
-</head>
+<!doctype html><html lang=en-us><head><title>https://iceberg.apache.org/spark-quickstart/</title><link rel=canonical href=https://iceberg.apache.org/spark-quickstart/><meta name=robots content="noindex"><meta charset=utf-8><meta http-equiv=refresh content="0; url=https://iceberg.apache.org/spark-quickstart/"></head></html>
\ No newline at end of file
diff --git a/hive-quickstart/index.html b/hive-quickstart/index.html
index 3f59cde7..bb141e9d 100644
--- a/hive-quickstart/index.html
+++ b/hive-quickstart/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.1</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.4.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
highlight some powerful features. You can learn more about Iceberg’s Hive runtime by checking out the <a href=../docs/latest/hive/>Hive</a> section.</p><ul><li><a href=#docker-images>Docker Images</a></li><li><a href=#creating-a-table>Creating a Table</a></li><li><a href=#writing-data-to-a-table>Writing Data to a Table</a></li><li><a href=#reading-data-from-a-table>Reading Data from a Table</a></li><li><a href=#next-steps>Next Steps</a></li></ul><h3 id=docker-images>Docker Images</ [...]
which provides a SQL-like interface to create and query Iceberg tables from your laptop. You need to install the <a href=https://www.docker.com/products/docker-desktop/>Docker Desktop</a>.</p><p>Take a look at the Tags tab in <a href="https://hub.docker.com/r/apache/hive/tags?page=1&ordering=-last_updated">Apache Hive docker images</a> to see the available Hive versions.</p><p>Set the version variable.</p><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#2728 [...]
</span></span></code></pre></div><p>Start the container, using the option <code>--platform linux/amd64</code> for a Mac with an M-Series chip:</p><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-sh data-lang=sh><span style=display:flex><span>docker run -d --platform linux/amd64 -p 10000:10000 -p 10002:10002 --env SERVICE_NAME<span style=color:#f92672>=</span>hiveserver2 --name hive4 apache/hive [...]
diff --git a/how-to-release/index.html b/how-to-release/index.html
index 48f413dd..102d9dc7 100644
--- a/how-to-release/index.html
+++ b/how-to-release/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.1</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.4.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
</span></span><span style=display:flex><span>cd icebergsvn
</span></span><span style=display:flex><span>echo <span style=color:#e6db74>""</span> >> KEYS <span style=color:#75715e># append a newline</span>
</span></span><span style=display:flex><span>gpg --list-sigs <YOUR KEY ID HERE> >> KEYS <span style=color:#75715e># append signatures</span>
@@ -156,10 +156,10 @@ It’s recommended to report the Java, Scala, Spark, Flink and Hive versions
license documentation.</p><h2 id=validating-a-source-release-candidate>Validating a source release candidate</h2><p>Release announcements include links to the following:</p><ul><li><strong>A source tarball</strong></li><li><strong>A signature (.asc)</strong></li><li><strong>A checksum (.sha512)</strong></li><li><strong>KEYS file</strong></li><li><strong>GitHub change comparison</strong></li></ul><p>After downloading the source tarball, signature, checksum, and KEYS file, here are instruc [...]
verify signatures, checksums, and documentation.</p><h3 id=verifying-signatures>Verifying Signatures</h3><p>First, import the keys.</p><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-bash data-lang=bash><span style=display:flex><span>curl https://dist.apache.org/repos/dist/dev/iceberg/KEYS -o KEYS
</span></span><span style=display:flex><span>gpg --import KEYS
-</span></span></code></pre></div><p>Next, verify the <code>.asc</code> file.</p><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-bash data-lang=bash><span style=display:flex><span>gpg --verify apache-iceberg-1.3.1.tar.gz.asc
-</span></span></code></pre></div><h3 id=verifying-checksums>Verifying Checksums</h3><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-bash data-lang=bash><span style=display:flex><span>shasum -a <span style=color:#ae81ff>512</span> --check apache-iceberg-1.3.1.tar.gz.sha512
-</span></span></code></pre></div><h3 id=verifying-license-documentation>Verifying License Documentation</h3><p>Untar the archive and change into the source directory.</p><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-bash data-lang=bash><span style=display:flex><span>tar xzf apache-iceberg-1.3.1.tar.gz
-</span></span><span style=display:flex><span>cd apache-iceberg-1.3.1
+</span></span></code></pre></div><p>Next, verify the <code>.asc</code> file.</p><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-bash data-lang=bash><span style=display:flex><span>gpg --verify apache-iceberg-1.4.0.tar.gz.asc
+</span></span></code></pre></div><h3 id=verifying-checksums>Verifying Checksums</h3><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-bash data-lang=bash><span style=display:flex><span>shasum -a <span style=color:#ae81ff>512</span> --check apache-iceberg-1.4.0.tar.gz.sha512
+</span></span></code></pre></div><h3 id=verifying-license-documentation>Verifying License Documentation</h3><p>Untar the archive and change into the source directory.</p><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-bash data-lang=bash><span style=display:flex><span>tar xzf apache-iceberg-1.4.0.tar.gz
+</span></span><span style=display:flex><span>cd apache-iceberg-1.4.0
</span></span></code></pre></div><p>Run RAT checks to validate license headers.</p><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-bash data-lang=bash><span style=display:flex><span>dev/check-license
</span></span></code></pre></div><h3 id=verifying-build-and-test>Verifying Build and Test</h3><p>To verify that the release candidate builds properly, run the following command.</p><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-bash data-lang=bash><span style=display:flex><span>./gradlew build
</span></span></code></pre></div><h2 id=testing-release-binaries>Testing release binaries</h2><p>Release announcements will also include a maven repository location. You can use this
@@ -182,21 +182,21 @@ location to test downstream dependencies by adding it to your maven or gradle bu
Replace <code>${MAVEN_URL}</code> with the URL provided in the release announcement</p><h3 id=verifying-with-spark>Verifying with Spark</h3><p>To verify using spark, start a <code>spark-shell</code> with a command like the following command (use the appropriate
spark-runtime jar for the Spark installation):</p><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-bash data-lang=bash><span style=display:flex><span>spark-shell <span style=color:#ae81ff>\
</span></span></span><span style=display:flex><span><span style=color:#ae81ff></span> --conf spark.jars.repositories<span style=color:#f92672>=</span><span style=color:#e6db74>${</span>MAVEN_URL<span style=color:#e6db74>}</span> <span style=color:#ae81ff>\
-</span></span></span><span style=display:flex><span><span style=color:#ae81ff></span> --packages org.apache.iceberg:iceberg-spark3-runtime:1.3.1 <span style=color:#ae81ff>\
+</span></span></span><span style=display:flex><span><span style=color:#ae81ff></span> --packages org.apache.iceberg:iceberg-spark3-runtime:1.4.0 <span style=color:#ae81ff>\
</span></span></span><span style=display:flex><span><span style=color:#ae81ff></span> --conf spark.sql.extensions<span style=color:#f92672>=</span>org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions <span style=color:#ae81ff>\
</span></span></span><span style=display:flex><span><span style=color:#ae81ff></span> --conf spark.sql.catalog.local<span style=color:#f92672>=</span>org.apache.iceberg.spark.SparkCatalog <span style=color:#ae81ff>\
</span></span></span><span style=display:flex><span><span style=color:#ae81ff></span> --conf spark.sql.catalog.local.type<span style=color:#f92672>=</span>hadoop <span style=color:#ae81ff>\
</span></span></span><span style=display:flex><span><span style=color:#ae81ff></span> --conf spark.sql.catalog.local.warehouse<span style=color:#f92672>=</span><span style=color:#e6db74>${</span>LOCAL_WAREHOUSE_PATH<span style=color:#e6db74>}</span> <span style=color:#ae81ff>\
</span></span></span><span style=display:flex><span><span style=color:#ae81ff></span> --conf spark.sql.catalog.local.default-namespace<span style=color:#f92672>=</span>default <span style=color:#ae81ff>\
</span></span></span><span style=display:flex><span><span style=color:#ae81ff></span> --conf spark.sql.defaultCatalog<span style=color:#f92672>=</span>local
-</span></span></code></pre></div><h3 id=verifying-with-flink>Verifying with Flink</h3><p>To verify using Flink, start a Flink SQL Client with the following command:</p><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-bash data-lang=bash><span style=display:flex><span>wget <span style=color:#e6db74>${</span>MAVEN_URL<span style=color:#e6db74>}</span>/iceberg-flink-runtime/1.3.1/iceberg-flink-run [...]
+</span></span></code></pre></div><h3 id=verifying-with-flink>Verifying with Flink</h3><p>To verify using Flink, start a Flink SQL Client with the following command:</p><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-bash data-lang=bash><span style=display:flex><span>wget <span style=color:#e6db74>${</span>MAVEN_URL<span style=color:#e6db74>}</span>/iceberg-flink-runtime/1.4.0/iceberg-flink-run [...]
</span></span><span style=display:flex><span>
</span></span><span style=display:flex><span>sql-client.sh embedded <span style=color:#ae81ff>\
-</span></span></span><span style=display:flex><span><span style=color:#ae81ff></span> -j iceberg-flink-runtime-1.3.1.jar <span style=color:#ae81ff>\
+</span></span></span><span style=display:flex><span><span style=color:#ae81ff></span> -j iceberg-flink-runtime-1.4.0.jar <span style=color:#ae81ff>\
</span></span></span><span style=display:flex><span><span style=color:#ae81ff></span> -j <span style=color:#e6db74>${</span>FLINK_CONNECTOR_PACKAGE<span style=color:#e6db74>}</span>-<span style=color:#e6db74>${</span>HIVE_VERSION<span style=color:#e6db74>}</span>_<span style=color:#e6db74>${</span>SCALA_VERSION<span style=color:#e6db74>}</span>-<span style=color:#e6db74>${</span>FLINK_VERSION<span style=color:#e6db74>}</span>.jar <span style=color:#ae81ff>\
</span></span></span><span style=display:flex><span><span style=color:#ae81ff></span> shell
</span></span></code></pre></div><h2 id=voting>Voting</h2><p>Votes are cast by replying to the release candidate announcement email on the dev mailing list
-with either <code>+1</code>, <code>0</code>, or <code>-1</code>.</p><blockquote><p>[ ] +1 Release this as Apache Iceberg 1.3.1
+with either <code>+1</code>, <code>0</code>, or <code>-1</code>.</p><blockquote><p>[ ] +1 Release this as Apache Iceberg 1.4.0
[ ] +0
[ ] -1 Do not release this because…</p></blockquote><p>In addition to your vote, it’s customary to specify if your vote is binding or non-binding. Only members
of the Project Management Committee have formally binding votes. If you’re unsure, you can specify that your
diff --git a/index.html b/index.html
index 9e5a72c5..48bc40b2 100644
--- a/index.html
+++ b/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.1</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.4.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
<span class=network-name>Community</span></a></li><li><a href=https://github.com/apache/iceberg class="btn btn-default btn-lg"><i class="fa fa-github fa-fw"></i>
<span class=network-name>github</span></a></li><li><a href=https://join.slack.com/t/apache-iceberg/shared_invite/zt-1jyaasx2a-TxE4z_ubxDkTFS7UFDHnjw class="btn btn-default btn-lg"><i class="fa fa-slack fa-fw"></i>
<span class=network-name>slack</span></a></li></ul></div></div></div></div></div></section><section id=about class="container content-section text-center"><div class=row><div class="col-lg-8 col-lg-offset-2"><br><br><h2>What is Iceberg?</h2><br>Iceberg is a high-performance format for huge analytic tables. Iceberg brings the reliability and simplicity of SQL tables to big data, while making it possible for engines like Spark, Trino, Flink, Presto, Hive and Impala to safely work with the [...]
diff --git a/index.xml b/index.xml
index 190e59f7..49f9c9bd 100644
--- a/index.xml
+++ b/index.xml
@@ -1,7 +1,7 @@
<?xml version="1.0" encoding="utf-8" standalone="yes"?><rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom"><channel><title>Apache Iceberg</title><link>https://iceberg.apache.org/</link><description>Recent content on Apache Iceberg</description><generator>Hugo -- gohugo.io</generator><language>en-us</language><atom:link href="https://iceberg.apache.org/index.xml" rel="self" type="application/rss+xml"/><item><title>Expressive SQL</title><link>https://iceberg.apache.org/services/exp [...]
Docker Images Creating a Table Writing Data to a Table Reading Data from a Table Next Steps Docker Images The fastest way to get started is to use Apache Hive images which provides a SQL-like interface to create and query Iceberg tables from your laptop.</description></item><item><title>Spark and Iceberg Quickstart</title><link>https://iceberg.apache.org/spark-quickstart/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><guid>https://iceberg.apache.org/spark-quickstart/</guid><des [...]
-Docker-Compose Creating a table Writing Data to a Table Reading Data from a Table Adding A Catalog Next Steps Docker-Compose The fastest way to get started is to use a docker-compose file that uses the tabulario/spark-iceberg image which contains a local Spark cluster with a configured Iceberg catalog.</description></item><item><title>Releases</title><link>https://iceberg.apache.org/releases/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><guid>https://iceberg.apache.org/release [...]
-1.3.1 source tar.gz &ndash; signature &ndash; sha512 1.3.1 Spark 3.4_2.12 runtime Jar &ndash; 3.4_2.13 1.3.1 Spark 3.3_2.12 runtime Jar &ndash; 3.3_2.13 1.3.1 Spark 3.2_2.12 runtime Jar &ndash; 3.2_2.13 1.3.1 Spark 3.1 runtime Jar 1.3.1 Flink 1.17 runtime Jar 1.3.1 Flink 1.16 runtime Jar 1.3.1 Flink 1.15 runtime Jar 1.3.1 Hive runtime Jar To use Iceberg in Spark or Flink, download the runtime JAR for your engine version and add it to the jars folder of your installati [...]
+Docker-Compose Creating a table Writing Data to a Table Reading Data from a Table Adding A Catalog Next Steps Docker-Compose The fastest way to get started is to use a docker-compose file that uses the tabulario/spark-iceberg image which contains a local Spark cluster with a configured Iceberg catalog.</description></item><item><title>Releases</title><link>https://iceberg.apache.org/releases/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><guid>https://iceberg.apache.org/release [...]
+1.4.0 source tar.gz &ndash; signature &ndash; sha512 1.4.0 Spark 3.5_2.12 runtime Jar &ndash; 3.5_2.13 1.4.0 Spark 3.4_2.12 runtime Jar &ndash; 3.4_2.13 1.4.0 Spark 3.3_2.12 runtime Jar &ndash; 3.3_2.13 1.4.0 Spark 3.2_2.12 runtime Jar &ndash; 3.2_2.13 1.4.0 Flink 1.17 runtime Jar 1.4.0 Flink 1.16 runtime Jar 1.4.0 Flink 1.15 runtime Jar 1.4.0 Hive runtime Jar 1.4.0 aws-bundle Jar 1.4.0 gcp-bundle Jar 1.4.0 azure-bundle Jar To use Iceberg in Spark or Flink, downlo [...]
Running Benchmarks on GitHub It is possible to run one or more Benchmarks via the JMH Benchmarks GH action on your own fork of the Iceberg repo.</description></item><item><title>Blogs</title><link>https://iceberg.apache.org/blogs/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><guid>https://iceberg.apache.org/blogs/</guid><description>Iceberg Blogs Here is a list of company blogs that talk about Iceberg. The blogs are ordered from most recent to oldest.
From Hive Tables to Iceberg Tables: Hassle-Free Date: July 14th, 2023, Company: Cloudera
Authors: Srinivas Rishindra Pothireddi
diff --git a/landingpagesearch.json b/landingpagesearch.json
index 27649031..d6a415d9 100644
--- a/landingpagesearch.json
+++ b/landingpagesearch.json
@@ -1 +1 @@
-[{"categories":null,"content":" Hive and Iceberg Quickstart This guide will get you up and running with an Iceberg and Hive environment, including sample code to highlight some powerful features. You can learn more about Iceberg’s Hive runtime by checking out the Hive section.\nDocker Images Creating a Table Writing Data to a Table Reading Data from a Table Next Steps Docker Images The fastest way to get started is to use Apache Hive images which provides a SQL-like interface to create a [...]
\ No newline at end of file
+[{"categories":null,"content":" Hive and Iceberg Quickstart This guide will get you up and running with an Iceberg and Hive environment, including sample code to highlight some powerful features. You can learn more about Iceberg’s Hive runtime by checking out the Hive section.\nDocker Images Creating a Table Writing Data to a Table Reading Data from a Table Next Steps Docker Images The fastest way to get started is to use Apache Hive images which provides a SQL-like interface to create a [...]
\ No newline at end of file
diff --git a/multi-engine-support/index.html b/multi-engine-support/index.html
index 109c55b0..8d371a8f 100644
--- a/multi-engine-support/index.html
+++ b/multi-engine-support/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.1</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.4.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
The community continuously improves Iceberg core library components to enable integrations with different compute engines that power analytics, business intelligence, machine learning, etc.
Connectors for Spark, Flink and Hive are maintained in the main Iceberg repository.</p><h2 id=multi-version-support>Multi-Version Support</h2><p>Processing engine connectors maintained in the iceberg repository are built for multiple versions.</p><p>For Spark and Flink, each new version that introduces backwards incompatible upgrade has its dedicated integration codebase and release artifacts.
For example, the code for Iceberg Spark 3.1 integration is under <code>/spark/v3.1</code> and the code for Iceberg Spark 3.2 integration is under <code>/spark/v3.2</code>.
@@ -13,8 +13,8 @@ New features in Iceberg could be developed against the latest features of an eng
When using Iceberg with these engines, the runtime jar is the only addition to the classpath needed in addition to vendor dependencies.
For example, to use Iceberg with Spark 3.2 and AWS integrations, <code>iceberg-spark-runtime-3.2_2.12</code> and AWS SDK dependencies are needed for the Spark installation.</p><p>Spark and Flink provide different runtime jars for each supported engine version.
Hive 2 and Hive 3 currently share the same runtime jar.
-The runtime jar names and latest version download links are listed in <a href=./multi-engine-support/#current-engine-version-lifecycle-status>the tables below</a>.</p><h3 id=engine-version-lifecycle>Engine Version Lifecycle</h3><p>Each engine version undergoes the following lifecycle stages:</p><ol><li><strong>Beta</strong>: a new engine version is supported, but still in the experimental stage. Maybe the engine version itself is still in preview (e.g. Spark <code>3.0.0-preview</code>), [...]
-Users should continuously upgrade their Flink version to stay up-to-date.</p><table><thead><tr><th>Version</th><th>Lifecycle Stage</th><th>Initial Iceberg Support</th><th>Latest Iceberg Support</th><th>Latest Runtime Jar</th></tr></thead><tbody><tr><td>1.11</td><td>End of Life</td><td>0.9.0</td><td>0.12.1</td><td><a href="https://search.maven.org/remotecontent?filepath=org/apache/iceberg/iceberg-flink-runtime/0.12.1/iceberg-flink-runtime-0.12.1.jar">iceberg-flink-runtime</a></td></tr><tr [...]
+The runtime jar names and latest version download links are listed in <a href=./multi-engine-support/#current-engine-version-lifecycle-status>the tables below</a>.</p><h3 id=engine-version-lifecycle>Engine Version Lifecycle</h3><p>Each engine version undergoes the following lifecycle stages:</p><ol><li><strong>Beta</strong>: a new engine version is supported, but still in the experimental stage. Maybe the engine version itself is still in preview (e.g. Spark <code>3.0.0-preview</code>), [...]
+Users should continuously upgrade their Flink version to stay up-to-date.</p><table><thead><tr><th>Version</th><th>Lifecycle Stage</th><th>Initial Iceberg Support</th><th>Latest Iceberg Support</th><th>Latest Runtime Jar</th></tr></thead><tbody><tr><td>1.11</td><td>End of Life</td><td>0.9.0</td><td>0.12.1</td><td><a href="https://search.maven.org/remotecontent?filepath=org/apache/iceberg/iceberg-flink-runtime/0.12.1/iceberg-flink-runtime-0.12.1.jar">iceberg-flink-runtime</a></td></tr><tr [...]
This allows the Iceberg support to evolve with the engine.
Projects such as <a href=https://trino.io/docs/current/connector/iceberg.html>Trino</a> and <a href=https://prestodb.io/docs/current/connector/iceberg.html>Presto</a> are good examples of such support strategy.</p><p>In this approach, an Iceberg version upgrade is needed for an engine to consume new Iceberg features.
To facilitate engine development against unreleased Iceberg features, a daily snapshot is published in the <a href=https://repository.apache.org/content/repositories/snapshots/org/apache/iceberg/>Apache snapshot repository</a>.</p><p>If bringing an engine directly to the Iceberg main repository is needed, please raise a discussion thread in the <a href=../community>Iceberg community</a>.</p></div><div id=toc class=markdown-body><div id=full><nav id=TableOfContents><ul><li><a href=#multi- [...]
diff --git a/puffin-spec/index.html b/puffin-spec/index.html
index d4f5d3c6..60f332e3 100644
--- a/puffin-spec/index.html
+++ b/puffin-spec/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.1</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.4.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
information such as indexes and statistics about data managed in an
Iceberg table that cannot be stored directly within the Iceberg manifest. A
Puffin file contains arbitrary pieces of information (here called “blobs”),
diff --git a/releases/index.html b/releases/index.html
index dae14847..42cafda1 100644
--- a/releases/index.html
+++ b/releases/index.html
@@ -3,20 +3,21 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.1</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
- compile 'org.apache.iceberg:iceberg-core:1.3.1'
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.4.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
+ compile 'org.apache.iceberg:iceberg-core:1.4.0'
}
</code></pre><p>You may also want to include <code>iceberg-parquet</code> for Parquet file support.</p><h3 id=maven>Maven</h3><p>To add a dependency on Iceberg in Maven, add the following to your <code>pom.xml</code>:</p><pre tabindex=0><code><dependencies>
...
<dependency>
<groupId>org.apache.iceberg</groupId>
<artifactId>iceberg-core</artifactId>
- <version>1.3.1</version>
+ <version>1.4.0</version>
</dependency>
...
</dependencies>
-</code></pre><h2 id=131-release>1.3.1 release</h2><p>Apache Iceberg 1.3.1 was released on July 25, 2023.
-The 1.3.1 release addresses various issues identified in the 1.3.0 release.</p><ul><li>Core<ul><li>Table Metadata parser now accepts null for fields: current-snapshot-id, properties, and snapshots (<a href=https://github.com/apache/iceberg/pull/8064>#8064</a>)</li></ul></li><li>Hive<ul><li>Fix HiveCatalog deleting metadata on failures in checking lock status (<a href=https://github.com/apache/iceberg/pull/7931>#7931</a>)</li></ul></li><li>Spark<ul><li>Fix RewritePositionDeleteFiles failu [...]
+</code></pre><h3 id=140-release>1.4.0 release</h3><p>Apache Iceberg 1.4.0 was released on October 4, 2023.
+The 1.4.0 release adds a variety of new features and bug fixes.</p><ul><li>API<ul><li>Implement bound expression sanitization (<a href=https://github.com/apache/iceberg/pull/8149>#8149</a>)</li><li>Remove overflow checks in <code>DefaultCounter</code> causing performance issues (<a href=https://github.com/apache/iceberg/pull/8297>#8297</a>)</li><li>Support incremental scanning with branch (<a href=https://github.com/apache/iceberg/pull/5984>#5984</a>)</li><li>Add a validation API to <cod [...]
+The 1.3.1 release addresses various issues identified in the 1.3.0 release.</p><ul><li>Core<ul><li>Table Metadata parser now accepts null for fields: current-snapshot-id, properties, and snapshots (<a href=https://github.com/apache/iceberg/pull/8064>#8064</a>)</li></ul></li><li>Hive<ul><li>Fix HiveCatalog deleting metadata on failures in checking lock status (<a href=https://github.com/apache/iceberg/pull/7931>#7931</a>)</li></ul></li><li>Spark<ul><li>Fix RewritePositionDeleteFiles failu [...]
The 1.3.0 release adds a variety of new features and bug fixes.</p><ul><li>Core<ul><li>Expose file and data sequence numbers in ContentFile (<a href=https://github.com/apache/iceberg/pull/7555>#7555</a>)</li><li>Improve bit density in object storage layout (<a href=https://github.com/apache/iceberg/pull/7128>#7128</a>)</li><li>Store split offsets for delete files (<a href=https://github.com/apache/iceberg/pull/7011>#7011</a>)</li><li>Readable metrics in entries metadata table (<a href=ht [...]
The 1.2.1 release is a patch release to address various issues identified in the prior release.
Here is an overview:</p><ul><li>CORE<ul><li>REST: fix previous locations for refs-only load <a href=https://github.com/apache/iceberg/pull/7284>#7284</a></li><li>Parse snapshot-id as long in remove-statistics update <a href=https://github.com/apache/iceberg/pull/7235>#7235</a></li></ul></li><li>Spark<ul><li>Broadcast table instead of file IO in rewrite manifests <a href=https://github.com/apache/iceberg/pull/7263>#7263</a></li><li>Revert “Spark: Add “Iceberg” prefix to [...]
@@ -27,7 +28,7 @@ and adds a variety of new features.
Here is an overview:</p><ul><li>Core<ul><li>Puffin statistics have been <a href=https://github.com/apache/iceberg/pull/4945>added to the Table API</a></li><li>Support for <a href=https://github.com/apache/iceberg/pull/5268>Table scan reporting</a>, which enables collection of statistics of the table scans.</li><li><a href=https://github.com/apache/iceberg/pull/6002>Add file sequence number to ManifestEntry</a></li><li><a href=https://github.com/apache/iceberg/pull/5037>Support register t [...]
Prior to the fix, filters would not be extracted and targeted merge conditions were not pushed down leading to degraded performance
for these targeted merge operations.</li><li><a href=https://github.com/apache/iceberg/pull/4024>#4024</a> fixes table creation in the root namespace of a Hadoop Catalog.</li></ul></li><li><p><strong>Flink</strong></p><ul><li><a href=https://github.com/apache/iceberg/pull/3986>#3986</a> fixes manifest location collisions when there are multiple committers
-in the same Flink job.</li></ul></li></ul><h3 id=0130>0.13.0</h3><p>Apache Iceberg 0.13.0 was released on February 4th, 2022.</p><ul><li>Git tag: <a href=https://github.com/apache/iceberg/releases/tag/apache-iceberg-0.13.0>0.13.0</a></li><li><a href=https://www.apache.org/dyn/closer.cgi/iceberg/apache-iceberg-0.13.0/apache-iceberg-0.13.0.tar.gz>0.13.0 source tar.gz</a> – <a href=https://downloads.apache.org/iceberg/apache-iceberg-0.13.0/apache-iceberg-0.13.0.tar.gz.asc>signature</a [...]
+in the same Flink job.</li></ul></li></ul><h3 id=0130>0.13.0</h3><p>Apache Iceberg 0.13.0 was released on February 4th, 2022.</p><ul><li>Git tag: <a href=https://github.com/apache/iceberg/releases/tag/apache-iceberg-0.13.0>0.13.0</a></li><li><a href=https://www.apache.org/dyn/closer.cgi/iceberg/apache-iceberg-0.13.0/apache-iceberg-0.13.0.tar.gz>0.13.0 source tar.gz</a> – <a href=https://downloads.apache.org/iceberg/apache-iceberg-0.13.0/apache-iceberg-0.13.0.tar.gz.asc>signature</a [...]
<script src=https://iceberg.apache.org//js/jquery.easing.min.js></script>
<script type=text/javascript src=https://iceberg.apache.org//js/search.js></script>
<script src=https://iceberg.apache.org//js/bootstrap.min.js></script>
diff --git a/roadmap/index.html b/roadmap/index.html
index 95dc2e05..8894aefa 100644
--- a/roadmap/index.html
+++ b/roadmap/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.1</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.4.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
This is based on the latest <a href=https://lists.apache.org/thread.html/r84e80216c259c81f824c6971504c321cd8c785774c489d52d4fc123f%40%3Cdev.iceberg.apache.org%3E>community priority discussion</a>.
Each high-level item links to a Github project board that tracks the current status.
Related design docs will be linked on the planning boards.</p><h1 id=priority-1>Priority 1</h1><ul><li>API: <a href=https://github.com/apache/iceberg/projects/3>Iceberg 1.0.0</a> [medium]</li><li>Python: <a href=https://github.com/apache/iceberg/projects/7>Pythonic refactor</a> [medium]</li><li>Spec: <a href=https://github.com/apache/iceberg/projects/16>Z-ordering / Space-filling curves</a> [medium]</li><li>Spec: <a href=https://github.com/apache/iceberg/projects/4>Snapshot tagging and b [...]
diff --git a/security/index.html b/security/index.html
index 8f19c2b4..c67b4608 100644
--- a/security/index.html
+++ b/security/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.1</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.4.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
Security Team</a> for reporting vulnerabilities.
Note that vulnerabilities should not be publicly disclosed until the project has
responded.</p><p>To report a possible security vulnerability, please email <a href=mailto:security@iceberg.apache.org><a href=mailto:security@iceberg.apache.org>security@iceberg.apache.org</a></a>.</p><h1 id=verifying-signed-releases>Verifying Signed Releases</h1><p>Please refer to the instructions on the <a href=https://www.apache.org/info/verification.html>Release Verification</a> page.</p></div><div id=toc class=markdown-body><div id=full><nav id=TableOfContents></nav></div></div></di [...]
diff --git a/services/data-compaction/index.html b/services/data-compaction/index.html
index 3c64b056..e074d332 100644
--- a/services/data-compaction/index.html
+++ b/services/data-compaction/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.1</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.4.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
<script src=https://iceberg.apache.org//js/jquery.easing.min.js></script>
<script type=text/javascript src=https://iceberg.apache.org//js/search.js></script>
<script src=https://iceberg.apache.org//js/bootstrap.min.js></script>
diff --git a/services/expressive-sql/index.html b/services/expressive-sql/index.html
index 95a9116d..2b9337e9 100644
--- a/services/expressive-sql/index.html
+++ b/services/expressive-sql/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.1</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.4.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
<span data-ty=input data-ty-cursor=▋ data-ty-prompt>USING (SELECT * FROM staging.nyc.taxis) st</span>
<span data-ty=input data-ty-cursor=▋ data-ty-prompt>ON pt.id = st.id</span>
<span data-ty=input data-ty-cursor=▋ data-ty-prompt>WHEN NOT MATCHED THEN INSERT *;</span>
diff --git a/services/hidden-partitioning/index.html b/services/hidden-partitioning/index.html
index e7939af1..76ab20f9 100644
--- a/services/hidden-partitioning/index.html
+++ b/services/hidden-partitioning/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.1</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.4.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
<script src=https://iceberg.apache.org//js/jquery.easing.min.js></script>
<script type=text/javascript src=https://iceberg.apache.org//js/search.js></script>
<script src=https://iceberg.apache.org//js/bootstrap.min.js></script>
diff --git a/services/schema-evolution/index.html b/services/schema-evolution/index.html
index 0c0ca1a1..7165c811 100644
--- a/services/schema-evolution/index.html
+++ b/services/schema-evolution/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.1</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.4.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
<span data-ty=input data-ty-cursor=▋ data-ty-prompt>ALTER COLUMN trip_distance</span>
<span data-ty=input data-ty-cursor=▋ data-ty-prompt data-ty-delay=2500>TYPE double;</span>
<span data-ty>Done!</span>
diff --git a/services/time-travel/index.html b/services/time-travel/index.html
index c6d79798..23867f52 100644
--- a/services/time-travel/index.html
+++ b/services/time-travel/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.1</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.4.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
<span data-ty>2,853,020</span>
<span data-ty=input data-ty-cursor=▋ data-ty-prompt="sql>">SELECT count(*) FROM nyc.taxis FOR VERSION AS OF 2188465307835585443</span>
<span data-ty>2,798,371</span>
diff --git a/spark-quickstart/index.html b/spark-quickstart/index.html
index 78e3f170..736eae22 100644
--- a/spark-quickstart/index.html
+++ b/spark-quickstart/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.1</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.4.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
highlight some powerful features. You can learn more about Iceberg’s Spark runtime by checking out the <a href=../docs/latest/spark-ddl/>Spark</a> section.</p><ul><li><a href=#docker-compose>Docker-Compose</a></li><li><a href=#creating-a-table>Creating a table</a></li><li><a href=#writing-data-to-a-table>Writing Data to a Table</a></li><li><a href=#reading-data-from-a-table>Reading Data from a Table</a></li><li><a href=#adding-a-catalog>Adding A Catalog</a></li><li><a href=#next-st [...]
which contains a local Spark cluster with a configured Iceberg catalog. To use this, you’ll need to install the <a href=https://docs.docker.com/get-docker/>Docker CLI</a> as well as the <a href=https://github.com/docker/compose-cli/blob/main/INSTALL.md>Docker Compose CLI</a>.</p><p>Once you have those, save the yaml below into a file named <code>docker-compose.yml</code>:</p><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-siz [...]
</span></span><span style=display:flex><span>
@@ -172,7 +172,7 @@ the <a href=../docs/latest/spark-configuration/#catalogs>Catalog</a> page in the
<label for=cli>CLI</label>
<input id=spark-defaults type=radio name=AddingACatalog onclick='selectExampleLanguage("spark-init","spark-defaults")'>
<label for=spark-defaults>spark-defaults.conf</label>
-<codeblock class=cli><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-sh data-lang=sh><span style=display:flex><span>spark-sql --packages org.apache.iceberg:iceberg-spark-runtime-3.2_2.12:1.3.1<span style=color:#ae81ff>\
+<codeblock class=cli><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-sh data-lang=sh><span style=display:flex><span>spark-sql --packages org.apache.iceberg:iceberg-spark-runtime-3.2_2.12:1.4.0<span style=color:#ae81ff>\
</span></span></span><span style=display:flex><span><span style=color:#ae81ff></span> --conf spark.sql.extensions<span style=color:#f92672>=</span>org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions <span style=color:#ae81ff>\
</span></span></span><span style=display:flex><span><span style=color:#ae81ff></span> --conf spark.sql.catalog.spark_catalog<span style=color:#f92672>=</span>org.apache.iceberg.spark.SparkSessionCatalog <span style=color:#ae81ff>\
</span></span></span><span style=display:flex><span><span style=color:#ae81ff></span> --conf spark.sql.catalog.spark_catalog.type<span style=color:#f92672>=</span>hive <span style=color:#ae81ff>\
@@ -180,7 +180,7 @@ the <a href=../docs/latest/spark-configuration/#catalogs>Catalog</a> page in the
</span></span></span><span style=display:flex><span><span style=color:#ae81ff></span> --conf spark.sql.catalog.local.type<span style=color:#f92672>=</span>hadoop <span style=color:#ae81ff>\
</span></span></span><span style=display:flex><span><span style=color:#ae81ff></span> --conf spark.sql.catalog.local.warehouse<span style=color:#f92672>=</span>$PWD/warehouse <span style=color:#ae81ff>\
</span></span></span><span style=display:flex><span><span style=color:#ae81ff></span> --conf spark.sql.defaultCatalog<span style=color:#f92672>=</span>local
-</span></span></code></pre></div></codeblock><codeblock class=spark-defaults><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-sh data-lang=sh><span style=display:flex><span>spark.jars.packages org.apache.iceberg:iceberg-spark-runtime-3.2_2.12:1.3.1
+</span></span></code></pre></div></codeblock><codeblock class=spark-defaults><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-sh data-lang=sh><span style=display:flex><span>spark.jars.packages org.apache.iceberg:iceberg-spark-runtime-3.2_2.12:1.4.0
</span></span><span style=display:flex><span>spark.sql.extensions org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions
</span></span><span style=display:flex><span>spark.sql.catalog.spark_catalog org.apache.iceberg.spark.SparkSessionCatalog
</span></span><span style=display:flex><span>spark.sql.catalog.spark_catalog.type hive
@@ -194,9 +194,9 @@ the <a href=../docs/latest/spark-configuration/#catalogs>Catalog</a> page in the
<label for=spark-shell>Spark-Shell</label>
<input id=pyspark type=radio name=AddIcebergToSpark onclick='selectExampleLanguage("spark-queries","pyspark")'>
<label for=pyspark>PySpark</label>
-<codeblock class=spark-sql><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-sh data-lang=sh><span style=display:flex><span>spark-sql --packages org.apache.iceberg:iceberg-spark-runtime-3.2_2.12:1.3.1
-</span></span></code></pre></div></codeblock><codeblock class=spark-shell><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-sh data-lang=sh><span style=display:flex><span>spark-shell --packages org.apache.iceberg:iceberg-spark-runtime-3.2_2.12:1.3.1
-</span></span></code></pre></div></codeblock><codeblock class=pyspark><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-sh data-lang=sh><span style=display:flex><span>pyspark --packages org.apache.iceberg:iceberg-spark-runtime-3.2_2.12:1.3.1
+<codeblock class=spark-sql><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-sh data-lang=sh><span style=display:flex><span>spark-sql --packages org.apache.iceberg:iceberg-spark-runtime-3.2_2.12:1.4.0
+</span></span></code></pre></div></codeblock><codeblock class=spark-shell><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-sh data-lang=sh><span style=display:flex><span>spark-shell --packages org.apache.iceberg:iceberg-spark-runtime-3.2_2.12:1.4.0
+</span></span></code></pre></div></codeblock><codeblock class=pyspark><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-sh data-lang=sh><span style=display:flex><span>pyspark --packages org.apache.iceberg:iceberg-spark-runtime-3.2_2.12:1.4.0
</span></span></code></pre></div></codeblock></div><div class=info>If you want to include Iceberg in your Spark installation, add the Iceberg Spark runtime to Spark’s <code>jars</code> folder.
You can download the runtime by visiting to the <a href=https://iceberg.apache.org/releases/>Releases</a> page.</div><h4 id=learn-more>Learn More</h4><p>Now that you’re up an running with Iceberg and Spark, check out the <a href=../docs/latest/spark-ddl/>Iceberg-Spark docs</a> to learn more!</p></div></div></div></section></body><script src=https://iceberg.apache.org//js/jquery-1.11.0.js></script>
<script src=https://iceberg.apache.org//js/jquery.easing.min.js></script>
diff --git a/spec/index.html b/spec/index.html
index 8fd1ff12..db5db670 100644
--- a/spec/index.html
+++ b/spec/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.1</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.4.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
</code></pre><p>Notes:</p><ol><li>Changing the number of buckets as a table grows is possible by evolving the partition spec.</li></ol><p>For hash function details by type, see Appendix B.</p><h4 id=truncate-transform-details>Truncate Transform Details</h4><table><thead><tr><th><strong>Type</strong></th><th><strong>Config</strong></th><th><strong>Truncate specification</strong></th><th><strong>Examples</strong></th></tr></thead><tbody><tr><td><strong><code>int</code></strong></td><td><co [...]
The <code>sequence_number</code> field represents the data sequence number and must never change after a file is added to the dataset. The data sequence number represents a relative age of the file content and should be used for planning which delete files apply to a data file.
The <code>file_sequence_number</code> field represents the sequence number of the snapshot that added the file and must also remain unchanged upon assigning at commit. The file sequence number can’t be used for pruning delete files as the data within the file may have an older data sequence number.
@@ -13,7 +13,7 @@ Tags are labels for individual snapshots. Branches are mutable named references
The snapshot expiration procedure removes snapshots from table metadata and applies the table’s retention policy.
Retention policy can be configured both globally and on snapshot reference through properties <code>min-snapshots-to-keep</code>, <code>max-snapshot-age-ms</code> and <code>max-ref-age-ms</code>.</p><p>When expiring snapshots, retention policies in table and snapshot references are evaluated in the following way:</p><ol><li>Start with an empty set of snapshots to retain</li><li>Remove any refs (other than main) where the referenced snapshot is older than <code>max-ref-age-ms</code></li>< [...]
ignore statistics information. Statistics support is not required to read the table correctly. A table can contain
-many statistics files associated with different table snapshots.</p><p>Statistics files metadata within <code>statistics</code> table metadata field is a struct with the following fields:</p><table><thead><tr><th>v1</th><th>v2</th><th>Field name</th><th>Type</th><th>Description</th></tr></thead><tbody><tr><td><em>required</em></td><td><em>required</em></td><td><strong><code>snapshot-id</code></strong></td><td><code>string</code></td><td>ID of the Iceberg table’s snapshot the statis [...]
+many statistics files associated with different table snapshots.</p><p>Statistics files metadata within <code>statistics</code> table metadata field is a struct with the following fields:</p><table><thead><tr><th>v1</th><th>v2</th><th>Field name</th><th>Type</th><th>Description</th></tr></thead><tbody><tr><td><em>required</em></td><td><em>required</em></td><td><strong><code>snapshot-id</code></strong></td><td><code>string</code></td><td>ID of the Iceberg table’s snapshot the statis [...]
</span></span><span style=display:flex><span>-------|-------------|---------
</span></span><span style=display:flex><span> 1 | marsupial | Koala
</span></span><span style=display:flex><span> 2 | toy | Teddy
diff --git a/talks/index.html b/talks/index.html
index 8b27b1cc..267507ba 100644
--- a/talks/index.html
+++ b/talks/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.1</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.4.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
<script src=https://iceberg.apache.org//js/jquery.easing.min.js></script>
<script type=text/javascript src=https://iceberg.apache.org//js/search.js></script>
<script src=https://iceberg.apache.org//js/bootstrap.min.js></script>
diff --git a/terms/index.html b/terms/index.html
index 459bd7c8..73ec71f7 100644
--- a/terms/index.html
+++ b/terms/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.1</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.4.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
<script src=https://iceberg.apache.org//js/jquery.easing.min.js></script>
<script type=text/javascript src=https://iceberg.apache.org//js/search.js></script>
<script src=https://iceberg.apache.org//js/bootstrap.min.js></script>
diff --git a/trademarks/index.html b/trademarks/index.html
index 1726749d..0d7008a7 100644
--- a/trademarks/index.html
+++ b/trademarks/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.1</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.4.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
<script src=https://iceberg.apache.org//js/jquery.easing.min.js></script>
<script type=text/javascript src=https://iceberg.apache.org//js/search.js></script>
<script src=https://iceberg.apache.org//js/bootstrap.min.js></script>
diff --git a/vendors/index.html b/vendors/index.html
index 8102ac23..b3af2e5a 100644
--- a/vendors/index.html
+++ b/vendors/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.1</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.4.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
<script src=https://iceberg.apache.org//js/jquery.easing.min.js></script>
<script type=text/javascript src=https://iceberg.apache.org//js/search.js></script>
<script src=https://iceberg.apache.org//js/bootstrap.min.js></script>
diff --git a/view-spec/index.html b/view-spec/index.html
index 12ad8d12..7fda8494 100644
--- a/view-spec/index.html
+++ b/view-spec/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.1</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.4.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
</span></span></code></pre></div><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-sql data-lang=sql><span style=display:flex><span><span style=color:#66d9ef>CREATE</span> <span style=color:#66d9ef>OR</span> <span style=color:#66d9ef>REPLACE</span> <span style=color:#66d9ef>VIEW</span> event_agg (
</span></span><span style=display:flex><span> event_count <span style=color:#66d9ef>COMMENT</span> <span style=color:#e6db74>'Count of events'</span>,
</span></span><span style=display:flex><span> event_date) <span style=color:#66d9ef>AS</span>