You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@iceberg.apache.org by gi...@apache.org on 2023/10/23 10:57:53 UTC
[iceberg-docs] branch asf-site updated: deploy: 1a8c288a453718ba546ed9f8de3ff80db1b9acf2
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/iceberg-docs.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 5fc34f85 deploy: 1a8c288a453718ba546ed9f8de3ff80db1b9acf2
5fc34f85 is described below
commit 5fc34f8567957af2664f725efe5b1ca5fe2c71eb
Author: Fokko <Fo...@users.noreply.github.com>
AuthorDate: Mon Oct 23 10:57:48 2023 +0000
deploy: 1a8c288a453718ba546ed9f8de3ff80db1b9acf2
---
about/about/index.html | 2 +-
benchmarks/index.html | 2 +-
blogs/index.html | 2 +-
catalog/index.html | 2 +-
common/index.xml | 4 ++--
community/index.html | 2 +-
contribute/index.html | 2 +-
gcm-stream-spec/index.html | 2 +-
getting-started/index.html | 20 +-------------------
hive-quickstart/index.html | 2 +-
how-to-release/index.html | 18 +++++++++---------
index.html | 2 +-
index.xml | 4 ++--
landingpagesearch.json | 2 +-
multi-engine-support/index.html | 6 +++---
puffin-spec/index.html | 2 +-
releases/index.html | 13 +++++++------
roadmap/index.html | 2 +-
security/index.html | 2 +-
services/data-compaction/index.html | 2 +-
services/expressive-sql/index.html | 2 +-
services/hidden-partitioning/index.html | 2 +-
services/schema-evolution/index.html | 2 +-
services/time-travel/index.html | 2 +-
spark-quickstart/index.html | 12 ++++++------
spec/index.html | 4 ++--
talks/index.html | 2 +-
terms/index.html | 2 +-
trademarks/index.html | 2 +-
vendors/index.html | 2 +-
view-spec/index.html | 2 +-
31 files changed, 55 insertions(+), 72 deletions(-)
diff --git a/about/about/index.html b/about/about/index.html
index 241d7f78..1770d6a9 100644
--- a/about/about/index.html
+++ b/about/about/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.4.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.4.1</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
<script src=https://iceberg.apache.org//js/jquery.easing.min.js></script>
<script type=text/javascript src=https://iceberg.apache.org//js/search.js></script>
<script src=https://iceberg.apache.org//js/bootstrap.min.js></script>
diff --git a/benchmarks/index.html b/benchmarks/index.html
index 54e41526..cab9e536 100644
--- a/benchmarks/index.html
+++ b/benchmarks/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.4.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.4.1</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
Also note that JMH benchmarks run within the same JVM as the system-under-test, so results might vary between runs.</p><h2 id=running-benchmarks-on-github>Running Benchmarks on GitHub</h2><p>It is possible to run one or more Benchmarks via the <strong>JMH Benchmarks</strong> GH action on your own fork of the Iceberg repo. This GH action takes the following inputs:</p><ul><li>The repository name where those benchmarks should be run against, such as <code>apache/iceberg</code> or <code>< [...]
It is likely that there is variability in results across different runs, therefore the benchmark results shouldn’t be used to form assumptions around production choices.</p><h2 id=running-benchmarks-locally>Running Benchmarks locally</h2><p>Below are the existing benchmarks shown with the actual commands on how to run them locally.</p><h3 id=icebergsourcenestedlistparquetdatawritebenchmark>IcebergSourceNestedListParquetDataWriteBenchmark</h3><p>A benchmark that evaluates the perfor [...]
column used in the filter predicate. The performance is compared to the built-in file source in Spark. To run this benchmark for either spark-2 or spark-3:</p><p><code>./gradlew :iceberg-spark:iceberg-spark[2|3]:jmh -PjmhIncludeRegex=IcebergSourceFlatParquetDataFilterBenchmark -PjmhOutputPath=benchmark/iceberg-source-flat-parquet-data-filter-benchmark-result.txt</code></p><h3 id=icebergsourcenestedparquetdatafilterbenchmark>IcebergSourceNestedParquetDataFilterBenchmark</h3><p>A benchmark [...]
diff --git a/blogs/index.html b/blogs/index.html
index af6ae2e4..4e7c626e 100644
--- a/blogs/index.html
+++ b/blogs/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.4.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.4.1</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
<script src=https://iceberg.apache.org//js/jquery.easing.min.js></script>
<script type=text/javascript src=https://iceberg.apache.org//js/search.js></script>
<script src=https://iceberg.apache.org//js/bootstrap.min.js></script>
diff --git a/catalog/index.html b/catalog/index.html
index f1801a3b..b7432bcc 100644
--- a/catalog/index.html
+++ b/catalog/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.4.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.4.1</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
it easier to make custom catalogs compatible with engines like Athena or Starburst without requiring the inclusion of a Jar into the classpath.</p></div><div id=toc class=markdown-body><div id=full><nav id=TableOfContents><ul><li><a href=#overview>Overview</a></li><li><a href=#catalog-implementations>Catalog Implementations</a></li><li><a href=#decoupling-using-the-rest-catalog>Decoupling Using the REST Catalog</a></li></ul></nav></div></div></div></div></section></body><script src=https [...]
<script src=https://iceberg.apache.org//js/jquery.easing.min.js></script>
<script type=text/javascript src=https://iceberg.apache.org//js/search.js></script>
diff --git a/common/index.xml b/common/index.xml
index 8b558b63..5cb97cfb 100644
--- a/common/index.xml
+++ b/common/index.xml
@@ -1,7 +1,7 @@
<?xml version="1.0" encoding="utf-8" standalone="yes"?><rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom"><channel><title>Commons on Apache Iceberg</title><link>https://iceberg.apache.org/common/</link><description>Recent content in Commons on Apache Iceberg</description><generator>Hugo -- gohugo.io</generator><language>en-us</language><atom:link href="https://iceberg.apache.org/common/index.xml" rel="self" type="application/rss+xml"/><item><title>Hive and Iceberg Quickstart</ti [...]
Docker Images Creating a Table Writing Data to a Table Reading Data from a Table Next Steps Docker Images The fastest way to get started is to use Apache Hive images which provides a SQL-like interface to create and query Iceberg tables from your laptop.</description></item><item><title>Spark and Iceberg Quickstart</title><link>https://iceberg.apache.org/spark-quickstart/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><guid>https://iceberg.apache.org/spark-quickstart/</guid><des [...]
-Docker-Compose Creating a table Writing Data to a Table Reading Data from a Table Adding A Catalog Next Steps Docker-Compose The fastest way to get started is to use a docker-compose file that uses the tabulario/spark-iceberg image which contains a local Spark cluster with a configured Iceberg catalog.</description></item><item><title>Releases</title><link>https://iceberg.apache.org/releases/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><guid>https://iceberg.apache.org/release [...]
-1.4.0 source tar.gz &ndash; signature &ndash; sha512 1.4.0 Spark 3.5_2.12 runtime Jar &ndash; 3.5_2.13 1.4.0 Spark 3.4_2.12 runtime Jar &ndash; 3.4_2.13 1.4.0 Spark 3.3_2.12 runtime Jar &ndash; 3.3_2.13 1.4.0 Spark 3.2_2.12 runtime Jar &ndash; 3.2_2.13 1.4.0 Flink 1.17 runtime Jar 1.4.0 Flink 1.16 runtime Jar 1.4.0 Flink 1.15 runtime Jar 1.4.0 Hive runtime Jar 1.4.0 aws-bundle Jar 1.4.0 gcp-bundle Jar 1.4.0 azure-bundle Jar To use Iceberg in Spark or Flink, downlo [...]
+Docker-Compose Creating a table Writing Data to a Table Reading Data from a Table Adding A Catalog Next Steps Docker-Compose The fastest way to get started is to use a docker-compose file that uses the tabulario/spark-iceberg image which contains a local Spark cluster with a configured Iceberg catalog.</description></item><item><title>Releases</title><link>https://iceberg.apache.org/releases/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><guid>https://iceberg.apache.org/release [...]
+1.4.1 source tar.gz &ndash; signature &ndash; sha512 1.4.1 Spark 3.5_2.12 runtime Jar &ndash; 3.5_2.13 1.4.1 Spark 3.4_2.12 runtime Jar &ndash; 3.4_2.13 1.4.1 Spark 3.3_2.12 runtime Jar &ndash; 3.3_2.13 1.4.1 Spark 3.2_2.12 runtime Jar &ndash; 3.2_2.13 1.4.1 Flink 1.17 runtime Jar 1.4.1 Flink 1.16 runtime Jar 1.4.1 Flink 1.15 runtime Jar 1.4.1 Hive runtime Jar 1.4.1 aws-bundle Jar 1.4.1 gcp-bundle Jar 1.4.1 azure-bundle Jar To use Iceberg in Spark or Flink, downlo [...]
Running Benchmarks on GitHub It is possible to run one or more Benchmarks via the JMH Benchmarks GH action on your own fork of the Iceberg repo.</description></item><item><title>Blogs</title><link>https://iceberg.apache.org/blogs/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><guid>https://iceberg.apache.org/blogs/</guid><description>Iceberg Blogs Here is a list of company blogs that talk about Iceberg. The blogs are ordered from most recent to oldest.
Apache Hive-4.x with Iceberg Branches &amp; Tags Date: October 12th, 2023, Company: Cloudera
Authors: Ayush Saxena
diff --git a/community/index.html b/community/index.html
index 2fb2e69e..5433aa22 100644
--- a/community/index.html
+++ b/community/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.4.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.4.1</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
and all who engage with the community are expected to be respectful, open, come with the best interests of the community in mind,
and abide by the Apache Foundation <a href=https://www.apache.org/foundation/policies/conduct.html>Code of Conduct</a>.</p><h3 id=participants-with-corporate-interests>Participants with Corporate Interests</h3><p>A wide range of corporate entities have interests that overlap in both features and frameworks related to Iceberg and while we
encourage engagement and contributions, the community is not a venue for marketing, solicitation, or recruitment.</p><p>Any vendor who wants to participate in the Apache Iceberg community Slack workspace should create a dedicated vendor channel
diff --git a/contribute/index.html b/contribute/index.html
index 7cbd1272..b1147996 100644
--- a/contribute/index.html
+++ b/contribute/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.4.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.4.1</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
these are hard rules and they’re meant as a collection of helpful suggestions to make contributing as seamless of an
experience as possible.</p><p>If you are thinking of contributing but first would like to discuss the change you wish to make, we welcome you to
head over to the <a href=https://iceberg.apache.org/community/>Community</a> page on the official Iceberg documentation site
diff --git a/gcm-stream-spec/index.html b/gcm-stream-spec/index.html
index f93b381a..4285e4f8 100644
--- a/gcm-stream-spec/index.html
+++ b/gcm-stream-spec/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.4.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.4.1</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
</code></pre><p>where</p><ul><li><code>Magic</code> is four bytes 0x41, 0x47, 0x53, 0x31 (“AGS1”, short for: AES GCM Stream, version 1)</li><li><code>BlockLength</code> is four bytes (little endian) integer keeping the length of the equal-size split blocks before encryption. The length is specified in bytes.</li><li><code>CipherBlockᵢ</code> is the i-th enciphered block in the file, with the structure defined below.</li></ul><h3 id=cipher-block-structure>Cipher Block structur [...]
<script src=https://iceberg.apache.org//js/jquery.easing.min.js></script>
<script type=text/javascript src=https://iceberg.apache.org//js/search.js></script>
diff --git a/getting-started/index.html b/getting-started/index.html
index b3c6e0d8..52cc0335 100644
--- a/getting-started/index.html
+++ b/getting-started/index.html
@@ -1,19 +1 @@
-<!--
- - Licensed to the Apache Software Foundation (ASF) under one or more
- - contributor license agreements. See the NOTICE file distributed with
- - this work for additional information regarding copyright ownership.
- - The ASF licenses this file to You under the Apache License, Version 2.0
- - (the "License"); you may not use this file except in compliance with
- - the License. You may obtain a copy of the License at
- -
- - http://www.apache.org/licenses/LICENSE-2.0
- -
- - Unless required by applicable law or agreed to in writing, software
- - distributed under the License is distributed on an "AS IS" BASIS,
- - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- - See the License for the specific language governing permissions and
- - limitations under the License.
- -->
-<head>
- <meta http-equiv="Refresh" content="0; url='/docs/latest/getting-started'" />
-</head>
+<!doctype html><html lang=en-us><head><title>https://iceberg.apache.org/spark-quickstart/</title><link rel=canonical href=https://iceberg.apache.org/spark-quickstart/><meta name=robots content="noindex"><meta charset=utf-8><meta http-equiv=refresh content="0; url=https://iceberg.apache.org/spark-quickstart/"></head></html>
\ No newline at end of file
diff --git a/hive-quickstart/index.html b/hive-quickstart/index.html
index 1d2a6322..7f92a6dd 100644
--- a/hive-quickstart/index.html
+++ b/hive-quickstart/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.4.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.4.1</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
highlight some powerful features. You can learn more about Iceberg’s Hive runtime by checking out the <a href=../docs/latest/hive/>Hive</a> section.</p><ul><li><a href=#docker-images>Docker Images</a></li><li><a href=#creating-a-table>Creating a Table</a></li><li><a href=#writing-data-to-a-table>Writing Data to a Table</a></li><li><a href=#reading-data-from-a-table>Reading Data from a Table</a></li><li><a href=#next-steps>Next Steps</a></li></ul><h3 id=docker-images>Docker Images</ [...]
which provides a SQL-like interface to create and query Iceberg tables from your laptop. You need to install the <a href=https://www.docker.com/products/docker-desktop/>Docker Desktop</a>.</p><p>Take a look at the Tags tab in <a href="https://hub.docker.com/r/apache/hive/tags?page=1&ordering=-last_updated">Apache Hive docker images</a> to see the available Hive versions.</p><p>Set the version variable.</p><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#2728 [...]
</span></span></code></pre></div><p>Start the container, using the option <code>--platform linux/amd64</code> for a Mac with an M-Series chip:</p><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-sh data-lang=sh><span style=display:flex><span>docker run -d --platform linux/amd64 -p 10000:10000 -p 10002:10002 --env SERVICE_NAME<span style=color:#f92672>=</span>hiveserver2 --name hive4 apache/hive [...]
diff --git a/how-to-release/index.html b/how-to-release/index.html
index fdb79876..ba6d3049 100644
--- a/how-to-release/index.html
+++ b/how-to-release/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.4.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.4.1</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
</span></span><span style=display:flex><span>cd icebergsvn
</span></span><span style=display:flex><span>echo <span style=color:#e6db74>""</span> >> KEYS <span style=color:#75715e># append a newline</span>
</span></span><span style=display:flex><span>gpg --list-sigs <YOUR KEY ID HERE> >> KEYS <span style=color:#75715e># append signatures</span>
@@ -156,10 +156,10 @@ It’s recommended to report the Java, Scala, Spark, Flink and Hive versions
license documentation.</p><h2 id=validating-a-source-release-candidate>Validating a source release candidate</h2><p>Release announcements include links to the following:</p><ul><li><strong>A source tarball</strong></li><li><strong>A signature (.asc)</strong></li><li><strong>A checksum (.sha512)</strong></li><li><strong>KEYS file</strong></li><li><strong>GitHub change comparison</strong></li></ul><p>After downloading the source tarball, signature, checksum, and KEYS file, here are instruc [...]
verify signatures, checksums, and documentation.</p><h3 id=verifying-signatures>Verifying Signatures</h3><p>First, import the keys.</p><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-bash data-lang=bash><span style=display:flex><span>curl https://dist.apache.org/repos/dist/dev/iceberg/KEYS -o KEYS
</span></span><span style=display:flex><span>gpg --import KEYS
-</span></span></code></pre></div><p>Next, verify the <code>.asc</code> file.</p><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-bash data-lang=bash><span style=display:flex><span>gpg --verify apache-iceberg-1.4.0.tar.gz.asc
-</span></span></code></pre></div><h3 id=verifying-checksums>Verifying Checksums</h3><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-bash data-lang=bash><span style=display:flex><span>shasum -a <span style=color:#ae81ff>512</span> --check apache-iceberg-1.4.0.tar.gz.sha512
-</span></span></code></pre></div><h3 id=verifying-license-documentation>Verifying License Documentation</h3><p>Untar the archive and change into the source directory.</p><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-bash data-lang=bash><span style=display:flex><span>tar xzf apache-iceberg-1.4.0.tar.gz
-</span></span><span style=display:flex><span>cd apache-iceberg-1.4.0
+</span></span></code></pre></div><p>Next, verify the <code>.asc</code> file.</p><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-bash data-lang=bash><span style=display:flex><span>gpg --verify apache-iceberg-1.4.1.tar.gz.asc
+</span></span></code></pre></div><h3 id=verifying-checksums>Verifying Checksums</h3><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-bash data-lang=bash><span style=display:flex><span>shasum -a <span style=color:#ae81ff>512</span> --check apache-iceberg-1.4.1.tar.gz.sha512
+</span></span></code></pre></div><h3 id=verifying-license-documentation>Verifying License Documentation</h3><p>Untar the archive and change into the source directory.</p><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-bash data-lang=bash><span style=display:flex><span>tar xzf apache-iceberg-1.4.1.tar.gz
+</span></span><span style=display:flex><span>cd apache-iceberg-1.4.1
</span></span></code></pre></div><p>Run RAT checks to validate license headers.</p><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-bash data-lang=bash><span style=display:flex><span>dev/check-license
</span></span></code></pre></div><h3 id=verifying-build-and-test>Verifying Build and Test</h3><p>To verify that the release candidate builds properly, run the following command.</p><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-bash data-lang=bash><span style=display:flex><span>./gradlew build
</span></span></code></pre></div><h2 id=testing-release-binaries>Testing release binaries</h2><p>Release announcements will also include a maven repository location. You can use this
@@ -182,21 +182,21 @@ location to test downstream dependencies by adding it to your maven or gradle bu
Replace <code>${MAVEN_URL}</code> with the URL provided in the release announcement</p><h3 id=verifying-with-spark>Verifying with Spark</h3><p>To verify using spark, start a <code>spark-shell</code> with a command like the following command (use the appropriate
spark-runtime jar for the Spark installation):</p><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-bash data-lang=bash><span style=display:flex><span>spark-shell <span style=color:#ae81ff>\
</span></span></span><span style=display:flex><span><span style=color:#ae81ff></span> --conf spark.jars.repositories<span style=color:#f92672>=</span><span style=color:#e6db74>${</span>MAVEN_URL<span style=color:#e6db74>}</span> <span style=color:#ae81ff>\
-</span></span></span><span style=display:flex><span><span style=color:#ae81ff></span> --packages org.apache.iceberg:iceberg-spark3-runtime:1.4.0 <span style=color:#ae81ff>\
+</span></span></span><span style=display:flex><span><span style=color:#ae81ff></span> --packages org.apache.iceberg:iceberg-spark3-runtime:1.4.1 <span style=color:#ae81ff>\
</span></span></span><span style=display:flex><span><span style=color:#ae81ff></span> --conf spark.sql.extensions<span style=color:#f92672>=</span>org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions <span style=color:#ae81ff>\
</span></span></span><span style=display:flex><span><span style=color:#ae81ff></span> --conf spark.sql.catalog.local<span style=color:#f92672>=</span>org.apache.iceberg.spark.SparkCatalog <span style=color:#ae81ff>\
</span></span></span><span style=display:flex><span><span style=color:#ae81ff></span> --conf spark.sql.catalog.local.type<span style=color:#f92672>=</span>hadoop <span style=color:#ae81ff>\
</span></span></span><span style=display:flex><span><span style=color:#ae81ff></span> --conf spark.sql.catalog.local.warehouse<span style=color:#f92672>=</span><span style=color:#e6db74>${</span>LOCAL_WAREHOUSE_PATH<span style=color:#e6db74>}</span> <span style=color:#ae81ff>\
</span></span></span><span style=display:flex><span><span style=color:#ae81ff></span> --conf spark.sql.catalog.local.default-namespace<span style=color:#f92672>=</span>default <span style=color:#ae81ff>\
</span></span></span><span style=display:flex><span><span style=color:#ae81ff></span> --conf spark.sql.defaultCatalog<span style=color:#f92672>=</span>local
-</span></span></code></pre></div><h3 id=verifying-with-flink>Verifying with Flink</h3><p>To verify using Flink, start a Flink SQL Client with the following command:</p><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-bash data-lang=bash><span style=display:flex><span>wget <span style=color:#e6db74>${</span>MAVEN_URL<span style=color:#e6db74>}</span>/iceberg-flink-runtime/1.4.0/iceberg-flink-run [...]
+</span></span></code></pre></div><h3 id=verifying-with-flink>Verifying with Flink</h3><p>To verify using Flink, start a Flink SQL Client with the following command:</p><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-bash data-lang=bash><span style=display:flex><span>wget <span style=color:#e6db74>${</span>MAVEN_URL<span style=color:#e6db74>}</span>/iceberg-flink-runtime/1.4.1/iceberg-flink-run [...]
</span></span><span style=display:flex><span>
</span></span><span style=display:flex><span>sql-client.sh embedded <span style=color:#ae81ff>\
-</span></span></span><span style=display:flex><span><span style=color:#ae81ff></span> -j iceberg-flink-runtime-1.4.0.jar <span style=color:#ae81ff>\
+</span></span></span><span style=display:flex><span><span style=color:#ae81ff></span> -j iceberg-flink-runtime-1.4.1.jar <span style=color:#ae81ff>\
</span></span></span><span style=display:flex><span><span style=color:#ae81ff></span> -j <span style=color:#e6db74>${</span>FLINK_CONNECTOR_PACKAGE<span style=color:#e6db74>}</span>-<span style=color:#e6db74>${</span>HIVE_VERSION<span style=color:#e6db74>}</span>_<span style=color:#e6db74>${</span>SCALA_VERSION<span style=color:#e6db74>}</span>-<span style=color:#e6db74>${</span>FLINK_VERSION<span style=color:#e6db74>}</span>.jar <span style=color:#ae81ff>\
</span></span></span><span style=display:flex><span><span style=color:#ae81ff></span> shell
</span></span></code></pre></div><h2 id=voting>Voting</h2><p>Votes are cast by replying to the release candidate announcement email on the dev mailing list
-with either <code>+1</code>, <code>0</code>, or <code>-1</code>.</p><blockquote><p>[ ] +1 Release this as Apache Iceberg 1.4.0
+with either <code>+1</code>, <code>0</code>, or <code>-1</code>.</p><blockquote><p>[ ] +1 Release this as Apache Iceberg 1.4.1
[ ] +0
[ ] -1 Do not release this because…</p></blockquote><p>In addition to your vote, it’s customary to specify if your vote is binding or non-binding. Only members
of the Project Management Committee have formally binding votes. If you’re unsure, you can specify that your
diff --git a/index.html b/index.html
index fa7f2581..241269b8 100644
--- a/index.html
+++ b/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.4.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.4.1</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
<span class=network-name>Community</span></a></li><li><a href=https://github.com/apache/iceberg class="btn btn-default btn-lg"><i class="fa fa-github fa-fw"></i>
<span class=network-name>github</span></a></li><li><a href=https://join.slack.com/t/apache-iceberg/shared_invite/zt-1znkcg5zm-7_FE~pcox347XwZE3GNfPg class="btn btn-default btn-lg"><i class="fa fa-slack fa-fw"></i>
<span class=network-name>slack</span></a></li></ul></div></div></div></div></div></section><section id=about class="container content-section text-center"><div class=row><div class="col-lg-8 col-lg-offset-2"><br><br><h2>What is Iceberg?</h2><br>Iceberg is a high-performance format for huge analytic tables. Iceberg brings the reliability and simplicity of SQL tables to big data, while making it possible for engines like Spark, Trino, Flink, Presto, Hive and Impala to safely work with the [...]
diff --git a/index.xml b/index.xml
index 06e472b6..4ebf2993 100644
--- a/index.xml
+++ b/index.xml
@@ -1,7 +1,7 @@
<?xml version="1.0" encoding="utf-8" standalone="yes"?><rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom"><channel><title>Apache Iceberg</title><link>https://iceberg.apache.org/</link><description>Recent content on Apache Iceberg</description><generator>Hugo -- gohugo.io</generator><language>en-us</language><atom:link href="https://iceberg.apache.org/index.xml" rel="self" type="application/rss+xml"/><item><title>Expressive SQL</title><link>https://iceberg.apache.org/services/exp [...]
Docker Images Creating a Table Writing Data to a Table Reading Data from a Table Next Steps Docker Images The fastest way to get started is to use Apache Hive images which provides a SQL-like interface to create and query Iceberg tables from your laptop.</description></item><item><title>Spark and Iceberg Quickstart</title><link>https://iceberg.apache.org/spark-quickstart/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><guid>https://iceberg.apache.org/spark-quickstart/</guid><des [...]
-Docker-Compose Creating a table Writing Data to a Table Reading Data from a Table Adding A Catalog Next Steps Docker-Compose The fastest way to get started is to use a docker-compose file that uses the tabulario/spark-iceberg image which contains a local Spark cluster with a configured Iceberg catalog.</description></item><item><title>Releases</title><link>https://iceberg.apache.org/releases/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><guid>https://iceberg.apache.org/release [...]
-1.4.0 source tar.gz &ndash; signature &ndash; sha512 1.4.0 Spark 3.5_2.12 runtime Jar &ndash; 3.5_2.13 1.4.0 Spark 3.4_2.12 runtime Jar &ndash; 3.4_2.13 1.4.0 Spark 3.3_2.12 runtime Jar &ndash; 3.3_2.13 1.4.0 Spark 3.2_2.12 runtime Jar &ndash; 3.2_2.13 1.4.0 Flink 1.17 runtime Jar 1.4.0 Flink 1.16 runtime Jar 1.4.0 Flink 1.15 runtime Jar 1.4.0 Hive runtime Jar 1.4.0 aws-bundle Jar 1.4.0 gcp-bundle Jar 1.4.0 azure-bundle Jar To use Iceberg in Spark or Flink, downlo [...]
+Docker-Compose Creating a table Writing Data to a Table Reading Data from a Table Adding A Catalog Next Steps Docker-Compose The fastest way to get started is to use a docker-compose file that uses the tabulario/spark-iceberg image which contains a local Spark cluster with a configured Iceberg catalog.</description></item><item><title>Releases</title><link>https://iceberg.apache.org/releases/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><guid>https://iceberg.apache.org/release [...]
+1.4.1 source tar.gz &ndash; signature &ndash; sha512 1.4.1 Spark 3.5_2.12 runtime Jar &ndash; 3.5_2.13 1.4.1 Spark 3.4_2.12 runtime Jar &ndash; 3.4_2.13 1.4.1 Spark 3.3_2.12 runtime Jar &ndash; 3.3_2.13 1.4.1 Spark 3.2_2.12 runtime Jar &ndash; 3.2_2.13 1.4.1 Flink 1.17 runtime Jar 1.4.1 Flink 1.16 runtime Jar 1.4.1 Flink 1.15 runtime Jar 1.4.1 Hive runtime Jar 1.4.1 aws-bundle Jar 1.4.1 gcp-bundle Jar 1.4.1 azure-bundle Jar To use Iceberg in Spark or Flink, downlo [...]
Running Benchmarks on GitHub It is possible to run one or more Benchmarks via the JMH Benchmarks GH action on your own fork of the Iceberg repo.</description></item><item><title>Blogs</title><link>https://iceberg.apache.org/blogs/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><guid>https://iceberg.apache.org/blogs/</guid><description>Iceberg Blogs Here is a list of company blogs that talk about Iceberg. The blogs are ordered from most recent to oldest.
Apache Hive-4.x with Iceberg Branches &amp; Tags Date: October 12th, 2023, Company: Cloudera
Authors: Ayush Saxena
diff --git a/landingpagesearch.json b/landingpagesearch.json
index 1d1825d0..3ab5c308 100644
--- a/landingpagesearch.json
+++ b/landingpagesearch.json
@@ -1 +1 @@
-[{"categories":null,"content":" Hive and Iceberg Quickstart This guide will get you up and running with an Iceberg and Hive environment, including sample code to highlight some powerful features. You can learn more about Iceberg’s Hive runtime by checking out the Hive section.\nDocker Images Creating a Table Writing Data to a Table Reading Data from a Table Next Steps Docker Images The fastest way to get started is to use Apache Hive images which provides a SQL-like interface to create a [...]
\ No newline at end of file
+[{"categories":null,"content":" Hive and Iceberg Quickstart This guide will get you up and running with an Iceberg and Hive environment, including sample code to highlight some powerful features. You can learn more about Iceberg’s Hive runtime by checking out the Hive section.\nDocker Images Creating a Table Writing Data to a Table Reading Data from a Table Next Steps Docker Images The fastest way to get started is to use Apache Hive images which provides a SQL-like interface to create a [...]
\ No newline at end of file
diff --git a/multi-engine-support/index.html b/multi-engine-support/index.html
index e4ea3ec3..da7a5135 100644
--- a/multi-engine-support/index.html
+++ b/multi-engine-support/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.4.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.4.1</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
The community continuously improves Iceberg core library components to enable integrations with different compute engines that power analytics, business intelligence, machine learning, etc.
Connectors for Spark, Flink and Hive are maintained in the main Iceberg repository.</p><h2 id=multi-version-support>Multi-Version Support</h2><p>Processing engine connectors maintained in the iceberg repository are built for multiple versions.</p><p>For Spark and Flink, each new version that introduces backwards incompatible upgrade has its dedicated integration codebase and release artifacts.
For example, the code for Iceberg Spark 3.1 integration is under <code>/spark/v3.1</code> and the code for Iceberg Spark 3.2 integration is under <code>/spark/v3.2</code>.
@@ -13,8 +13,8 @@ New features in Iceberg could be developed against the latest features of an eng
When using Iceberg with these engines, the runtime jar is the only addition to the classpath needed in addition to vendor dependencies.
For example, to use Iceberg with Spark 3.2 and AWS integrations, <code>iceberg-spark-runtime-3.2_2.12</code> and AWS SDK dependencies are needed for the Spark installation.</p><p>Spark and Flink provide different runtime jars for each supported engine version.
Hive 2 and Hive 3 currently share the same runtime jar.
-The runtime jar names and latest version download links are listed in <a href=./multi-engine-support/#current-engine-version-lifecycle-status>the tables below</a>.</p><h3 id=engine-version-lifecycle>Engine Version Lifecycle</h3><p>Each engine version undergoes the following lifecycle stages:</p><ol><li><strong>Beta</strong>: a new engine version is supported, but still in the experimental stage. Maybe the engine version itself is still in preview (e.g. Spark <code>3.0.0-preview</code>), [...]
-Users should continuously upgrade their Flink version to stay up-to-date.</p><table><thead><tr><th>Version</th><th>Lifecycle Stage</th><th>Initial Iceberg Support</th><th>Latest Iceberg Support</th><th>Latest Runtime Jar</th></tr></thead><tbody><tr><td>1.11</td><td>End of Life</td><td>0.9.0</td><td>0.12.1</td><td><a href="https://search.maven.org/remotecontent?filepath=org/apache/iceberg/iceberg-flink-runtime/0.12.1/iceberg-flink-runtime-0.12.1.jar">iceberg-flink-runtime</a></td></tr><tr [...]
+The runtime jar names and latest version download links are listed in <a href=./multi-engine-support/#current-engine-version-lifecycle-status>the tables below</a>.</p><h3 id=engine-version-lifecycle>Engine Version Lifecycle</h3><p>Each engine version undergoes the following lifecycle stages:</p><ol><li><strong>Beta</strong>: a new engine version is supported, but still in the experimental stage. Maybe the engine version itself is still in preview (e.g. Spark <code>3.0.0-preview</code>), [...]
+Users should continuously upgrade their Flink version to stay up-to-date.</p><table><thead><tr><th>Version</th><th>Lifecycle Stage</th><th>Initial Iceberg Support</th><th>Latest Iceberg Support</th><th>Latest Runtime Jar</th></tr></thead><tbody><tr><td>1.11</td><td>End of Life</td><td>0.9.0</td><td>0.12.1</td><td><a href="https://search.maven.org/remotecontent?filepath=org/apache/iceberg/iceberg-flink-runtime/0.12.1/iceberg-flink-runtime-0.12.1.jar">iceberg-flink-runtime</a></td></tr><tr [...]
This allows the Iceberg support to evolve with the engine.
Projects such as <a href=https://trino.io/docs/current/connector/iceberg.html>Trino</a> and <a href=https://prestodb.io/docs/current/connector/iceberg.html>Presto</a> are good examples of such support strategy.</p><p>In this approach, an Iceberg version upgrade is needed for an engine to consume new Iceberg features.
To facilitate engine development against unreleased Iceberg features, a daily snapshot is published in the <a href=https://repository.apache.org/content/repositories/snapshots/org/apache/iceberg/>Apache snapshot repository</a>.</p><p>If bringing an engine directly to the Iceberg main repository is needed, please raise a discussion thread in the <a href=../community>Iceberg community</a>.</p></div><div id=toc class=markdown-body><div id=full><nav id=TableOfContents><ul><li><a href=#multi- [...]
diff --git a/puffin-spec/index.html b/puffin-spec/index.html
index 04077fd4..05763f20 100644
--- a/puffin-spec/index.html
+++ b/puffin-spec/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.4.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.4.1</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
information such as indexes and statistics about data managed in an
Iceberg table that cannot be stored directly within the Iceberg manifest. A
Puffin file contains arbitrary pieces of information (here called “blobs”),
diff --git a/releases/index.html b/releases/index.html
index 09cacd24..0c8d0b09 100644
--- a/releases/index.html
+++ b/releases/index.html
@@ -3,20 +3,21 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.4.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
- compile 'org.apache.iceberg:iceberg-core:1.4.0'
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.4.1</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
+ compile 'org.apache.iceberg:iceberg-core:1.4.1'
}
</code></pre><p>You may also want to include <code>iceberg-parquet</code> for Parquet file support.</p><h3 id=maven>Maven</h3><p>To add a dependency on Iceberg in Maven, add the following to your <code>pom.xml</code>:</p><pre tabindex=0><code><dependencies>
...
<dependency>
<groupId>org.apache.iceberg</groupId>
<artifactId>iceberg-core</artifactId>
- <version>1.4.0</version>
+ <version>1.4.1</version>
</dependency>
...
</dependencies>
-</code></pre><h3 id=140-release>1.4.0 release</h3><p>Apache Iceberg 1.4.0 was released on October 4, 2023.
-The 1.4.0 release adds a variety of new features and bug fixes.</p><ul><li>API<ul><li>Implement bound expression sanitization (<a href=https://github.com/apache/iceberg/pull/8149>#8149</a>)</li><li>Remove overflow checks in <code>DefaultCounter</code> causing performance issues (<a href=https://github.com/apache/iceberg/pull/8297>#8297</a>)</li><li>Support incremental scanning with branch (<a href=https://github.com/apache/iceberg/pull/5984>#5984</a>)</li><li>Add a validation API to <cod [...]
+</code></pre><h3 id=141-release>1.4.1 Release</h3><p>Apache Iceberg 1.4.1 was released on October 23, 2023.
+The 1.4.1 release addresses various issues identified in the 1.4.0 release.</p><ul><li>Core<ul><li>Core: Do not use a lazy split offset list in manifests (<a href=https://github.com/apache/iceberg/pull/8834>#8834</a>)</li><li>Core: Ignore split offsets when the last split offset is past the file length (<a href=https://github.com/apache/iceberg/pull/8860>#8860</a>)</li></ul></li><li>AWS<ul><li>Avoid static global credentials provider which doesn’t play well with lifecycle managemen [...]
+The 1.4.0 release adds a variety of new features and bug fixes.</p><ul><li>API<ul><li>Implement bound expression sanitization (<a href=https://github.com/apache/iceberg/pull/8149>#8149</a>)</li><li>Remove overflow checks in <code>DefaultCounter</code> causing performance issues (<a href=https://github.com/apache/iceberg/pull/8297>#8297</a>)</li><li>Support incremental scanning with branch (<a href=https://github.com/apache/iceberg/pull/5984>#5984</a>)</li><li>Add a validation API to <cod [...]
The 1.3.1 release addresses various issues identified in the 1.3.0 release.</p><ul><li>Core<ul><li>Table Metadata parser now accepts null for fields: current-snapshot-id, properties, and snapshots (<a href=https://github.com/apache/iceberg/pull/8064>#8064</a>)</li></ul></li><li>Hive<ul><li>Fix HiveCatalog deleting metadata on failures in checking lock status (<a href=https://github.com/apache/iceberg/pull/7931>#7931</a>)</li></ul></li><li>Spark<ul><li>Fix RewritePositionDeleteFiles failu [...]
The 1.3.0 release adds a variety of new features and bug fixes.</p><ul><li>Core<ul><li>Expose file and data sequence numbers in ContentFile (<a href=https://github.com/apache/iceberg/pull/7555>#7555</a>)</li><li>Improve bit density in object storage layout (<a href=https://github.com/apache/iceberg/pull/7128>#7128</a>)</li><li>Store split offsets for delete files (<a href=https://github.com/apache/iceberg/pull/7011>#7011</a>)</li><li>Readable metrics in entries metadata table (<a href=ht [...]
The 1.2.1 release is a patch release to address various issues identified in the prior release.
@@ -28,7 +29,7 @@ and adds a variety of new features.
Here is an overview:</p><ul><li>Core<ul><li>Puffin statistics have been <a href=https://github.com/apache/iceberg/pull/4945>added to the Table API</a></li><li>Support for <a href=https://github.com/apache/iceberg/pull/5268>Table scan reporting</a>, which enables collection of statistics of the table scans.</li><li><a href=https://github.com/apache/iceberg/pull/6002>Add file sequence number to ManifestEntry</a></li><li><a href=https://github.com/apache/iceberg/pull/5037>Support register t [...]
Prior to the fix, filters would not be extracted and targeted merge conditions were not pushed down leading to degraded performance
for these targeted merge operations.</li><li><a href=https://github.com/apache/iceberg/pull/4024>#4024</a> fixes table creation in the root namespace of a Hadoop Catalog.</li></ul></li><li><p><strong>Flink</strong></p><ul><li><a href=https://github.com/apache/iceberg/pull/3986>#3986</a> fixes manifest location collisions when there are multiple committers
-in the same Flink job.</li></ul></li></ul><h3 id=0130>0.13.0</h3><p>Apache Iceberg 0.13.0 was released on February 4th, 2022.</p><ul><li>Git tag: <a href=https://github.com/apache/iceberg/releases/tag/apache-iceberg-0.13.0>0.13.0</a></li><li><a href=https://www.apache.org/dyn/closer.cgi/iceberg/apache-iceberg-0.13.0/apache-iceberg-0.13.0.tar.gz>0.13.0 source tar.gz</a> – <a href=https://downloads.apache.org/iceberg/apache-iceberg-0.13.0/apache-iceberg-0.13.0.tar.gz.asc>signature</a [...]
+in the same Flink job.</li></ul></li></ul><h3 id=0130>0.13.0</h3><p>Apache Iceberg 0.13.0 was released on February 4th, 2022.</p><ul><li>Git tag: <a href=https://github.com/apache/iceberg/releases/tag/apache-iceberg-0.13.0>0.13.0</a></li><li><a href=https://www.apache.org/dyn/closer.cgi/iceberg/apache-iceberg-0.13.0/apache-iceberg-0.13.0.tar.gz>0.13.0 source tar.gz</a> – <a href=https://downloads.apache.org/iceberg/apache-iceberg-0.13.0/apache-iceberg-0.13.0.tar.gz.asc>signature</a [...]
<script src=https://iceberg.apache.org//js/jquery.easing.min.js></script>
<script type=text/javascript src=https://iceberg.apache.org//js/search.js></script>
<script src=https://iceberg.apache.org//js/bootstrap.min.js></script>
diff --git a/roadmap/index.html b/roadmap/index.html
index c30a6710..8f3fa434 100644
--- a/roadmap/index.html
+++ b/roadmap/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.4.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.4.1</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
This is based on the latest <a href=https://lists.apache.org/thread.html/r84e80216c259c81f824c6971504c321cd8c785774c489d52d4fc123f%40%3Cdev.iceberg.apache.org%3E>community priority discussion</a>.
Each high-level item links to a Github project board that tracks the current status.
Related design docs will be linked on the planning boards.</p><h1 id=priority-1>Priority 1</h1><ul><li>API: <a href=https://github.com/apache/iceberg/projects/3>Iceberg 1.0.0</a> [medium]</li><li>Python: <a href=https://github.com/apache/iceberg/projects/7>Pythonic refactor</a> [medium]</li><li>Spec: <a href=https://github.com/apache/iceberg/projects/16>Z-ordering / Space-filling curves</a> [medium]</li><li>Spec: <a href=https://github.com/apache/iceberg/projects/4>Snapshot tagging and b [...]
diff --git a/security/index.html b/security/index.html
index a9069908..feb5d5d1 100644
--- a/security/index.html
+++ b/security/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.4.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.4.1</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
Security Team</a> for reporting vulnerabilities.
Note that vulnerabilities should not be publicly disclosed until the project has
responded.</p><p>To report a possible security vulnerability, please email <a href=mailto:security@iceberg.apache.org><a href=mailto:security@iceberg.apache.org>security@iceberg.apache.org</a></a>.</p><h1 id=verifying-signed-releases>Verifying Signed Releases</h1><p>Please refer to the instructions on the <a href=https://www.apache.org/info/verification.html>Release Verification</a> page.</p></div><div id=toc class=markdown-body><div id=full><nav id=TableOfContents></nav></div></div></di [...]
diff --git a/services/data-compaction/index.html b/services/data-compaction/index.html
index 1ea801b1..6a0e2558 100644
--- a/services/data-compaction/index.html
+++ b/services/data-compaction/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.4.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.4.1</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
<script src=https://iceberg.apache.org//js/jquery.easing.min.js></script>
<script type=text/javascript src=https://iceberg.apache.org//js/search.js></script>
<script src=https://iceberg.apache.org//js/bootstrap.min.js></script>
diff --git a/services/expressive-sql/index.html b/services/expressive-sql/index.html
index 77972e89..45d69548 100644
--- a/services/expressive-sql/index.html
+++ b/services/expressive-sql/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.4.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.4.1</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
<span data-ty=input data-ty-cursor=▋ data-ty-prompt>USING (SELECT * FROM staging.nyc.taxis) st</span>
<span data-ty=input data-ty-cursor=▋ data-ty-prompt>ON pt.id = st.id</span>
<span data-ty=input data-ty-cursor=▋ data-ty-prompt>WHEN NOT MATCHED THEN INSERT *;</span>
diff --git a/services/hidden-partitioning/index.html b/services/hidden-partitioning/index.html
index 4222912b..2b1762b2 100644
--- a/services/hidden-partitioning/index.html
+++ b/services/hidden-partitioning/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.4.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.4.1</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
<script src=https://iceberg.apache.org//js/jquery.easing.min.js></script>
<script type=text/javascript src=https://iceberg.apache.org//js/search.js></script>
<script src=https://iceberg.apache.org//js/bootstrap.min.js></script>
diff --git a/services/schema-evolution/index.html b/services/schema-evolution/index.html
index 97280eb9..c6c56f50 100644
--- a/services/schema-evolution/index.html
+++ b/services/schema-evolution/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.4.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.4.1</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
<span data-ty=input data-ty-cursor=▋ data-ty-prompt>ALTER COLUMN trip_distance</span>
<span data-ty=input data-ty-cursor=▋ data-ty-prompt data-ty-delay=2500>TYPE double;</span>
<span data-ty>Done!</span>
diff --git a/services/time-travel/index.html b/services/time-travel/index.html
index 93672ce1..393e1bb6 100644
--- a/services/time-travel/index.html
+++ b/services/time-travel/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.4.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.4.1</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
<span data-ty>2,853,020</span>
<span data-ty=input data-ty-cursor=▋ data-ty-prompt="sql>">SELECT count(*) FROM nyc.taxis FOR VERSION AS OF 2188465307835585443</span>
<span data-ty>2,798,371</span>
diff --git a/spark-quickstart/index.html b/spark-quickstart/index.html
index b47121a8..605ef61a 100644
--- a/spark-quickstart/index.html
+++ b/spark-quickstart/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.4.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.4.1</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
highlight some powerful features. You can learn more about Iceberg’s Spark runtime by checking out the <a href=../docs/latest/spark-ddl/>Spark</a> section.</p><ul><li><a href=#docker-compose>Docker-Compose</a></li><li><a href=#creating-a-table>Creating a table</a></li><li><a href=#writing-data-to-a-table>Writing Data to a Table</a></li><li><a href=#reading-data-from-a-table>Reading Data from a Table</a></li><li><a href=#adding-a-catalog>Adding A Catalog</a></li><li><a href=#next-st [...]
which contains a local Spark cluster with a configured Iceberg catalog. To use this, you’ll need to install the <a href=https://docs.docker.com/get-docker/>Docker CLI</a> as well as the <a href=https://github.com/docker/compose-cli/blob/main/INSTALL.md>Docker Compose CLI</a>.</p><p>Once you have those, save the yaml below into a file named <code>docker-compose.yml</code>:</p><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-siz [...]
</span></span><span style=display:flex><span>
@@ -172,7 +172,7 @@ the <a href=../docs/latest/spark-configuration/#catalogs>Catalog</a> page in the
<label for=cli>CLI</label>
<input id=spark-defaults type=radio name=AddingACatalog onclick='selectExampleLanguage("spark-init","spark-defaults")'>
<label for=spark-defaults>spark-defaults.conf</label>
-<codeblock class=cli><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-sh data-lang=sh><span style=display:flex><span>spark-sql --packages org.apache.iceberg:iceberg-spark-runtime-3.2_2.12:1.4.0<span style=color:#ae81ff>\
+<codeblock class=cli><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-sh data-lang=sh><span style=display:flex><span>spark-sql --packages org.apache.iceberg:iceberg-spark-runtime-3.2_2.12:1.4.1<span style=color:#ae81ff>\
</span></span></span><span style=display:flex><span><span style=color:#ae81ff></span> --conf spark.sql.extensions<span style=color:#f92672>=</span>org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions <span style=color:#ae81ff>\
</span></span></span><span style=display:flex><span><span style=color:#ae81ff></span> --conf spark.sql.catalog.spark_catalog<span style=color:#f92672>=</span>org.apache.iceberg.spark.SparkSessionCatalog <span style=color:#ae81ff>\
</span></span></span><span style=display:flex><span><span style=color:#ae81ff></span> --conf spark.sql.catalog.spark_catalog.type<span style=color:#f92672>=</span>hive <span style=color:#ae81ff>\
@@ -180,7 +180,7 @@ the <a href=../docs/latest/spark-configuration/#catalogs>Catalog</a> page in the
</span></span></span><span style=display:flex><span><span style=color:#ae81ff></span> --conf spark.sql.catalog.local.type<span style=color:#f92672>=</span>hadoop <span style=color:#ae81ff>\
</span></span></span><span style=display:flex><span><span style=color:#ae81ff></span> --conf spark.sql.catalog.local.warehouse<span style=color:#f92672>=</span>$PWD/warehouse <span style=color:#ae81ff>\
</span></span></span><span style=display:flex><span><span style=color:#ae81ff></span> --conf spark.sql.defaultCatalog<span style=color:#f92672>=</span>local
-</span></span></code></pre></div></codeblock><codeblock class=spark-defaults><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-sh data-lang=sh><span style=display:flex><span>spark.jars.packages org.apache.iceberg:iceberg-spark-runtime-3.2_2.12:1.4.0
+</span></span></code></pre></div></codeblock><codeblock class=spark-defaults><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-sh data-lang=sh><span style=display:flex><span>spark.jars.packages org.apache.iceberg:iceberg-spark-runtime-3.2_2.12:1.4.1
</span></span><span style=display:flex><span>spark.sql.extensions org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions
</span></span><span style=display:flex><span>spark.sql.catalog.spark_catalog org.apache.iceberg.spark.SparkSessionCatalog
</span></span><span style=display:flex><span>spark.sql.catalog.spark_catalog.type hive
@@ -194,9 +194,9 @@ the <a href=../docs/latest/spark-configuration/#catalogs>Catalog</a> page in the
<label for=spark-shell>Spark-Shell</label>
<input id=pyspark type=radio name=AddIcebergToSpark onclick='selectExampleLanguage("spark-queries","pyspark")'>
<label for=pyspark>PySpark</label>
-<codeblock class=spark-sql><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-sh data-lang=sh><span style=display:flex><span>spark-sql --packages org.apache.iceberg:iceberg-spark-runtime-3.2_2.12:1.4.0
-</span></span></code></pre></div></codeblock><codeblock class=spark-shell><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-sh data-lang=sh><span style=display:flex><span>spark-shell --packages org.apache.iceberg:iceberg-spark-runtime-3.2_2.12:1.4.0
-</span></span></code></pre></div></codeblock><codeblock class=pyspark><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-sh data-lang=sh><span style=display:flex><span>pyspark --packages org.apache.iceberg:iceberg-spark-runtime-3.2_2.12:1.4.0
+<codeblock class=spark-sql><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-sh data-lang=sh><span style=display:flex><span>spark-sql --packages org.apache.iceberg:iceberg-spark-runtime-3.2_2.12:1.4.1
+</span></span></code></pre></div></codeblock><codeblock class=spark-shell><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-sh data-lang=sh><span style=display:flex><span>spark-shell --packages org.apache.iceberg:iceberg-spark-runtime-3.2_2.12:1.4.1
+</span></span></code></pre></div></codeblock><codeblock class=pyspark><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-sh data-lang=sh><span style=display:flex><span>pyspark --packages org.apache.iceberg:iceberg-spark-runtime-3.2_2.12:1.4.1
</span></span></code></pre></div></codeblock></div><div class=info>If you want to include Iceberg in your Spark installation, add the Iceberg Spark runtime to Spark’s <code>jars</code> folder.
You can download the runtime by visiting to the <a href=https://iceberg.apache.org/releases/>Releases</a> page.</div><h4 id=learn-more>Learn More</h4><p>Now that you’re up an running with Iceberg and Spark, check out the <a href=../docs/latest/spark-ddl/>Iceberg-Spark docs</a> to learn more!</p></div></div></div></section></body><script src=https://iceberg.apache.org//js/jquery-1.11.0.js></script>
<script src=https://iceberg.apache.org//js/jquery.easing.min.js></script>
diff --git a/spec/index.html b/spec/index.html
index 4ff029a7..3e86b5df 100644
--- a/spec/index.html
+++ b/spec/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.4.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.4.1</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
</code></pre><p>Notes:</p><ol><li>Changing the number of buckets as a table grows is possible by evolving the partition spec.</li></ol><p>For hash function details by type, see Appendix B.</p><h4 id=truncate-transform-details>Truncate Transform Details</h4><table><thead><tr><th><strong>Type</strong></th><th><strong>Config</strong></th><th><strong>Truncate specification</strong></th><th><strong>Examples</strong></th></tr></thead><tbody><tr><td><strong><code>int</code></strong></td><td><co [...]
The <code>sequence_number</code> field represents the data sequence number and must never change after a file is added to the dataset. The data sequence number represents a relative age of the file content and should be used for planning which delete files apply to a data file.
The <code>file_sequence_number</code> field represents the sequence number of the snapshot that added the file and must also remain unchanged upon assigning at commit. The file sequence number can’t be used for pruning delete files as the data within the file may have an older data sequence number.
@@ -13,7 +13,7 @@ Tags are labels for individual snapshots. Branches are mutable named references
The snapshot expiration procedure removes snapshots from table metadata and applies the table’s retention policy.
Retention policy can be configured both globally and on snapshot reference through properties <code>min-snapshots-to-keep</code>, <code>max-snapshot-age-ms</code> and <code>max-ref-age-ms</code>.</p><p>When expiring snapshots, retention policies in table and snapshot references are evaluated in the following way:</p><ol><li>Start with an empty set of snapshots to retain</li><li>Remove any refs (other than main) where the referenced snapshot is older than <code>max-ref-age-ms</code></li>< [...]
ignore statistics information. Statistics support is not required to read the table correctly. A table can contain
-many statistics files associated with different table snapshots.</p><p>Statistics files metadata within <code>statistics</code> table metadata field is a struct with the following fields:</p><table><thead><tr><th>v1</th><th>v2</th><th>Field name</th><th>Type</th><th>Description</th></tr></thead><tbody><tr><td><em>required</em></td><td><em>required</em></td><td><strong><code>snapshot-id</code></strong></td><td><code>string</code></td><td>ID of the Iceberg table’s snapshot the statis [...]
+many statistics files associated with different table snapshots.</p><p>Statistics files metadata within <code>statistics</code> table metadata field is a struct with the following fields:</p><table><thead><tr><th>v1</th><th>v2</th><th>Field name</th><th>Type</th><th>Description</th></tr></thead><tbody><tr><td><em>required</em></td><td><em>required</em></td><td><strong><code>snapshot-id</code></strong></td><td><code>string</code></td><td>ID of the Iceberg table’s snapshot the statis [...]
</span></span><span style=display:flex><span>-------|-------------|---------
</span></span><span style=display:flex><span> 1 | marsupial | Koala
</span></span><span style=display:flex><span> 2 | toy | Teddy
diff --git a/talks/index.html b/talks/index.html
index 0ed3268e..84fcb0b6 100644
--- a/talks/index.html
+++ b/talks/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.4.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.4.1</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
<script src=https://iceberg.apache.org//js/jquery.easing.min.js></script>
<script type=text/javascript src=https://iceberg.apache.org//js/search.js></script>
<script src=https://iceberg.apache.org//js/bootstrap.min.js></script>
diff --git a/terms/index.html b/terms/index.html
index c41e6e2e..c9006d3c 100644
--- a/terms/index.html
+++ b/terms/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.4.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.4.1</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
<script src=https://iceberg.apache.org//js/jquery.easing.min.js></script>
<script type=text/javascript src=https://iceberg.apache.org//js/search.js></script>
<script src=https://iceberg.apache.org//js/bootstrap.min.js></script>
diff --git a/trademarks/index.html b/trademarks/index.html
index 5a96420e..6ac7497c 100644
--- a/trademarks/index.html
+++ b/trademarks/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.4.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.4.1</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
<script src=https://iceberg.apache.org//js/jquery.easing.min.js></script>
<script type=text/javascript src=https://iceberg.apache.org//js/search.js></script>
<script src=https://iceberg.apache.org//js/bootstrap.min.js></script>
diff --git a/vendors/index.html b/vendors/index.html
index 1344b8c2..0d368af8 100644
--- a/vendors/index.html
+++ b/vendors/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.4.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.4.1</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
<script src=https://iceberg.apache.org//js/jquery.easing.min.js></script>
<script type=text/javascript src=https://iceberg.apache.org//js/search.js></script>
<script src=https://iceberg.apache.org//js/bootstrap.min.js></script>
diff --git a/view-spec/index.html b/view-spec/index.html
index 7479573e..a3bcbae0 100644
--- a/view-spec/index.html
+++ b/view-spec/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.4.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.4.1</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=/docs/latest>latest</a></li> [...]
</span></span></code></pre></div><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-sql data-lang=sql><span style=display:flex><span><span style=color:#66d9ef>CREATE</span> <span style=color:#66d9ef>OR</span> <span style=color:#66d9ef>REPLACE</span> <span style=color:#66d9ef>VIEW</span> event_agg (
</span></span><span style=display:flex><span> event_count <span style=color:#66d9ef>COMMENT</span> <span style=color:#e6db74>'Count of events'</span>,
</span></span><span style=display:flex><span> event_date) <span style=color:#66d9ef>AS</span>