You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@iceberg.apache.org by bl...@apache.org on 2019/09/13 17:42:28 UTC
[incubator-iceberg] branch asf-site updated: Deployed c5542826 with
MkDocs version: 1.0.4
This is an automated email from the ASF dual-hosted git repository.
blue pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/incubator-iceberg.git
The following commit(s) were added to refs/heads/asf-site by this push:
new d3361fc Deployed c5542826 with MkDocs version: 1.0.4
d3361fc is described below
commit d3361fc96be1301c03670abafb3e998a341e5ef9
Author: Ryan Blue <bl...@apache.org>
AuthorDate: Fri Sep 13 09:42:20 2019 -0800
Deployed c5542826 with MkDocs version: 1.0.4
---
404.html | 6 +
api-quickstart/index.html | 6 +
api/index.html | 8 +-
community/index.html | 6 +
configuration/index.html | 6 +
{partitioning => custom-catalog}/index.html | 198 ++++++++++++++++++++--------
evolution/index.html | 6 +
getting-started/index.html | 6 +
index.html | 8 +-
partitioning/index.html | 6 +
performance/index.html | 6 +
presto/index.html | 6 +
reliability/index.html | 6 +
schemas/index.html | 6 +
sitemap.xml | 35 ++---
sitemap.xml.gz | Bin 216 -> 215 bytes
snapshots/index.html | 6 +
spark/index.html | 6 +
spec/index.html | 56 +++++---
terms/index.html | 8 +-
why-iceberg/index.html | 6 +
21 files changed, 302 insertions(+), 95 deletions(-)
diff --git a/404.html b/404.html
index 2905c3c..35a241a 100644
--- a/404.html
+++ b/404.html
@@ -180,6 +180,12 @@
</li>
+
+<li >
+ <a href="/custom-catalog/">Custom Catalog</a>
+</li>
+
+
</ul>
</li>
diff --git a/api-quickstart/index.html b/api-quickstart/index.html
index 860553f..8e7f967 100644
--- a/api-quickstart/index.html
+++ b/api-quickstart/index.html
@@ -180,6 +180,12 @@
</li>
+
+<li >
+ <a href="../custom-catalog/">Custom Catalog</a>
+</li>
+
+
</ul>
</li>
diff --git a/api/index.html b/api/index.html
index a1e18e3..fcd21e0 100644
--- a/api/index.html
+++ b/api/index.html
@@ -180,6 +180,12 @@
</li>
+
+<li >
+ <a href="../custom-catalog/">Custom Catalog</a>
+</li>
+
+
</ul>
</li>
@@ -235,7 +241,7 @@
</a>
</li>
<li >
- <a rel="next" href="../terms/">
+ <a rel="next" href="../custom-catalog/">
Next <i class="fas fa-arrow-right"></i>
</a>
</li>
diff --git a/community/index.html b/community/index.html
index 192267d..50eaeac 100644
--- a/community/index.html
+++ b/community/index.html
@@ -180,6 +180,12 @@
</li>
+
+<li >
+ <a href="../custom-catalog/">Custom Catalog</a>
+</li>
+
+
</ul>
</li>
diff --git a/configuration/index.html b/configuration/index.html
index c8048be..29d9dcc 100644
--- a/configuration/index.html
+++ b/configuration/index.html
@@ -180,6 +180,12 @@
</li>
+
+<li >
+ <a href="../custom-catalog/">Custom Catalog</a>
+</li>
+
+
</ul>
</li>
diff --git a/partitioning/index.html b/custom-catalog/index.html
similarity index 59%
copy from partitioning/index.html
copy to custom-catalog/index.html
index e038c15..3fbba39 100644
--- a/partitioning/index.html
+++ b/custom-catalog/index.html
@@ -11,7 +11,7 @@
<link rel="../img/favicon.ico">
- <title>Partitioning - Apache Iceberg (incubating)</title>
+ <title>Custom Catalog - Apache Iceberg (incubating)</title>
<link rel="stylesheet" href="//use.fontawesome.com/releases/v5.5.0/css/all.css" integrity="sha384-B4dIYHKNBt8Bc12p+WXckhzcICo0wtJAoU8YZTY5qE0Id1GSseTk6S+L3BlXeVIU" crossorigin="anonymous">
@@ -83,7 +83,7 @@
- <li class="dropdown active">
+ <li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown">User docs <b class="caret"></b></a>
<ul class="dropdown-menu">
@@ -94,8 +94,8 @@
-<li class="active">
- <a href="./">Partitioning</a>
+<li >
+ <a href="../partitioning/">Partitioning</a>
</li>
@@ -152,7 +152,7 @@
- <li class="dropdown">
+ <li class="dropdown active">
<a href="#" class="dropdown-toggle" data-toggle="dropdown">Java <b class="caret"></b></a>
<ul class="dropdown-menu">
@@ -180,6 +180,12 @@
</li>
+
+<li class="active">
+ <a href="./">Custom Catalog</a>
+</li>
+
+
</ul>
</li>
@@ -230,12 +236,12 @@
<ul class="nav navbar-nav navbar-right">
<li >
- <a rel="prev" href="../schemas/">
+ <a rel="prev" href="../api/">
<i class="fas fa-arrow-left"></i> Previous
</a>
</li>
<li >
- <a rel="next" href="../configuration/">
+ <a rel="next" href="../terms/">
Next <i class="fas fa-arrow-right"></i>
</a>
</li>
@@ -249,70 +255,146 @@
<div class="col-md-3"><div class="bs-sidebar hidden-print affix well" role="complementary">
<ul class="nav bs-sidenav">
- <li class="first-level active"><a href="#what-is-partitioning">What is partitioning?</a></li>
- <li class="first-level "><a href="#what-does-iceberg-do-differently">What does Iceberg do differently?</a></li>
- <li class="second-level"><a href="#partitioning-in-hive">Partitioning in Hive</a></li>
+ <li class="first-level active"><a href="#custom-catalog-implementation">Custom Catalog Implementation</a></li>
+ <li class="second-level"><a href="#custom-table-operations-implementation">Custom table operations implementation</a></li>
- <li class="second-level"><a href="#problems-with-hive-partitioning">Problems with Hive partitioning</a></li>
+ <li class="second-level"><a href="#custom-table-implementation">Custom table implementation</a></li>
- <li class="second-level"><a href="#icebergs-hidden-partitioning">Iceberg's hidden partitioning</a></li>
+ <li class="second-level"><a href="#custom-icebergsource">Custom IcebergSource</a></li>
</ul>
</div></div>
<div class="col-md-9" role="main">
-<h2 id="what-is-partitioning">What is partitioning?<a class="headerlink" href="#what-is-partitioning" title="Permanent link">¶</a></h2>
-<p>Partitioning is a way to make queries faster by grouping similar rows together when writing.</p>
-<p>For example, queries for log entries from a <code>logs</code> table would usually include a time range, like this query for logs between 10 and 12 AM:</p>
-<pre><code class="sql">SELECT level, message FROM logs
-WHERE event_time BETWEEN '2018-12-01 10:00:00' AND '2018-12-01 12:00:00'
-</code></pre>
-
-<p>Configuring the <code>logs</code> table to partition by the date of <code>event_time</code> will group log events into files with the same event date. Iceberg keeps track of that date and will use it to skip files for other dates that don’t have useful data.</p>
-<p>Iceberg can partition timestamps by year, month, day, and hour granularity. It can also use a categorical column, like <code>level</code> in this logs example, to store rows together and speed up queries.</p>
-<h2 id="what-does-iceberg-do-differently">What does Iceberg do differently?<a class="headerlink" href="#what-does-iceberg-do-differently" title="Permanent link">¶</a></h2>
-<p>Other tables formats like Hive support partitioning, but Iceberg supports <em>hidden partitioning</em>.</p>
+<h1 id="custom-catalog-implementation">Custom Catalog Implementation<a class="headerlink" href="#custom-catalog-implementation" title="Permanent link">¶</a></h1>
+<p>It’s possible to read an iceberg table either from an hdfs path or from a hive table. It’s also possible to use a custom metastore in place of hive. The steps to do that are as follows.</p>
<ul>
-<li>Iceberg handles the tedious and error-prone task of producing partition values for rows in a table.</li>
-<li>Iceberg avoids reading unnecessary partitions automatically. Consumers don’t need to know how the table is partitioned and add extra filters to their queries.</li>
-<li>Iceberg partition layouts can evolve as needed.</li>
+<li><a href="#custom-table-operations-implementation">Custom TableOperations</a></li>
+<li><a href="#custom-table-implementation">Custom Catalog</a></li>
+<li><a href="#custom-icebergsource">Custom IcebergSource</a></li>
</ul>
-<h3 id="partitioning-in-hive">Partitioning in Hive<a class="headerlink" href="#partitioning-in-hive" title="Permanent link">¶</a></h3>
-<p>To demonstrate the difference, consider how Hive would handle a <code>logs</code> table.</p>
-<p>In Hive, partitions are explicit and appear as a column, so the <code>logs</code> table would have a column called <code>event_date</code>. When writing, an insert needs to supply the data for the <code>event_date</code> column:</p>
-<pre><code class="sql">INSERT INTO logs PARTITION (event_date)
- SELECT level, message, event_time, format_time(event_time, 'YYYY-MM-dd')
- FROM unstructured_log_source
+<h3 id="custom-table-operations-implementation">Custom table operations implementation<a class="headerlink" href="#custom-table-operations-implementation" title="Permanent link">¶</a></h3>
+<p>Extend <code>BaseMetastoreTableOperations</code> to provide implementation on how to read and write metadata</p>
+<p>Example:</p>
+<pre><code class="java">class CustomTableOperations extends BaseMetastoreTableOperations {
+ private String dbName;
+ private String tableName;
+ private Configuration conf;
+ private FileIO fileIO;
+
+ protected CustomTableOperations(Configuration conf, String dbName, String tableName) {
+ this.conf = conf;
+ this.dbName = dbName;
+ this.tableName = tableName;
+ }
+
+ // The doRefresh method should provide implementation on how to get the metadata location
+ @Override
+ public void doRefresh() {
+
+ // Example custom service which returns the metadata location given a dbName and tableName
+ String metadataLocation = CustomService.getMetadataForTable(conf, dbName, tableName);
+
+ // When updating from a metadata file location, call the helper method
+ refreshFromMetadataLocation(metadataLocation);
+
+ }
+
+ // The doCommit method should provide implementation on how to update with metadata location atomically
+ @Override
+ public void doCommit(TableMetadata base, TableMetadata metadata) {
+ String oldMetadataLocation = base.location();
+
+ // Write new metadata using helper method
+ String newMetadataLocation = writeNewMetadata(metadata, currentVersion() + 1);
+
+ // Example custom service which updates the metadata location for the given db and table atomically
+ CustomService.updateMetadataLocation(dbName, tableName, oldMetadataLocation, newMetadataLocation);
+
+ }
+
+ // The io method provides a FileIO which is used to read and write the table metadata files
+ @Override
+ public FileIO io() {
+ if (fileIO == null) {
+ fileIO = new HadoopFileIO(conf);
+ }
+ return fileIO;
+ }
+}
</code></pre>
-<p>Similarly, queries that search through the <code>logs</code> table must have an <code>event_date</code> filter in addition to an <code>event_time</code> filter.</p>
-<pre><code class="sql">SELECT level, count(1) as count FROM logs
-WHERE event_time BETWEEN '2018-12-01 10:00:00' AND '2018-12-01 12:00:00'
- AND event_date = '2018-12-01'
+<h3 id="custom-table-implementation">Custom table implementation<a class="headerlink" href="#custom-table-implementation" title="Permanent link">¶</a></h3>
+<p>Extend <code>BaseMetastoreCatalog</code> to provide default warehouse locations and instantiate <code>CustomTableOperations</code></p>
+<p>Example:</p>
+<pre><code class="java">public class CustomCatalog extends BaseMetastoreCatalog {
+
+ private Configuration configuration;
+
+ public CustomCatalog(Configuration configuration) {
+ this.configuration = configuration;
+ }
+
+ @Override
+ protected TableOperations newTableOps(TableIdentifier tableIdentifier) {
+ String dbName = tableIdentifier.namespace().level(0);
+ String tableName = tableIdentifier.name();
+ // instantiate the CustomTableOperations
+ return new CustomTableOperations(configuration, dbName, tableName);
+ }
+
+ @Override
+ protected String defaultWarehouseLocation(TableIdentifier tableIdentifier) {
+
+ // Can choose to use any other configuration name
+ String tableLocation = configuration.get("custom.iceberg.warehouse.location");
+
+ // Can be an s3 or hdfs path
+ if (tableLocation == null) {
+ throw new RuntimeException("custom.iceberg.warehouse.location configuration not set!");
+ }
+
+ return String.format(
+ "%s/%s.db/%s", tableLocation,
+ tableIdentifier.namespace().levels()[0],
+ tableIdentifier.name());
+ }
+
+ @Override
+ public boolean dropTable(TableIdentifier identifier, boolean purge) {
+ // Example service to delete table
+ CustomService.deleteTable(identifier.namepsace().level(0), identifier.name());
+ }
+
+ @Override
+ public void renameTable(TableIdentifier from, TableIdentifier to) {
+ Preconditions.checkArgument(from.namespace().level(0).equals(to.namespace().level(0)),
+ "Cannot move table between databases");
+ // Example service to rename table
+ CustomService.renameTable(from.namepsace().level(0), from.name(), to.name());
+ }
+}
</code></pre>
-<p>If the <code>event_date</code> filter were missing, Hive would scan through every file in the table because it doesn’t know that the <code>event_time</code> column is related to the <code>event_date</code> column.</p>
-<h3 id="problems-with-hive-partitioning">Problems with Hive partitioning<a class="headerlink" href="#problems-with-hive-partitioning" title="Permanent link">¶</a></h3>
-<p>Hive must be given partition values. In the logs example, it doesn’t know the relationship bewteen <code>event_time</code> and <code>event_date</code>.</p>
-<p>This leads to several problems:</p>
-<ul>
-<li>Hive can’t validate partition values – it is up to the writer to produce the correct value<ul>
-<li>Using the wrong format, <code>2018-12-01</code> instead of <code>20181201</code>, produces silently incorrect results, not query failures</li>
-<li>Using the wrong source column, like <code>processing_time</code>, or time zone also causes incorrect results, not failures</li>
-</ul>
-</li>
-<li>It is up to the user to write queries correctly<ul>
-<li>Using the wrong format also leads to silently incorrect results</li>
-<li>Users that don’t understand a table’s physical layout get needlessly slow queries – Hive can’t translate filters automatically</li>
-</ul>
-</li>
-<li>Working queries are tied to the table’s partitioning scheme, so partitioning configuration cannot be changed without breaking queries</li>
-</ul>
-<h3 id="icebergs-hidden-partitioning">Iceberg’s hidden partitioning<a class="headerlink" href="#icebergs-hidden-partitioning" title="Permanent link">¶</a></h3>
-<p>Iceberg produces partition values by taking a column value and optionally transforming it. Iceberg is responsible for converting <code>event_time</code> into <code>event_date</code>, and keeps track of the relationship.</p>
-<p>Table partitioning is configured using these relationships. The <code>logs</code> table would be partitioned by <code>date(event_time)</code> and <code>level</code>.</p>
-<p>Because Iceberg doesn’t require user-maintained partition columns, it can hide partitioning. Partition values are produced correctly every time and always used to speed up queries, when possible. Producers and consumers wouldn’t even see <code>event_date</code>.</p>
-<p>Most importantly, queries no longer depend on a table’s physical layout. With a separation between physical and logical, Iceberg tables can evolve partition schemes over time as data volume changes. Misconfigured tables can be fixed without an expensive migration.</p></div>
+<h3 id="custom-icebergsource">Custom IcebergSource<a class="headerlink" href="#custom-icebergsource" title="Permanent link">¶</a></h3>
+<p>Extend <code>IcebergSource</code> and provide implementation to read from <code>CustomCatalog</code></p>
+<p>Example:</p>
+<pre><code class="java">public class CustomIcebergSource extends IcebergSource {
+
+ @Override
+ protected Table findTable(DataSourceOptions options, Configuration conf) {
+ Optional<String> path = options.get("path");
+ Preconditions.checkArgument(path.isPresent(), "Cannot open table: path is not set");
+
+ // Read table from CustomCatalog
+ CustomCatalog catalog = new CustomCatalog(conf);
+ TableIdentifier tableIdentifier = TableIdentifier.parse(path.get());
+ return catalog.loadTable(tableIdentifier);
+ }
+}
+</code></pre>
+
+<p>Register the <code>CustomIcebergSource</code> by updating <code>META-INF/services/org.apache.spark.sql.sources.DataSourceRegister</code> with its fully qualified name</p></div>
</div>
diff --git a/evolution/index.html b/evolution/index.html
index da31bc9..3d9b173 100644
--- a/evolution/index.html
+++ b/evolution/index.html
@@ -180,6 +180,12 @@
</li>
+
+<li >
+ <a href="../custom-catalog/">Custom Catalog</a>
+</li>
+
+
</ul>
</li>
diff --git a/getting-started/index.html b/getting-started/index.html
index b2826a1..2ddfa78 100644
--- a/getting-started/index.html
+++ b/getting-started/index.html
@@ -180,6 +180,12 @@
</li>
+
+<li >
+ <a href="../custom-catalog/">Custom Catalog</a>
+</li>
+
+
</ul>
</li>
diff --git a/index.html b/index.html
index a3af56c..e0ef717 100644
--- a/index.html
+++ b/index.html
@@ -180,6 +180,12 @@
</li>
+
+<li >
+ <a href="custom-catalog/">Custom Catalog</a>
+</li>
+
+
</ul>
</li>
@@ -368,5 +374,5 @@
<!--
MkDocs version : 1.0.4
-Build Date UTC : 2019-08-24 18:59:28
+Build Date UTC : 2019-09-13 17:42:20
-->
diff --git a/partitioning/index.html b/partitioning/index.html
index e038c15..71d9835 100644
--- a/partitioning/index.html
+++ b/partitioning/index.html
@@ -180,6 +180,12 @@
</li>
+
+<li >
+ <a href="../custom-catalog/">Custom Catalog</a>
+</li>
+
+
</ul>
</li>
diff --git a/performance/index.html b/performance/index.html
index 831d75c..0c0c9d9 100644
--- a/performance/index.html
+++ b/performance/index.html
@@ -180,6 +180,12 @@
</li>
+
+<li >
+ <a href="../custom-catalog/">Custom Catalog</a>
+</li>
+
+
</ul>
</li>
diff --git a/presto/index.html b/presto/index.html
index be0dec7..4c3f3d4 100644
--- a/presto/index.html
+++ b/presto/index.html
@@ -180,6 +180,12 @@
</li>
+
+<li >
+ <a href="../custom-catalog/">Custom Catalog</a>
+</li>
+
+
</ul>
</li>
diff --git a/reliability/index.html b/reliability/index.html
index 374966b..96047a0 100644
--- a/reliability/index.html
+++ b/reliability/index.html
@@ -180,6 +180,12 @@
</li>
+
+<li >
+ <a href="../custom-catalog/">Custom Catalog</a>
+</li>
+
+
</ul>
</li>
diff --git a/schemas/index.html b/schemas/index.html
index 8a71d8a..1e657d2 100644
--- a/schemas/index.html
+++ b/schemas/index.html
@@ -180,6 +180,12 @@
</li>
+
+<li >
+ <a href="../custom-catalog/">Custom Catalog</a>
+</li>
+
+
</ul>
</li>
diff --git a/sitemap.xml b/sitemap.xml
index 0ae974f..d3bd216 100644
--- a/sitemap.xml
+++ b/sitemap.xml
@@ -2,42 +2,42 @@
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
<url>
<loc>None</loc>
- <lastmod>2019-08-24</lastmod>
+ <lastmod>2019-09-13</lastmod>
<changefreq>daily</changefreq>
</url>
<url>
<loc>None</loc>
- <lastmod>2019-08-24</lastmod>
+ <lastmod>2019-09-13</lastmod>
<changefreq>daily</changefreq>
</url>
<url>
<loc>None</loc>
- <lastmod>2019-08-24</lastmod>
+ <lastmod>2019-09-13</lastmod>
<changefreq>daily</changefreq>
</url>
<url>
<loc>None</loc>
- <lastmod>2019-08-24</lastmod>
+ <lastmod>2019-09-13</lastmod>
<changefreq>daily</changefreq>
</url>
<url>
<loc>None</loc>
- <lastmod>2019-08-24</lastmod>
+ <lastmod>2019-09-13</lastmod>
<changefreq>daily</changefreq>
</url>
<url>
<loc>None</loc>
- <lastmod>2019-08-24</lastmod>
+ <lastmod>2019-09-13</lastmod>
<changefreq>daily</changefreq>
</url>
<url>
<loc>None</loc>
- <lastmod>2019-08-24</lastmod>
+ <lastmod>2019-09-13</lastmod>
<changefreq>daily</changefreq>
</url>
<url>
<loc>None</loc>
- <lastmod>2019-08-24</lastmod>
+ <lastmod>2019-09-13</lastmod>
<changefreq>daily</changefreq>
</url>
<url>
@@ -47,17 +47,17 @@
</url>
<url>
<loc>None</loc>
- <lastmod>2019-08-24</lastmod>
+ <lastmod>2019-09-13</lastmod>
<changefreq>daily</changefreq>
</url>
<url>
<loc>None</loc>
- <lastmod>2019-08-24</lastmod>
+ <lastmod>2019-09-13</lastmod>
<changefreq>daily</changefreq>
</url>
<url>
<loc>None</loc>
- <lastmod>2019-08-24</lastmod>
+ <lastmod>2019-09-13</lastmod>
<changefreq>daily</changefreq>
</url>
<url>
@@ -67,12 +67,12 @@
</url>
<url>
<loc>None</loc>
- <lastmod>2019-08-24</lastmod>
+ <lastmod>2019-09-13</lastmod>
<changefreq>daily</changefreq>
</url>
<url>
<loc>None</loc>
- <lastmod>2019-08-24</lastmod>
+ <lastmod>2019-09-13</lastmod>
<changefreq>daily</changefreq>
</url>
<url>
@@ -81,18 +81,23 @@
<changefreq>daily</changefreq>
</url>
<url>
+ <loc>None</loc>
+ <lastmod>2019-09-13</lastmod>
+ <changefreq>daily</changefreq>
+ </url>
+ <url>
<loc></loc>
<changefreq>daily</changefreq>
</url>
<url>
<loc>None</loc>
- <lastmod>2019-08-24</lastmod>
+ <lastmod>2019-09-13</lastmod>
<changefreq>daily</changefreq>
</url>
<url>
<loc>None</loc>
- <lastmod>2019-08-24</lastmod>
+ <lastmod>2019-09-13</lastmod>
<changefreq>daily</changefreq>
</url>
<url>
diff --git a/sitemap.xml.gz b/sitemap.xml.gz
index 4107ede..8e66317 100644
Binary files a/sitemap.xml.gz and b/sitemap.xml.gz differ
diff --git a/snapshots/index.html b/snapshots/index.html
index bf58bda..7ad26c5 100644
--- a/snapshots/index.html
+++ b/snapshots/index.html
@@ -180,6 +180,12 @@
</li>
+
+<li >
+ <a href="../custom-catalog/">Custom Catalog</a>
+</li>
+
+
</ul>
</li>
diff --git a/spark/index.html b/spark/index.html
index ac9563d..59bb085 100644
--- a/spark/index.html
+++ b/spark/index.html
@@ -180,6 +180,12 @@
</li>
+
+<li >
+ <a href="../custom-catalog/">Custom Catalog</a>
+</li>
+
+
</ul>
</li>
diff --git a/spec/index.html b/spec/index.html
index 5d906bc..63d754a 100644
--- a/spec/index.html
+++ b/spec/index.html
@@ -180,6 +180,12 @@
</li>
+
+<li >
+ <a href="../custom-catalog/">Custom Catalog</a>
+</li>
+
+
</ul>
</li>
@@ -486,7 +492,7 @@ Timestamps <em>without time zone</em> represent a date and time of day regardles
<td><strong><code>day</code></strong></td>
<td>Extract a date or timestamp day, as days from 1970-01-01</td>
<td><code>date</code>, <code>timestamp(tz)</code></td>
-<td><code>int</code></td>
+<td><code>date</code></td>
</tr>
<tr>
<td><strong><code>hour</code></strong></td>
@@ -1319,11 +1325,6 @@ Timestamps <em>without time zone</em> represent a date and time of day regardles
</thead>
<tbody>
<tr>
-<td><strong><code>boolean</code></strong></td>
-<td><code>false: hashInt(0)</code>, <code>true: hashInt(1)</code></td>
-<td><code>true</code> ￫ <code>1392991556</code></td>
-</tr>
-<tr>
<td><strong><code>int</code></strong></td>
<td><code>hashLong(long(v))</code> [1]</td>
<td><code>34</code> ￫ <code>2017239379</code></td>
@@ -1334,18 +1335,8 @@ Timestamps <em>without time zone</em> represent a date and time of day regardles
<td><code>34L</code> ￫ <code>2017239379</code></td>
</tr>
<tr>
-<td><strong><code>float</code></strong></td>
-<td><code>hashDouble(double(v))</code> [2]</td>
-<td><code>1.0F</code> ￫ <code>-142385009</code></td>
-</tr>
-<tr>
-<td><strong><code>double</code></strong></td>
-<td><code>hashLong(doubleToLongBits(v))</code></td>
-<td><code>1.0D</code> ￫ <code>-142385009</code></td>
-</tr>
-<tr>
<td><strong><code>decimal(P,S)</code></strong></td>
-<td><code>hashBytes(minBigEndian(unscaled(v)))</code>[3]</td>
+<td><code>hashBytes(minBigEndian(unscaled(v)))</code>[2]</td>
<td><code>14.20</code> ￫ <code>-500754589</code></td>
</tr>
<tr>
@@ -1375,7 +1366,7 @@ Timestamps <em>without time zone</em> represent a date and time of day regardles
</tr>
<tr>
<td><strong><code>uuid</code></strong></td>
-<td><code>hashBytes(uuidBytes(v))</code> [4]</td>
+<td><code>hashBytes(uuidBytes(v))</code> [3]</td>
<td><code>f79c3e09-677c-4bbd-a479-3f349cb785e7</code> ￫ <code>1488055340</code></td>
</tr>
<tr>
@@ -1390,14 +1381,41 @@ Timestamps <em>without time zone</em> represent a date and time of day regardles
</tr>
</tbody>
</table>
+<p>The types below are not currently valid for bucketing, and so are not hashed. However, if that changes and a hash value is needed, the following table shall apply:</p>
+<table>
+<thead>
+<tr>
+<th>Primitive type</th>
+<th>Hash specification</th>
+<th>Test value</th>
+</tr>
+</thead>
+<tbody>
+<tr>
+<td><strong><code>boolean</code></strong></td>
+<td><code>false: hashInt(0)</code>, <code>true: hashInt(1)</code></td>
+<td><code>true</code> ￫ <code>1392991556</code></td>
+</tr>
+<tr>
+<td><strong><code>float</code></strong></td>
+<td><code>hashDouble(double(v))</code> [4]</td>
+<td><code>1.0F</code> ￫ <code>-142385009</code></td>
+</tr>
+<tr>
+<td><strong><code>double</code></strong></td>
+<td><code>hashLong(doubleToLongBits(v))</code></td>
+<td><code>1.0D</code> ￫ <code>-142385009</code></td>
+</tr>
+</tbody>
+</table>
<p>Notes:</p>
<ol>
<li>Integer and long hash results must be identical for all integer values. This ensures that schema evolution does not change bucket partition values if integer types are promoted.</li>
-<li>Float hash values are the result of hashing the float cast to double to ensure that schema evolution does not change hash values if float types are promoted. Note that floating point types are not valid source values for partitioning.</li>
<li>Decimal values are hashed using the minimum number of bytes required to hold the unscaled value as a two’s complement big-endian; this representation does not include padding bytes required for storage in a fixed-length array.
Hash results are not dependent on decimal scale, which is part of the type, not the data value.</li>
<li>UUIDs are encoded using big endian. The test UUID for the example above is: <code>f79c3e09-677c-4bbd-a479-3f349cb785e7</code>. This UUID encoded as a byte array is:
<code>F7 9C 3E 09 67 7C 4B BD A4 79 3F 34 9C B7 85 E7</code></li>
+<li>Float hash values are the result of hashing the float cast to double to ensure that schema evolution does not change hash values if float types are promoted.</li>
</ol>
<h2 id="appendix-c-json-serialization">Appendix C: JSON serialization<a class="headerlink" href="#appendix-c-json-serialization" title="Permanent link">¶</a></h2>
<h3 id="schemas">Schemas<a class="headerlink" href="#schemas" title="Permanent link">¶</a></h3>
diff --git a/terms/index.html b/terms/index.html
index 90c0c8b..0b92e44 100644
--- a/terms/index.html
+++ b/terms/index.html
@@ -180,6 +180,12 @@
</li>
+
+<li >
+ <a href="../custom-catalog/">Custom Catalog</a>
+</li>
+
+
</ul>
</li>
@@ -230,7 +236,7 @@
<ul class="nav navbar-nav navbar-right">
<li >
- <a rel="prev" href="../api/">
+ <a rel="prev" href="../custom-catalog/">
<i class="fas fa-arrow-left"></i> Previous
</a>
</li>
diff --git a/why-iceberg/index.html b/why-iceberg/index.html
index e0fce29..94bd08a 100644
--- a/why-iceberg/index.html
+++ b/why-iceberg/index.html
@@ -180,6 +180,12 @@
</li>
+
+<li >
+ <a href="../custom-catalog/">Custom Catalog</a>
+</li>
+
+
</ul>
</li>