You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@iceberg.apache.org by gi...@apache.org on 2022/09/09 19:47:47 UTC
[iceberg-docs] branch asf-site updated: deploy: d6f1ad88ca18f3b0ab81342c0d8e6b17da634778
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/iceberg-docs.git
The following commit(s) were added to refs/heads/asf-site by this push:
new c24c93b2 deploy: d6f1ad88ca18f3b0ab81342c0d8e6b17da634778
c24c93b2 is described below
commit c24c93b28531732bf4452b1402f6f5860df5713b
Author: danielcweeks <da...@users.noreply.github.com>
AuthorDate: Fri Sep 9 19:47:42 2022 +0000
deploy: d6f1ad88ca18f3b0ab81342c0d8e6b17da634778
---
getting-started/index.html | 20 +-------------------
landingpagesearch.json | 2 +-
spec/index.html | 8 +++++---
3 files changed, 7 insertions(+), 23 deletions(-)
diff --git a/getting-started/index.html b/getting-started/index.html
index b3c6e0d8..52cc0335 100644
--- a/getting-started/index.html
+++ b/getting-started/index.html
@@ -1,19 +1 @@
-<!--
- - Licensed to the Apache Software Foundation (ASF) under one or more
- - contributor license agreements. See the NOTICE file distributed with
- - this work for additional information regarding copyright ownership.
- - The ASF licenses this file to You under the Apache License, Version 2.0
- - (the "License"); you may not use this file except in compliance with
- - the License. You may obtain a copy of the License at
- -
- - http://www.apache.org/licenses/LICENSE-2.0
- -
- - Unless required by applicable law or agreed to in writing, software
- - distributed under the License is distributed on an "AS IS" BASIS,
- - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- - See the License for the specific language governing permissions and
- - limitations under the License.
- -->
-<head>
- <meta http-equiv="Refresh" content="0; url='/docs/latest/getting-started'" />
-</head>
+<!doctype html><html lang=en-us><head><title>https://iceberg.apache.org/spark-quickstart/</title><link rel=canonical href=https://iceberg.apache.org/spark-quickstart/><meta name=robots content="noindex"><meta charset=utf-8><meta http-equiv=refresh content="0; url=https://iceberg.apache.org/spark-quickstart/"></head></html>
\ No newline at end of file
diff --git a/landingpagesearch.json b/landingpagesearch.json
index 67869141..42ede0c8 100644
--- a/landingpagesearch.json
+++ b/landingpagesearch.json
@@ -1 +1 @@
-[{"categories":null,"content":" Spark and Iceberg Quickstart This guide will get you up and running with an Iceberg and Spark environment, including sample code to highlight some powerful features. You can learn more about Iceberg’s Spark runtime by checking out the Spark section.\nDocker-Compose Creating a table Writing Data to a Table Reading Data from a Table Adding A Catalog Next Steps Docker-Compose The fastest way to get started is to use a docker-compose file that uses the the tab [...]
\ No newline at end of file
+[{"categories":null,"content":" Spark and Iceberg Quickstart This guide will get you up and running with an Iceberg and Spark environment, including sample code to highlight some powerful features. You can learn more about Iceberg’s Spark runtime by checking out the Spark section.\nDocker-Compose Creating a table Writing Data to a Table Reading Data from a Table Adding A Catalog Next Steps Docker-Compose The fastest way to get started is to use a docker-compose file that uses the the tab [...]
\ No newline at end of file
diff --git a/spec/index.html b/spec/index.html
index 95f60e02..908bb98e 100644
--- a/spec/index.html
+++ b/spec/index.html
@@ -7,7 +7,9 @@
</code></pre><p>Notes:</p><ol><li>Changing the number of buckets as a table grows is possible by evolving the partition spec.</li></ol><p>For hash function details by type, see Appendix B.</p><h4 id=truncate-transform-details>Truncate Transform Details</h4><table><thead><tr><th><strong>Type</strong></th><th><strong>Config</strong></th><th><strong>Truncate specification</strong></th><th><strong>Examples</strong></th></tr></thead><tbody><tr><td><strong><code>int</code></strong></td><td><co [...]
Tags are labels for individual snapshots. Branches are mutable named references that can be updated by committing a new snapshot as the branch’s referenced snapshot using the <a href=#commit-conflict-resolution-and-retry>Commit Conflict Resolution and Retry</a> procedures.</p><p>The snapshot reference object records all the information of a reference including snapshot ID, reference type and <a href=#snapshot-retention-policy>Snapshot Retention Policy</a>.</p><table><thead><tr><th> [...]
The snapshot expiration procedure removes snapshots from table metadata and applies the table’s retention policy.
-Retention policy can be configured both globally and on snapshot reference through properties <code>min-snapshots-to-keep</code>, <code>max-snapshot-age-ms</code> and <code>max-ref-age-ms</code>.</p><p>When expiring snapshots, retention policies in table and snapshot references are evaluated in the following way:</p><ol><li>Start with an empty set of snapshots to retain</li><li>Remove any refs (other than main) where the referenced snapshot is older than <code>max-ref-age-ms</code></li>< [...]
+Retention policy can be configured both globally and on snapshot reference through properties <code>min-snapshots-to-keep</code>, <code>max-snapshot-age-ms</code> and <code>max-ref-age-ms</code>.</p><p>When expiring snapshots, retention policies in table and snapshot references are evaluated in the following way:</p><ol><li>Start with an empty set of snapshots to retain</li><li>Remove any refs (other than main) where the referenced snapshot is older than <code>max-ref-age-ms</code></li>< [...]
+ignore statistics information. Statistics support is not required to read the table correctly. A table can contain
+many statistics files associated with different table snapshots.</p><p>Statistics files metadata within <code>statistics</code> table metadata field is a struct with the following fields:</p><table><thead><tr><th>v1</th><th>v2</th><th>Field name</th><th>Type</th><th>Description</th></tr></thead><tbody><tr><td><em>required</em></td><td><em>required</em></td><td><strong><code>snapshot-id</code></strong></td><td><code>string</code></td><td>ID of the Iceberg table’s snapshot the statis [...]
</span></span><span style=display:flex><span>-------|-------------|---------
</span></span><span style=display:flex><span> 1 | marsupial | Koala
</span></span><span style=display:flex><span> 2 | toy | Teddy
@@ -28,7 +30,7 @@ Retention policy can be configured both globally and on snapshot reference throu
</span></span><span style=display:flex><span> 1: id | 2: category | 3: name
</span></span><span style=display:flex><span>-------|-------------|---------
</span></span><span style=display:flex><span> 4 | NULL | Polar
-</span></span></code></pre></div><p>If a delete column in an equality delete file is later dropped from the table, it must still be used when applying the equality deletes. If a column was added to a table and later used as a delete column in an equality delete file, the column value is read for older data files using normal projection rules (defaults to <code>null</code>).</p><h4 id=delete-file-stats>Delete File Stats</h4><p>Manifests hold the same statistics for delete files and data f [...]
+</span></span></code></pre></div><p>If a delete column in an equality delete file is later dropped from the table, it must still be used when applying the equality deletes. If a column was added to a table and later used as a delete column in an equality delete file, the column value is read for older data files using normal projection rules (defaults to <code>null</code>).</p><h4 id=delete-file-stats>Delete File Stats</h4><p>Manifests hold the same statistics for delete files and data f [...]
Hash results are not dependent on decimal scale, which is part of the type, not the data value.</li><li>UUIDs are encoded using big endian. The test UUID for the example above is: <code>f79c3e09-677c-4bbd-a479-3f349cb785e7</code>. This UUID encoded as a byte array is:
<code>F7 9C 3E 09 67 7C 4B BD A4 79 3F 34 9C B7 85 E7</code></li><li>Float hash values are the result of hashing the float cast to double to ensure that schema evolution does not change hash values if float types are promoted.</li></ol><h2 id=appendix-c-json-serialization>Appendix C: JSON serialization</h2><h3 id=schemas>Schemas</h3><p>Schemas are serialized as a JSON object with the same fields as a struct in the table below, and the following additional fields:</p><table><thead><tr><th [...]
</span></span><span style=display:flex><span> { <span style=color:#f92672>"field-id"</span>: <span style=color:#ae81ff>2</span>, <span style=color:#f92672>"names"</span>: [<span style=color:#e6db74>"data"</span>] },
@@ -36,7 +38,7 @@ Hash results are not dependent on decimal scale, which is part of the type, not
</span></span><span style=display:flex><span> { <span style=color:#f92672>"field-id"</span>: <span style=color:#ae81ff>4</span>, <span style=color:#f92672>"names"</span>: [<span style=color:#e6db74>"latitude"</span>, <span style=color:#e6db74>"lat"</span>] },
</span></span><span style=display:flex><span> { <span style=color:#f92672>"field-id"</span>: <span style=color:#ae81ff>5</span>, <span style=color:#f92672>"names"</span>: [<span style=color:#e6db74>"longitude"</span>, <span style=color:#e6db74>"long"</span>] }
</span></span><span style=display:flex><span> ] } ]
-</span></span></code></pre></div><h2 id=appendix-d-single-value-serialization>Appendix D: Single-value serialization</h2><h3 id=binary-single-value-serialization>Binary single-value serialization</h3><p>This serialization scheme is for storing single values as individual binary values in the lower and upper bounds maps of manifest files.</p><table><thead><tr><th>Type</th><th>Binary serialization</th></tr></thead><tbody><tr><td><strong><code>boolean</code></strong></td><td><code>0x00</cod [...]
+</span></span></code></pre></div><h2 id=appendix-d-single-value-serialization>Appendix D: Single-value serialization</h2><h3 id=binary-single-value-serialization>Binary single-value serialization</h3><p>This serialization scheme is for storing single values as individual binary values in the lower and upper bounds maps of manifest files.</p><table><thead><tr><th>Type</th><th>Binary serialization</th></tr></thead><tbody><tr><td><strong><code>boolean</code></strong></td><td><code>0x00</cod [...]
<script src=https://iceberg.apache.org//js/jquery.easing.min.js></script>
<script type=text/javascript src=https://iceberg.apache.org//js/search.js></script>
<script src=https://iceberg.apache.org//js/bootstrap.min.js></script>