You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@orc.apache.org by om...@apache.org on 2016/06/11 18:15:54 UTC
[2/6] orc git commit: Pushing update for ORC-65
http://git-wip-us.apache.org/repos/asf/orc/blob/1eb37b72/docs/spec-intro.html
----------------------------------------------------------------------
diff --git a/docs/spec-intro.html b/docs/spec-intro.html
index d0637d3..b5036f1 100644
--- a/docs/spec-intro.html
+++ b/docs/spec-intro.html
@@ -105,25 +105,19 @@
- <option value="/docs/index.html">Background</option>
-
-
+ <option value="/docs/index.html">Background</option>
-
-
-
-
@@ -140,21 +134,23 @@
-
+
-
+
+ <option value="/docs/adopters.html">ORC Adopters</option>
+
- <option value="/docs/types.html">Types</option>
+
-
+
-
+
@@ -171,7 +167,7 @@
- <option value="/docs/indexes.html">Indexes</option>
+
@@ -189,8 +185,6 @@
- <option value="/docs/acid.html">ACID support</option>
-
@@ -216,16 +210,7 @@
-
-
- </optgroup>
- <optgroup label="Hive Usage">
-
-
-
-
-
@@ -238,12 +223,12 @@
- <option value="/docs/hive-ddl.html">Hive DDL</option>
+ <option value="/docs/types.html">Types</option>
-
+
-
+
@@ -255,9 +240,7 @@
-
-
-
+
@@ -268,7 +251,7 @@
- <option value="/docs/hive-config.html">Hive Configuration</option>
+ <option value="/docs/indexes.html">Indexes</option>
@@ -287,19 +270,14 @@
-
-
- </optgroup>
- <optgroup label="Format Specification">
-
-
+
-
+ <option value="/docs/acid.html">ACID support</option>
@@ -319,17 +297,11 @@
- <option value="/docs/spec-intro.html">Introduction</option>
-
-
-
-
-
@@ -337,8 +309,6 @@
- <option value="/docs/file-tail.html">File Tail</option>
-
@@ -348,24 +318,35 @@
+
+
+ </optgroup>
+ <optgroup label="Installing">
+
+
+
+
+
+ <option value="/docs/building.html">Building ORC</option>
+
-
+
-
+
- <option value="/docs/compression.html">Compression</option>
+
@@ -411,8 +392,6 @@
- <option value="/docs/run-length.html">Run Length Encoding</option>
-
@@ -422,9 +401,11 @@
-
+
+ <option value="/docs/releases.html">Releases</option>
+
-
+
@@ -436,7 +417,16 @@
+
+
+ </optgroup>
+ <optgroup label="Using in Hive">
+
+
+
+
+
@@ -449,21 +439,23 @@
- <option value="/docs/stripes.html">Stripes</option>
+
-
+
+ <option value="/docs/hive-ddl.html">Hive DDL</option>
+
-
+
- <option value="/docs/encodings.html">Column Encodings</option>
+
@@ -478,6 +470,10 @@
+
+
+
+
@@ -486,13 +482,15 @@
-
+
-
+
+ <option value="/docs/hive-config.html">Hive Configuration</option>
+
@@ -509,7 +507,7 @@
- <option value="/docs/spec-index.html">Indexes</option>
+
@@ -522,97 +520,41 @@
</optgroup>
- </select>
-</div>
-
-
- <div class="unit four-fifths">
- <article>
- <h1>Introduction</h1>
- <p>Hive\u2019s RCFile was the standard format for storing tabular data in
-Hadoop for several years. However, RCFile has limitations because it
-treats each column as a binary blob without semantics. In Hive 0.11 we
-added a new file format named Optimized Row Columnar (ORC) file that
-uses and retains the type information from the table definition. ORC
-uses type specific readers and writers that provide light weight
-compression techniques such as dictionary encoding, bit packing, delta
-encoding, and run length encoding \u2013 resulting in dramatically smaller
-files. Additionally, ORC can apply generic compression using zlib, or
-Snappy on top of the lightweight compression for even smaller
-files. However, storage savings are only part of the gain. ORC
-supports projection, which selects subsets of the columns for reading,
-so that queries reading only one column read only the required
-bytes. Furthermore, ORC files include light weight indexes that
-include the minimum and maximum values for each column in each set of
-10,000 rows and the entire file. Using pushdown filters from Hive, the
-file reader can skip entire sets of rows that aren\u2019t important for
-this query.</p>
-
-<p><img src="/img/OrcFileLayout.png" alt="ORC file structure" /></p>
-
-
-
-
-
+ <optgroup label="Using in MapReduce">
+
-
+
-
-
+
-
+
+
-
+
+
-
+
+
-
+
+
- <div class="section-nav">
- <div class="left align-right">
-
-
-
- <a href="/docs/hive-config.html" class="prev">Back</a>
-
- </div>
- <div class="right align-left">
-
-
-
- <a href="/docs/file-tail.html" class="next">Next</a>
-
- </div>
- </div>
- <div class="clear"></div>
-
- </article>
- </div>
-
- <div class="unit one-fifth hide-on-mobiles">
- <aside>
+
- <h4>Overview</h4>
+ <option value="/docs/mapred.html">Using in MapRed</option>
-
-<ul>
-
-
-
-
-
@@ -626,15 +568,9 @@ this query.</p>
- <li class=""><a href="/docs/index.html">Background</a></li>
-
-
-
-
-
@@ -662,17 +598,9 @@ this query.</p>
- <li class=""><a href="/docs/types.html">Types</a></li>
-
-
-
-
-
-
+ <option value="/docs/mapreduce.html">Using in MapReduce</option>
-
-
@@ -687,8 +615,11 @@ this query.</p>
+
+
+ </optgroup>
- <li class=""><a href="/docs/indexes.html">Indexes</a></li>
+ <optgroup label="Using ORC Core">
@@ -697,27 +628,17 @@ this query.</p>
-
-
-
- <li class=""><a href="/docs/acid.html">ACID support</a></li>
-
-
-
-</ul>
-
- <h4>Hive Usage</h4>
+
-
-<ul>
-
-
+
+ <option value="/docs/core-java.html">Using Core Java</option>
+
-
+
@@ -730,16 +651,12 @@ this query.</p>
- <li class=""><a href="/docs/hive-ddl.html">Hive DDL</a></li>
-
-
-
-
+
-
+
@@ -750,24 +667,21 @@ this query.</p>
- <li class=""><a href="/docs/hive-config.html">Hive Configuration</a></li>
-
-
+
-</ul>
+ </optgroup>
- <h4>Format Specification</h4>
-
+ <optgroup label="Tools">
+
-<ul>
-
+
@@ -790,19 +704,952 @@ this query.</p>
- <li class="current"><a href="/docs/spec-intro.html">Introduction</a></li>
-
-
-
-
+
+
+
+
+
+
+
+
+ <option value="/docs/tools.html">Tools</option>
+
+
+
+
+
+
+ </optgroup>
+
+ <optgroup label="Format Specification">
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <option value="/docs/spec-intro.html">Introduction</option>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <option value="/docs/file-tail.html">File Tail</option>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <option value="/docs/compression.html">Compression</option>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <option value="/docs/run-length.html">Run Length Encoding</option>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <option value="/docs/stripes.html">Stripes</option>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <option value="/docs/encodings.html">Column Encodings</option>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <option value="/docs/spec-index.html">Indexes</option>
+
+
+
+
+
+
+
+
+
+
+
+
+ </optgroup>
+
+ </select>
+</div>
+
+
+ <div class="unit four-fifths">
+ <article>
+ <h1>Introduction</h1>
+ <p>Hive\u2019s RCFile was the standard format for storing tabular data in
+Hadoop for several years. However, RCFile has limitations because it
+treats each column as a binary blob without semantics. In Hive 0.11 we
+added a new file format named Optimized Row Columnar (ORC) file that
+uses and retains the type information from the table definition. ORC
+uses type specific readers and writers that provide light weight
+compression techniques such as dictionary encoding, bit packing, delta
+encoding, and run length encoding \u2013 resulting in dramatically smaller
+files. Additionally, ORC can apply generic compression using zlib, or
+Snappy on top of the lightweight compression for even smaller
+files. However, storage savings are only part of the gain. ORC
+supports projection, which selects subsets of the columns for reading,
+so that queries reading only one column read only the required
+bytes. Furthermore, ORC files include light weight indexes that
+include the minimum and maximum values for each column in each set of
+10,000 rows and the entire file. Using pushdown filters from Hive, the
+file reader can skip entire sets of rows that aren\u2019t important for
+this query.</p>
+
+<p><img src="/img/OrcFileLayout.png" alt="ORC file structure" /></p>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <div class="section-nav">
+ <div class="left align-right">
+
+
+
+ <a href="/docs/tools.html" class="prev">Back</a>
+
+ </div>
+ <div class="right align-left">
+
+
+
+ <a href="/docs/file-tail.html" class="next">Next</a>
+
+ </div>
+ </div>
+ <div class="clear"></div>
+
+
+ </article>
+ </div>
+
+ <div class="unit one-fifth hide-on-mobiles">
+ <aside>
+
+ <h4>Overview</h4>
+
+
+<ul>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <li class=""><a href="/docs/index.html">Background</a></li>
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <li class=""><a href="/docs/adopters.html">ORC Adopters</a></li>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <li class=""><a href="/docs/types.html">Types</a></li>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <li class=""><a href="/docs/indexes.html">Indexes</a></li>
+
+
+
+
+
+
+
+
+
+
+
+ <li class=""><a href="/docs/acid.html">ACID support</a></li>
+
+
+
+</ul>
+
+
+ <h4>Installing</h4>
+
+
+<ul>
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <li class=""><a href="/docs/building.html">Building ORC</a></li>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <li class=""><a href="/docs/releases.html">Releases</a></li>
+
+
+
+</ul>
+
+
+ <h4>Using in Hive</h4>
+
+
+<ul>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <li class=""><a href="/docs/hive-ddl.html">Hive DDL</a></li>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <li class=""><a href="/docs/hive-config.html">Hive Configuration</a></li>
+
+
+
+</ul>
+
+
+ <h4>Using in MapReduce</h4>
+
+
+<ul>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <li class=""><a href="/docs/mapred.html">Using in MapRed</a></li>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <li class=""><a href="/docs/mapreduce.html">Using in MapReduce</a></li>
+
+
+
+</ul>
+
+
+ <h4>Using ORC Core</h4>
+
+
+<ul>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <li class=""><a href="/docs/core-java.html">Using Core Java</a></li>
+
+
+
+</ul>
+
+
+ <h4>Tools</h4>
+
+
+<ul>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <li class=""><a href="/docs/tools.html">Tools</a></li>
+
+
+
+</ul>
+
+
+ <h4>Format Specification</h4>
+
+
+<ul>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <li class="current"><a href="/docs/spec-intro.html">Introduction</a></li>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
@@ -822,6 +1669,10 @@ this query.</p>
+
+
+
+
<li class=""><a href="/docs/compression.html">Compression</a></li>
@@ -850,6 +1701,18 @@ this query.</p>
+
+
+
+
+
+
+
+
+
+
+
+
<li class=""><a href="/docs/run-length.html">Run Length Encoding</a></li>
@@ -884,6 +1747,18 @@ this query.</p>
+
+
+
+
+
+
+
+
+
+
+
+
<li class=""><a href="/docs/stripes.html">Stripes</a></li>
@@ -900,6 +1775,12 @@ this query.</p>
+
+
+
+
+
+
<li class=""><a href="/docs/encodings.html">Column Encodings</a></li>
@@ -930,6 +1811,18 @@ this query.</p>
+
+
+
+
+
+
+
+
+
+
+
+
<li class=""><a href="/docs/spec-index.html">Indexes</a></li>
http://git-wip-us.apache.org/repos/asf/orc/blob/1eb37b72/docs/stripes.html
----------------------------------------------------------------------
diff --git a/docs/stripes.html b/docs/stripes.html
index c9d9b92..678b13f 100644
--- a/docs/stripes.html
+++ b/docs/stripes.html
@@ -105,6 +105,12 @@
+
+
+
+
+
+
<option value="/docs/index.html">Background</option>
@@ -120,6 +126,60 @@
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <option value="/docs/adopters.html">ORC Adopters</option>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
@@ -149,6 +209,20 @@
+
+
+
+
+
+
+
+
+
+
+
+
+
+
<option value="/docs/types.html">Types</option>
@@ -171,6 +245,12 @@
+
+
+
+
+
+
<option value="/docs/indexes.html">Indexes</option>
@@ -184,6 +264,14 @@
+
+
+
+
+
+
+
+
@@ -216,11 +304,25 @@
+
+
+
+
+
+
+
+
+
+
+
+
+
+
</optgroup>
- <optgroup label="Hive Usage">
+ <optgroup label="Installing">
@@ -232,13 +334,27 @@
+ <option value="/docs/building.html">Building ORC</option>
+
- <option value="/docs/hive-ddl.html">Hive DDL</option>
+
+
+
+
+
+
+
+
+
+
+
+
+
@@ -268,7 +384,17 @@
- <option value="/docs/hive-config.html">Hive Configuration</option>
+
+
+
+
+
+
+
+
+
+
+
@@ -276,6 +402,10 @@
+ <option value="/docs/releases.html">Releases</option>
+
+
+
@@ -291,7 +421,7 @@
</optgroup>
- <optgroup label="Format Specification">
+ <optgroup label="Using in Hive">
@@ -315,11 +445,25 @@
+ <option value="/docs/hive-ddl.html">Hive DDL</option>
+
- <option value="/docs/spec-intro.html">Introduction</option>
+
+
+
+
+
+
+
+
+
+
+
+
+
@@ -337,7 +481,17 @@
- <option value="/docs/file-tail.html">File Tail</option>
+
+
+
+
+
+
+
+
+ <option value="/docs/hive-config.html">Hive Configuration</option>
+
+
@@ -358,6 +512,17 @@
+
+
+
+
+
+
+ </optgroup>
+
+ <optgroup label="Using in MapReduce">
+
+
@@ -365,7 +530,11 @@
- <option value="/docs/compression.html">Compression</option>
+
+
+
+
+
@@ -381,6 +550,16 @@
+ <option value="/docs/mapred.html">Using in MapRed</option>
+
+
+
+
+
+
+
+
+
@@ -411,7 +590,7 @@
- <option value="/docs/run-length.html">Run Length Encoding</option>
+
@@ -419,12 +598,14 @@
+ <option value="/docs/mapreduce.html">Using in MapReduce</option>
+
-
+
-
+
@@ -434,7 +615,16 @@
+
+
+ </optgroup>
+ <optgroup label="Using ORC Core">
+
+
+
+
+
@@ -445,25 +635,27 @@
+ <option value="/docs/core-java.html">Using Core Java</option>
+
- <option value="/docs/stripes.html">Stripes</option>
+
-
+
-
+
- <option value="/docs/encodings.html">Column Encodings</option>
+
@@ -476,7 +668,16 @@
+
+
+ </optgroup>
+ <optgroup label="Tools">
+
+
+
+
+
@@ -486,9 +687,9 @@
-
+
-
+
@@ -509,12 +710,14 @@
- <option value="/docs/spec-index.html">Indexes</option>
+
+ <option value="/docs/tools.html">Tools</option>
+
@@ -522,38 +725,367 @@
</optgroup>
- </select>
-</div>
-
-
- <div class="unit four-fifths">
- <article>
- <h1>Stripes</h1>
- <p>The body of ORC files consists of a series of stripes. Stripes are
-large (typically ~200MB) and independent of each other and are often
-processed by different tasks. The defining characteristic for columnar
-storage formats is that the data for each column is stored separately
-and that reading data out of the file should be proportional to the
-number of columns read.</p>
-
-<p>In ORC files, each column is stored in several streams that are stored
-next to each other in the file. For example, an integer column is
-represented as two streams PRESENT, which uses one with a bit per
-value recording if the value is non-null, and DATA, which records the
-non-null values. If all of a column\u2019s values in a stripe are non-null,
-the PRESENT stream is omitted from the stripe. For binary data, ORC
-uses three streams PRESENT, DATA, and LENGTH, which stores the length
-of each value. The details of each type will be presented in the
-following subsections.</p>
+ <optgroup label="Format Specification">
+
-<h1 id="stripe-footer">Stripe Footer</h1>
-<p>The stripe footer contains the encoding of each column and the
-directory of the streams including their location.</p>
+
-<p><code>message StripeFooter {
- // the location of each stream
- repeated Stream streams = 1;
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <option value="/docs/spec-intro.html">Introduction</option>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <option value="/docs/file-tail.html">File Tail</option>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <option value="/docs/compression.html">Compression</option>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <option value="/docs/run-length.html">Run Length Encoding</option>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <option value="/docs/stripes.html">Stripes</option>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <option value="/docs/encodings.html">Column Encodings</option>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <option value="/docs/spec-index.html">Indexes</option>
+
+
+
+
+
+
+
+
+
+
+
+
+ </optgroup>
+
+ </select>
+</div>
+
+
+ <div class="unit four-fifths">
+ <article>
+ <h1>Stripes</h1>
+ <p>The body of ORC files consists of a series of stripes. Stripes are
+large (typically ~200MB) and independent of each other and are often
+processed by different tasks. The defining characteristic for columnar
+storage formats is that the data for each column is stored separately
+and that reading data out of the file should be proportional to the
+number of columns read.</p>
+
+<p>In ORC files, each column is stored in several streams that are stored
+next to each other in the file. For example, an integer column is
+represented as two streams PRESENT, which uses one with a bit per
+value recording if the value is non-null, and DATA, which records the
+non-null values. If all of a column\u2019s values in a stripe are non-null,
+the PRESENT stream is omitted from the stripe. For binary data, ORC
+uses three streams PRESENT, DATA, and LENGTH, which stores the length
+of each value. The details of each type will be presented in the
+following subsections.</p>
+
+<h1 id="stripe-footer">Stripe Footer</h1>
+
+<p>The stripe footer contains the encoding of each column and the
+directory of the streams including their location.</p>
+
+<p><code>message StripeFooter {
+ // the location of each stream
+ repeated Stream streams = 1;
// the encoding of each column
repeated ColumnEncoding columns = 2;
}
@@ -611,70 +1143,299 @@ further refined as to whether they use RLE v1 or v2.</p>
}
</code></p>
-
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <div class="section-nav">
+ <div class="left align-right">
+
+
+
+ <a href="/docs/run-length.html" class="prev">Back</a>
+
+ </div>
+ <div class="right align-left">
+
+
+
+ <a href="/docs/encodings.html" class="next">Next</a>
+
+ </div>
+ </div>
+ <div class="clear"></div>
+
+
+ </article>
+ </div>
+
+ <div class="unit one-fifth hide-on-mobiles">
+ <aside>
+
+ <h4>Overview</h4>
+
+
+<ul>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <li class=""><a href="/docs/index.html">Background</a></li>
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <li class=""><a href="/docs/adopters.html">ORC Adopters</a></li>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <li class=""><a href="/docs/types.html">Types</a></li>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <li class=""><a href="/docs/indexes.html">Indexes</a></li>
+
-
+
+
+ <li class=""><a href="/docs/acid.html">ACID support</a></li>
+
+
+
+</ul>
+
+
+ <h4>Installing</h4>
+
+
+<ul>
+
+
+
-
+
+
+ <li class=""><a href="/docs/building.html">Building ORC</a></li>
+
+
+
+
+
-
+
+
-
+
+
-
+
+
+
+
+
+
+
+
+
+
- <div class="section-nav">
- <div class="left align-right">
-
-
-
- <a href="/docs/run-length.html" class="prev">Back</a>
-
- </div>
- <div class="right align-left">
-
-
-
- <a href="/docs/encodings.html" class="next">Next</a>
-
- </div>
- </div>
- <div class="clear"></div>
+
+
+
+
+ <li class=""><a href="/docs/releases.html">Releases</a></li>
+
- </article>
- </div>
- <div class="unit one-fifth hide-on-mobiles">
- <aside>
+</ul>
+
- <h4>Overview</h4>
+ <h4>Using in Hive</h4>
<ul>
@@ -699,7 +1460,11 @@ further refined as to whether they use RLE v1 or v2.</p>
- <li class=""><a href="/docs/index.html">Background</a></li>
+
+
+
+
+ <li class=""><a href="/docs/hive-ddl.html">Hive DDL</a></li>
@@ -725,6 +1490,38 @@ further refined as to whether they use RLE v1 or v2.</p>
+ <li class=""><a href="/docs/hive-config.html">Hive Configuration</a></li>
+
+
+
+</ul>
+
+
+ <h4>Using in MapReduce</h4>
+
+
+<ul>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
@@ -735,7 +1532,7 @@ further refined as to whether they use RLE v1 or v2.</p>
- <li class=""><a href="/docs/types.html">Types</a></li>
+ <li class=""><a href="/docs/mapred.html">Using in MapRed</a></li>
@@ -761,10 +1558,28 @@ further refined as to whether they use RLE v1 or v2.</p>
- <li class=""><a href="/docs/indexes.html">Indexes</a></li>
+
+
+
+
+
+
+
+
+
+
+ <li class=""><a href="/docs/mapreduce.html">Using in MapReduce</a></li>
+</ul>
+
+
+ <h4>Using ORC Core</h4>
+
+
+<ul>
+
@@ -773,14 +1588,22 @@ further refined as to whether they use RLE v1 or v2.</p>
- <li class=""><a href="/docs/acid.html">ACID support</a></li>
+
+
+
+
+
+
+
+
+ <li class=""><a href="/docs/core-java.html">Using Core Java</a></li>
</ul>
- <h4>Hive Usage</h4>
+ <h4>Tools</h4>
<ul>
@@ -803,16 +1626,12 @@ further refined as to whether they use RLE v1 or v2.</p>
- <li class=""><a href="/docs/hive-ddl.html">Hive DDL</a></li>
-
-
-
-
+
-
+
@@ -823,7 +1642,17 @@ further refined as to whether they use RLE v1 or v2.</p>
- <li class=""><a href="/docs/hive-config.html">Hive Configuration</a></li>
+
+
+
+
+
+
+
+
+
+
+ <li class=""><a href="/docs/tools.html">Tools</a></li>
@@ -863,6 +1692,18 @@ further refined as to whether they use RLE v1 or v2.</p>
+
+
+
+
+
+
+
+
+
+
+
+
<li class=""><a href="/docs/spec-intro.html">Introduction</a></li>
@@ -881,6 +1722,12 @@ further refined as to whether they use RLE v1 or v2.</p>
+
+
+
+
+
+
<li class=""><a href="/docs/file-tail.html">File Tail</a></li>
@@ -895,6 +1742,10 @@ further refined as to whether they use RLE v1 or v2.</p>
+
+
+
+
<li class=""><a href="/docs/compression.html">Compression</a></li>
@@ -923,6 +1774,18 @@ further refined as to whether they use RLE v1 or v2.</p>
+
+
+
+
+
+
+
+
+
+
+
+
<li class=""><a href="/docs/run-length.html">Run Length Encoding</a></li>
@@ -957,6 +1820,18 @@ further refined as to whether they use RLE v1 or v2.</p>
+
+
+
+
+
+
+
+
+
+
+
+
<li class="current"><a href="/docs/stripes.html">Stripes</a></li>
@@ -973,6 +1848,12 @@ further refined as to whether they use RLE v1 or v2.</p>
+
+
+
+
+
+
<li class=""><a href="/docs/encodings.html">Column Encodings</a></li>
@@ -1003,6 +1884,18 @@ further refined as to whether they use RLE v1 or v2.</p>
+
+
+
+
+
+
+
+
+
+
+
+
<li class=""><a href="/docs/spec-index.html">Indexes</a></li>
http://git-wip-us.apache.org/repos/asf/orc/blob/1eb37b72/docs/tools.html
----------------------------------------------------------------------
diff --git a/docs/tools.html b/docs/tools.html
new file mode 100644
index 0000000..fdd3c08
--- /dev/null
+++ b/docs/tools.html
@@ -0,0 +1,2111 @@
+<!DOCTYPE HTML>
+<html lang="en-US">
+<head>
+ <meta charset="UTF-8">
+ <title>Tools</title>
+ <meta name="viewport" content="width=device-width,initial-scale=1">
+ <meta name="generator" content="Jekyll v2.4.0">
+ <link rel="stylesheet" href="//fonts.googleapis.com/css?family=Lato:300,300italic,400,400italic,700,700italic,900">
+ <link rel="stylesheet" href="/css/screen.css">
+ <link rel="icon" type="image/x-icon" href="/favicon.ico">
+ <!--[if lt IE 9]>
+ <script src="/js/html5shiv.min.js"></script>
+ <script src="/js/respond.min.js"></script>
+ <![endif]-->
+</head>
+
+
+<body class="wrap">
+ <header role="banner">
+ <nav class="mobile-nav show-on-mobiles">
+ <ul>
+ <li class="">
+ <a href="/">Home</a>
+ </li>
+ <li class="current">
+ <a href="/docs/">Doc<span class="show-on-mobiles">s</span>
+ <span class="hide-on-mobiles">umentation</span></a>
+ </li>
+ <li class="">
+ <a href="/talks/">Talks</a>
+ </li>
+ <li class="">
+ <a href="/news/">News</a>
+ </li>
+ <li class="">
+ <a href="/help/">Help</a>
+ </li>
+ <li class="">
+ <a href="/develop/">Develop</a>
+ </li>
+</ul>
+
+ </nav>
+ <div class="grid">
+ <div class="unit one-third center-on-mobiles">
+ <h1>
+ <a href="/">
+ <span class="sr-only">Apache ORC</span>
+ <img src="/img/logo.png" width="249" height="101" alt="ORC Logo">
+ </a>
+ </h1>
+ </div>
+ <nav class="main-nav unit two-thirds hide-on-mobiles">
+ <ul>
+ <li class="">
+ <a href="/">Home</a>
+ </li>
+ <li class="current">
+ <a href="/docs/">Doc<span class="show-on-mobiles">s</span>
+ <span class="hide-on-mobiles">umentation</span></a>
+ </li>
+ <li class="">
+ <a href="/talks/">Talks</a>
+ </li>
+ <li class="">
+ <a href="/news/">News</a>
+ </li>
+ <li class="">
+ <a href="/help/">Help</a>
+ </li>
+ <li class="">
+ <a href="/develop/">Develop</a>
+ </li>
+</ul>
+
+ </nav>
+ </div>
+</header>
+
+
+ <section class="docs">
+ <div class="grid">
+
+ <div class="docs-nav-mobile unit whole show-on-mobiles">
+ <select onchange="if (this.value) window.location.href=this.value">
+ <option value="">Navigate the docs\u2026</option>
+
+ <optgroup label="Overview">
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <option value="/docs/index.html">Background</option>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <option value="/docs/adopters.html">ORC Adopters</option>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <option value="/docs/types.html">Types</option>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <option value="/docs/indexes.html">Indexes</option>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <option value="/docs/acid.html">ACID support</option>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ </optgroup>
+
+ <optgroup label="Installing">
+
+
+
+
+
+
+
+
+
+
+
+ <option value="/docs/building.html">Building ORC</option>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <option value="/docs/releases.html">Releases</option>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ </optgroup>
+
+ <optgroup label="Using in Hive">
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <option value="/docs/hive-ddl.html">Hive DDL</option>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <option value="/docs/hive-config.html">Hive Configuration</option>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ </optgroup>
+
+ <optgroup label="Using in MapReduce">
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <option value="/docs/mapred.html">Using in MapRed</option>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <option value="/docs/mapreduce.html">Using in MapReduce</option>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ </optgroup>
+
+ <optgroup label="Using ORC Core">
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <option value="/docs/core-java.html">Using Core Java</option>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ </optgroup>
+
+ <optgroup label="Tools">
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <option value="/docs/tools.html">Tools</option>
+
+
+
+
+
+
+ </optgroup>
+
+ <optgroup label="Format Specification">
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <option value="/docs/spec-intro.html">Introduction</option>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <option value="/docs/file-tail.html">File Tail</option>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <option value="/docs/compression.html">Compression</option>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <option value="/docs/run-length.html">Run Length Encoding</option>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <option value="/docs/stripes.html">Stripes</option>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <option value="/docs/encodings.html">Column Encodings</option>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <option value="/docs/spec-index.html">Indexes</option>
+
+
+
+
+
+
+
+
+
+
+
+
+ </optgroup>
+
+ </select>
+</div>
+
+
+ <div class="unit four-fifths">
+ <article>
+ <h1>Tools</h1>
+ <h2 id="orc-contents">orc-contents</h2>
+
+<p>Displays the contents of the ORC file as a JSON document. With the
+<code>columns</code> argument only the selected columns are printed.</p>
+
+<pre><code class="language-shell">% orc-contents [--columns=1,2,...] <filename>
+</code></pre>
+
+<p>If you run it on the example file TestOrcFile.test1.orc, you\u2019ll see (without
+the line breaks within each record):</p>
+
+<pre><code class="language-shell">% orc-contents examples/TestOrcFile.test1.orc
+{"boolean1": false, "byte1": 1, "short1": 1024, "int1": 65536, \\
+ "long1": 9223372036854775807, "float1": 1, "double1": -15, \\
+ "bytes1": [0, 1, 2, 3, 4], "string1": "hi", "middle": \\
+ {"list": [{"int1": 1, "string1": "bye"}, \\
+ {"int1": 2, "string1": "sigh"}]}, \\
+ "list": [{"int1": 3, "string1": "good"}, \\
+ {"int1": 4, "string1": "bad"}], \\
+ "map": []}
+{"boolean1": true, "byte1": 100, "short1": 2048, "int1": 65536,
+ "long1": 9223372036854775807, "float1": 2, "double1": -5, \\
+ "bytes1": [], "string1": "bye", \\
+ "middle": {"list": [{"int1": 1, "string1": "bye"}, \\
+ {"int1": 2, "string1": "sigh"}]}, \\
+ "list": [{"int1": 100000000, "string1": "cat"}, \\
+ {"int1": -100000, "string1": "in"}, \\
+ {"int1": 1234, "string1": "hat"}], \\
+ "map": [{"key": "chani", "value": {"int1": 5, "string1": "chani"}}, \\
+ {"key": "mauddib", \\
+ "value": {"int1": 1, "string1": "mauddib"}}]}
+</code></pre>
+
+<h2 id="orc-metadata">orc-metadata</h2>
+
+<p>Displays the metadata of the ORC file as a JSON document. With the
+<code>verbose</code> argument additional information about the layout of the file
+is also printed.</p>
+
+<pre><code class="language-shell">% orc-metadata [-v] <filename>
+</code></pre>
+
+<p>If you run it on the example file TestOrcFile.test1.orc, you\u2019ll see:</p>
+
+<pre><code class="language-shell">% orc-metadata examples/TestOrcFile.test1.orc
+{ "name": "../examples/TestOrcFile.test1.orc",
+ "type": "struct<boolean1:boolean,byte1:tinyint,short1:smallint,
+int1:int,long1:bigint,float1:float,double1:double,bytes1:binary,
+string1:string,middle:struct<list:array<struct<int1:int,string1:
+string>>>,list:array<struct<int1:int,string1:string>>,map:map<
+string,struct<int1:int,string1:string>>>",
+ "rows": 2,
+ "stripe count": 1,
+ "format": "0.12", "writer version": "HIVE-8732",
+ "compression": "zlib", "compression block": 10000,
+ "file length": 1711,
+ "content": 1015, "stripe stats": 250, "footer": 421, "postscript": 24,
+ "row index stride": 10000,
+ "user metadata": {
+ },
+ "stripes": [
+ { "stripe": 0, "rows": 2,
+ "offset": 3, "length": 1012,
+ "index": 570, "data": 243, "footer": 199
+ }
+ ]
+}
+</code></pre>
+
+<h2 id="file-dump">file-dump</h2>
+
+<p>The org.apache.orc.tools.FileDump Java class, which is available via Hive as:</p>
+
+<pre><code class="language-shell">% hive --orcfiledump [-j] [-p] [-d] [-t] [--rowindex <cols>]
+ [--recover] [--skip-dump] [--backup-path <new path>] <file>
+</code></pre>
+
+<dl>
+ <dt>-j</dt>
+ <dd>format the output in JSON</dd>
+ <dt>-p</dt>
+ <dd>pretty print the output</dd>
+ <dt>-d</dt>
+ <dd>print the data instead of the metadata</dd>
+ <dt>-t</dt>
+ <dd>print the timezone</dd>
+ <dt>\u2013rowindex</dt>
+ <dd>print the row indexes for the comma separated list of column ids</dd>
+ <dt>\u2013recover</dt>
+ <dd>skip over corrupted values in the ORC file</dd>
+ <dt>\u2013skip-dump</dt>
+ <dd>skip dumping the metadata</dd>
+ <dt>\u2013backup-path</dt>
+ <dd>a new path to write the recovered file to</dd>
+</dl>
+
+<p>An example of the output is given below:</p>
+
+<pre><code class="language-shell">% hive --orcfiledump examples/TestOrcFile.test1.orc
+Processing data file examples/TestOrcFile.test1.orc [length: 1711]
+Structure for examples/TestOrcFile.test1.orc
+File Version: 0.12 with HIVE_8732
+Rows: 2
+Compression: ZLIB
+Compression size: 10000
+Type: struct<boolean1:boolean,byte1:tinyint,short1:smallint,int1:int,
+long1:bigint,float1:float,double1:double,bytes1:binary,string1:string,
+middle:struct<list:array<struct<int1:int,string1:string>>>,list:array<
+struct<int1:int,string1:string>>,map:map<string,struct<int1:int,string1:
+string>>>
+
+Stripe Statistics:
+ Stripe 1:
+ Column 0: count: 2 hasNull: false
+ Column 1: count: 2 hasNull: false true: 1
+ Column 2: count: 2 hasNull: false min: 1 max: 100 sum: 101
+ Column 3: count: 2 hasNull: false min: 1024 max: 2048 sum: 3072
+ Column 4: count: 2 hasNull: false min: 65536 max: 65536 sum: 131072
+ Column 5: count: 2 hasNull: false min: 9223372036854775807 max: 9223372036854775807
+ Column 6: count: 2 hasNull: false min: 1.0 max: 2.0 sum: 3.0
+ Column 7: count: 2 hasNull: false min: -15.0 max: -5.0 sum: -20.0
+ Column 8: count: 2 hasNull: false sum: 5
+ Column 9: count: 2 hasNull: false min: bye max: hi sum: 5
+ Column 10: count: 2 hasNull: false
+ Column 11: count: 2 hasNull: false
+ Column 12: count: 4 hasNull: false
+ Column 13: count: 4 hasNull: false min: 1 max: 2 sum: 6
+ Column 14: count: 4 hasNull: false min: bye max: sigh sum: 14
+ Column 15: count: 2 hasNull: false
+ Column 16: count: 5 hasNull: false
+ Column 17: count: 5 hasNull: false min: -100000 max: 100000000 sum: 99901241
+ Column 18: count: 5 hasNull: false min: bad max: in sum: 15
+ Column 19: count: 2 hasNull: false
+ Column 20: count: 2 hasNull: false min: chani max: mauddib sum: 12
+ Column 21: count: 2 hasNull: false
+ Column 22: count: 2 hasNull: false min: 1 max: 5 sum: 6
+ Column 23: count: 2 hasNull: false min: chani max: mauddib sum: 12
+
+File Statistics:
+ Column 0: count: 2 hasNull: false
+ Column 1: count: 2 hasNull: false true: 1
+ Column 2: count: 2 hasNull: false min: 1 max: 100 sum: 101
+ Column 3: count: 2 hasNull: false min: 1024 max: 2048 sum: 3072
+ Column 4: count: 2 hasNull: false min: 65536 max: 65536 sum: 131072
+ Column 5: count: 2 hasNull: false min: 9223372036854775807 max: 9223372036854775807
+ Column 6: count: 2 hasNull: false min: 1.0 max: 2.0 sum: 3.0
+ Column 7: count: 2 hasNull: false min: -15.0 max: -5.0 sum: -20.0
+ Column 8: count: 2 hasNull: false sum: 5
+ Column 9: count: 2 hasNull: false min: bye max: hi sum: 5
+ Column 10: count: 2 hasNull: false
+ Column 11: count: 2 hasNull: false
+ Column 12: count: 4 hasNull: false
+ Column 13: count: 4 hasNull: false min: 1 max: 2 sum: 6
+ Column 14: count: 4 hasNull: false min: bye max: sigh sum: 14
+ Column 15: count: 2 hasNull: false
+ Column 16: count: 5 hasNull: false
+ Column 17: count: 5 hasNull: false min: -100000 max: 100000000 sum: 99901241
+ Column 18: count: 5 hasNull: false min: bad max: in sum: 15
+ Column 19: count: 2 hasNull: false
+ Column 20: count: 2 hasNull: false min: chani max: mauddib sum: 12
+ Column 21: count: 2 hasNull: false
+ Column 22: count: 2 hasNull: false min: 1 max: 5 sum: 6
+ Column 23: count: 2 hasNull: false min: chani max: mauddib sum: 12
+
+Stripes:
+ Stripe: offset: 3 data: 243 rows: 2 tail: 199 index: 570
+ Stream: column 0 section ROW_INDEX start: 3 length 11
+ Stream: column 1 section ROW_INDEX start: 14 length 22
+ Stream: column 2 section ROW_INDEX start: 36 length 26
+ Stream: column 3 section ROW_INDEX start: 62 length 27
+ Stream: column 4 section ROW_INDEX start: 89 length 30
+ Stream: column 5 section ROW_INDEX start: 119 length 28
+ Stream: column 6 section ROW_INDEX start: 147 length 34
+ Stream: column 7 section ROW_INDEX start: 181 length 34
+ Stream: column 8 section ROW_INDEX start: 215 length 21
+ Stream: column 9 section ROW_INDEX start: 236 length 30
+ Stream: column 10 section ROW_INDEX start: 266 length 11
+ Stream: column 11 section ROW_INDEX start: 277 length 16
+ Stream: column 12 section ROW_INDEX start: 293 length 11
+ Stream: column 13 section ROW_INDEX start: 304 length 24
+ Stream: column 14 section ROW_INDEX start: 328 length 31
+ Stream: column 15 section ROW_INDEX start: 359 length 16
+ Stream: column 16 section ROW_INDEX start: 375 length 11
+ Stream: column 17 section ROW_INDEX start: 386 length 32
+ Stream: column 18 section ROW_INDEX start: 418 length 30
+ Stream: column 19 section ROW_INDEX start: 448 length 16
+ Stream: column 20 section ROW_INDEX start: 464 length 37
+ Stream: column 21 section ROW_INDEX start: 501 length 11
+ Stream: column 22 section ROW_INDEX start: 512 length 24
+ Stream: column 23 section ROW_INDEX start: 536 length 37
+ Stream: column 1 section DATA start: 573 length 5
+ Stream: column 2 section DATA start: 578 length 6
+ Stream: column 3 section DATA start: 584 length 9
+ Stream: column 4 section DATA start: 593 length 11
+ Stream: column 5 section DATA start: 604 length 12
+ Stream: column 6 section DATA start: 616 length 11
+ Stream: column 7 section DATA start: 627 length 15
+ Stream: column 8 section DATA start: 642 length 8
+ Stream: column 8 section LENGTH start: 650 length 6
+ Stream: column 9 section DATA start: 656 length 8
+ Stream: column 9 section LENGTH start: 664 length 6
+ Stream: column 11 section LENGTH start: 670 length 6
+ Stream: column 13 section DATA start: 676 length 7
+ Stream: column 14 section DATA start: 683 length 6
+ Stream: column 14 section LENGTH start: 689 length 6
+ Stream: column 14 section DICTIONARY_DATA start: 695 length 10
+ Stream: column 15 section LENGTH start: 705 length 6
+ Stream: column 17 section DATA start: 711 length 25
+ Stream: column 18 section DATA start: 736 length 18
+ Stream: column 18 section LENGTH start: 754 length 8
+ Stream: column 19 section LENGTH start: 762 length 6
+ Stream: column 20 section DATA start: 768 length 15
+ Stream: column 20 section LENGTH start: 783 length 6
+ Stream: column 22 section DATA start: 789 length 6
+ Stream: column 23 section DATA start: 795 length 15
+ Stream: column 23 section LENGTH start: 810 length 6
+ Encoding column 0: DIRECT
+ Encoding column 1: DIRECT
+ Encoding column 2: DIRECT
+ Encoding column 3: DIRECT_V2
+ Encoding column 4: DIRECT_V2
+ Encoding column 5: DIRECT_V2
+ Encoding column 6: DIRECT
+ Encoding column 7: DIRECT
+ Encoding column 8: DIRECT_V2
+ Encoding column 9: DIRECT_V2
+ Encoding column 10: DIRECT
+ Encoding column 11: DIRECT_V2
+ Encoding column 12: DIRECT
+ Encoding column 13: DIRECT_V2
+ Encoding column 14: DICTIONARY_V2[2]
+ Encoding column 15: DIRECT_V2
+ Encoding column 16: DIRECT
+ Encoding column 17: DIRECT_V2
+ Encoding column 18: DIRECT_V2
+ Encoding column 19: DIRECT_V2
+ Encoding column 20: DIRECT_V2
+ Encoding column 21: DIRECT
+ Encoding column 22: DIRECT_V2
+ Encoding column 23: DIRECT_V2
+
+File length: 1711 bytes
+Padding length: 0 bytes
+Padding ratio: 0%
+______________________________________________________________________
+</code></pre>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <div class="section-nav">
+ <div class="left align-right">
+
+
+
+ <a href="/docs/core-java.html" class="prev">Back</a>
+
+ </div>
+ <div class="right align-left">
+
+
+
+ <a href="/docs/spec-intro.html" class="next">Next</a>
+
+ </div>
+ </div>
+ <div class="clear"></div>
+
+
+ </article>
+ </div>
+
+ <div class="unit one-fifth hide-on-mobiles">
+ <aside>
+
+ <h4>Overview</h4>
+
+
+<ul>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <li class=""><a href="/docs/index.html">Background</a></li>
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <li class=""><a href="/docs/adopters.html">ORC Adopters</a></li>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <li class=""><a href="/docs/types.html">Types</a></li>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <li class=""><a href="/docs/indexes.html">Indexes</a></li>
+
+
+
+
+
+
+
+
+
+
+
+ <li class=""><a href="/docs/acid.html">ACID support</a></li>
+
+
+
+</ul>
+
+
+ <h4>Installing</h4>
+
+
+<ul>
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <li class=""><a href="/docs/building.html">Building ORC</a></li>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <li class=""><a href="/docs/releases.html">Releases</a></li>
+
+
+
+</ul>
+
+
+ <h4>Using in Hive</h4>
+
+
+<ul>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <li class=""><a href="/docs/hive-ddl.html">Hive DDL</a></li>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <li class=""><a href="/docs/hive-config.html">Hive Configuration</a></li>
+
+
+
+</ul>
+
+
+ <h4>Using in MapReduce</h4>
+
+
+<ul>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <li class=""><a href="/docs/mapred.html">Using in MapRed</a></li>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <li class=""><a href="/docs/mapreduce.html">Using in MapReduce</a></li>
+
+
+
+</ul>
+
+
+ <h4>Using ORC Core</h4>
+
+
+<ul>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <li class=""><a href="/docs/core-java.html">Using Core Java</a></li>
+
+
+
+</ul>
+
+
+ <h4>Tools</h4>
+
+
+<ul>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <li class="current"><a href="/docs/tools.html">Tools</a></li>
+
+
+
+</ul>
+
+
+ <h4>Format Specification</h4>
+
+
+<ul>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <li class=""><a href="/docs/spec-intro.html">Introduction</a></li>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <li class=""><a href="/docs/file-tail.html">File Tail</a></li>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <li class=""><a href="/docs/compression.html">Compression</a></li>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <li class=""><a href="/docs/run-length.html">Run Length Encoding</a></li>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <li class=""><a href="/docs/stripes.html">Stripes</a></li>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <li class=""><a href="/docs/encodings.html">Column Encodings</a></li>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <li class=""><a href="/docs/spec-index.html">Indexes</a></li>
+
+
+
+</ul>
+
+
+ </aside>
+</div>
+
+
+ <div class="clear"></div>
+
+ </div>
+ </section>
+
+
+ <footer role="contentinfo">
+ <p>The contents of this website are © 2016
+ <a href="https://www.apache.org/">Apache Software Foundation</a>
+ under the terms of the <a
+ href="https://www.apache.org/licenses/LICENSE-2.0.html">
+ Apache License v2</a>. Apache ORC and its logo are trademarks
+ of the Apache Software Foundation.</p>
+</footer>
+
+ <script>
+ var anchorForId = function (id) {
+ var anchor = document.createElement("a");
+ anchor.className = "header-link";
+ anchor.href = "#" + id;
+ anchor.innerHTML = "<span class=\"sr-only\">Permalink</span><i class=\"fa fa-link\"></i>";
+ anchor.title = "Permalink";
+ return anchor;
+ };
+
+ var linkifyAnchors = function (level, containingElement) {
+ var headers = containingElement.getElementsByTagName("h" + level);
+ for (var h = 0; h < headers.length; h++) {
+ var header = headers[h];
+
+ if (typeof header.id !== "undefined" && header.id !== "") {
+ header.appendChild(anchorForId(header.id));
+ }
+ }
+ };
+
+ document.onreadystatechange = function () {
+ if (this.readyState === "complete") {
+ var contentBlock = document.getElementsByClassName("docs")[0] || document.getElementsByClassName("news")[0];
+ if (!contentBlock) {
+ return;
+ }
+ for (var level = 1; level <= 6; level++) {
+ linkifyAnchors(level, contentBlock);
+ }
+ }
+ };
+</script>
+
+
+</body>
+</html>
http://git-wip-us.apache.org/repos/asf/orc/blob/1eb37b72/docs/types.html
----------------------------------------------------------------------
diff --git a/docs/types.html b/docs/types.html
index b7eaab8..d0fbca2 100644
--- a/docs/types.html
+++ b/docs/types.html
@@ -105,25 +105,19 @@
- <option value="/docs/index.html">Background</option>
-
-
+ <option value="/docs/index.html">Background</option>
-
-
-
-
@@ -140,21 +134,23 @@
-
+
-
+
+ <option value="/docs/adopters.html">ORC Adopters</option>
+
- <option value="/docs/types.html">Types</option>
+
-
+
-
+
@@ -171,7 +167,7 @@
- <option value="/docs/indexes.html">Indexes</option>
+
@@ -189,8 +185,6 @@
- <option value="/docs/acid.html">ACID support</option>
-
@@ -216,16 +210,7 @@
-
-
- </optgroup>
- <optgroup label="Hive Usage">
-
-
-
-
-
@@ -238,12 +223,12 @@
- <option value="/docs/hive-ddl.html">Hive DDL</option>
+ <option value="/docs/types.html">Types</option>
-
+
-
+
@@ -255,9 +240,7 @@
-
-
-
+
@@ -268,7 +251,7 @@
- <option value="/docs/hive-config.html">Hive Configuration</option>
+ <option value="/docs/indexes.html">Indexes</option>
@@ -287,19 +270,14 @@
-
-
- </optgroup>
- <optgroup label="Format Specification">
-
-
+
-
+ <option value="/docs/acid.html">ACID support</option>
@@ -319,17 +297,11 @@
- <option value="/docs/spec-intro.html">Introduction</option>
-
-
-
-
-
@@ -337,8 +309,6 @@
- <option value="/docs/file-tail.html">File Tail</option>
-
@@ -348,24 +318,35 @@
+
+
+ </optgroup>
+ <optgroup label="Installing">
+
+
+
+
+
+ <option value="/docs/building.html">Building ORC</option>
+
-
+
-
+
- <option value="/docs/compression.html">Compression</option>
+
@@ -411,8 +392,6 @@
- <option value="/docs/run-length.html">Run Length Encoding</option>
-
@@ -422,12 +401,8 @@
-
-
-
-
-
+ <option value="/docs/releases.html">Releases</option>
@@ -442,28 +417,37 @@
+
+
+ </optgroup>
+ <optgroup label="Using in Hive">
+
+
+
-
+
- <option value="/docs/stripes.html">Stripes</option>
+
-
+
-
+
- <option value="/docs/encodings.html">Column Encodings</option>
+ <option value="/docs/hive-ddl.html">Hive DDL</option>
+
+
@@ -505,11 +489,13 @@
+ <option value="/docs/hive-config.html">Hive Configuration</option>
+
- <option value="/docs/spec-index.html">Indexes</option>
+
@@ -518,143 +504,31 @@
-
-
- </optgroup>
- </select>
-</div>
-
-
- <div class="unit four-fifths">
- <article>
- <h1>Types</h1>
- <p>ORC files are completely self-describing and do not depend on the Hive
-Metastore or any other external metadata. The file includes all of the
-type and encoding information for the objects stored in the file. Because the
-file is self-contained, it does not depend on the user\u2019s environment to
-correctly interpret the file\u2019s contents.</p>
-
-<p>ORC provides a rich set of scalar and compound types:</p>
-
-<ul>
- <li>Integer
- <ul>
- <li>boolean (1 bit)</li>
- <li>tinyint (8 bit)</li>
- <li>smallint (16 bit)</li>
- <li>int (32 bit)</li>
- <li>bigint (64 bit)</li>
- </ul>
- </li>
- <li>Floating point
- <ul>
- <li>float</li>
- <li>double</li>
- </ul>
- </li>
- <li>String types
- <ul>
- <li>string</li>
- <li>char</li>
- <li>varchar</li>
- </ul>
- </li>
- <li>Binary blobs
- <ul>
- <li>binary</li>
- </ul>
- </li>
- <li>Date/time
- <ul>
- <li>timestamp</li>
- <li>date</li>
- </ul>
- </li>
- <li>Compound types
- <ul>
- <li>struct</li>
- <li>list</li>
- <li>map</li>
- <li>union</li>
- </ul>
- </li>
-</ul>
-
-<p>All ORC file are logically sequences of identically typed objects. Hive
-always uses a struct with a field for each of the top-level columns as
-the root object type, but that is not required. All types in ORC can take
-null values including the compound types.</p>
-
-<p>Compound types have children columns that hold the values for their
-sub-elements. For example, a struct column has one child column for
-each field of the struct. Lists always have a single child column for
-the element values and maps always have two child columns. Union
-columns have one child column for each of the variants.</p>
-
-<p>Given the following definition of the table Foobar, the columns in the
-file would form the given tree.</p>
-
-<p><code>create table Foobar (
- myInt int,
- myMap map<string,
- struct<myString : string,
- myDouble: double>>,
- myTime timestamp
-);
-</code></p>
-
-<p><img src="/img/TreeWriters.png" alt="ORC column structure" /></p>
-
-
-
-
-
-
-
-
+
-
+
+
+
+
- <div class="section-nav">
- <div class="left align-right">
-
-
-
- <a href="/docs/index.html" class="prev">Back</a>
-
- </div>
- <div class="right align-left">
-
-
-
- <a href="/docs/indexes.html" class="next">Next</a>
-
- </div>
- </div>
- <div class="clear"></div>
+
- </article>
- </div>
- <div class="unit one-fifth hide-on-mobiles">
- <aside>
-
- <h4>Overview</h4>
+ </optgroup>
+ <optgroup label="Using in MapReduce">
+
-<ul>
-
-
@@ -668,23 +542,15 @@ file would form the given tree.</p>
- <li class=""><a href="/docs/index.html">Background</a></li>
-
-
-
-
-
-
-
-
+ <option value="/docs/mapred.html">Using in MapRed</option>
@@ -703,17 +569,13 @@ file would form the given tree.</p>
-
- <li class="current"><a href="/docs/types.html">Types</a></li>
-
-
-
+
@@ -730,16 +592,729 @@ file would form the given tree.</p>
- <li class=""><a href="/docs/indexes.html">Indexes</a></li>
-
-
-
-
+
-
+
+ <option value="/docs/mapreduce.html">Using in MapReduce</option>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ </optgroup>
+
+ <optgroup label="Using ORC Core">
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <option value="/docs/core-java.html">Using Core Java</option>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ </optgroup>
+
+ <optgroup label="Tools">
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <option value="/docs/tools.html">Tools</option>
+
+
+
+
+
+
+ </optgroup>
+
+ <optgroup label="Format Specification">
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <option value="/docs/spec-intro.html">Introduction</option>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <option value="/docs/file-tail.html">File Tail</option>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <option value="/docs/compression.html">Compression</option>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <option value="/docs/run-length.html">Run Length Encoding</option>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <option value="/docs/stripes.html">Stripes</option>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <option value="/docs/encodings.html">Column Encodings</option>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <option value="/docs/spec-index.html">Indexes</option>
+
+
+
+
+
+
+
+
+
+
+
+
+ </optgroup>
+
+ </select>
+</div>
+
+
+ <div class="unit four-fifths">
+ <article>
+ <h1>Types</h1>
+ <p>ORC files are completely self-describing and do not depend on the Hive
+Metastore or any other external metadata. The file includes all of the
+type and encoding information for the objects stored in the file. Because the
+file is self-contained, it does not depend on the user\u2019s environment to
+correctly interpret the file\u2019s contents.</p>
+
+<p>ORC provides a rich set of scalar and compound types:</p>
+
+<ul>
+ <li>Integer
+ <ul>
+ <li>boolean (1 bit)</li>
+ <li>tinyint (8 bit)</li>
+ <li>smallint (16 bit)</li>
+ <li>int (32 bit)</li>
+ <li>bigint (64 bit)</li>
+ </ul>
+ </li>
+ <li>Floating point
+ <ul>
+ <li>float</li>
+ <li>double</li>
+ </ul>
+ </li>
+ <li>String types
+ <ul>
+ <li>string</li>
+ <li>char</li>
+ <li>varchar</li>
+ </ul>
+ </li>
+ <li>Binary blobs
+ <ul>
+ <li>binary</li>
+ </ul>
+ </li>
+ <li>Date/time
+ <ul>
+ <li>timestamp</li>
+ <li>date</li>
+ </ul>
+ </li>
+ <li>Compound types
+ <ul>
+ <li>struct</li>
+ <li>list</li>
+ <li>map</li>
+ <li>union</li>
+ </ul>
+ </li>
+</ul>
+
+<p>All ORC file are logically sequences of identically typed objects. Hive
+always uses a struct with a field for each of the top-level columns as
+the root object type, but that is not required. All types in ORC can take
+null values including the compound types.</p>
+
+<p>Compound types have children columns that hold the values for their
+sub-elements. For example, a struct column has one child column for
+each field of the struct. Lists always have a single child column for
+the element values and maps always have two child columns. Union
+columns have one child column for each of the variants.</p>
+
+<p>Given the following definition of the table Foobar, the columns in the
+file would form the given tree.</p>
+
+<p><code>create table Foobar (
+ myInt int,
+ myMap map<string,
+ struct<myString : string,
+ myDouble: double>>,
+ myTime timestamp
+);
+</code></p>
+
+<p><img src="/img/TreeWriters.png" alt="ORC column structure" /></p>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <div class="section-nav">
+ <div class="left align-right">
+
+
+
+ <a href="/docs/adopters.html" class="prev">Back</a>
+
+ </div>
+ <div class="right align-left">
+
+
+
+ <a href="/docs/indexes.html" class="next">Next</a>
+
+ </div>
+ </div>
+ <div class="clear"></div>
+
+
+ </article>
+ </div>
+
+ <div class="unit one-fifth hide-on-mobiles">
+ <aside>
+
+ <h4>Overview</h4>
+
+
+<ul>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <li class=""><a href="/docs/index.html">Background</a></li>
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <li class=""><a href="/docs/adopters.html">ORC Adopters</a></li>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <li class="current"><a href="/docs/types.html">Types</a></li>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <li class=""><a href="/docs/indexes.html">Indexes</a></li>
+
+
+
+
+
+
+
+
+
<li class=""><a href="/docs/acid.html">ACID support</a></li>
@@ -749,7 +1324,69 @@ file would form the given tree.</p>
</ul>
- <h4>Hive Usage</h4>
+ <h4>Installing</h4>
+
+
+<ul>
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <li class=""><a href="/docs/building.html">Building ORC</a></li>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <li class=""><a href="/docs/releases.html">Releases</a></li>
+
+
+
+</ul>
+
+
+ <h4>Using in Hive</h4>
<ul>
@@ -772,6 +1409,12 @@ file would form the given tree.</p>
+
+
+
+
+
+
<li class=""><a href="/docs/hive-ddl.html">Hive DDL</a></li>
@@ -792,6 +1435,12 @@ file would form the given tree.</p>
+
+
+
+
+
+
<li class=""><a href="/docs/hive-config.html">Hive Configuration</a></li>
@@ -799,6 +1448,168 @@ file would form the given tree.</p>
</ul>
+ <h4>Using in MapReduce</h4>
+
+
+<ul>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <li class=""><a href="/docs/mapred.html">Using in MapRed</a></li>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <li class=""><a href="/docs/mapreduce.html">Using in MapReduce</a></li>
+
+
+
+</ul>
+
+
+ <h4>Using ORC Core</h4>
+
+
+<ul>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <li class=""><a href="/docs/core-java.html">Using Core Java</a></li>
+
+
+
+</ul>
+
+
+ <h4>Tools</h4>
+
+
+<ul>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <li class=""><a href="/docs/tools.html">Tools</a></li>
+
+
+
+</ul>
+
+
<h4>Format Specification</h4>
@@ -832,6 +1643,18 @@ file would form the given tree.</p>
+
+
+
+
+
+
+
+
+
+
+
+
<li class=""><a href="/docs/spec-intro.html">Introduction</a></li>
@@ -850,6 +1673,12 @@ file would form the given tree.</p>
+
+
+
+
+
+
<li class=""><a href="/docs/file-tail.html">File Tail</a></li>
@@ -864,6 +1693,10 @@ file would form the given tree.</p>
+
+
+
+
<li class=""><a href="/docs/compression.html">Compression</a></li>
@@ -892,6 +1725,18 @@ file would form the given tree.</p>
+
+
+
+
+
+
+
+
+
+
+
+
<li class=""><a href="/docs/run-length.html">Run Length Encoding</a></li>
@@ -926,6 +1771,18 @@ file would form the given tree.</p>
+
+
+
+
+
+
+
+
+
+
+
+
<li class=""><a href="/docs/stripes.html">Stripes</a></li>
@@ -942,6 +1799,12 @@ file would form the given tree.</p>
+
+
+
+
+
+
<li class=""><a href="/docs/encodings.html">Column Encodings</a></li>
@@ -972,6 +1835,18 @@ file would form the given tree.</p>
+
+
+
+
+
+
+
+
+
+
+
+
<li class=""><a href="/docs/spec-index.html">Indexes</a></li>
http://git-wip-us.apache.org/repos/asf/orc/blob/1eb37b72/downloads/index.html
----------------------------------------------------------------------
diff --git a/downloads/index.html b/downloads/index.html
deleted file mode 100644
index 04040a9..0000000
--- a/downloads/index.html
+++ /dev/null
@@ -1,144 +0,0 @@
-<!DOCTYPE HTML>
-<html lang="en-US">
-<head>
- <meta charset="UTF-8">
- <title>Downloads</title>
- <meta name="viewport" content="width=device-width,initial-scale=1">
- <meta name="generator" content="Jekyll v2.4.0">
- <link rel="stylesheet" href="//fonts.googleapis.com/css?family=Lato:300,300italic,400,400italic,700,700italic,900">
- <link rel="stylesheet" href="/css/screen.css">
- <link rel="icon" type="image/x-icon" href="/favicon.ico">
- <!--[if lt IE 9]>
- <script src="/js/html5shiv.min.js"></script>
- <script src="/js/respond.min.js"></script>
- <![endif]-->
-</head>
-
-
-<body class="wrap">
- <header role="banner">
- <nav class="mobile-nav show-on-mobiles">
- <ul>
- <li class="">
- <a href="/">Home</a>
- </li>
- <li class="">
- <a href="/docs/">Doc<span class="show-on-mobiles">s</span>
- <span class="hide-on-mobiles">umentation</span></a>
- </li>
- <li class="">
- <a href="/talks/">Talks</a>
- </li>
- <li class="">
- <a href="/news/">News</a>
- </li>
- <li class="">
- <a href="/help/">Help</a>
- </li>
- <li class="">
- <a href="/develop/">Develop</a>
- </li>
-</ul>
-
- </nav>
- <div class="grid">
- <div class="unit one-third center-on-mobiles">
- <h1>
- <a href="/">
- <span class="sr-only">Apache ORC</span>
- <img src="/img/logo.png" width="249" height="101" alt="ORC Logo">
- </a>
- </h1>
- </div>
- <nav class="main-nav unit two-thirds hide-on-mobiles">
- <ul>
- <li class="">
- <a href="/">Home</a>
- </li>
- <li class="">
- <a href="/docs/">Doc<span class="show-on-mobiles">s</span>
- <span class="hide-on-mobiles">umentation</span></a>
- </li>
- <li class="">
- <a href="/talks/">Talks</a>
- </li>
- <li class="">
- <a href="/news/">News</a>
- </li>
- <li class="">
- <a href="/help/">Help</a>
- </li>
- <li class="">
- <a href="/develop/">Develop</a>
- </li>
-</ul>
-
- </nav>
- </div>
-</header>
-
-
- <section class="standalone">
- <div class="grid">
-
- <div class="unit whole">
- <article>
- <h1>Downloads</h1>
- <p>We haven\u2019t made any releases as a separate project yet. Please download
-the Hive 1.1 release and use the hive-exec.jar.</p>
-
- </article>
- </div>
-
- <div class="clear"></div>
-
- </div>
-</section>
-
-
- <footer role="contentinfo">
- <p>The contents of this website are © 2016
- <a href="https://www.apache.org/">Apache Software Foundation</a>
- under the terms of the <a
- href="https://www.apache.org/licenses/LICENSE-2.0.html">
- Apache License v2</a>. Apache ORC and its logo are trademarks
- of the Apache Software Foundation.</p>
-</footer>
-
- <script>
- var anchorForId = function (id) {
- var anchor = document.createElement("a");
- anchor.className = "header-link";
- anchor.href = "#" + id;
- anchor.innerHTML = "<span class=\"sr-only\">Permalink</span><i class=\"fa fa-link\"></i>";
- anchor.title = "Permalink";
- return anchor;
- };
-
- var linkifyAnchors = function (level, containingElement) {
- var headers = containingElement.getElementsByTagName("h" + level);
- for (var h = 0; h < headers.length; h++) {
- var header = headers[h];
-
- if (typeof header.id !== "undefined" && header.id !== "") {
- header.appendChild(anchorForId(header.id));
- }
- }
- };
-
- document.onreadystatechange = function () {
- if (this.readyState === "complete") {
- var contentBlock = document.getElementsByClassName("docs")[0] || document.getElementsByClassName("news")[0];
- if (!contentBlock) {
- return;
- }
- for (var level = 1; level <= 6; level++) {
- linkifyAnchors(level, contentBlock);
- }
- }
- };
-</script>
-
-
-</body>
-</html>
http://git-wip-us.apache.org/repos/asf/orc/blob/1eb37b72/img/asf_logo.png
----------------------------------------------------------------------
diff --git a/img/asf_logo.png b/img/asf_logo.png
new file mode 100644
index 0000000..b20bb7f
Binary files /dev/null and b/img/asf_logo.png differ
http://git-wip-us.apache.org/repos/asf/orc/blob/1eb37b72/index.html
----------------------------------------------------------------------
diff --git a/index.html b/index.html
index 25fcf43..85b2f49 100644
--- a/index.html
+++ b/index.html
@@ -109,6 +109,53 @@
<div class="clear"></div>
</div>
</section>
+<section class="quickstart">
+ <div class="grid">
+ <div class="unit golden-small center-on-mobiles">
+ <p class = "side">Download the <a href="docs/releases.html">ORC 1.1.0</a>
+ release.</p>
+ </div>
+ <div class="unit golden-large code">
+ <p class="title">Quickstart Documentation</p>
+ <ul class="shell">
+ <li><a href="docs/hive-ddl.html">Using with Hive</a></li>
+ <li><a href="docs/mapred.html">Using with Hadoop MapRed</a></li>
+ <li><a href="docs/mapreduce.html">Using with Hadoop MapReduce</a></li>
+ <li><a href="docs/core-java.html">Using Core ORC from Java</a></li>
+ </ul>
+ </div>
+ <div class="clear"></div>
+ </div>
+</section>
+<section class="apache">
+ <div class="grid">
+ <div class="unit whole">
+ <div class="grid pane">
+ <div class="unit whole center-on-mobiles">
+ <a href="http://www.apache.org">
+ <img src="img/asf_logo.png" width="429" height="175"
+ alt="Apache Software Foundation"></a>
+ <div class="pane-content">
+ <h2 class="center-on-mobiles">ORC is an <strong>Apache</strong>
+ project.</h2>
+ <p>Apache is a non-profit organization helping open-source
+ software projects released under the
+ <a href="http://www.apache.org/licenses/">Apache License</a>
+ and managed with
+ <a href="http://www.apache.org/foundation/how-it-works.html">
+ open governance</a>. We'd like to
+ <a href="http://www.apache.org/foundation/thanks.html">thank
+ </a> the Apache sponsors and encourage others to
+ <a href="http://www.apache.org/foundation/sponsorship.html">
+ donate</a>.
+ </p>
+ </div>
+ </div>
+ <div class="clear"></div>
+ </div>
+ </div>
+ </div>
+</section>
<footer role="contentinfo">
http://git-wip-us.apache.org/repos/asf/orc/blob/1eb37b72/news/2015/04/22/orc-as-tlp/index.html
----------------------------------------------------------------------
diff --git a/news/2015/04/22/orc-as-tlp/index.html b/news/2015/04/22/orc-as-tlp/index.html
index 34e268e..fe34f79 100644
--- a/news/2015/04/22/orc-as-tlp/index.html
+++ b/news/2015/04/22/orc-as-tlp/index.html
@@ -87,6 +87,8 @@
<option value="/news/">Home</option>
<optgroup label="v1.x">
+ <option value="/news/2016/06/10/ORC-1.1.0/">ORC 1.1.0 Released</option>
+
<option value="/news/2016/01/25/ORC-1.0.0/">ORC 1.0.0 Released</option>
<option value="/news/2015/11/19/aliaksei-on-pmc/">ORC adds Aliaksei Sandryhaila to PMC</option>
@@ -181,6 +183,10 @@ together with the Java implementation.</p>
<ul>
<li class="">
+ <a href="/news/2016/06/10/ORC-1.1.0/">Version 1.1.0</a>
+ </li>
+
+ <li class="">
<a href="/news/2016/01/25/ORC-1.0.0/">Version 1.0.0</a>
</li>
@@ -191,6 +197,8 @@ together with the Java implementation.</p>
+
+
<li class="">
<a href="/news/2015/11/19/aliaksei-on-pmc/">ORC adds Aliaksei Sandryhaila to PMC</a>
</li>
http://git-wip-us.apache.org/repos/asf/orc/blob/1eb37b72/news/2015/05/11/new-committers/index.html
----------------------------------------------------------------------
diff --git a/news/2015/05/11/new-committers/index.html b/news/2015/05/11/new-committers/index.html
index 131bb3a..3699e25 100644
--- a/news/2015/05/11/new-committers/index.html
+++ b/news/2015/05/11/new-committers/index.html
@@ -87,6 +87,8 @@
<option value="/news/">Home</option>
<optgroup label="v1.x">
+ <option value="/news/2016/06/10/ORC-1.1.0/">ORC 1.1.0 Released</option>
+
<option value="/news/2016/01/25/ORC-1.0.0/">ORC 1.0.0 Released</option>
<option value="/news/2015/11/19/aliaksei-on-pmc/">ORC adds Aliaksei Sandryhaila to PMC</option>
@@ -157,6 +159,10 @@ for their work on ORC. Welcome all!</p>
<ul>
<li class="">
+ <a href="/news/2016/06/10/ORC-1.1.0/">Version 1.1.0</a>
+ </li>
+
+ <li class="">
<a href="/news/2016/01/25/ORC-1.0.0/">Version 1.0.0</a>
</li>
@@ -167,6 +173,8 @@ for their work on ORC. Welcome all!</p>
+
+
<li class="">
<a href="/news/2015/11/19/aliaksei-on-pmc/">ORC adds Aliaksei Sandryhaila to PMC</a>
</li>
http://git-wip-us.apache.org/repos/asf/orc/blob/1eb37b72/news/2015/06/26/new-logo/index.html
----------------------------------------------------------------------
diff --git a/news/2015/06/26/new-logo/index.html b/news/2015/06/26/new-logo/index.html
index 9e35cbf..9de7f76 100644
--- a/news/2015/06/26/new-logo/index.html
+++ b/news/2015/06/26/new-logo/index.html
@@ -87,6 +87,8 @@
<option value="/news/">Home</option>
<optgroup label="v1.x">
+ <option value="/news/2016/06/10/ORC-1.1.0/">ORC 1.1.0 Released</option>
+
<option value="/news/2016/01/25/ORC-1.0.0/">ORC 1.0.0 Released</option>
<option value="/news/2015/11/19/aliaksei-on-pmc/">ORC adds Aliaksei Sandryhaila to PMC</option>
@@ -150,6 +152,10 @@
<ul>
<li class="">
+ <a href="/news/2016/06/10/ORC-1.1.0/">Version 1.1.0</a>
+ </li>
+
+ <li class="">
<a href="/news/2016/01/25/ORC-1.0.0/">Version 1.0.0</a>
</li>
@@ -160,6 +166,8 @@
+
+
<li class="">
<a href="/news/2015/11/19/aliaksei-on-pmc/">ORC adds Aliaksei Sandryhaila to PMC</a>
</li>
http://git-wip-us.apache.org/repos/asf/orc/blob/1eb37b72/news/2015/11/19/aliaksei-on-pmc/index.html
----------------------------------------------------------------------
diff --git a/news/2015/11/19/aliaksei-on-pmc/index.html b/news/2015/11/19/aliaksei-on-pmc/index.html
index 4060654..ba79637 100644
--- a/news/2015/11/19/aliaksei-on-pmc/index.html
+++ b/news/2015/11/19/aliaksei-on-pmc/index.html
@@ -87,6 +87,8 @@
<option value="/news/">Home</option>
<optgroup label="v1.x">
+ <option value="/news/2016/06/10/ORC-1.1.0/">ORC 1.1.0 Released</option>
+
<option value="/news/2016/01/25/ORC-1.0.0/">ORC 1.0.0 Released</option>
<option value="/news/2015/11/19/aliaksei-on-pmc/">ORC adds Aliaksei Sandryhaila to PMC</option>
@@ -153,6 +155,10 @@ more.</p>
<ul>
<li class="">
+ <a href="/news/2016/06/10/ORC-1.1.0/">Version 1.1.0</a>
+ </li>
+
+ <li class="">
<a href="/news/2016/01/25/ORC-1.0.0/">Version 1.0.0</a>
</li>
@@ -163,6 +169,8 @@ more.</p>
+
+
<li class="current">
<a href="/news/2015/11/19/aliaksei-on-pmc/">ORC adds Aliaksei Sandryhaila to PMC</a>
</li>