You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@orc.apache.org by om...@apache.org on 2018/05/17 22:12:55 UTC

[01/11] orc git commit: Deploy docs with CVE.

Repository: orc
Updated Branches:
  refs/heads/asf-site f5aaac5f0 -> e58d3fc43


http://git-wip-us.apache.org/repos/asf/orc/blob/e58d3fc4/specification/ORCv2/index.html
----------------------------------------------------------------------
diff --git a/specification/ORCv2/index.html b/specification/ORCv2/index.html
new file mode 100644
index 0000000..0c7a42f
--- /dev/null
+++ b/specification/ORCv2/index.html
@@ -0,0 +1,1339 @@
+<!DOCTYPE HTML>
+<html lang="en-US">
+<head>
+  <meta charset="UTF-8">
+  <title>Evolving Draft for ORC Specification v2</title>
+  <meta name="viewport" content="width=device-width,initial-scale=1">
+  <meta name="generator" content="Jekyll v3.7.3">
+  <link rel="stylesheet" href="//fonts.googleapis.com/css?family=Lato:300,300italic,400,400italic,700,700italic,900">
+  <link rel="stylesheet" href="/css/screen.css">
+  <link rel="icon" type="image/x-icon" href="/favicon.ico">
+  <!--[if lt IE 9]>
+  <script src="/js/html5shiv.min.js"></script>
+  <script src="/js/respond.min.js"></script>
+  <![endif]-->
+</head>
+
+
+<body class="wrap">
+  <header role="banner">
+  <nav class="mobile-nav show-on-mobiles">
+    <ul>
+  <li class="">
+    <a href="/">Home</a>
+  </li>
+  <li class="">
+    <a href="/docs/"><span class="show-on-mobiles">Docs</span>
+                     <span class="hide-on-mobiles">Documentation</span></a>
+  </li>
+  <li class="">
+    <a href="/talks/">Talks</a>
+  </li>
+  <li class="">
+    <a href="/news/">News</a>
+  </li>
+  <li class="">
+    <a href="/help/">Help</a>
+  </li>
+  <li class="">
+    <a href="/develop/">Develop</a>
+  </li>
+</ul>
+
+  </nav>
+  <div class="grid">
+    <div class="unit one-third center-on-mobiles">
+      <h1>
+        <a href="/">
+          <span class="sr-only">Apache ORC</span>
+          <img src="/img/logo.png" width="249" height="101" alt="ORC Logo">
+        </a>
+      </h1>
+    </div>
+    <nav class="main-nav unit two-thirds hide-on-mobiles">
+      <ul>
+  <li class="">
+    <a href="/">Home</a>
+  </li>
+  <li class="">
+    <a href="/docs/"><span class="show-on-mobiles">Docs</span>
+                     <span class="hide-on-mobiles">Documentation</span></a>
+  </li>
+  <li class="">
+    <a href="/talks/">Talks</a>
+  </li>
+  <li class="">
+    <a href="/news/">News</a>
+  </li>
+  <li class="">
+    <a href="/help/">Help</a>
+  </li>
+  <li class="">
+    <a href="/develop/">Develop</a>
+  </li>
+</ul>
+
+    </nav>
+  </div>
+</header>
+
+
+  <section class="standalone">
+  <div class="grid">
+
+    <div class="unit whole">
+      <article>
+        <h1>Evolving Draft for ORC Specification v2</h1>
+        <p>This specification is rapidly evolving and should only be used for
+developers on the project.</p>
+
+<h1 id="to-do-items">TO DO items</h1>
+
+<p>The list of things that we plan to change:</p>
+
+<ul>
+  <li>Move decimal encoding to RLEv3 and remove variable length encoding.</li>
+  <li>Create a better float/double encoding that splits mantissa and
+exponent.</li>
+  <li>Create a dictionary encoding for float, double, and decimal.</li>
+  <li>Create RLEv3:
+    <ul>
+      <li>64 and 128 bit variants</li>
+      <li>Zero suppression</li>
+      <li>Evaluate the rle subformats</li>
+    </ul>
+  </li>
+  <li>Group stripe data into stripelets to enable Async IO for reads.</li>
+  <li>Reorder stripe data into (stripe metadata, index, dictionary, data)</li>
+  <li>Stop sorting dictionaries and record the sort order separately in the index.</li>
+  <li>Remove use of RLEv1 and RLEv2.</li>
+  <li>Remove non-utf8 bloom filter.</li>
+  <li>Use numeric value for decimal statistics and bloom filter.</li>
+  <li>Add Zstd with dictionary.</li>
+</ul>
+
+<h1 id="motivation">Motivation</h1>
+
+<p>Hive’s RCFile was the standard format for storing tabular data in
+Hadoop for several years. However, RCFile has limitations because it
+treats each column as a binary blob without semantics. In Hive 0.11 we
+added a new file format named Optimized Row Columnar (ORC) file that
+uses and retains the type information from the table definition. ORC
+uses type specific readers and writers that provide light weight
+compression techniques such as dictionary encoding, bit packing, delta
+encoding, and run length encoding – resulting in dramatically smaller
+files. Additionally, ORC can apply generic compression using zlib, or
+Snappy on top of the lightweight compression for even smaller
+files. However, storage savings are only part of the gain. ORC
+supports projection, which selects subsets of the columns for reading,
+so that queries reading only one column read only the required
+bytes. Furthermore, ORC files include light weight indexes that
+include the minimum and maximum values for each column in each set of
+10,000 rows and the entire file. Using pushdown filters from Hive, the
+file reader can skip entire sets of rows that aren’t important for
+this query.</p>
+
+<p><img src="http://localhost:4000/img/OrcFileLayout.png" alt="ORC file structure" /></p>
+
+<h1 id="file-tail">File Tail</h1>
+
+<p>Since HDFS does not support changing the data in a file after it is
+written, ORC stores the top level index at the end of the file. The
+overall structure of the file is given in the figure above.  The
+file’s tail consists of 3 parts; the file metadata, file footer and
+postscript.</p>
+
+<p>The metadata for ORC is stored using
+<a href="https://s.apache.org/protobuf_encoding">Protocol Buffers</a>, which provides
+the ability to add new fields without breaking readers. This document
+incorporates the Protobuf definition from the
+<a href="https://s.apache.org/orc_proto">ORC source code</a> and the
+reader is encouraged to review the Protobuf encoding if they need to
+understand the byte-level encoding</p>
+
+<h2 id="postscript">Postscript</h2>
+
+<p>The Postscript section provides the necessary information to interpret
+the rest of the file including the length of the file’s Footer and
+Metadata sections, the version of the file, and the kind of general
+compression used (eg. none, zlib, or snappy). The Postscript is never
+compressed and ends one byte before the end of the file. The version
+stored in the Postscript is the lowest version of Hive that is
+guaranteed to be able to read the file and it stored as a sequence of
+the major and minor version. This file version is encoded as [0,12].</p>
+
+<p>The process of reading an ORC file works backwards through the
+file. Rather than making multiple short reads, the ORC reader reads
+the last 16k bytes of the file with the hope that it will contain both
+the Footer and Postscript sections. The final byte of the file
+contains the serialized length of the Postscript, which must be less
+than 256 bytes. Once the Postscript is parsed, the compressed
+serialized length of the Footer is known and it can be decompressed
+and parsed.</p>
+
+<p>```message PostScript {
+ // the length of the footer section in bytes
+ optional uint64 footerLength = 1;
+ // the kind of generic compression used
+ optional CompressionKind compression = 2;
+ // the maximum size of each compression chunk
+ optional uint64 compressionBlockSize = 3;
+ // the version of the writer
+ repeated uint32 version = 4 [packed = true];
+ // the length of the metadata section in bytes
+ optional uint64 metadataLength = 5;
+ // the fixed string “ORC”
+ optional string magic = 8000;
+}</p>
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
+```enum CompressionKind {
+ NONE = 0;
+ ZLIB = 1;
+ SNAPPY = 2;
+ LZO = 3;
+ LZ4 = 4;
+ ZSTD = 5;
+}
+</code></pre></div></div>
+
+<h2 id="footer">Footer</h2>
+
+<p>The Footer section contains the layout of the body of the file, the
+type schema information, the number of rows, and the statistics about
+each of the columns.</p>
+
+<p>The file is broken in to three parts- Header, Body, and Tail. The
+Header consists of the bytes “ORC’’ to support tools that want to
+scan the front of the file to determine the type of the file. The Body
+contains the rows and indexes, and the Tail gives the file level
+information as described in this section.</p>
+
+<p>```message Footer {
+ // the length of the file header in bytes (always 3)
+ optional uint64 headerLength = 1;
+ // the length of the file header and body in bytes
+ optional uint64 contentLength = 2;
+ // the information about the stripes
+ repeated StripeInformation stripes = 3;
+ // the schema information
+ repeated Type types = 4;
+ // the user metadata that was added
+ repeated UserMetadataItem metadata = 5;
+ // the total number of rows in the file
+ optional uint64 numberOfRows = 6;
+ // the statistics of each column across the file
+ repeated ColumnStatistics statistics = 7;
+ // the maximum number of rows in each index entry
+ optional uint32 rowIndexStride = 8;
+}</p>
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
+### Stripe Information
+
+The body of the file is divided into stripes. Each stripe is self
+contained and may be read using only its own bytes combined with the
+file's Footer and Postscript. Each stripe contains only entire rows so
+that rows never straddle stripe boundaries. Stripes have three
+sections: a set of indexes for the rows within the stripe, the data
+itself, and a stripe footer. Both the indexes and the data sections
+are divided by columns so that only the data for the required columns
+needs to be read.
+
+```message StripeInformation {
+ // the start of the stripe within the file
+ optional uint64 offset = 1;
+ // the length of the indexes in bytes
+ optional uint64 indexLength = 2;
+ // the length of the data in bytes
+ optional uint64 dataLength = 3;
+ // the length of the footer in bytes
+ optional uint64 footerLength = 4;
+ // the number of rows in the stripe
+ optional uint64 numberOfRows = 5;
+}
+</code></pre></div></div>
+
+<h3 id="type-information">Type Information</h3>
+
+<p>All of the rows in an ORC file must have the same schema. Logically
+the schema is expressed as a tree as in the figure below, where
+the compound types have subcolumns under them.</p>
+
+<p><img src="http://localhost:4000/img/TreeWriters.png" alt="ORC column structure" /></p>
+
+<p>The equivalent Hive DDL would be:</p>
+
+<p>```create table Foobar (
+ myInt int,
+ myMap map&lt;string,
+ struct&lt;myString : string,
+ myDouble: double»,
+ myTime timestamp
+);</p>
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
+The type tree is flattened in to a list via a pre-order traversal
+where each type is assigned the next id. Clearly the root of the type
+tree is always type id 0. Compound types have a field named subtypes
+that contains the list of their children's type ids.
+
+```message Type {
+ enum Kind {
+ BOOLEAN = 0;
+ BYTE = 1;
+ SHORT = 2;
+ INT = 3;
+ LONG = 4;
+ FLOAT = 5;
+ DOUBLE = 6;
+ STRING = 7;
+ BINARY = 8;
+ TIMESTAMP = 9;
+ LIST = 10;
+ MAP = 11;
+ STRUCT = 12;
+ UNION = 13;
+ DECIMAL = 14;
+ DATE = 15;
+ VARCHAR = 16;
+ CHAR = 17;
+ }
+ // the kind of this type
+ required Kind kind = 1;
+ // the type ids of any subcolumns for list, map, struct, or union
+ repeated uint32 subtypes = 2 [packed=true];
+ // the list of field names for struct
+ repeated string fieldNames = 3;
+ // the maximum length of the type for varchar or char in UTF-8 characters
+ optional uint32 maximumLength = 4;
+ // the precision and scale for decimal
+ optional uint32 precision = 5;
+ optional uint32 scale = 6;
+}
+</code></pre></div></div>
+
+<h3 id="column-statistics">Column Statistics</h3>
+
+<p>The goal of the column statistics is that for each column, the writer
+records the count and depending on the type other useful fields. For
+most of the primitive types, it records the minimum and maximum
+values; and for numeric types it additionally stores the sum.
+From Hive 1.1.0 onwards, the column statistics will also record if
+there are any null values within the row group by setting the hasNull flag.
+The hasNull flag is used by ORC’s predicate pushdown to better answer
+‘IS NULL’ queries.</p>
+
+<p>```message ColumnStatistics {
+ // the number of values
+ optional uint64 numberOfValues = 1;
+ // At most one of these has a value for any column
+ optional IntegerStatistics intStatistics = 2;
+ optional DoubleStatistics doubleStatistics = 3;
+ optional StringStatistics stringStatistics = 4;
+ optional BucketStatistics bucketStatistics = 5;
+ optional DecimalStatistics decimalStatistics = 6;
+ optional DateStatistics dateStatistics = 7;
+ optional BinaryStatistics binaryStatistics = 8;
+ optional TimestampStatistics timestampStatistics = 9;
+ optional bool hasNull = 10;
+}</p>
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
+For integer types (tinyint, smallint, int, bigint), the column
+statistics includes the minimum, maximum, and sum. If the sum
+overflows long at any point during the calculation, no sum is
+recorded.
+
+```message IntegerStatistics {
+ optional sint64 minimum = 1;
+ optional sint64 maximum = 2;
+ optional sint64 sum = 3;
+}
+</code></pre></div></div>
+
+<p>For floating point types (float, double), the column statistics
+include the minimum, maximum, and sum. If the sum overflows a double,
+no sum is recorded.</p>
+
+<p>```message DoubleStatistics {
+ optional double minimum = 1;
+ optional double maximum = 2;
+ optional double sum = 3;
+}</p>
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
+For strings, the minimum value, maximum value, and the sum of the
+lengths of the values are recorded.
+
+```message StringStatistics {
+ optional string minimum = 1;
+ optional string maximum = 2;
+ // sum will store the total length of all strings
+ optional sint64 sum = 3;
+}
+</code></pre></div></div>
+
+<p>For booleans, the statistics include the count of false and true values.</p>
+
+<p>```message BucketStatistics {
+ repeated uint64 count = 1 [packed=true];
+}</p>
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
+For decimals, the minimum, maximum, and sum are stored.
+
+```message DecimalStatistics {
+ optional string minimum = 1;
+ optional string maximum = 2;
+ optional string sum = 3;
+}
+</code></pre></div></div>
+
+<p>Date columns record the minimum and maximum values as the number of
+days since the epoch (1/1/2015).</p>
+
+<p>```message DateStatistics {
+ // min,max values saved as days since epoch
+ optional sint32 minimum = 1;
+ optional sint32 maximum = 2;
+}</p>
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
+Timestamp columns record the minimum and maximum values as the number of
+milliseconds since the epoch (1/1/2015).
+
+```message TimestampStatistics {
+ // min,max values saved as milliseconds since epoch
+ optional sint64 minimum = 1;
+ optional sint64 maximum = 2;
+}
+</code></pre></div></div>
+
+<p>Binary columns store the aggregate number of bytes across all of the values.</p>
+
+<p>```message BinaryStatistics {
+ // sum will store the total binary blob length
+ optional sint64 sum = 1;
+}</p>
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
+### User Metadata
+
+The user can add arbitrary key/value pairs to an ORC file as it is
+written. The contents of the keys and values are completely
+application defined, but the key is a string and the value is
+binary. Care should be taken by applications to make sure that their
+keys are unique and in general should be prefixed with an organization
+code.
+
+```message UserMetadataItem {
+ // the user defined key
+ required string name = 1;
+ // the user defined binary value
+ required bytes value = 2;
+}
+</code></pre></div></div>
+
+<h3 id="file-metadata">File Metadata</h3>
+
+<p>The file Metadata section contains column statistics at the stripe
+level granularity. These statistics enable input split elimination
+based on the predicate push-down evaluated per a stripe.</p>
+
+<p>```message StripeStatistics {
+ repeated ColumnStatistics colStats = 1;
+}</p>
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
+```message Metadata {
+ repeated StripeStatistics stripeStats = 1;
+}
+</code></pre></div></div>
+
+<h1 id="compression">Compression</h1>
+
+<p>If the ORC file writer selects a generic compression codec (zlib or
+snappy), every part of the ORC file except for the Postscript is
+compressed with that codec. However, one of the requirements for ORC
+is that the reader be able to skip over compressed bytes without
+decompressing the entire stream. To manage this, ORC writes compressed
+streams in chunks with headers as in the figure below.
+To handle uncompressable data, if the compressed data is larger than
+the original, the original is stored and the isOriginal flag is
+set. Each header is 3 bytes long with (compressedLength * 2 +
+isOriginal) stored as a little endian value. For example, the header
+for a chunk that compressed to 100,000 bytes would be [0x40, 0x0d,
+0x03]. The header for 5 bytes that did not compress would be [0x0b,
+0x00, 0x00]. Each compression chunk is compressed independently so
+that as long as a decompressor starts at the top of a header, it can
+start decompressing without the previous bytes.</p>
+
+<p><img src="http://localhost:4000/img/CompressionStream.png" alt="compression streams" /></p>
+
+<p>The default compression chunk size is 256K, but writers can choose
+their own value. Larger chunks lead to better compression, but require
+more memory. The chunk size is recorded in the Postscript so that
+readers can allocate appropriately sized buffers. Readers are
+guaranteed that no chunk will expand to more than the compression chunk
+size.</p>
+
+<p>ORC files without generic compression write each stream directly
+with no headers.</p>
+
+<h1 id="run-length-encoding">Run Length Encoding</h1>
+
+<h2 id="base-128-varint">Base 128 Varint</h2>
+
+<p>Variable width integer encodings take advantage of the fact that most
+numbers are small and that having smaller encodings for small numbers
+shrinks the overall size of the data. ORC uses the varint format from
+Protocol Buffers, which writes data in little endian format using the
+low 7 bits of each byte. The high bit in each byte is set if the
+number continues into the next byte.</p>
+
+<table>
+  <thead>
+    <tr>
+      <th style="text-align: left">Unsigned Original</th>
+      <th style="text-align: left">Serialized</th>
+    </tr>
+  </thead>
+  <tbody>
+    <tr>
+      <td style="text-align: left">0</td>
+      <td style="text-align: left">0x00</td>
+    </tr>
+    <tr>
+      <td style="text-align: left">1</td>
+      <td style="text-align: left">0x01</td>
+    </tr>
+    <tr>
+      <td style="text-align: left">127</td>
+      <td style="text-align: left">0x7f</td>
+    </tr>
+    <tr>
+      <td style="text-align: left">128</td>
+      <td style="text-align: left">0x80, 0x01</td>
+    </tr>
+    <tr>
+      <td style="text-align: left">129</td>
+      <td style="text-align: left">0x81, 0x01</td>
+    </tr>
+    <tr>
+      <td style="text-align: left">16,383</td>
+      <td style="text-align: left">0xff, 0x7f</td>
+    </tr>
+    <tr>
+      <td style="text-align: left">16,384</td>
+      <td style="text-align: left">0x80, 0x80, 0x01</td>
+    </tr>
+    <tr>
+      <td style="text-align: left">16,385</td>
+      <td style="text-align: left">0x81, 0x80, 0x01</td>
+    </tr>
+  </tbody>
+</table>
+
+<p>For signed integer types, the number is converted into an unsigned
+number using a zigzag encoding. Zigzag encoding moves the sign bit to
+the least significant bit using the expression (val « 1) ^ (val »
+63) and derives its name from the fact that positive and negative
+numbers alternate once encoded. The unsigned number is then serialized
+as above.</p>
+
+<table>
+  <thead>
+    <tr>
+      <th style="text-align: left">Signed Original</th>
+      <th style="text-align: left">Unsigned</th>
+    </tr>
+  </thead>
+  <tbody>
+    <tr>
+      <td style="text-align: left">0</td>
+      <td style="text-align: left">0</td>
+    </tr>
+    <tr>
+      <td style="text-align: left">-1</td>
+      <td style="text-align: left">1</td>
+    </tr>
+    <tr>
+      <td style="text-align: left">1</td>
+      <td style="text-align: left">2</td>
+    </tr>
+    <tr>
+      <td style="text-align: left">-2</td>
+      <td style="text-align: left">3</td>
+    </tr>
+    <tr>
+      <td style="text-align: left">2</td>
+      <td style="text-align: left">4</td>
+    </tr>
+  </tbody>
+</table>
+
+<h2 id="byte-run-length-encoding">Byte Run Length Encoding</h2>
+
+<p>For byte streams, ORC uses a very light weight encoding of identical
+values.</p>
+
+<ul>
+  <li>Run - a sequence of at least 3 identical values</li>
+  <li>Literals - a sequence of non-identical values</li>
+</ul>
+
+<p>The first byte of each group of values is a header than determines
+whether it is a run (value between 0 to 127) or literal list (value
+between -128 to -1). For runs, the control byte is the length of the
+run minus the length of the minimal run (3) and the control byte for
+literal lists is the negative length of the list. For example, a
+hundred 0’s is encoded as [0x61, 0x00] and the sequence 0x44, 0x45
+would be encoded as [0xfe, 0x44, 0x45]. The next group can choose
+either of the encodings.</p>
+
+<h2 id="boolean-run-length-encoding">Boolean Run Length Encoding</h2>
+
+<p>For encoding boolean types, the bits are put in the bytes from most
+significant to least significant. The bytes are encoded using byte run
+length encoding as described in the previous section. For example,
+the byte sequence [0xff, 0x80] would be one true followed by
+seven false values.</p>
+
+<h2 id="integer-run-length-encoding-version-1">Integer Run Length Encoding, version 1</h2>
+
+<p>In Hive 0.11 ORC files used Run Length Encoding version 1 (RLEv1),
+which provides a lightweight compression of signed or unsigned integer
+sequences. RLEv1 has two sub-encodings:</p>
+
+<ul>
+  <li>Run - a sequence of values that differ by a small fixed delta</li>
+  <li>Literals - a sequence of varint encoded values</li>
+</ul>
+
+<p>Runs start with an initial byte of 0x00 to 0x7f, which encodes the
+length of the run - 3. A second byte provides the fixed delta in the
+range of -128 to 127. Finally, the first value of the run is encoded
+as a base 128 varint.</p>
+
+<p>For example, if the sequence is 100 instances of 7 the encoding would
+start with 100 - 3, followed by a delta of 0, and a varint of 7 for
+an encoding of [0x61, 0x00, 0x07]. To encode the sequence of numbers
+running from 100 to 1, the first byte is 100 - 3, the delta is -1,
+and the varint is 100 for an encoding of [0x61, 0xff, 0x64].</p>
+
+<p>Literals start with an initial byte of 0x80 to 0xff, which corresponds
+to the negative of number of literals in the sequence. Following the
+header byte, the list of N varints is encoded. Thus, if there are
+no runs, the overhead is 1 byte for each 128 integers. The first 5
+prime numbers [2, 3, 4, 7, 11] would encoded as [0xfb, 0x02, 0x03,
+0x04, 0x07, 0xb].</p>
+
+<h2 id="integer-run-length-encoding-version-2">Integer Run Length Encoding, version 2</h2>
+
+<p>In Hive 0.12, ORC introduced Run Length Encoding version 2 (RLEv2),
+which has improved compression and fixed bit width encodings for
+faster expansion. RLEv2 uses four sub-encodings based on the data:</p>
+
+<ul>
+  <li>Short Repeat - used for short sequences with repeated values</li>
+  <li>Direct - used for random sequences with a fixed bit width</li>
+  <li>Patched Base - used for random sequences with a variable bit width</li>
+  <li>Delta - used for monotonically increasing or decreasing sequences</li>
+</ul>
+
+<h3 id="short-repeat">Short Repeat</h3>
+
+<p>The short repeat encoding is used for short repeating integer
+sequences with the goal of minimizing the overhead of the header. All
+of the bits listed in the header are from the first byte to the last
+and from most significant bit to least significant bit. If the type is
+signed, the value is zigzag encoded.</p>
+
+<ul>
+  <li>1 byte header
+    <ul>
+      <li>2 bits for encoding type (0)</li>
+      <li>3 bits for width (W) of repeating value (1 to 8 bytes)</li>
+      <li>3 bits for repeat count (3 to 10 values)</li>
+    </ul>
+  </li>
+  <li>W bytes in big endian format, which is zigzag encoded if they type
+is signed</li>
+</ul>
+
+<p>The unsigned sequence of [10000, 10000, 10000, 10000, 10000] would be
+serialized with short repeat encoding (0), a width of 2 bytes (1), and
+repeat count of 5 (2) as [0x0a, 0x27, 0x10].</p>
+
+<h3 id="direct">Direct</h3>
+
+<p>The direct encoding is used for integer sequences whose values have a
+relatively constant bit width. It encodes the values directly using a
+fixed width big endian encoding. The width of the values is encoded
+using the table below.</p>
+
+<p>The 5 bit width encoding table for RLEv2:</p>
+
+<table>
+  <thead>
+    <tr>
+      <th style="text-align: left">Width in Bits</th>
+      <th style="text-align: left">Encoded Value</th>
+      <th style="text-align: left">Notes</th>
+    </tr>
+  </thead>
+  <tbody>
+    <tr>
+      <td style="text-align: left">0</td>
+      <td style="text-align: left">0</td>
+      <td style="text-align: left">for delta encoding</td>
+    </tr>
+    <tr>
+      <td style="text-align: left">1</td>
+      <td style="text-align: left">0</td>
+      <td style="text-align: left">for non-delta encoding</td>
+    </tr>
+    <tr>
+      <td style="text-align: left">2</td>
+      <td style="text-align: left">1</td>
+      <td style="text-align: left"> </td>
+    </tr>
+    <tr>
+      <td style="text-align: left">4</td>
+      <td style="text-align: left">3</td>
+      <td style="text-align: left"> </td>
+    </tr>
+    <tr>
+      <td style="text-align: left">8</td>
+      <td style="text-align: left">7</td>
+      <td style="text-align: left"> </td>
+    </tr>
+    <tr>
+      <td style="text-align: left">16</td>
+      <td style="text-align: left">15</td>
+      <td style="text-align: left"> </td>
+    </tr>
+    <tr>
+      <td style="text-align: left">24</td>
+      <td style="text-align: left">23</td>
+      <td style="text-align: left"> </td>
+    </tr>
+    <tr>
+      <td style="text-align: left">32</td>
+      <td style="text-align: left">27</td>
+      <td style="text-align: left"> </td>
+    </tr>
+    <tr>
+      <td style="text-align: left">40</td>
+      <td style="text-align: left">28</td>
+      <td style="text-align: left"> </td>
+    </tr>
+    <tr>
+      <td style="text-align: left">48</td>
+      <td style="text-align: left">29</td>
+      <td style="text-align: left"> </td>
+    </tr>
+    <tr>
+      <td style="text-align: left">56</td>
+      <td style="text-align: left">30</td>
+      <td style="text-align: left"> </td>
+    </tr>
+    <tr>
+      <td style="text-align: left">64</td>
+      <td style="text-align: left">31</td>
+      <td style="text-align: left"> </td>
+    </tr>
+    <tr>
+      <td style="text-align: left">3</td>
+      <td style="text-align: left">2</td>
+      <td style="text-align: left">deprecated</td>
+    </tr>
+    <tr>
+      <td style="text-align: left">5 &lt;= x &lt;= 7</td>
+      <td style="text-align: left">x - 1</td>
+      <td style="text-align: left">deprecated</td>
+    </tr>
+    <tr>
+      <td style="text-align: left">9 &lt;= x &lt;= 15</td>
+      <td style="text-align: left">x - 1</td>
+      <td style="text-align: left">deprecated</td>
+    </tr>
+    <tr>
+      <td style="text-align: left">17 &lt;= x &lt;= 21</td>
+      <td style="text-align: left">x - 1</td>
+      <td style="text-align: left">deprecated</td>
+    </tr>
+    <tr>
+      <td style="text-align: left">26</td>
+      <td style="text-align: left">24</td>
+      <td style="text-align: left">deprecated</td>
+    </tr>
+    <tr>
+      <td style="text-align: left">28</td>
+      <td style="text-align: left">25</td>
+      <td style="text-align: left">deprecated</td>
+    </tr>
+    <tr>
+      <td style="text-align: left">30</td>
+      <td style="text-align: left">26</td>
+      <td style="text-align: left">deprecated</td>
+    </tr>
+  </tbody>
+</table>
+
+<ul>
+  <li>2 bytes header
+    <ul>
+      <li>2 bits for encoding type (1)</li>
+      <li>5 bits for encoded width (W) of values (1 to 64 bits) using the 5 bit
+width encoding table</li>
+      <li>9 bits for length (L) (1 to 512 values)</li>
+    </ul>
+  </li>
+  <li>W * L bits (padded to the next byte) encoded in big endian format, which is
+zigzag encoding if the type is signed</li>
+</ul>
+
+<p>The unsigned sequence of [23713, 43806, 57005, 48879] would be
+serialized with direct encoding (1), a width of 16 bits (15), and
+length of 4 (3) as [0x5e, 0x03, 0x5c, 0xa1, 0xab, 0x1e, 0xde, 0xad,
+0xbe, 0xef].</p>
+
+<h3 id="patched-base">Patched Base</h3>
+
+<p>The patched base encoding is used for integer sequences whose bit
+widths varies a lot. The minimum signed value of the sequence is found
+and subtracted from the other values. The bit width of those adjusted
+values is analyzed and the 90 percentile of the bit width is chosen
+as W. The 10\% of values larger than W use patches from a patch list
+to set the additional bits. Patches are encoded as a list of gaps in
+the index values and the additional value bits.</p>
+
+<ul>
+  <li>4 bytes header
+    <ul>
+      <li>2 bits for encoding type (2)</li>
+      <li>5 bits for encoded width (W) of values (1 to 64 bits) using the 5 bit
+  width encoding table</li>
+      <li>9 bits for length (L) (1 to 512 values)</li>
+      <li>3 bits for base value width (BW) (1 to 8 bytes)</li>
+      <li>5 bits for patch width (PW) (1 to 64 bits) using  the 5 bit width
+encoding table</li>
+      <li>3 bits for patch gap width (PGW) (1 to 8 bits)</li>
+      <li>5 bits for patch list length (PLL) (0 to 31 patches)</li>
+    </ul>
+  </li>
+  <li>Base value (BW bytes) - The base value is stored as a big endian value
+with negative values marked by the most significant bit set. If it that
+bit is set, the entire value is negated.</li>
+  <li>Data values (W * L bits padded to the byte) - A sequence of W bit positive
+values that are added to the base value.</li>
+  <li>Data values (W * L bits padded to the byte) - A sequence of W bit positive
+values that are added to the base value.</li>
+  <li>Patch list (PLL * (PGW + PW) bytes) - A list of patches for values
+that didn’t fit within W bits. Each entry in the list consists of a
+gap, which is the number of elements skipped from the previous
+patch, and a patch value. Patches are applied by logically or’ing
+the data values with the relevant patch shifted W bits left. If a
+patch is 0, it was introduced to skip over more than 255 items. The
+combined length of each patch (PGW + PW) must be less or equal to
+64.</li>
+</ul>
+
+<p>The unsigned sequence of [2030, 2000, 2020, 1000000, 2040, 2050, 2060, 2070,
+2080, 2090, 2100, 2110, 2120, 2130, 2140, 2150, 2160, 2170, 2180, 2190]
+has a minimum of 2000, which makes the adjusted
+sequence [30, 0, 20, 998000, 40, 50, 60, 70, 80, 90, 100, 110, 120, 130, 140,
+150, 160, 170, 180, 190]. It has an
+encoding of patched base (2), a bit width of 8 (7), a length of 20
+(19), a base value width of 2 bytes (1), a patch width of 12 bits (11),
+patch gap width of 2 bits (1), and a patch list length of 1 (1). The
+base value is 2000 and the combined result is [0x8e, 0x13, 0x2b, 0x21, 0x07,
+0xd0, 0x1e, 0x00, 0x14, 0x70, 0x28, 0x32, 0x3c, 0x46, 0x50, 0x5a, 0x64, 0x6e,
+0x78, 0x82, 0x8c, 0x96, 0xa0, 0xaa, 0xb4, 0xbe, 0xfc, 0xe8]</p>
+
+<h3 id="delta">Delta</h3>
+
+<p>The Delta encoding is used for monotonically increasing or decreasing
+sequences. The first two numbers in the sequence can not be identical,
+because the encoding is using the sign of the first delta to determine
+if the series is increasing or decreasing.</p>
+
+<ul>
+  <li>2 bytes header
+    <ul>
+      <li>2 bits for encoding type (3)</li>
+      <li>5 bits for encoded width (W) of deltas (0 to 64 bits) using the 5 bit
+width encoding table</li>
+      <li>9 bits for run length (L) (1 to 512 values)</li>
+    </ul>
+  </li>
+  <li>Base value - encoded as (signed or unsigned) varint</li>
+  <li>Delta base - encoded as signed varint</li>
+  <li>Delta values $W * (L - 2)$ bytes - encode each delta after the first
+one. If the delta base is positive, the sequence is increasing and if it is
+negative the sequence is decreasing.</li>
+</ul>
+
+<p>The unsigned sequence of [2, 3, 5, 7, 11, 13, 17, 19, 23, 29] would be
+serialized with delta encoding (3), a width of 4 bits (3), length of
+10 (9), a base of 2 (2), and first delta of 1 (2). The resulting
+sequence is [0xc6, 0x09, 0x02, 0x02, 0x22, 0x42, 0x42, 0x46].</p>
+
+<h1 id="stripes">Stripes</h1>
+
+<p>The body of ORC files consists of a series of stripes. Stripes are
+large (typically ~200MB) and independent of each other and are often
+processed by different tasks. The defining characteristic for columnar
+storage formats is that the data for each column is stored separately
+and that reading data out of the file should be proportional to the
+number of columns read.</p>
+
+<p>In ORC files, each column is stored in several streams that are stored
+next to each other in the file. For example, an integer column is
+represented as two streams PRESENT, which uses one with a bit per
+value recording if the value is non-null, and DATA, which records the
+non-null values. If all of a column’s values in a stripe are non-null,
+the PRESENT stream is omitted from the stripe. For binary data, ORC
+uses three streams PRESENT, DATA, and LENGTH, which stores the length
+of each value. The details of each type will be presented in the
+following subsections.</p>
+
+<h2 id="stripe-footer">Stripe Footer</h2>
+
+<p>The stripe footer contains the encoding of each column and the
+directory of the streams including their location.</p>
+
+<p>```message StripeFooter {
+ // the location of each stream
+ repeated Stream streams = 1;
+ // the encoding of each column
+ repeated ColumnEncoding columns = 2;
+}</p>
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
+To describe each stream, ORC stores the kind of stream, the column id,
+and the stream's size in bytes. The details of what is stored in each stream
+depends on the type and encoding of the column.
+
+```message Stream {
+ enum Kind {
+ // boolean stream of whether the next value is non-null
+ PRESENT = 0;
+ // the primary data stream
+ DATA = 1;
+ // the length of each value for variable length data
+ LENGTH = 2;
+ // the dictionary blob
+ DICTIONARY\_DATA = 3;
+ // deprecated prior to Hive 0.11
+ // It was used to store the number of instances of each value in the
+ // dictionary
+ DICTIONARY_COUNT = 4;
+ // a secondary data stream
+ SECONDARY = 5;
+ // the index for seeking to particular row groups
+ ROW_INDEX = 6;
+ // original bloom filters used before ORC-101
+ BLOOM_FILTER = 7;
+ // bloom filters that consistently use utf8
+ BLOOM_FILTER_UTF8 = 8;
+ }
+ required Kind kind = 1;
+ // the column id
+ optional uint32 column = 2;
+ // the number of bytes in the file
+ optional uint64 length = 3;
+}
+</code></pre></div></div>
+
+<p>Depending on their type several options for encoding are possible. The
+encodings are divided into direct or dictionary-based categories and
+further refined as to whether they use RLE v1 or v2.</p>
+
+<p>```message ColumnEncoding {
+ enum Kind {
+ // the encoding is mapped directly to the stream using RLE v1
+ DIRECT = 0;
+ // the encoding uses a dictionary of unique values using RLE v1
+ DICTIONARY = 1;
+ // the encoding is direct using RLE v2
+ DIRECT_V2 = 2;
+ // the encoding is dictionary-based using RLE v2
+ DICTIONARY_V2 = 3;
+ }
+ required Kind kind = 1;
+ // for dictionary encodings, record the size of the dictionary
+ optional uint32 dictionarySize = 2;
+}</p>
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
+# Column Encodings
+
+## SmallInt, Int, and BigInt Columns
+
+All of the 16, 32, and 64 bit integer column types use the same set of
+potential encodings, which is basically whether they use RLE v1 or
+v2. If the PRESENT stream is not included, all of the values are
+present. For values that have false bits in the present stream, no
+values are included in the data stream.
+
+Encoding  | Stream Kind | Optional | Contents
+:-------- | :---------- | :------- | :-------
+DIRECT    | PRESENT     | Yes      | Boolean RLE
+          | DATA        | No       | Signed Integer RLE v1
+DIRECT_V2 | PRESENT     | Yes      | Boolean RLE
+          | DATA        | No       | Signed Integer RLE v2
+
+## Float and Double Columns
+
+Floating point types are stored using IEEE 754 floating point bit
+layout. Float columns use 4 bytes per value and double columns use 8
+bytes.
+
+Encoding  | Stream Kind | Optional | Contents
+:-------- | :---------- | :------- | :-------
+DIRECT    | PRESENT     | Yes      | Boolean RLE
+          | DATA        | No       | IEEE 754 floating point representation
+
+## String, Char, and VarChar Columns
+
+String, char, and varchar columns may be encoded either using a
+dictionary encoding or a direct encoding. A direct encoding should be
+preferred when there are many distinct values. In all of the
+encodings, the PRESENT stream encodes whether the value is null. The
+Java ORC writer automatically picks the encoding after the first row
+group (10,000 rows).
+
+For direct encoding the UTF-8 bytes are saved in the DATA stream and
+the length of each value is written into the LENGTH stream. In direct
+encoding, if the values were ["Nevada", "California"]; the DATA
+would be "NevadaCalifornia" and the LENGTH would be [6, 10].
+
+For dictionary encodings the dictionary is sorted and UTF-8 bytes of
+each unique value are placed into DICTIONARY_DATA. The length of each
+item in the dictionary is put into the LENGTH stream. The DATA stream
+consists of the sequence of references to the dictionary elements.
+
+In dictionary encoding, if the values were ["Nevada",
+"California", "Nevada", "California", and "Florida"]; the
+DICTIONARY_DATA would be "CaliforniaFloridaNevada" and LENGTH would
+be [10, 7, 6]. The DATA would be [2, 0, 2, 0, 1].
+
+Encoding      | Stream Kind     | Optional | Contents
+:------------ | :-------------- | :------- | :-------
+DIRECT        | PRESENT         | Yes      | Boolean RLE
+              | DATA            | No       | String contents
+              | LENGTH          | No       | Unsigned Integer RLE v1
+DICTIONARY    | PRESENT         | Yes      | Boolean RLE
+              | DATA            | No       | Unsigned Integer RLE v1
+              | DICTIONARY_DATA | No       | String contents
+              | LENGTH          | No       | Unsigned Integer RLE v1
+DIRECT_V2     | PRESENT         | Yes      | Boolean RLE
+              | DATA            | No       | String contents
+              | LENGTH          | No       | Unsigned Integer RLE v2
+DICTIONARY_V2 | PRESENT         | Yes      | Boolean RLE
+              | DATA            | No       | Unsigned Integer RLE v2
+              | DICTIONARY_DATA | No       | String contents
+              | LENGTH          | No       | Unsigned Integer RLE v2
+
+## Boolean Columns
+
+Boolean columns are rare, but have a simple encoding.
+
+Encoding      | Stream Kind     | Optional | Contents
+:------------ | :-------------- | :------- | :-------
+DIRECT        | PRESENT         | Yes      | Boolean RLE
+              | DATA            | No       | Boolean RLE
+
+## TinyInt Columns
+
+TinyInt (byte) columns use byte run length encoding.
+
+Encoding      | Stream Kind     | Optional | Contents
+:------------ | :-------------- | :------- | :-------
+DIRECT        | PRESENT         | Yes      | Boolean RLE
+              | DATA            | No       | Byte RLE
+
+## Binary Columns
+
+Binary data is encoded with a PRESENT stream, a DATA stream that records
+the contents, and a LENGTH stream that records the number of bytes per a
+value.
+
+Encoding      | Stream Kind     | Optional | Contents
+:------------ | :-------------- | :------- | :-------
+DIRECT        | PRESENT         | Yes      | Boolean RLE
+              | DATA            | No       | String contents
+              | LENGTH          | No       | Unsigned Integer RLE v1
+DIRECT_V2     | PRESENT         | Yes      | Boolean RLE
+              | DATA            | No       | String contents
+              | LENGTH          | No       | Unsigned Integer RLE v2
+
+## Decimal Columns
+
+Since Hive 0.13, all decimals have had fixed precision and scale.
+The goal is to use RLEv3 for the value and use the fixed scale from
+the type. As an interim solution, we are using RLE v2 for short decimals
+(precision &lt;= 18) and the old encoding for long decimals.
+
+Encoding      | Stream Kind     | Optional | Contents
+:------------ | :-------------- | :------- | :-------
+DIRECT        | PRESENT         | Yes      | Boolean RLE
+              | DATA            | No       | Signed Integer RLE v2
+DIRECT_V2     | PRESENT         | Yes      | Boolean RLE
+              | DATA            | No       | Unbounded base 128 varints
+              | SECONDARY       | No       | Unsigned Integer RLE v2
+
+
+## Date Columns
+
+Date data is encoded with a PRESENT stream, a DATA stream that records
+the number of days after January 1, 1970 in UTC.
+
+Encoding      | Stream Kind     | Optional | Contents
+:------------ | :-------------- | :------- | :-------
+DIRECT        | PRESENT         | Yes      | Boolean RLE
+              | DATA            | No       | Signed Integer RLE v1
+DIRECT_V2     | PRESENT         | Yes      | Boolean RLE
+              | DATA            | No       | Signed Integer RLE v2
+
+## Timestamp Columns
+
+Timestamp records times down to nanoseconds as a PRESENT stream that
+records non-null values, a DATA stream that records the number of
+seconds after 1 January 2015, and a SECONDARY stream that records the
+number of nanoseconds.
+
+Because the number of nanoseconds often has a large number of trailing
+zeros, the number has trailing decimal zero digits removed and the
+last three bits are used to record how many zeros were removed. Thus
+1000 nanoseconds would be serialized as 0x0b and 100000 would be
+serialized as 0x0d.
+
+Encoding      | Stream Kind     | Optional | Contents
+:------------ | :-------------- | :------- | :-------
+DIRECT        | PRESENT         | Yes      | Boolean RLE
+              | DATA            | No       | Signed Integer RLE v1
+              | SECONDARY       | No       | Unsigned Integer RLE v1
+DIRECT_V2     | PRESENT         | Yes      | Boolean RLE
+              | DATA            | No       | Signed Integer RLE v2
+              | SECONDARY       | No       | Unsigned Integer RLE v2
+
+## Struct Columns
+
+Structs have no data themselves and delegate everything to their child
+columns except for their PRESENT stream. They have a child column
+for each of the fields.
+
+Encoding      | Stream Kind     | Optional | Contents
+:------------ | :-------------- | :------- | :-------
+DIRECT        | PRESENT         | Yes      | Boolean RLE
+
+## List Columns
+
+Lists are encoded as the PRESENT stream and a length stream with
+number of items in each list. They have a single child column for the
+element values.
+
+Encoding      | Stream Kind     | Optional | Contents
+:------------ | :-------------- | :------- | :-------
+DIRECT        | PRESENT         | Yes      | Boolean RLE
+              | LENGTH          | No       | Unsigned Integer RLE v1
+DIRECT_V2     | PRESENT         | Yes      | Boolean RLE
+              | LENGTH          | No       | Unsigned Integer RLE v2
+
+## Map Columns
+
+Maps are encoded as the PRESENT stream and a length stream with number
+of items in each list. They have a child column for the key and
+another child column for the value.
+
+Encoding      | Stream Kind     | Optional | Contents
+:------------ | :-------------- | :------- | :-------
+DIRECT        | PRESENT         | Yes      | Boolean RLE
+              | LENGTH          | No       | Unsigned Integer RLE v1
+DIRECT_V2     | PRESENT         | Yes      | Boolean RLE
+              | LENGTH          | No       | Unsigned Integer RLE v2
+
+## Union Columns
+
+Unions are encoded as the PRESENT stream and a tag stream that controls which
+potential variant is used. They have a child column for each variant of the
+union. Currently ORC union types are limited to 256 variants, which matches
+the Hive type model.
+
+Encoding      | Stream Kind     | Optional | Contents
+:------------ | :-------------- | :------- | :-------
+DIRECT        | PRESENT         | Yes      | Boolean RLE
+              | DIRECT          | No       | Byte RLE
+
+# Indexes
+
+## Row Group Index
+
+The row group indexes consist of a ROW_INDEX stream for each primitive
+column that has an entry for each row group. Row groups are controlled
+by the writer and default to 10,000 rows. Each RowIndexEntry gives the
+position of each stream for the column and the statistics for that row
+group.
+
+The index streams are placed at the front of the stripe, because in
+the default case of streaming they do not need to be read. They are
+only loaded when either predicate push down is being used or the
+reader seeks to a particular row.
+
+```message RowIndexEntry {
+ repeated uint64 positions = 1 [packed=true];
+ optional ColumnStatistics statistics = 2;
+}
+</code></pre></div></div>
+
+<p>```message RowIndex {
+ repeated RowIndexEntry entry = 1;
+}</p>
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
+To record positions, each stream needs a sequence of numbers. For
+uncompressed streams, the position is the byte offset of the RLE run's
+start location followed by the number of values that need to be
+consumed from the run. In compressed streams, the first number is the
+start of the compression chunk in the stream, followed by the number
+of decompressed bytes that need to be consumed, and finally the number
+of values consumed in the RLE.
+
+For columns with multiple streams, the sequences of positions in each
+stream are concatenated. That was an unfortunate decision on my part
+that we should fix at some point, because it makes code that uses the
+indexes error-prone.
+
+Because dictionaries are accessed randomly, there is not a position to
+record for the dictionary and the entire dictionary must be read even
+if only part of a stripe is being read.
+
+## Bloom Filter Index
+
+Bloom Filters are added to ORC indexes from Hive 1.2.0 onwards.
+Predicate pushdown can make use of bloom filters to better prune
+the row groups that do not satisfy the filter condition.
+The bloom filter indexes consist of a BLOOM_FILTER stream for each
+column specified through 'orc.bloom.filter.columns' table properties.
+A BLOOM_FILTER stream records a bloom filter entry for each row
+group (default to 10,000 rows) in a column. Only the row groups that
+satisfy min/max row index evaluation will be evaluated against the
+bloom filter index.
+
+Each BloomFilterEntry stores the number of hash functions ('k') used
+and the bitset backing the bloom filter. The original encoding (pre
+ORC-101) of bloom filters used the bitset field encoded as a repeating
+sequence of longs in the bitset field with a little endian encoding
+(0x1 is bit 0 and 0x2 is bit 1.) After ORC-101, the encoding is a
+sequence of bytes with a little endian encoding in the utf8bitset field.
+
+```message BloomFilter {
+ optional uint32 numHashFunctions = 1;
+ repeated fixed64 bitset = 2;
+ optional bytes utf8bitset = 3;
+}
+</code></pre></div></div>
+
+<p><code class="highlighter-rouge">message BloomFilterIndex {
+ repeated BloomFilter bloomFilter = 1;
+}
+</code></p>
+
+<p>Bloom filter internally uses two different hash functions to map a key
+to a position in the bit set. For tinyint, smallint, int, bigint, float
+and double types, Thomas Wang’s 64-bit integer hash function is used.
+Floats are converted to IEEE-754 32 bit representation
+(using Java’s Float.floatToIntBits(float)). Similary, Doubles are
+converted to IEEE-754 64 bit representation (using Java’s
+Double.doubleToLongBits(double)). All these primitive types
+are cast to long base type before being passed on to the hash function.
+For strings and binary types, Murmur3 64 bit hash algorithm is used.
+The 64 bit variant of Murmur3 considers only the most significant
+8 bytes of Murmur3 128-bit algorithm. The 64 bit hashcode generated
+from the above algorithms is used as a base to derive ‘k’ different
+hash functions. We use the idea mentioned in the paper “Less Hashing,
+Same Performance: Building a Better Bloom Filter” by Kirsch et. al. to
+quickly compute the k hashcodes.</p>
+
+<p>The algorithm for computing k hashcodes and setting the bit position
+in a bloom filter is as follows:</p>
+
+<ol>
+  <li>Get 64 bit base hash code from Murmur3 or Thomas Wang’s hash algorithm.</li>
+  <li>Split the above hashcode into two 32-bit hashcodes (say hash1 and hash2).</li>
+  <li>k’th hashcode is obtained by (where k &gt; 0):
+    <ul>
+      <li>combinedHash = hash1 + (k * hash2)</li>
+    </ul>
+  </li>
+  <li>If combinedHash is negative flip all the bits:
+    <ul>
+      <li>combinedHash = ~combinedHash</li>
+    </ul>
+  </li>
+  <li>Bit set position is obtained by performing modulo with m:
+    <ul>
+      <li>position = combinedHash % m</li>
+    </ul>
+  </li>
+  <li>Set the position in bit set. The LSB 6 bits identifies the long index
+within bitset and bit position within the long uses little endian order.
+    <ul>
+      <li>bitset[position »&gt; 6] |= (1L « position);</li>
+    </ul>
+  </li>
+</ol>
+
+<p>Bloom filter streams are interlaced with row group indexes. This placement
+makes it convenient to read the bloom filter stream and row index stream
+together in single read operation.</p>
+
+<p><img src="http://localhost:4000/img/BloomFilter.png" alt="bloom filter" /></p>
+
+      </article>
+    </div>
+
+    <div class="clear"></div>
+
+  </div>
+</section>
+
+
+  <footer role="contentinfo">
+  <p>The contents of this website are &copy;&nbsp;2018
+     <a href="https://www.apache.org/">Apache Software Foundation</a>
+     under the terms of the <a
+      href="https://www.apache.org/licenses/LICENSE-2.0.html">
+      Apache&nbsp;License&nbsp;v2</a>. Apache ORC and its logo are trademarks
+      of the Apache Software Foundation.</p>
+</footer>
+
+  <script>
+  var anchorForId = function (id) {
+    var anchor = document.createElement("a");
+    anchor.className = "header-link";
+    anchor.href      = "#" + id;
+    anchor.innerHTML = "<span class=\"sr-only\">Permalink</span><i class=\"fa fa-link\"></i>";
+    anchor.title = "Permalink";
+    return anchor;
+  };
+
+  var linkifyAnchors = function (level, containingElement) {
+    var headers = containingElement.getElementsByTagName("h" + level);
+    for (var h = 0; h < headers.length; h++) {
+      var header = headers[h];
+
+      if (typeof header.id !== "undefined" && header.id !== "") {
+        header.appendChild(anchorForId(header.id));
+      }
+    }
+  };
+
+  document.onreadystatechange = function () {
+    if (this.readyState === "complete") {
+      var contentBlock = document.getElementsByClassName("docs")[0] || document.getElementsByClassName("news")[0];
+      if (!contentBlock) {
+        return;
+      }
+      for (var level = 1; level <= 6; level++) {
+        linkifyAnchors(level, contentBlock);
+      }
+    }
+  };
+</script>
+
+
+</body>
+</html>

http://git-wip-us.apache.org/repos/asf/orc/blob/e58d3fc4/specification/index.html
----------------------------------------------------------------------
diff --git a/specification/index.html b/specification/index.html
index 3c3a5fe..69d45cf 100644
--- a/specification/index.html
+++ b/specification/index.html
@@ -4,7 +4,7 @@
   <meta charset="UTF-8">
   <title>ORC Specification</title>
   <meta name="viewport" content="width=device-width,initial-scale=1">
-  <meta name="generator" content="Jekyll v2.4.0">
+  <meta name="generator" content="Jekyll v3.7.3">
   <link rel="stylesheet" href="//fonts.googleapis.com/css?family=Lato:300,300italic,400,400italic,700,700italic,900">
   <link rel="stylesheet" href="/css/screen.css">
   <link rel="icon" type="image/x-icon" href="/favicon.ico">
@@ -87,8 +87,8 @@
         <p>There have been two released ORC file versions:</p>
 
 <ul>
-  <li><a href="ORCv0.html">ORC v0</a> was released in Hive 0.11.</li>
-  <li><a href="ORCv1.html">ORC v1</a> was released in Hive 0.12 and ORC 1.x.</li>
+  <li><a href="ORCv0">ORC v0</a> was released in Hive 0.11.</li>
+  <li><a href="ORCv1">ORC v1</a> was released in Hive 0.12 and ORC 1.x.</li>
 </ul>
 
 <p>Each version of the library will detect the format version and use
@@ -99,7 +99,7 @@ clusters can read correctly.</p>
 <p>We are working on a new version of the file format:</p>
 
 <ul>
-  <li><a href="ORCv2.html">ORC v2</a> is a work in progress and is rapidly evolving.</li>
+  <li><a href="ORCv2">ORC v2</a> is a work in progress and is rapidly evolving.</li>
 </ul>
 
       </article>

http://git-wip-us.apache.org/repos/asf/orc/blob/e58d3fc4/talks/index.html
----------------------------------------------------------------------
diff --git a/talks/index.html b/talks/index.html
index 8b48a8b..6014a98 100644
--- a/talks/index.html
+++ b/talks/index.html
@@ -4,7 +4,7 @@
   <meta charset="UTF-8">
   <title>ORC Talks</title>
   <meta name="viewport" content="width=device-width,initial-scale=1">
-  <meta name="generator" content="Jekyll v2.4.0">
+  <meta name="generator" content="Jekyll v3.7.3">
   <link rel="stylesheet" href="//fonts.googleapis.com/css?family=Lato:300,300italic,400,400italic,700,700italic,900">
   <link rel="stylesheet" href="/css/screen.css">
   <link rel="icon" type="image/x-icon" href="/favicon.ico">


[11/11] orc git commit: Deploy docs with CVE.

Posted by om...@apache.org.
Deploy docs with CVE.

Signed-off-by: Owen O'Malley <om...@apache.org>


Project: http://git-wip-us.apache.org/repos/asf/orc/repo
Commit: http://git-wip-us.apache.org/repos/asf/orc/commit/e58d3fc4
Tree: http://git-wip-us.apache.org/repos/asf/orc/tree/e58d3fc4
Diff: http://git-wip-us.apache.org/repos/asf/orc/diff/e58d3fc4

Branch: refs/heads/asf-site
Commit: e58d3fc43ac357ef6e4c1de47861334d7dd8a16b
Parents: f5aaac5
Author: Owen O'Malley <om...@apache.org>
Authored: Thu May 17 15:12:26 2018 -0700
Committer: Owen O'Malley <om...@apache.org>
Committed: Thu May 17 15:12:43 2018 -0700

----------------------------------------------------------------------
 develop/bylaws.html                         |  448 ------
 develop/bylaws/index.html                   |  448 ++++++
 develop/coding.html                         |  186 ---
 develop/coding/index.html                   |  186 +++
 develop/committers.html                     |  359 -----
 develop/committers/index.html               |  359 +++++
 develop/index.html                          |   52 +-
 develop/make-release.html                   |  324 ----
 develop/make-release/index.html             |  324 ++++
 docs/acid.html                              |   97 +-
 docs/adopters.html                          |  112 +-
 docs/building.html                          |  132 +-
 docs/core-cpp.html                          |  384 +++--
 docs/core-java.html                         |  562 ++++---
 docs/cpp-tools.html                         |  318 ++--
 docs/hive-config.html                       |  100 +-
 docs/hive-ddl.html                          |  120 +-
 docs/index.html                             |   92 +-
 docs/indexes.html                           |   92 +-
 docs/java-tools.html                        |  200 ++-
 docs/mapred.html                            |  284 ++--
 docs/mapreduce.html                         |  262 ++--
 docs/releases.html                          |  100 +-
 docs/types.html                             |   96 +-
 help/index.html                             |    6 +-
 index.html                                  |    2 +-
 news/2015/04/22/orc-as-tlp/index.html       |    2 +-
 news/2015/05/11/new-committers/index.html   |    2 +-
 news/2015/06/26/new-logo/index.html         |    4 +-
 news/2015/11/19/aliaksei-on-pmc/index.html  |    2 +-
 news/2016/01/25/ORC-1.0.0/index.html        |    9 +-
 news/2016/06/10/ORC-1.1.0/index.html        |   10 +-
 news/2016/06/13/ORC-1.1.1/index.html        |   10 +-
 news/2016/06/28/file-format-talk/index.html |    2 +-
 news/2016/07/08/ORC-1.1.2/index.html        |   10 +-
 news/2016/08/25/ORC-1.2.0/index.html        |    8 +-
 news/2016/10/05/ORC-1.2.1/index.html        |    6 +-
 news/2016/12/01/ORC-1.2.2/index.html        |    6 +-
 news/2016/12/12/ORC-1.2.3/index.html        |    6 +-
 news/2016/12/15/new-committers/index.html   |    2 +-
 news/2017/01/04/gopal-on-pmc/index.html     |    2 +-
 news/2017/01/23/ORC-1.3.0/index.html        |    9 +-
 news/2017/02/03/ORC-1.3.1/index.html        |    6 +-
 news/2017/02/13/ORC-1.3.2/index.html        |    4 +-
 news/2017/02/21/ORC-1.3.3/index.html        |    4 +-
 news/2017/05/08/ORC-1.4.0/index.html        |    4 +-
 news/2017/05/16/new-committer/index.html    |    2 +-
 news/2017/09/06/new-pmc/index.html          |    2 +-
 news/2017/10/16/ORC-1.3.4/index.html        |    4 +-
 news/2017/10/16/ORC-1.4.1/index.html        |    9 +-
 news/2018/01/23/ORC-1.4.2/index.html        |    9 +-
 news/2018/02/09/ORC-1.4.3/index.html        |    6 +-
 news/2018/05/14/ORC-1.4.4/index.html        |    2 +-
 news/2018/05/14/ORC-1.5.0/index.html        |    2 +-
 news/index.html                             |   90 +-
 news/releases/index.html                    |   88 +-
 security/CVE-2018-8015/index.html           |  187 +++
 security/index.html                         |    6 +-
 specification/ORCv0.html                    | 1260 ----------------
 specification/ORCv0/index.html              |  954 ++++++++++++
 specification/ORCv1.html                    | 1744 ---------------------
 specification/ORCv1/index.html              | 1317 ++++++++++++++++
 specification/ORCv2.html                    | 1760 ----------------------
 specification/ORCv2/index.html              | 1339 ++++++++++++++++
 specification/index.html                    |    8 +-
 talks/index.html                            |    2 +-
 66 files changed, 6572 insertions(+), 7972 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/orc/blob/e58d3fc4/develop/bylaws.html
----------------------------------------------------------------------
diff --git a/develop/bylaws.html b/develop/bylaws.html
deleted file mode 100644
index e8e0ae8..0000000
--- a/develop/bylaws.html
+++ /dev/null
@@ -1,448 +0,0 @@
-<!DOCTYPE HTML>
-<html lang="en-US">
-<head>
-  <meta charset="UTF-8">
-  <title>Apache ORC Bylaws</title>
-  <meta name="viewport" content="width=device-width,initial-scale=1">
-  <meta name="generator" content="Jekyll v2.4.0">
-  <link rel="stylesheet" href="//fonts.googleapis.com/css?family=Lato:300,300italic,400,400italic,700,700italic,900">
-  <link rel="stylesheet" href="/css/screen.css">
-  <link rel="icon" type="image/x-icon" href="/favicon.ico">
-  <!--[if lt IE 9]>
-  <script src="/js/html5shiv.min.js"></script>
-  <script src="/js/respond.min.js"></script>
-  <![endif]-->
-</head>
-
-
-<body class="wrap">
-  <header role="banner">
-  <nav class="mobile-nav show-on-mobiles">
-    <ul>
-  <li class="">
-    <a href="/">Home</a>
-  </li>
-  <li class="">
-    <a href="/docs/"><span class="show-on-mobiles">Docs</span>
-                     <span class="hide-on-mobiles">Documentation</span></a>
-  </li>
-  <li class="">
-    <a href="/talks/">Talks</a>
-  </li>
-  <li class="">
-    <a href="/news/">News</a>
-  </li>
-  <li class="">
-    <a href="/help/">Help</a>
-  </li>
-  <li class="current">
-    <a href="/develop/">Develop</a>
-  </li>
-</ul>
-
-  </nav>
-  <div class="grid">
-    <div class="unit one-third center-on-mobiles">
-      <h1>
-        <a href="/">
-          <span class="sr-only">Apache ORC</span>
-          <img src="/img/logo.png" width="249" height="101" alt="ORC Logo">
-        </a>
-      </h1>
-    </div>
-    <nav class="main-nav unit two-thirds hide-on-mobiles">
-      <ul>
-  <li class="">
-    <a href="/">Home</a>
-  </li>
-  <li class="">
-    <a href="/docs/"><span class="show-on-mobiles">Docs</span>
-                     <span class="hide-on-mobiles">Documentation</span></a>
-  </li>
-  <li class="">
-    <a href="/talks/">Talks</a>
-  </li>
-  <li class="">
-    <a href="/news/">News</a>
-  </li>
-  <li class="">
-    <a href="/help/">Help</a>
-  </li>
-  <li class="current">
-    <a href="/develop/">Develop</a>
-  </li>
-</ul>
-
-    </nav>
-  </div>
-</header>
-
-
-  <section class="standalone">
-  <div class="grid">
-
-    <div class="unit whole">
-      <article>
-        <h1>Apache ORC Bylaws</h1>
-        <h2 id="introduction">Introduction</h2>
-
-<p>This document defines the bylaws under which the Apache ORC project
-operates. It defines the roles and responsibilities of the project,
-who may vote, how voting works, how conflicts are resolved, etc.</p>
-
-<p>ORC is a project of the Apache Software Foundation (ASF) and the foundation
-holds the trademark on the name “ORC” and copyright on the combined
-code base. The <a href="https://www.apache.org/foundation/faq.html">Apache Foundation
-FAQ</a> and
-<a href="https://www.apache.org/foundation/how-it-works.html">How-It-Works</a>
-explain the operation and background of the foundation.</p>
-
-<p>Apache has a <a href="https://www.apache.org/foundation/policies/conduct.html">code of
-conduct</a> that
-it expects its members to follow. In particular:</p>
-
-<ul>
-  <li>
-    <p>Be <strong>open and welcoming</strong>. It is important that we grow and
-encourage the community of users and developers for our project.</p>
-  </li>
-  <li>
-    <p>Be <strong>collaborative</strong>. Working together on the open mailing lists and
-bug database to make decisions helps the project grow.</p>
-  </li>
-  <li>
-    <p>Be <strong>respectful</strong> of others. Everyone is volunteering their time and
-efforts to work on this project. Please be respectful of everyone
-and their views.</p>
-  </li>
-</ul>
-
-<p>ORC is typical of Apache projects in that it operates under a set of
-principles, known collectively as the “Apache Way”. If you are new to
-Apache development, please refer to the <a href="https://incubator.apache.org/">Incubator
-project</a> for more
-information on how Apache projects operate.</p>
-
-<h2 id="roles-and-responsibilities">Roles and Responsibilities</h2>
-
-<p>Apache projects define a set of roles with associated rights and
-responsibilities. These roles govern what tasks an individual may
-perform within the project. The roles are defined in the following
-sections.</p>
-
-<h3 id="users">Users</h3>
-
-<p>The most important participants in the project are people who use our
-software. The majority of our developers start out as users and guide
-their development efforts from the user’s perspective.  Users
-contribute to the Apache projects by providing feedback to developers
-in the form of bug reports and feature suggestions. As well, users
-participate in the Apache community by helping other users on mailing
-lists and user support forums.</p>
-
-<h3 id="contributors">Contributors</h3>
-
-<p>Contributors include all of the volunteers who donate time, code,
-documentation,
-or resources to the ORC Project. A contributor that makes sustained,
-welcome contributions to the project may be invited to become a
-committer, though the exact timing of such invitations depends on many
-factors.</p>
-
-<h3 id="committers">Committers</h3>
-
-<p>The project’s committers are responsible for the project’s technical
-management. Committers have the right to commit to the project’s git
-repository. Committers may cast binding votes on any technical
-discussion.</p>
-
-<p>Committer access is by invitation only and must be approved by
-consensus approval of the active Project Management Committee (PMC)
-members.</p>
-
-<p>If a committer wishes to leave the project or does not contribute to
-the project in any form for six months, the PMC may make them emeritus.
-Emeritus committers lose their ability to commit code or cast binding
-votes. An emeritus committer may
-request reinstatement of commit access from the PMC. Such
-reinstatement is subject to consensus approval of active PMC members.</p>
-
-<p>All Apache committers are required to have a signed <a href="https://www.apache.org/licenses/icla.txt">Individual
-Contributor License
-Agreement</a> (ICLA) on file
-with the Apache Software Foundation. There is a <a href="https://www.apache.org/dev/committers.html">Committer
-FAQ</a> which provides more
-details on the requirements for Committers.</p>
-
-<p>A committer who makes a
-sustained contribution to the project may be invited to become a
-member of the PMC. The form of contribution
-is not limited to code. It can also include code review, helping out
-users on the mailing lists, documentation, testing, etc.</p>
-
-<h3 id="release-manager">Release Manager</h3>
-
-<p>A Release Manager (RM) is a committer who volunteers to produce a
-Release Candidate. The RM shall publish a Release Plan on the
-dev mailing list stating the branch from which they intend to
-make a Release Candidate.</p>
-
-<h3 id="project-management-committee">Project Management Committee</h3>
-
-<p>The Project Management Committee (PMC) for Apache ORC was created by
-the Apache Board in April 2015 when ORC moved out of Hive and became a
-top level project at Apache. The PMC is responsible to the board and
-the ASF for the management and oversight of the Apache ORC
-codebase. The responsibilities of the PMC include</p>
-
-<ul>
-  <li>
-    <p>Deciding what is distributed as products of the Apache ORC
-project. In particular all releases must be approved by the PMC.</p>
-  </li>
-  <li>
-    <p>Maintaining the project’s shared resources, including the codebase
-repository, mailing lists, and websites.</p>
-  </li>
-  <li>
-    <p>Speaking on behalf of the project.</p>
-  </li>
-  <li>
-    <p>Resolving license disputes regarding products of the project</p>
-  </li>
-  <li>
-    <p>Nominating new PMC members and committers</p>
-  </li>
-  <li>
-    <p>Maintaining these bylaws and other guidelines of the project</p>
-  </li>
-</ul>
-
-<p>Membership of the PMC is by invitation only and must be approved by a
-consensus approval of active PMC members.</p>
-
-<p>A PMC member is considered
-emeritus by their own declaration or by not contributing in any form
-to the project for over six months. An emeritus member may request
-reinstatement to the PMC. Such reinstatement is subject to consensus
-approval of the active PMC members.</p>
-
-<p>The chair of the PMC is appointed by the ASF board. The chair is an
-office holder of the Apache Software Foundation (Vice President,
-Apache ORC) and has primary responsibility to the board for the
-management of the project within the scope of the ORC PMC. The
-chair reports to the board quarterly on developments within the ORC
-project.</p>
-
-<p>When the project desires a new PMC chair, the PMC votes to recommend a
-new chair using <a href="https://wiki.apache.org/general/BoardVoting">Single Transferable
-Vote</a> voting. The decision
-must be ratified by the Apache board.</p>
-
-<h2 id="decision-making">Decision Making</h2>
-
-<p>Within the ORC project, different types of decisions require
-different forms of approval. For example, the previous section
-describes several decisions which require “consensus approval.”
-This section defines how voting is performed, the types of
-approvals, and which types of decision require which type of approval.</p>
-
-<h3 id="voting">Voting</h3>
-
-<p>Decisions regarding the project are made by votes on the primary
-project development mailing list (dev@orc.apache.org). Where
-necessary, PMC voting may take place on the private ORC PMC mailing
-list. Votes are clearly indicated by subject line starting with
-[VOTE]. Votes may contain multiple items for approval and these should
-be clearly separated. Voting is carried out by replying to the vote
-mail. Voting may take five flavors:</p>
-
-<ul>
-  <li>
-    <p><strong>+1</strong> – “Yes,” “Agree,” or “the action should be performed.” In general,
-this vote also indicates a willingness on the behalf of the voter in
-“making it happen.”</p>
-  </li>
-  <li>
-    <p><strong>+0</strong> – This vote indicates a willingness for the action under
-consideration to go ahead. The voter, however, will not be able to
-help.</p>
-  </li>
-  <li>
-    <p><strong>0</strong> – The voter is neutral on the topic under discussion.</p>
-  </li>
-  <li>
-    <p><strong>-0</strong> – This vote indicates that the voter does not, in general, agree
- with the proposed action but is not concerned enough to prevent the
- action going ahead.</p>
-  </li>
-  <li>
-    <p><strong>-1</strong> – This is a negative vote. On issues where consensus is required,
- this vote counts as a veto. All vetoes must contain an explanation
- of why the veto is appropriate. Vetoes with no explanation are
- void. It may also be appropriate for a -1 vote to include an
- alternative course of action.</p>
-  </li>
-</ul>
-
-<p>All participants in the ORC project are encouraged to show their
-agreement for or against a particular action by voting, regardless of
-whether their vote is binding. Nonbinding votes are useful for
-encouraging discussion and understanding the scope of opinions within
-the project.</p>
-
-<h3 id="approvals">Approvals</h3>
-
-<p>These are the types of approvals that can be sought. Different actions
-require different types of approvals.</p>
-
-<ul>
-  <li>
-    <p><strong>Consensus Approval</strong> – Consensus approval requires 3 binding +1
-votes and no binding vetoes.</p>
-  </li>
-  <li>
-    <p><strong>Lazy Consensus</strong> – Lazy consensus requires at least one +1 vote and
-no -1 votes (‘silence gives assent’).</p>
-  </li>
-  <li>
-    <p><strong>Lazy Majority</strong> – A lazy majority vote requires 3 binding +1 votes
- and more binding +1 votes than -1 votes.</p>
-  </li>
-  <li>
-    <p><strong>Lazy 2/3 Majority</strong> – Lazy 2/3 majority votes requires at least 3
-votes and twice as many +1 votes as -1 votes.</p>
-  </li>
-</ul>
-
-<h3 id="vetoes">Vetoes</h3>
-
-<p>A valid, binding veto cannot be overruled. If a veto is cast, it must
-be accompanied by a valid reason explaining the reasons for the
-veto. The validity of a veto, if challenged, can be confirmed by
-anyone who has a binding vote. This does not necessarily signify
-agreement with the veto - merely that the veto is valid.  If you
-disagree with a valid veto, you must lobby the person casting the veto
-to withdraw their veto. If a veto is not withdrawn, any action that
-has already been taken  must be reversed in a timely manner.</p>
-
-<h3 id="actions">Actions</h3>
-
-<p>This section describes the various actions which are undertaken within
-the project, the corresponding approval required for that action and
-those who have binding votes over the action.</p>
-
-<h4 id="code-change">Code Change</h4>
-
-<p>A change made to a codebase of the project requires <em>lazy consensus</em>
-of active committers other than the author of the patch. We can commit
-changes before they have been reviewed, although we prefer to get
-reviews first.</p>
-
-<h4 id="product-release">Product Release</h4>
-
-<p>To make a release, the release manager creates a release candidate and
-a vote requiring a <em>lazy majority</em> of the active PMC members is
-required. Once the vote passes, the release candidate becomes an
-official release.</p>
-
-<h4 id="adoption-of-new-codebase">Adoption of New Codebase</h4>
-
-<p>When the codebase for an existing, released product is to be replaced
-with an alternative codebase, it requires a <em>lazy 2/3 majority</em> of PMC
-members. This also covers the creation of new sub-projects and
-submodules within the project.</p>
-
-<h4 id="new-committer">New Committer</h4>
-
-<p>When a new committer is proposed for the project, <em>consensus approval</em>
-of the active PMC members is required.</p>
-
-<h4 id="new-pmc-member">New PMC Member</h4>
-
-<p>To promote a committer to a PMC member requires <em>consensus approval</em>
-of active PMC members.</p>
-
-<p>If the vote passes, the Apache Board must be notified to make the change
-official.</p>
-
-<h4 id="committer-removal">Committer Removal</h4>
-
-<p>Removal of commit privileges requires a <em>lazy 2/3 majority</em> of active
-PMC members.</p>
-
-<h4 id="pmc-member-removal">PMC Member Removal</h4>
-
-<p>Removing a PMC member requires a <em>lazy 2/3 majority</em> of active PMC
-members, excluding the member in question.</p>
-
-<p>If the vote passes, the Apache Board must be notified to make the change
-official.</p>
-
-<h4 id="modifying-bylaws">Modifying Bylaws</h4>
-
-<p>Modifying this document requires a <em>lazy majority</em> of active PMC members.</p>
-
-<h3 id="voting-timeframes">Voting Timeframes</h3>
-
-<p>Votes are open for a minimum period of 72 hours to allow all active
-voters time to consider the vote. For holiday weekends or conferences,
-consider using a longer vote window. Votes relating to code changes are
-not subject to a strict timetable but should be made as timely as
-possible.</p>
-
-      </article>
-    </div>
-
-    <div class="clear"></div>
-
-  </div>
-</section>
-
-
-  <footer role="contentinfo">
-  <p>The contents of this website are &copy;&nbsp;2018
-     <a href="https://www.apache.org/">Apache Software Foundation</a>
-     under the terms of the <a
-      href="https://www.apache.org/licenses/LICENSE-2.0.html">
-      Apache&nbsp;License&nbsp;v2</a>. Apache ORC and its logo are trademarks
-      of the Apache Software Foundation.</p>
-</footer>
-
-  <script>
-  var anchorForId = function (id) {
-    var anchor = document.createElement("a");
-    anchor.className = "header-link";
-    anchor.href      = "#" + id;
-    anchor.innerHTML = "<span class=\"sr-only\">Permalink</span><i class=\"fa fa-link\"></i>";
-    anchor.title = "Permalink";
-    return anchor;
-  };
-
-  var linkifyAnchors = function (level, containingElement) {
-    var headers = containingElement.getElementsByTagName("h" + level);
-    for (var h = 0; h < headers.length; h++) {
-      var header = headers[h];
-
-      if (typeof header.id !== "undefined" && header.id !== "") {
-        header.appendChild(anchorForId(header.id));
-      }
-    }
-  };
-
-  document.onreadystatechange = function () {
-    if (this.readyState === "complete") {
-      var contentBlock = document.getElementsByClassName("docs")[0] || document.getElementsByClassName("news")[0];
-      if (!contentBlock) {
-        return;
-      }
-      for (var level = 1; level <= 6; level++) {
-        linkifyAnchors(level, contentBlock);
-      }
-    }
-  };
-</script>
-
-
-</body>
-</html>

http://git-wip-us.apache.org/repos/asf/orc/blob/e58d3fc4/develop/bylaws/index.html
----------------------------------------------------------------------
diff --git a/develop/bylaws/index.html b/develop/bylaws/index.html
new file mode 100644
index 0000000..bd78ba2
--- /dev/null
+++ b/develop/bylaws/index.html
@@ -0,0 +1,448 @@
+<!DOCTYPE HTML>
+<html lang="en-US">
+<head>
+  <meta charset="UTF-8">
+  <title>Apache ORC Bylaws</title>
+  <meta name="viewport" content="width=device-width,initial-scale=1">
+  <meta name="generator" content="Jekyll v3.7.3">
+  <link rel="stylesheet" href="//fonts.googleapis.com/css?family=Lato:300,300italic,400,400italic,700,700italic,900">
+  <link rel="stylesheet" href="/css/screen.css">
+  <link rel="icon" type="image/x-icon" href="/favicon.ico">
+  <!--[if lt IE 9]>
+  <script src="/js/html5shiv.min.js"></script>
+  <script src="/js/respond.min.js"></script>
+  <![endif]-->
+</head>
+
+
+<body class="wrap">
+  <header role="banner">
+  <nav class="mobile-nav show-on-mobiles">
+    <ul>
+  <li class="">
+    <a href="/">Home</a>
+  </li>
+  <li class="">
+    <a href="/docs/"><span class="show-on-mobiles">Docs</span>
+                     <span class="hide-on-mobiles">Documentation</span></a>
+  </li>
+  <li class="">
+    <a href="/talks/">Talks</a>
+  </li>
+  <li class="">
+    <a href="/news/">News</a>
+  </li>
+  <li class="">
+    <a href="/help/">Help</a>
+  </li>
+  <li class="current">
+    <a href="/develop/">Develop</a>
+  </li>
+</ul>
+
+  </nav>
+  <div class="grid">
+    <div class="unit one-third center-on-mobiles">
+      <h1>
+        <a href="/">
+          <span class="sr-only">Apache ORC</span>
+          <img src="/img/logo.png" width="249" height="101" alt="ORC Logo">
+        </a>
+      </h1>
+    </div>
+    <nav class="main-nav unit two-thirds hide-on-mobiles">
+      <ul>
+  <li class="">
+    <a href="/">Home</a>
+  </li>
+  <li class="">
+    <a href="/docs/"><span class="show-on-mobiles">Docs</span>
+                     <span class="hide-on-mobiles">Documentation</span></a>
+  </li>
+  <li class="">
+    <a href="/talks/">Talks</a>
+  </li>
+  <li class="">
+    <a href="/news/">News</a>
+  </li>
+  <li class="">
+    <a href="/help/">Help</a>
+  </li>
+  <li class="current">
+    <a href="/develop/">Develop</a>
+  </li>
+</ul>
+
+    </nav>
+  </div>
+</header>
+
+
+  <section class="standalone">
+  <div class="grid">
+
+    <div class="unit whole">
+      <article>
+        <h1>Apache ORC Bylaws</h1>
+        <h2 id="introduction">Introduction</h2>
+
+<p>This document defines the bylaws under which the Apache ORC project
+operates. It defines the roles and responsibilities of the project,
+who may vote, how voting works, how conflicts are resolved, etc.</p>
+
+<p>ORC is a project of the Apache Software Foundation (ASF) and the foundation
+holds the trademark on the name “ORC” and copyright on the combined
+code base. The <a href="https://www.apache.org/foundation/faq.html">Apache Foundation
+FAQ</a> and
+<a href="https://www.apache.org/foundation/how-it-works.html">How-It-Works</a>
+explain the operation and background of the foundation.</p>
+
+<p>Apache has a <a href="https://www.apache.org/foundation/policies/conduct.html">code of
+conduct</a> that
+it expects its members to follow. In particular:</p>
+
+<ul>
+  <li>
+    <p>Be <strong>open and welcoming</strong>. It is important that we grow and
+encourage the community of users and developers for our project.</p>
+  </li>
+  <li>
+    <p>Be <strong>collaborative</strong>. Working together on the open mailing lists and
+bug database to make decisions helps the project grow.</p>
+  </li>
+  <li>
+    <p>Be <strong>respectful</strong> of others. Everyone is volunteering their time and
+efforts to work on this project. Please be respectful of everyone
+and their views.</p>
+  </li>
+</ul>
+
+<p>ORC is typical of Apache projects in that it operates under a set of
+principles, known collectively as the “Apache Way”. If you are new to
+Apache development, please refer to the <a href="https://incubator.apache.org/">Incubator
+project</a> for more
+information on how Apache projects operate.</p>
+
+<h2 id="roles-and-responsibilities">Roles and Responsibilities</h2>
+
+<p>Apache projects define a set of roles with associated rights and
+responsibilities. These roles govern what tasks an individual may
+perform within the project. The roles are defined in the following
+sections.</p>
+
+<h3 id="users">Users</h3>
+
+<p>The most important participants in the project are people who use our
+software. The majority of our developers start out as users and guide
+their development efforts from the user’s perspective.  Users
+contribute to the Apache projects by providing feedback to developers
+in the form of bug reports and feature suggestions. As well, users
+participate in the Apache community by helping other users on mailing
+lists and user support forums.</p>
+
+<h3 id="contributors">Contributors</h3>
+
+<p>Contributors include all of the volunteers who donate time, code,
+documentation,
+or resources to the ORC Project. A contributor that makes sustained,
+welcome contributions to the project may be invited to become a
+committer, though the exact timing of such invitations depends on many
+factors.</p>
+
+<h3 id="committers">Committers</h3>
+
+<p>The project’s committers are responsible for the project’s technical
+management. Committers have the right to commit to the project’s git
+repository. Committers may cast binding votes on any technical
+discussion.</p>
+
+<p>Committer access is by invitation only and must be approved by
+consensus approval of the active Project Management Committee (PMC)
+members.</p>
+
+<p>If a committer wishes to leave the project or does not contribute to
+the project in any form for six months, the PMC may make them emeritus.
+Emeritus committers lose their ability to commit code or cast binding
+votes. An emeritus committer may
+request reinstatement of commit access from the PMC. Such
+reinstatement is subject to consensus approval of active PMC members.</p>
+
+<p>All Apache committers are required to have a signed <a href="https://www.apache.org/licenses/icla.txt">Individual
+Contributor License
+Agreement</a> (ICLA) on file
+with the Apache Software Foundation. There is a <a href="https://www.apache.org/dev/committers.html">Committer
+FAQ</a> which provides more
+details on the requirements for Committers.</p>
+
+<p>A committer who makes a
+sustained contribution to the project may be invited to become a
+member of the PMC. The form of contribution
+is not limited to code. It can also include code review, helping out
+users on the mailing lists, documentation, testing, etc.</p>
+
+<h3 id="release-manager">Release Manager</h3>
+
+<p>A Release Manager (RM) is a committer who volunteers to produce a
+Release Candidate. The RM shall publish a Release Plan on the
+dev mailing list stating the branch from which they intend to
+make a Release Candidate.</p>
+
+<h3 id="project-management-committee">Project Management Committee</h3>
+
+<p>The Project Management Committee (PMC) for Apache ORC was created by
+the Apache Board in April 2015 when ORC moved out of Hive and became a
+top level project at Apache. The PMC is responsible to the board and
+the ASF for the management and oversight of the Apache ORC
+codebase. The responsibilities of the PMC include</p>
+
+<ul>
+  <li>
+    <p>Deciding what is distributed as products of the Apache ORC
+project. In particular all releases must be approved by the PMC.</p>
+  </li>
+  <li>
+    <p>Maintaining the project’s shared resources, including the codebase
+repository, mailing lists, and websites.</p>
+  </li>
+  <li>
+    <p>Speaking on behalf of the project.</p>
+  </li>
+  <li>
+    <p>Resolving license disputes regarding products of the project</p>
+  </li>
+  <li>
+    <p>Nominating new PMC members and committers</p>
+  </li>
+  <li>
+    <p>Maintaining these bylaws and other guidelines of the project</p>
+  </li>
+</ul>
+
+<p>Membership of the PMC is by invitation only and must be approved by a
+consensus approval of active PMC members.</p>
+
+<p>A PMC member is considered
+emeritus by their own declaration or by not contributing in any form
+to the project for over six months. An emeritus member may request
+reinstatement to the PMC. Such reinstatement is subject to consensus
+approval of the active PMC members.</p>
+
+<p>The chair of the PMC is appointed by the ASF board. The chair is an
+office holder of the Apache Software Foundation (Vice President,
+Apache ORC) and has primary responsibility to the board for the
+management of the project within the scope of the ORC PMC. The
+chair reports to the board quarterly on developments within the ORC
+project.</p>
+
+<p>When the project desires a new PMC chair, the PMC votes to recommend a
+new chair using <a href="https://wiki.apache.org/general/BoardVoting">Single Transferable
+Vote</a> voting. The decision
+must be ratified by the Apache board.</p>
+
+<h2 id="decision-making">Decision Making</h2>
+
+<p>Within the ORC project, different types of decisions require
+different forms of approval. For example, the previous section
+describes several decisions which require “consensus approval.”
+This section defines how voting is performed, the types of
+approvals, and which types of decision require which type of approval.</p>
+
+<h3 id="voting">Voting</h3>
+
+<p>Decisions regarding the project are made by votes on the primary
+project development mailing list (dev@orc.apache.org). Where
+necessary, PMC voting may take place on the private ORC PMC mailing
+list. Votes are clearly indicated by subject line starting with
+[VOTE]. Votes may contain multiple items for approval and these should
+be clearly separated. Voting is carried out by replying to the vote
+mail. Voting may take five flavors:</p>
+
+<ul>
+  <li>
+    <p><strong>+1</strong> – “Yes,” “Agree,” or “the action should be performed.” In general,
+this vote also indicates a willingness on the behalf of the voter in
+“making it happen.”</p>
+  </li>
+  <li>
+    <p><strong>+0</strong> – This vote indicates a willingness for the action under
+consideration to go ahead. The voter, however, will not be able to
+help.</p>
+  </li>
+  <li>
+    <p><strong>0</strong> – The voter is neutral on the topic under discussion.</p>
+  </li>
+  <li>
+    <p><strong>-0</strong> – This vote indicates that the voter does not, in general, agree
+ with the proposed action but is not concerned enough to prevent the
+ action going ahead.</p>
+  </li>
+  <li>
+    <p><strong>-1</strong> – This is a negative vote. On issues where consensus is required,
+ this vote counts as a veto. All vetoes must contain an explanation
+ of why the veto is appropriate. Vetoes with no explanation are
+ void. It may also be appropriate for a -1 vote to include an
+ alternative course of action.</p>
+  </li>
+</ul>
+
+<p>All participants in the ORC project are encouraged to show their
+agreement for or against a particular action by voting, regardless of
+whether their vote is binding. Nonbinding votes are useful for
+encouraging discussion and understanding the scope of opinions within
+the project.</p>
+
+<h3 id="approvals">Approvals</h3>
+
+<p>These are the types of approvals that can be sought. Different actions
+require different types of approvals.</p>
+
+<ul>
+  <li>
+    <p><strong>Consensus Approval</strong> – Consensus approval requires 3 binding +1
+votes and no binding vetoes.</p>
+  </li>
+  <li>
+    <p><strong>Lazy Consensus</strong> – Lazy consensus requires at least one +1 vote and
+no -1 votes (‘silence gives assent’).</p>
+  </li>
+  <li>
+    <p><strong>Lazy Majority</strong> – A lazy majority vote requires 3 binding +1 votes
+ and more binding +1 votes than -1 votes.</p>
+  </li>
+  <li>
+    <p><strong>Lazy 2/3 Majority</strong> – Lazy 2/3 majority votes requires at least 3
+votes and twice as many +1 votes as -1 votes.</p>
+  </li>
+</ul>
+
+<h3 id="vetoes">Vetoes</h3>
+
+<p>A valid, binding veto cannot be overruled. If a veto is cast, it must
+be accompanied by a valid reason explaining the reasons for the
+veto. The validity of a veto, if challenged, can be confirmed by
+anyone who has a binding vote. This does not necessarily signify
+agreement with the veto - merely that the veto is valid.  If you
+disagree with a valid veto, you must lobby the person casting the veto
+to withdraw their veto. If a veto is not withdrawn, any action that
+has already been taken  must be reversed in a timely manner.</p>
+
+<h3 id="actions">Actions</h3>
+
+<p>This section describes the various actions which are undertaken within
+the project, the corresponding approval required for that action and
+those who have binding votes over the action.</p>
+
+<h4 id="code-change">Code Change</h4>
+
+<p>A change made to a codebase of the project requires <em>lazy consensus</em>
+of active committers other than the author of the patch. We can commit
+changes before they have been reviewed, although we prefer to get
+reviews first.</p>
+
+<h4 id="product-release">Product Release</h4>
+
+<p>To make a release, the release manager creates a release candidate and
+a vote requiring a <em>lazy majority</em> of the active PMC members is
+required. Once the vote passes, the release candidate becomes an
+official release.</p>
+
+<h4 id="adoption-of-new-codebase">Adoption of New Codebase</h4>
+
+<p>When the codebase for an existing, released product is to be replaced
+with an alternative codebase, it requires a <em>lazy 2/3 majority</em> of PMC
+members. This also covers the creation of new sub-projects and
+submodules within the project.</p>
+
+<h4 id="new-committer">New Committer</h4>
+
+<p>When a new committer is proposed for the project, <em>consensus approval</em>
+of the active PMC members is required.</p>
+
+<h4 id="new-pmc-member">New PMC Member</h4>
+
+<p>To promote a committer to a PMC member requires <em>consensus approval</em>
+of active PMC members.</p>
+
+<p>If the vote passes, the Apache Board must be notified to make the change
+official.</p>
+
+<h4 id="committer-removal">Committer Removal</h4>
+
+<p>Removal of commit privileges requires a <em>lazy 2/3 majority</em> of active
+PMC members.</p>
+
+<h4 id="pmc-member-removal">PMC Member Removal</h4>
+
+<p>Removing a PMC member requires a <em>lazy 2/3 majority</em> of active PMC
+members, excluding the member in question.</p>
+
+<p>If the vote passes, the Apache Board must be notified to make the change
+official.</p>
+
+<h4 id="modifying-bylaws">Modifying Bylaws</h4>
+
+<p>Modifying this document requires a <em>lazy majority</em> of active PMC members.</p>
+
+<h3 id="voting-timeframes">Voting Timeframes</h3>
+
+<p>Votes are open for a minimum period of 72 hours to allow all active
+voters time to consider the vote. For holiday weekends or conferences,
+consider using a longer vote window. Votes relating to code changes are
+not subject to a strict timetable but should be made as timely as
+possible.</p>
+
+      </article>
+    </div>
+
+    <div class="clear"></div>
+
+  </div>
+</section>
+
+
+  <footer role="contentinfo">
+  <p>The contents of this website are &copy;&nbsp;2018
+     <a href="https://www.apache.org/">Apache Software Foundation</a>
+     under the terms of the <a
+      href="https://www.apache.org/licenses/LICENSE-2.0.html">
+      Apache&nbsp;License&nbsp;v2</a>. Apache ORC and its logo are trademarks
+      of the Apache Software Foundation.</p>
+</footer>
+
+  <script>
+  var anchorForId = function (id) {
+    var anchor = document.createElement("a");
+    anchor.className = "header-link";
+    anchor.href      = "#" + id;
+    anchor.innerHTML = "<span class=\"sr-only\">Permalink</span><i class=\"fa fa-link\"></i>";
+    anchor.title = "Permalink";
+    return anchor;
+  };
+
+  var linkifyAnchors = function (level, containingElement) {
+    var headers = containingElement.getElementsByTagName("h" + level);
+    for (var h = 0; h < headers.length; h++) {
+      var header = headers[h];
+
+      if (typeof header.id !== "undefined" && header.id !== "") {
+        header.appendChild(anchorForId(header.id));
+      }
+    }
+  };
+
+  document.onreadystatechange = function () {
+    if (this.readyState === "complete") {
+      var contentBlock = document.getElementsByClassName("docs")[0] || document.getElementsByClassName("news")[0];
+      if (!contentBlock) {
+        return;
+      }
+      for (var level = 1; level <= 6; level++) {
+        linkifyAnchors(level, contentBlock);
+      }
+    }
+  };
+</script>
+
+
+</body>
+</html>

http://git-wip-us.apache.org/repos/asf/orc/blob/e58d3fc4/develop/coding.html
----------------------------------------------------------------------
diff --git a/develop/coding.html b/develop/coding.html
deleted file mode 100644
index 579bfc7..0000000
--- a/develop/coding.html
+++ /dev/null
@@ -1,186 +0,0 @@
-<!DOCTYPE HTML>
-<html lang="en-US">
-<head>
-  <meta charset="UTF-8">
-  <title>Coding Guidelines</title>
-  <meta name="viewport" content="width=device-width,initial-scale=1">
-  <meta name="generator" content="Jekyll v2.4.0">
-  <link rel="stylesheet" href="//fonts.googleapis.com/css?family=Lato:300,300italic,400,400italic,700,700italic,900">
-  <link rel="stylesheet" href="/css/screen.css">
-  <link rel="icon" type="image/x-icon" href="/favicon.ico">
-  <!--[if lt IE 9]>
-  <script src="/js/html5shiv.min.js"></script>
-  <script src="/js/respond.min.js"></script>
-  <![endif]-->
-</head>
-
-
-<body class="wrap">
-  <header role="banner">
-  <nav class="mobile-nav show-on-mobiles">
-    <ul>
-  <li class="">
-    <a href="/">Home</a>
-  </li>
-  <li class="">
-    <a href="/docs/"><span class="show-on-mobiles">Docs</span>
-                     <span class="hide-on-mobiles">Documentation</span></a>
-  </li>
-  <li class="">
-    <a href="/talks/">Talks</a>
-  </li>
-  <li class="">
-    <a href="/news/">News</a>
-  </li>
-  <li class="">
-    <a href="/help/">Help</a>
-  </li>
-  <li class="current">
-    <a href="/develop/">Develop</a>
-  </li>
-</ul>
-
-  </nav>
-  <div class="grid">
-    <div class="unit one-third center-on-mobiles">
-      <h1>
-        <a href="/">
-          <span class="sr-only">Apache ORC</span>
-          <img src="/img/logo.png" width="249" height="101" alt="ORC Logo">
-        </a>
-      </h1>
-    </div>
-    <nav class="main-nav unit two-thirds hide-on-mobiles">
-      <ul>
-  <li class="">
-    <a href="/">Home</a>
-  </li>
-  <li class="">
-    <a href="/docs/"><span class="show-on-mobiles">Docs</span>
-                     <span class="hide-on-mobiles">Documentation</span></a>
-  </li>
-  <li class="">
-    <a href="/talks/">Talks</a>
-  </li>
-  <li class="">
-    <a href="/news/">News</a>
-  </li>
-  <li class="">
-    <a href="/help/">Help</a>
-  </li>
-  <li class="current">
-    <a href="/develop/">Develop</a>
-  </li>
-</ul>
-
-    </nav>
-  </div>
-</header>
-
-
-  <section class="standalone">
-  <div class="grid">
-
-    <div class="unit whole">
-      <article>
-        <h1>Coding Guidelines</h1>
-        <h2 id="general-rules">General rules</h2>
-
-<ul>
-  <li>All files must have an Apache copyright header at the top of the file.</li>
-  <li>Code should be removed rather than commented out.</li>
-  <li>All public functions should have javadoc comments.</li>
-  <li>Always use braces to surround branches.</li>
-  <li>try-finally should be avoided.</li>
-</ul>
-
-<h2 id="formatting">Formatting</h2>
-
-<ul>
-  <li>All files must have an 80 character maximum line length.</li>
-  <li>Indentation should be 2 spaces.</li>
-  <li>Files should use spaces instead of tabs.</li>
-  <li>Wrapping lines
-    <ul>
-      <li>Break after a comma.</li>
-      <li>Break before an operator.</li>
-      <li>Prefer higher-level breaks to lower-level breaks.</li>
-      <li>Align the new line with beginning of the expression at the same level
-on the previous line.</li>
-      <li>If the above rules lead to confusing code, just indent 8 spaces.</li>
-    </ul>
-  </li>
-  <li>One variable declaration per a line.</li>
-</ul>
-
-<h2 id="naming">Naming</h2>
-
-<ul>
-  <li>Packages should be all lowercase.
-    <ul>
-      <li>C++ code should be in <code>orc::</code>.</li>
-      <li>Java code should be in <code>org.apache.orc</code>, except for compatibility classes,
-which should be in <code>org.apache.hadoop.hive.ql.io.orc</code>.</li>
-    </ul>
-  </li>
-  <li>Classes should be in mixed case.</li>
-  <li>Variables should be in camel case.</li>
-  <li>Constants should be in upper case.</li>
-</ul>
-
-
-      </article>
-    </div>
-
-    <div class="clear"></div>
-
-  </div>
-</section>
-
-
-  <footer role="contentinfo">
-  <p>The contents of this website are &copy;&nbsp;2018
-     <a href="https://www.apache.org/">Apache Software Foundation</a>
-     under the terms of the <a
-      href="https://www.apache.org/licenses/LICENSE-2.0.html">
-      Apache&nbsp;License&nbsp;v2</a>. Apache ORC and its logo are trademarks
-      of the Apache Software Foundation.</p>
-</footer>
-
-  <script>
-  var anchorForId = function (id) {
-    var anchor = document.createElement("a");
-    anchor.className = "header-link";
-    anchor.href      = "#" + id;
-    anchor.innerHTML = "<span class=\"sr-only\">Permalink</span><i class=\"fa fa-link\"></i>";
-    anchor.title = "Permalink";
-    return anchor;
-  };
-
-  var linkifyAnchors = function (level, containingElement) {
-    var headers = containingElement.getElementsByTagName("h" + level);
-    for (var h = 0; h < headers.length; h++) {
-      var header = headers[h];
-
-      if (typeof header.id !== "undefined" && header.id !== "") {
-        header.appendChild(anchorForId(header.id));
-      }
-    }
-  };
-
-  document.onreadystatechange = function () {
-    if (this.readyState === "complete") {
-      var contentBlock = document.getElementsByClassName("docs")[0] || document.getElementsByClassName("news")[0];
-      if (!contentBlock) {
-        return;
-      }
-      for (var level = 1; level <= 6; level++) {
-        linkifyAnchors(level, contentBlock);
-      }
-    }
-  };
-</script>
-
-
-</body>
-</html>

http://git-wip-us.apache.org/repos/asf/orc/blob/e58d3fc4/develop/coding/index.html
----------------------------------------------------------------------
diff --git a/develop/coding/index.html b/develop/coding/index.html
new file mode 100644
index 0000000..fc7da4a
--- /dev/null
+++ b/develop/coding/index.html
@@ -0,0 +1,186 @@
+<!DOCTYPE HTML>
+<html lang="en-US">
+<head>
+  <meta charset="UTF-8">
+  <title>Coding Guidelines</title>
+  <meta name="viewport" content="width=device-width,initial-scale=1">
+  <meta name="generator" content="Jekyll v3.7.3">
+  <link rel="stylesheet" href="//fonts.googleapis.com/css?family=Lato:300,300italic,400,400italic,700,700italic,900">
+  <link rel="stylesheet" href="/css/screen.css">
+  <link rel="icon" type="image/x-icon" href="/favicon.ico">
+  <!--[if lt IE 9]>
+  <script src="/js/html5shiv.min.js"></script>
+  <script src="/js/respond.min.js"></script>
+  <![endif]-->
+</head>
+
+
+<body class="wrap">
+  <header role="banner">
+  <nav class="mobile-nav show-on-mobiles">
+    <ul>
+  <li class="">
+    <a href="/">Home</a>
+  </li>
+  <li class="">
+    <a href="/docs/"><span class="show-on-mobiles">Docs</span>
+                     <span class="hide-on-mobiles">Documentation</span></a>
+  </li>
+  <li class="">
+    <a href="/talks/">Talks</a>
+  </li>
+  <li class="">
+    <a href="/news/">News</a>
+  </li>
+  <li class="">
+    <a href="/help/">Help</a>
+  </li>
+  <li class="current">
+    <a href="/develop/">Develop</a>
+  </li>
+</ul>
+
+  </nav>
+  <div class="grid">
+    <div class="unit one-third center-on-mobiles">
+      <h1>
+        <a href="/">
+          <span class="sr-only">Apache ORC</span>
+          <img src="/img/logo.png" width="249" height="101" alt="ORC Logo">
+        </a>
+      </h1>
+    </div>
+    <nav class="main-nav unit two-thirds hide-on-mobiles">
+      <ul>
+  <li class="">
+    <a href="/">Home</a>
+  </li>
+  <li class="">
+    <a href="/docs/"><span class="show-on-mobiles">Docs</span>
+                     <span class="hide-on-mobiles">Documentation</span></a>
+  </li>
+  <li class="">
+    <a href="/talks/">Talks</a>
+  </li>
+  <li class="">
+    <a href="/news/">News</a>
+  </li>
+  <li class="">
+    <a href="/help/">Help</a>
+  </li>
+  <li class="current">
+    <a href="/develop/">Develop</a>
+  </li>
+</ul>
+
+    </nav>
+  </div>
+</header>
+
+
+  <section class="standalone">
+  <div class="grid">
+
+    <div class="unit whole">
+      <article>
+        <h1>Coding Guidelines</h1>
+        <h2 id="general-rules">General rules</h2>
+
+<ul>
+  <li>All files must have an Apache copyright header at the top of the file.</li>
+  <li>Code should be removed rather than commented out.</li>
+  <li>All public functions should have javadoc comments.</li>
+  <li>Always use braces to surround branches.</li>
+  <li>try-finally should be avoided.</li>
+</ul>
+
+<h2 id="formatting">Formatting</h2>
+
+<ul>
+  <li>All files must have an 80 character maximum line length.</li>
+  <li>Indentation should be 2 spaces.</li>
+  <li>Files should use spaces instead of tabs.</li>
+  <li>Wrapping lines
+    <ul>
+      <li>Break after a comma.</li>
+      <li>Break before an operator.</li>
+      <li>Prefer higher-level breaks to lower-level breaks.</li>
+      <li>Align the new line with beginning of the expression at the same level
+on the previous line.</li>
+      <li>If the above rules lead to confusing code, just indent 8 spaces.</li>
+    </ul>
+  </li>
+  <li>One variable declaration per a line.</li>
+</ul>
+
+<h2 id="naming">Naming</h2>
+
+<ul>
+  <li>Packages should be all lowercase.
+    <ul>
+      <li>C++ code should be in <code class="highlighter-rouge">orc::</code>.</li>
+      <li>Java code should be in <code class="highlighter-rouge">org.apache.orc</code>, except for compatibility classes,
+which should be in <code class="highlighter-rouge">org.apache.hadoop.hive.ql.io.orc</code>.</li>
+    </ul>
+  </li>
+  <li>Classes should be in mixed case.</li>
+  <li>Variables should be in camel case.</li>
+  <li>Constants should be in upper case.</li>
+</ul>
+
+
+      </article>
+    </div>
+
+    <div class="clear"></div>
+
+  </div>
+</section>
+
+
+  <footer role="contentinfo">
+  <p>The contents of this website are &copy;&nbsp;2018
+     <a href="https://www.apache.org/">Apache Software Foundation</a>
+     under the terms of the <a
+      href="https://www.apache.org/licenses/LICENSE-2.0.html">
+      Apache&nbsp;License&nbsp;v2</a>. Apache ORC and its logo are trademarks
+      of the Apache Software Foundation.</p>
+</footer>
+
+  <script>
+  var anchorForId = function (id) {
+    var anchor = document.createElement("a");
+    anchor.className = "header-link";
+    anchor.href      = "#" + id;
+    anchor.innerHTML = "<span class=\"sr-only\">Permalink</span><i class=\"fa fa-link\"></i>";
+    anchor.title = "Permalink";
+    return anchor;
+  };
+
+  var linkifyAnchors = function (level, containingElement) {
+    var headers = containingElement.getElementsByTagName("h" + level);
+    for (var h = 0; h < headers.length; h++) {
+      var header = headers[h];
+
+      if (typeof header.id !== "undefined" && header.id !== "") {
+        header.appendChild(anchorForId(header.id));
+      }
+    }
+  };
+
+  document.onreadystatechange = function () {
+    if (this.readyState === "complete") {
+      var contentBlock = document.getElementsByClassName("docs")[0] || document.getElementsByClassName("news")[0];
+      if (!contentBlock) {
+        return;
+      }
+      for (var level = 1; level <= 6; level++) {
+        linkifyAnchors(level, contentBlock);
+      }
+    }
+  };
+</script>
+
+
+</body>
+</html>

http://git-wip-us.apache.org/repos/asf/orc/blob/e58d3fc4/develop/committers.html
----------------------------------------------------------------------
diff --git a/develop/committers.html b/develop/committers.html
deleted file mode 100644
index 4831481..0000000
--- a/develop/committers.html
+++ /dev/null
@@ -1,359 +0,0 @@
-<!DOCTYPE HTML>
-<html lang="en-US">
-<head>
-  <meta charset="UTF-8">
-  <title>Project Members</title>
-  <meta name="viewport" content="width=device-width,initial-scale=1">
-  <meta name="generator" content="Jekyll v2.4.0">
-  <link rel="stylesheet" href="//fonts.googleapis.com/css?family=Lato:300,300italic,400,400italic,700,700italic,900">
-  <link rel="stylesheet" href="/css/screen.css">
-  <link rel="icon" type="image/x-icon" href="/favicon.ico">
-  <!--[if lt IE 9]>
-  <script src="/js/html5shiv.min.js"></script>
-  <script src="/js/respond.min.js"></script>
-  <![endif]-->
-</head>
-
-
-<body class="wrap">
-  <header role="banner">
-  <nav class="mobile-nav show-on-mobiles">
-    <ul>
-  <li class="">
-    <a href="/">Home</a>
-  </li>
-  <li class="">
-    <a href="/docs/"><span class="show-on-mobiles">Docs</span>
-                     <span class="hide-on-mobiles">Documentation</span></a>
-  </li>
-  <li class="">
-    <a href="/talks/">Talks</a>
-  </li>
-  <li class="">
-    <a href="/news/">News</a>
-  </li>
-  <li class="">
-    <a href="/help/">Help</a>
-  </li>
-  <li class="current">
-    <a href="/develop/">Develop</a>
-  </li>
-</ul>
-
-  </nav>
-  <div class="grid">
-    <div class="unit one-third center-on-mobiles">
-      <h1>
-        <a href="/">
-          <span class="sr-only">Apache ORC</span>
-          <img src="/img/logo.png" width="249" height="101" alt="ORC Logo">
-        </a>
-      </h1>
-    </div>
-    <nav class="main-nav unit two-thirds hide-on-mobiles">
-      <ul>
-  <li class="">
-    <a href="/">Home</a>
-  </li>
-  <li class="">
-    <a href="/docs/"><span class="show-on-mobiles">Docs</span>
-                     <span class="hide-on-mobiles">Documentation</span></a>
-  </li>
-  <li class="">
-    <a href="/talks/">Talks</a>
-  </li>
-  <li class="">
-    <a href="/news/">News</a>
-  </li>
-  <li class="">
-    <a href="/help/">Help</a>
-  </li>
-  <li class="current">
-    <a href="/develop/">Develop</a>
-  </li>
-</ul>
-
-    </nav>
-  </div>
-</header>
-
-
-  <section class="standalone">
-  <div class="grid">
-
-    <div class="unit whole">
-      <article>
-        <h1>Project Members</h1>
-        <h2 id="project-members">Project Members</h2>
-
-<table>
-  <thead>
-    <tr>
-      <th style="text-align: left">Name</th>
-      <th style="text-align: left">Apache Id</th>
-      <th style="text-align: left">Role</th>
-    </tr>
-  </thead>
-  <tbody>
-    <tr>
-      <td style="text-align: left">Aliaksei Sandryhaila</td>
-      <td style="text-align: left">asandryh</td>
-      <td style="text-align: left">PMC</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">Chris Douglas</td>
-      <td style="text-align: left">cdouglas</td>
-      <td style="text-align: left">PMC</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">Chinna Rao Lalam</td>
-      <td style="text-align: left">chinnaraol</td>
-      <td style="text-align: left">Committer</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">Chaoyu Tang</td>
-      <td style="text-align: left">ctang</td>
-      <td style="text-align: left">Committer</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">Carl Steinbach</td>
-      <td style="text-align: left">cws</td>
-      <td style="text-align: left">Committer</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">Daniel Dai</td>
-      <td style="text-align: left">daijy</td>
-      <td style="text-align: left">Committer</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">Deepak Majeti</td>
-      <td style="text-align: left">mdeepak</td>
-      <td style="text-align: left">PMC</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">Eugene Koifman</td>
-      <td style="text-align: left">ekoifman</td>
-      <td style="text-align: left">PMC</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">Gang Wu</td>
-      <td style="text-align: left">gangwu</td>
-      <td style="text-align: left">Committer</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">Alan Gates</td>
-      <td style="text-align: left">gates</td>
-      <td style="text-align: left">PMC</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">Gopal Vijayaraghavan</td>
-      <td style="text-align: left">gopalv</td>
-      <td style="text-align: left">PMC</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">Gunther Hagleitner</td>
-      <td style="text-align: left">gunther</td>
-      <td style="text-align: left">Committer</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">Ashutosh Chauhan</td>
-      <td style="text-align: left">hashutosh</td>
-      <td style="text-align: left">Committer</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">Jesus Camacho Rodriguez</td>
-      <td style="text-align: left">jcamacho</td>
-      <td style="text-align: left">Committer</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">Jason Dere</td>
-      <td style="text-align: left">jdere</td>
-      <td style="text-align: left">Committer</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">Jimmy Xiang</td>
-      <td style="text-align: left">jxiang</td>
-      <td style="text-align: left">Committer</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">Kevin Wilfong</td>
-      <td style="text-align: left">kevinwilfong</td>
-      <td style="text-align: left">Committer</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">Lars Francke</td>
-      <td style="text-align: left">larsfrancke</td>
-      <td style="text-align: left">Committer</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">Lefty Leverenz</td>
-      <td style="text-align: left">leftyl</td>
-      <td style="text-align: left">PMC</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">Rui Li</td>
-      <td style="text-align: left">lirui</td>
-      <td style="text-align: left">Committer</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">Mithun Radhakrishnan</td>
-      <td style="text-align: left">mithun</td>
-      <td style="text-align: left">Committer</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">Matthew McCline</td>
-      <td style="text-align: left">mmccline</td>
-      <td style="text-align: left">Committer</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">Naveen Gangam</td>
-      <td style="text-align: left">ngangam</td>
-      <td style="text-align: left">Committer</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">Owen O’Malley</td>
-      <td style="text-align: left">omalley</td>
-      <td style="text-align: left">PMC</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">Prasanth Jayachandran</td>
-      <td style="text-align: left">prasanthj</td>
-      <td style="text-align: left">PMC</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">Pengcheng Xiong</td>
-      <td style="text-align: left">pxiong</td>
-      <td style="text-align: left">Committer</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">Rajesh Balamohan</td>
-      <td style="text-align: left">rbalamohan</td>
-      <td style="text-align: left">Committer</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">Sergey Shelukhin</td>
-      <td style="text-align: left">sershe</td>
-      <td style="text-align: left">Committer</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">Sergio Pena</td>
-      <td style="text-align: left">spena</td>
-      <td style="text-align: left">Committer</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">Siddharth Seth</td>
-      <td style="text-align: left">sseth</td>
-      <td style="text-align: left">Committer</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">Stephen Walkauskas</td>
-      <td style="text-align: left">swalkaus</td>
-      <td style="text-align: left">Committer</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">Vaibhav Gumashta</td>
-      <td style="text-align: left">vgumashta</td>
-      <td style="text-align: left">Committer</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">Wei Zheng</td>
-      <td style="text-align: left">weiz</td>
-      <td style="text-align: left">Committer</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">Xiening Dai</td>
-      <td style="text-align: left">xndai</td>
-      <td style="text-align: left">Committer</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">Xuefu Zhang</td>
-      <td style="text-align: left">xuefu</td>
-      <td style="text-align: left">Committer</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">Ferdinand Xu</td>
-      <td style="text-align: left">xuf</td>
-      <td style="text-align: left">Committer</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">Yongzhi Chen</td>
-      <td style="text-align: left">ychena</td>
-      <td style="text-align: left">Committer</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">Aihua Xu</td>
-      <td style="text-align: left">zihuaxu</td>
-      <td style="text-align: left">Committer</td>
-    </tr>
-  </tbody>
-</table>
-
-<p>Companies with employees that are committers:</p>
-
-<ul>
-  <li>Alibaba</li>
-  <li>Cloudera</li>
-  <li>Facebook</li>
-  <li>Hortonworks</li>
-  <li>Intel</li>
-  <li>LinkedIn</li>
-  <li>Microsoft</li>
-  <li>Vertica</li>
-  <li>Yahoo</li>
-</ul>
-
-      </article>
-    </div>
-
-    <div class="clear"></div>
-
-  </div>
-</section>
-
-
-  <footer role="contentinfo">
-  <p>The contents of this website are &copy;&nbsp;2018
-     <a href="https://www.apache.org/">Apache Software Foundation</a>
-     under the terms of the <a
-      href="https://www.apache.org/licenses/LICENSE-2.0.html">
-      Apache&nbsp;License&nbsp;v2</a>. Apache ORC and its logo are trademarks
-      of the Apache Software Foundation.</p>
-</footer>
-
-  <script>
-  var anchorForId = function (id) {
-    var anchor = document.createElement("a");
-    anchor.className = "header-link";
-    anchor.href      = "#" + id;
-    anchor.innerHTML = "<span class=\"sr-only\">Permalink</span><i class=\"fa fa-link\"></i>";
-    anchor.title = "Permalink";
-    return anchor;
-  };
-
-  var linkifyAnchors = function (level, containingElement) {
-    var headers = containingElement.getElementsByTagName("h" + level);
-    for (var h = 0; h < headers.length; h++) {
-      var header = headers[h];
-
-      if (typeof header.id !== "undefined" && header.id !== "") {
-        header.appendChild(anchorForId(header.id));
-      }
-    }
-  };
-
-  document.onreadystatechange = function () {
-    if (this.readyState === "complete") {
-      var contentBlock = document.getElementsByClassName("docs")[0] || document.getElementsByClassName("news")[0];
-      if (!contentBlock) {
-        return;
-      }
-      for (var level = 1; level <= 6; level++) {
-        linkifyAnchors(level, contentBlock);
-      }
-    }
-  };
-</script>
-
-
-</body>
-</html>

http://git-wip-us.apache.org/repos/asf/orc/blob/e58d3fc4/develop/committers/index.html
----------------------------------------------------------------------
diff --git a/develop/committers/index.html b/develop/committers/index.html
new file mode 100644
index 0000000..f8e6c85
--- /dev/null
+++ b/develop/committers/index.html
@@ -0,0 +1,359 @@
+<!DOCTYPE HTML>
+<html lang="en-US">
+<head>
+  <meta charset="UTF-8">
+  <title>Project Members</title>
+  <meta name="viewport" content="width=device-width,initial-scale=1">
+  <meta name="generator" content="Jekyll v3.7.3">
+  <link rel="stylesheet" href="//fonts.googleapis.com/css?family=Lato:300,300italic,400,400italic,700,700italic,900">
+  <link rel="stylesheet" href="/css/screen.css">
+  <link rel="icon" type="image/x-icon" href="/favicon.ico">
+  <!--[if lt IE 9]>
+  <script src="/js/html5shiv.min.js"></script>
+  <script src="/js/respond.min.js"></script>
+  <![endif]-->
+</head>
+
+
+<body class="wrap">
+  <header role="banner">
+  <nav class="mobile-nav show-on-mobiles">
+    <ul>
+  <li class="">
+    <a href="/">Home</a>
+  </li>
+  <li class="">
+    <a href="/docs/"><span class="show-on-mobiles">Docs</span>
+                     <span class="hide-on-mobiles">Documentation</span></a>
+  </li>
+  <li class="">
+    <a href="/talks/">Talks</a>
+  </li>
+  <li class="">
+    <a href="/news/">News</a>
+  </li>
+  <li class="">
+    <a href="/help/">Help</a>
+  </li>
+  <li class="current">
+    <a href="/develop/">Develop</a>
+  </li>
+</ul>
+
+  </nav>
+  <div class="grid">
+    <div class="unit one-third center-on-mobiles">
+      <h1>
+        <a href="/">
+          <span class="sr-only">Apache ORC</span>
+          <img src="/img/logo.png" width="249" height="101" alt="ORC Logo">
+        </a>
+      </h1>
+    </div>
+    <nav class="main-nav unit two-thirds hide-on-mobiles">
+      <ul>
+  <li class="">
+    <a href="/">Home</a>
+  </li>
+  <li class="">
+    <a href="/docs/"><span class="show-on-mobiles">Docs</span>
+                     <span class="hide-on-mobiles">Documentation</span></a>
+  </li>
+  <li class="">
+    <a href="/talks/">Talks</a>
+  </li>
+  <li class="">
+    <a href="/news/">News</a>
+  </li>
+  <li class="">
+    <a href="/help/">Help</a>
+  </li>
+  <li class="current">
+    <a href="/develop/">Develop</a>
+  </li>
+</ul>
+
+    </nav>
+  </div>
+</header>
+
+
+  <section class="standalone">
+  <div class="grid">
+
+    <div class="unit whole">
+      <article>
+        <h1>Project Members</h1>
+        <h2 id="project-members">Project Members</h2>
+
+<table>
+  <thead>
+    <tr>
+      <th style="text-align: left">Name</th>
+      <th style="text-align: left">Apache Id</th>
+      <th style="text-align: left">Role</th>
+    </tr>
+  </thead>
+  <tbody>
+    <tr>
+      <td style="text-align: left">Aliaksei Sandryhaila</td>
+      <td style="text-align: left">asandryh</td>
+      <td style="text-align: left">PMC</td>
+    </tr>
+    <tr>
+      <td style="text-align: left">Chris Douglas</td>
+      <td style="text-align: left">cdouglas</td>
+      <td style="text-align: left">PMC</td>
+    </tr>
+    <tr>
+      <td style="text-align: left">Chinna Rao Lalam</td>
+      <td style="text-align: left">chinnaraol</td>
+      <td style="text-align: left">Committer</td>
+    </tr>
+    <tr>
+      <td style="text-align: left">Chaoyu Tang</td>
+      <td style="text-align: left">ctang</td>
+      <td style="text-align: left">Committer</td>
+    </tr>
+    <tr>
+      <td style="text-align: left">Carl Steinbach</td>
+      <td style="text-align: left">cws</td>
+      <td style="text-align: left">Committer</td>
+    </tr>
+    <tr>
+      <td style="text-align: left">Daniel Dai</td>
+      <td style="text-align: left">daijy</td>
+      <td style="text-align: left">Committer</td>
+    </tr>
+    <tr>
+      <td style="text-align: left">Deepak Majeti</td>
+      <td style="text-align: left">mdeepak</td>
+      <td style="text-align: left">PMC</td>
+    </tr>
+    <tr>
+      <td style="text-align: left">Eugene Koifman</td>
+      <td style="text-align: left">ekoifman</td>
+      <td style="text-align: left">PMC</td>
+    </tr>
+    <tr>
+      <td style="text-align: left">Gang Wu</td>
+      <td style="text-align: left">gangwu</td>
+      <td style="text-align: left">Committer</td>
+    </tr>
+    <tr>
+      <td style="text-align: left">Alan Gates</td>
+      <td style="text-align: left">gates</td>
+      <td style="text-align: left">PMC</td>
+    </tr>
+    <tr>
+      <td style="text-align: left">Gopal Vijayaraghavan</td>
+      <td style="text-align: left">gopalv</td>
+      <td style="text-align: left">PMC</td>
+    </tr>
+    <tr>
+      <td style="text-align: left">Gunther Hagleitner</td>
+      <td style="text-align: left">gunther</td>
+      <td style="text-align: left">Committer</td>
+    </tr>
+    <tr>
+      <td style="text-align: left">Ashutosh Chauhan</td>
+      <td style="text-align: left">hashutosh</td>
+      <td style="text-align: left">Committer</td>
+    </tr>
+    <tr>
+      <td style="text-align: left">Jesus Camacho Rodriguez</td>
+      <td style="text-align: left">jcamacho</td>
+      <td style="text-align: left">Committer</td>
+    </tr>
+    <tr>
+      <td style="text-align: left">Jason Dere</td>
+      <td style="text-align: left">jdere</td>
+      <td style="text-align: left">Committer</td>
+    </tr>
+    <tr>
+      <td style="text-align: left">Jimmy Xiang</td>
+      <td style="text-align: left">jxiang</td>
+      <td style="text-align: left">Committer</td>
+    </tr>
+    <tr>
+      <td style="text-align: left">Kevin Wilfong</td>
+      <td style="text-align: left">kevinwilfong</td>
+      <td style="text-align: left">Committer</td>
+    </tr>
+    <tr>
+      <td style="text-align: left">Lars Francke</td>
+      <td style="text-align: left">larsfrancke</td>
+      <td style="text-align: left">Committer</td>
+    </tr>
+    <tr>
+      <td style="text-align: left">Lefty Leverenz</td>
+      <td style="text-align: left">leftyl</td>
+      <td style="text-align: left">PMC</td>
+    </tr>
+    <tr>
+      <td style="text-align: left">Rui Li</td>
+      <td style="text-align: left">lirui</td>
+      <td style="text-align: left">Committer</td>
+    </tr>
+    <tr>
+      <td style="text-align: left">Mithun Radhakrishnan</td>
+      <td style="text-align: left">mithun</td>
+      <td style="text-align: left">Committer</td>
+    </tr>
+    <tr>
+      <td style="text-align: left">Matthew McCline</td>
+      <td style="text-align: left">mmccline</td>
+      <td style="text-align: left">Committer</td>
+    </tr>
+    <tr>
+      <td style="text-align: left">Naveen Gangam</td>
+      <td style="text-align: left">ngangam</td>
+      <td style="text-align: left">Committer</td>
+    </tr>
+    <tr>
+      <td style="text-align: left">Owen O’Malley</td>
+      <td style="text-align: left">omalley</td>
+      <td style="text-align: left">PMC</td>
+    </tr>
+    <tr>
+      <td style="text-align: left">Prasanth Jayachandran</td>
+      <td style="text-align: left">prasanthj</td>
+      <td style="text-align: left">PMC</td>
+    </tr>
+    <tr>
+      <td style="text-align: left">Pengcheng Xiong</td>
+      <td style="text-align: left">pxiong</td>
+      <td style="text-align: left">Committer</td>
+    </tr>
+    <tr>
+      <td style="text-align: left">Rajesh Balamohan</td>
+      <td style="text-align: left">rbalamohan</td>
+      <td style="text-align: left">Committer</td>
+    </tr>
+    <tr>
+      <td style="text-align: left">Sergey Shelukhin</td>
+      <td style="text-align: left">sershe</td>
+      <td style="text-align: left">Committer</td>
+    </tr>
+    <tr>
+      <td style="text-align: left">Sergio Pena</td>
+      <td style="text-align: left">spena</td>
+      <td style="text-align: left">Committer</td>
+    </tr>
+    <tr>
+      <td style="text-align: left">Siddharth Seth</td>
+      <td style="text-align: left">sseth</td>
+      <td style="text-align: left">Committer</td>
+    </tr>
+    <tr>
+      <td style="text-align: left">Stephen Walkauskas</td>
+      <td style="text-align: left">swalkaus</td>
+      <td style="text-align: left">Committer</td>
+    </tr>
+    <tr>
+      <td style="text-align: left">Vaibhav Gumashta</td>
+      <td style="text-align: left">vgumashta</td>
+      <td style="text-align: left">Committer</td>
+    </tr>
+    <tr>
+      <td style="text-align: left">Wei Zheng</td>
+      <td style="text-align: left">weiz</td>
+      <td style="text-align: left">Committer</td>
+    </tr>
+    <tr>
+      <td style="text-align: left">Xiening Dai</td>
+      <td style="text-align: left">xndai</td>
+      <td style="text-align: left">Committer</td>
+    </tr>
+    <tr>
+      <td style="text-align: left">Xuefu Zhang</td>
+      <td style="text-align: left">xuefu</td>
+      <td style="text-align: left">Committer</td>
+    </tr>
+    <tr>
+      <td style="text-align: left">Ferdinand Xu</td>
+      <td style="text-align: left">xuf</td>
+      <td style="text-align: left">Committer</td>
+    </tr>
+    <tr>
+      <td style="text-align: left">Yongzhi Chen</td>
+      <td style="text-align: left">ychena</td>
+      <td style="text-align: left">Committer</td>
+    </tr>
+    <tr>
+      <td style="text-align: left">Aihua Xu</td>
+      <td style="text-align: left">zihuaxu</td>
+      <td style="text-align: left">Committer</td>
+    </tr>
+  </tbody>
+</table>
+
+<p>Companies with employees that are committers:</p>
+
+<ul>
+  <li>Alibaba</li>
+  <li>Cloudera</li>
+  <li>Facebook</li>
+  <li>Hortonworks</li>
+  <li>Intel</li>
+  <li>LinkedIn</li>
+  <li>Microsoft</li>
+  <li>Vertica</li>
+  <li>Yahoo</li>
+</ul>
+
+      </article>
+    </div>
+
+    <div class="clear"></div>
+
+  </div>
+</section>
+
+
+  <footer role="contentinfo">
+  <p>The contents of this website are &copy;&nbsp;2018
+     <a href="https://www.apache.org/">Apache Software Foundation</a>
+     under the terms of the <a
+      href="https://www.apache.org/licenses/LICENSE-2.0.html">
+      Apache&nbsp;License&nbsp;v2</a>. Apache ORC and its logo are trademarks
+      of the Apache Software Foundation.</p>
+</footer>
+
+  <script>
+  var anchorForId = function (id) {
+    var anchor = document.createElement("a");
+    anchor.className = "header-link";
+    anchor.href      = "#" + id;
+    anchor.innerHTML = "<span class=\"sr-only\">Permalink</span><i class=\"fa fa-link\"></i>";
+    anchor.title = "Permalink";
+    return anchor;
+  };
+
+  var linkifyAnchors = function (level, containingElement) {
+    var headers = containingElement.getElementsByTagName("h" + level);
+    for (var h = 0; h < headers.length; h++) {
+      var header = headers[h];
+
+      if (typeof header.id !== "undefined" && header.id !== "") {
+        header.appendChild(anchorForId(header.id));
+      }
+    }
+  };
+
+  document.onreadystatechange = function () {
+    if (this.readyState === "complete") {
+      var contentBlock = document.getElementsByClassName("docs")[0] || document.getElementsByClassName("news")[0];
+      if (!contentBlock) {
+        return;
+      }
+      for (var level = 1; level <= 6; level++) {
+        linkifyAnchors(level, contentBlock);
+      }
+    }
+  };
+</script>
+
+
+</body>
+</html>

http://git-wip-us.apache.org/repos/asf/orc/blob/e58d3fc4/develop/index.html
----------------------------------------------------------------------
diff --git a/develop/index.html b/develop/index.html
index d9224d0..5a08cf0 100644
--- a/develop/index.html
+++ b/develop/index.html
@@ -4,7 +4,7 @@
   <meta charset="UTF-8">
   <title>Developing</title>
   <meta name="viewport" content="width=device-width,initial-scale=1">
-  <meta name="generator" content="Jekyll v2.4.0">
+  <meta name="generator" content="Jekyll v3.7.3">
   <link rel="stylesheet" href="//fonts.googleapis.com/css?family=Lato:300,300italic,400,400italic,700,700italic,900">
   <link rel="stylesheet" href="/css/screen.css">
   <link rel="icon" type="image/x-icon" href="/favicon.ico">
@@ -105,11 +105,11 @@ publicly archived and work well asynchronously across timezones.</p>
 lists for ORC:</p>
 
 <ul>
-  <li><a href="&#109;&#097;&#105;&#108;&#116;&#111;:&#100;&#101;&#118;&#064;&#111;&#114;&#099;&#046;&#097;&#112;&#097;&#099;&#104;&#101;&#046;&#111;&#114;&#103;">&#100;&#101;&#118;&#064;&#111;&#114;&#099;&#046;&#097;&#112;&#097;&#099;&#104;&#101;&#046;&#111;&#114;&#103;</a> - Development discussions
+  <li><a href="mailto:dev@orc.apache.org">dev@orc.apache.org</a> - Development discussions
 with archive <a href="https://mail-archives.apache.org/mod_mbox/orc-dev/">here</a></li>
-  <li><a href="&#109;&#097;&#105;&#108;&#116;&#111;:&#105;&#115;&#115;&#117;&#101;&#115;&#064;&#111;&#114;&#099;&#046;&#097;&#112;&#097;&#099;&#104;&#101;&#046;&#111;&#114;&#103;">&#105;&#115;&#115;&#117;&#101;&#115;&#064;&#111;&#114;&#099;&#046;&#097;&#112;&#097;&#099;&#104;&#101;&#046;&#111;&#114;&#103;</a> - Bug tracking
+  <li><a href="mailto:issues@orc.apache.org">issues@orc.apache.org</a> - Bug tracking
 with archive <a href="https://mail-archives.apache.org/mod_mbox/orc-issues/">here</a></li>
-  <li><a href="&#109;&#097;&#105;&#108;&#116;&#111;:&#099;&#111;&#109;&#109;&#105;&#116;&#115;&#064;&#111;&#114;&#099;&#046;&#097;&#112;&#097;&#099;&#104;&#101;&#046;&#111;&#114;&#103;">&#099;&#111;&#109;&#109;&#105;&#116;&#115;&#064;&#111;&#114;&#099;&#046;&#097;&#112;&#097;&#099;&#104;&#101;&#046;&#111;&#114;&#103;</a> - Git tracking
+  <li><a href="mailto:commits@orc.apache.org">commits@orc.apache.org</a> - Git tracking
 with archive <a href="https://mail-archives.apache.org/mod_mbox/orc-commits/">here</a></li>
 </ul>
 
@@ -127,17 +127,17 @@ discussion of the change.</p>
 <p>ORC uses git for version control. Get the source code and configure it
 to fetch the pull requests also:</p>
 
-<pre><code>% git clone -o apache git@github.com:apache/orc.git
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>% git clone -o apache git@github.com:apache/orc.git
 % cd orc
 % git config --add remote.apache.fetch '+refs/pull/*/head:refs/remotes/apache/pr/*'
-</code></pre>
+</code></pre></div></div>
 
 <p>Pull requests will be named “apache/pr/999” for pull request 999.</p>
 
 <p>If you are a committer, add the push url:</p>
 
-<pre><code>% git remote set-url --push apache https://git-wip-us.apache.org/repos/asf/orc.git
-</code></pre>
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>% git remote set-url --push apache https://git-wip-us.apache.org/repos/asf/orc.git
+</code></pre></div></div>
 
 <p>The important branches are:</p>
 
@@ -178,25 +178,25 @@ committer’s review. However, most changes should be reviewed first.</p>
 
 <p>Fetch the current state of the project:</p>
 
-<pre><code>% git fetch apache
-</code></pre>
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>% git fetch apache
+</code></pre></div></div>
 
 <p>Switch to the branch:</p>
 
-<pre><code>% git checkout apache/pr/999
-</code></pre>
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>% git checkout apache/pr/999
+</code></pre></div></div>
 
 <p>You’ll want to rebase it and make it a single commit by squashing
 the commits into a single commit.</p>
 
-<pre><code>% git rebase -i apache/master
-</code></pre>
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>% git rebase -i apache/master
+</code></pre></div></div>
 
 <p>Update the commit message to sign it using your GPG key and close the
 pull request:</p>
 
-<pre><code>% git commit --amend -s -S
-</code></pre>
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>% git commit --amend -s -S
+</code></pre></div></div>
 
 <p>Ensure the first line of the commit starts with the jira number
 (eg. ORC-123) and includes a description of what was changed. Also add
@@ -206,39 +206,39 @@ claiming to have fixed the problem, the form “Closes #999” also works.</p>
 
 <p>Finally, push the result to Apache:</p>
 
-<pre><code>% git push apache HEAD:master
-</code></pre>
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>% git push apache HEAD:master
+</code></pre></div></div>
 
 <h2 id="creating-a-gpg-key">Creating a GPG key</h2>
 
 <p>When you become a committer, you should create a 4096 bit GPG key.</p>
 
-<pre><code>% gpg --full-gen-key
-</code></pre>
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>% gpg --full-gen-key
+</code></pre></div></div>
 
 <p>Use 4096 bits and your Apache email address. Once it is created,
 you’ll need to get your key fingerprint. Avoid using the short
 fingerprint (eg. 3D0C92B9), because it is possible to generate fake
 keys that have the same short fingerprint as the real key.</p>
 
-<pre><code>% gpg --list-secret-keys --keyid-format LONG
-</code></pre>
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>% gpg --list-secret-keys --keyid-format LONG
+</code></pre></div></div>
 
 <p>Your key fingerprint is the string after “rsa4096/”. Example output
 for the key with fingerprint 1209E7F13D0C92B9 looks like:</p>
 
-<pre><code>/Users/owen/.gnupg/pubring.gpg
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>/Users/owen/.gnupg/pubring.gpg
 ------------------------------
 sec   rsa4096/1209E7F13D0C92B9 2010-02-23 [SC]
       47660BC98BC433F01E5C90581209E7F13D0C92B9
 uid                 [ultimate] Owen O'Malley &lt;omalley@apache.org&gt;
-</code></pre>
+</code></pre></div></div>
 
 <p>Now publish your public key to one of the public keyservers. I usually use
 hkp://pgp.mit.edu, although any of them will work.</p>
 
-<pre><code>gpg --send-key &lt;your key fingerprint&gt;
-</code></pre>
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>gpg --send-key &lt;your key fingerprint&gt;
+</code></pre></div></div>
 
 <p>Next, you need to update the <a href="https://id.apache.org">Apache account
 database</a> with your new key. Login to add your

http://git-wip-us.apache.org/repos/asf/orc/blob/e58d3fc4/develop/make-release.html
----------------------------------------------------------------------
diff --git a/develop/make-release.html b/develop/make-release.html
deleted file mode 100644
index e8354f8..0000000
--- a/develop/make-release.html
+++ /dev/null
@@ -1,324 +0,0 @@
-<!DOCTYPE HTML>
-<html lang="en-US">
-<head>
-  <meta charset="UTF-8">
-  <title>How To Release ORC</title>
-  <meta name="viewport" content="width=device-width,initial-scale=1">
-  <meta name="generator" content="Jekyll v2.4.0">
-  <link rel="stylesheet" href="//fonts.googleapis.com/css?family=Lato:300,300italic,400,400italic,700,700italic,900">
-  <link rel="stylesheet" href="/css/screen.css">
-  <link rel="icon" type="image/x-icon" href="/favicon.ico">
-  <!--[if lt IE 9]>
-  <script src="/js/html5shiv.min.js"></script>
-  <script src="/js/respond.min.js"></script>
-  <![endif]-->
-</head>
-
-
-<body class="wrap">
-  <header role="banner">
-  <nav class="mobile-nav show-on-mobiles">
-    <ul>
-  <li class="">
-    <a href="/">Home</a>
-  </li>
-  <li class="">
-    <a href="/docs/"><span class="show-on-mobiles">Docs</span>
-                     <span class="hide-on-mobiles">Documentation</span></a>
-  </li>
-  <li class="">
-    <a href="/talks/">Talks</a>
-  </li>
-  <li class="">
-    <a href="/news/">News</a>
-  </li>
-  <li class="">
-    <a href="/help/">Help</a>
-  </li>
-  <li class="current">
-    <a href="/develop/">Develop</a>
-  </li>
-</ul>
-
-  </nav>
-  <div class="grid">
-    <div class="unit one-third center-on-mobiles">
-      <h1>
-        <a href="/">
-          <span class="sr-only">Apache ORC</span>
-          <img src="/img/logo.png" width="249" height="101" alt="ORC Logo">
-        </a>
-      </h1>
-    </div>
-    <nav class="main-nav unit two-thirds hide-on-mobiles">
-      <ul>
-  <li class="">
-    <a href="/">Home</a>
-  </li>
-  <li class="">
-    <a href="/docs/"><span class="show-on-mobiles">Docs</span>
-                     <span class="hide-on-mobiles">Documentation</span></a>
-  </li>
-  <li class="">
-    <a href="/talks/">Talks</a>
-  </li>
-  <li class="">
-    <a href="/news/">News</a>
-  </li>
-  <li class="">
-    <a href="/help/">Help</a>
-  </li>
-  <li class="current">
-    <a href="/develop/">Develop</a>
-  </li>
-</ul>
-
-    </nav>
-  </div>
-</header>
-
-
-  <section class="standalone">
-  <div class="grid">
-
-    <div class="unit whole">
-      <article>
-        <h1>How To Release ORC</h1>
-        <h2 id="preparing-for-release">Preparing for release</h2>
-
-<p>Set version so that it isn’t a SNAPSHOT.</p>
-
-<pre><code>% git checkout branch-X.Y
-% edit CMakeLists.txt
-% (mkdir build; cd build; cmake ..)
-</code></pre>
-
-<p>Commit the changes back to Apache along with a tag for the release candidate.</p>
-
-<pre><code>% git commit -s -S -am 'Preparing for release X.Y.Z'
-% git remote add apache https://git-wip-us.apache.org/repos/asf/orc.git
-% git push apache branch-X.Y
-% git tag release-X.Y.Zrc0
-% git push apache release-X.Y.Zrc0
-</code></pre>
-
-<p>Generate the source tarball and checksums for the release.</p>
-
-<pre><code>% wget https://github.com/apache/orc/archive/release-X.Y.Zrc0.tar.gz
-% tar xzf release-X.Y.Zrc0.tar.gz
-% mv orc-release-X.Y.Zrc0 orc-X.Y.Z
-% tar czf orc-X.Y.Zrc0.tar.gz orc-X.Y.Z
-% shasum -a 256 orc-X.Y.Zrc0.tar.gz &gt; orc-X.Y.Zrc0.tar.gz.sha256
-% gpg --detach-sig --armor orc-X.Y.Zrc0.tar.gz
-</code></pre>
-
-<p>Copy the artifacts into your personal Apache website.</p>
-
-<pre><code>% sftp &lt;apacheid&gt;@home.apache.org
-sftp&gt; cd public_html
-sftp&gt; mkdir orc-X.Y.Zrc0
-sftp&gt; cd orc-X.Y.Zrc0
-sftp&gt; put orc-X.Y.Zrc0*
-sftp&gt; quit
-</code></pre>
-
-<p>Make sure your GPG key exists <a href="https://dist.apache.org/repos/dist/release/orc/KEYS">here</a> for others to verify signature in RC</p>
-
-<p>Click the version to release (X.Y.Z) <a href="https://issues.apache.org/jira/projects/ORC?selectedItem=com.atlassian.jira.jira-projects-plugin:release-page">here</a>
-to get the list of jiras that are fixed in X.Y.Z</p>
-
-<p>Send email with the vote:</p>
-
-<pre><code>To: dev@orc.apache.org
-Subject: [VOTE] Should we release ORC X.Y.Zrc0?
-
-All,
-
-Should we release the following artifacts as ORC X.Y.Z?
-
-tar: http://home.apache.org/~omalley/orc-X.Y.Zrc0/
-tag: https://github.com/apache/orc/releases/tag/release-X.Y.Zrc0
-jiras: https://issues.apache.org/jira/browse/ORC/fixforversion/&lt;fixid&gt;
-
-Thanks!
-</code></pre>
-
-<h2 id="to-promote-a-release-candidate-rc-to-a-real-release">To promote a release candidate (RC) to a real release.</h2>
-
-<p>Update the final tag and remove the rc tag.</p>
-
-<pre><code>% git tag rel/release-X.Y.Z -s
-% git push apache rel/release-X.Y.Z
-% git push apache :release-X.Y.Zrc0
-</code></pre>
-
-<p>Publish the artifacts to Maven central staging. Make sure to have this <a href="http://www.apache.org/dev/publishing-maven-artifacts.html#dev-env">setup</a> for Apache releases</p>
-
-<pre><code>% cd java
-% mvn -Papache-release clean deploy
-</code></pre>
-
-<p>Publish from the staging area:</p>
-
-<ul>
-  <li>login to <a href="https://repository.apache.org/index.html#stagingRepositories">Maven staging</a></li>
-  <li>find your staging repository (search for org.apache.org)</li>
-  <li>close it</li>
-  <li>release it</li>
-</ul>
-
-<p>Publish the artifacts to Apache’s dist area.</p>
-
-<pre><code>In a svn clone of https://dist.apache.org/repos/dist/release/orc
-% mkdir orc-X.Y.Z
-% cd orc-X.Y.Z
-copy release artifacts with a rename from orc-X.Y.Zrc0* to orc-X.Y.Z*
-% svn add .
-% svn commit ---username &lt;apacheid&gt; -m "ORC X.Y.Z Release"
-
-We keep the latest patch release for each of the last two branches, so remove
-extra releases (say I.J.K) from the Apache dist area.
-
-% cd ..
-% svn rm orc-I.J.K
-% svn commit --username &lt;apacheid&gt; -m "Removed old release ORC I.J.K"
-</code></pre>
-
-<p>Update the release branch with the version for the next release.</p>
-
-<pre><code>edit CMakeLists.txt to change version to X.Y.(Z+1)-SNAPSHOT
-% cd build
-% cmake ..
-% git commit -a -s -S -am 'Preparing branch for post X.Y.Z development'
-% git push apache branch-X.Y
-</code></pre>
-
-<p>Update the site with the new release.</p>
-
-<ul>
-  <li>Check out the master branch (git co apache/master)</li>
-</ul>
-
-<pre><code>Change directory in to site.
-% pwd
-&lt;path-to-master-src&gt;
-% cd site
-% mkdir target
-% cd target
-Set up site/target to be a separate git workspace that tracks the asf-site branch.
-% git init
-% git remote add origin https://git-wip-us.apache.org/repos/asf/orc.git -t asf-site
-% git fetch origin
-% git checkout asf-site
-% cd ..
-% bundle install
-</code></pre>
-<ul>
-  <li>edit site/_data/releases.yml to add new release
-    <ul>
-      <li>update the state for the releases to match the changes in the Apache dist
-        <ul>
-          <li>latest = new release (only one of these!)</li>
-          <li>stable = other release still in dist</li>
-          <li>archived = removed from dist</li>
-        </ul>
-      </li>
-    </ul>
-  </li>
-  <li>create a new file _posts/YYYY-MM-DD-ORC-X.Y.Z.md for the news section</li>
-  <li>Run “bundle exec jekyll serve”</li>
-  <li>Check the website on http://0.0.0.0:4000/</li>
-  <li>If it looks good, use git add (from within site directory) to add the new files and commit to master with a message of “update site for X.Y.Z”.</li>
-</ul>
-
-<pre><code>% pwd
-&lt;path-to-master-src&gt;/site
-% git commit -am "Update site for X.Y.Z"
-% git push origin master
-</code></pre>
-
-<ul>
-  <li>Change directory into site/target for publishing the site.</li>
-  <li>Add the new files that you just generated.
-    <ul>
-      <li>This assumes you’ve set up site/target to be a separate git workspace that tracks the asf-site branch.</li>
-    </ul>
-  </li>
-  <li>Commit to asf-site to publish the updated site.</li>
-</ul>
-
-<pre><code>% cd target
-% pwd
-&lt;path-to-master-src&gt;/site/target
-% git commit -am "Publish site for X.Y.Z"
-% git push origin asf-site
-</code></pre>
-
-<p>Update ORC’s jira to reflect the released version.</p>
-
-<ul>
-  <li>Select the resolved issues and bulk transition them to closed with following query <a href="https://issues.apache.org/jira/issues/?filter=-1">here</a>.
-    <ul>
-      <li>query: project = ORC AND fixVersion = X.Y.Z and status = Resolved ORDER BY created desc</li>
-    </ul>
-  </li>
-  <li>Mark the version as released and set the date <a href="https://issues.apache.org/jira/projects/ORC?selectedItem=com.atlassian.jira.jira-projects-plugin:release-page">here</a>.</li>
-</ul>
-
-<p>It usually take up to 24 hours for the apache dist mirrors and maven central to update with the new release.</p>
-
-      </article>
-    </div>
-
-    <div class="clear"></div>
-
-  </div>
-</section>
-
-
-  <footer role="contentinfo">
-  <p>The contents of this website are &copy;&nbsp;2018
-     <a href="https://www.apache.org/">Apache Software Foundation</a>
-     under the terms of the <a
-      href="https://www.apache.org/licenses/LICENSE-2.0.html">
-      Apache&nbsp;License&nbsp;v2</a>. Apache ORC and its logo are trademarks
-      of the Apache Software Foundation.</p>
-</footer>
-
-  <script>
-  var anchorForId = function (id) {
-    var anchor = document.createElement("a");
-    anchor.className = "header-link";
-    anchor.href      = "#" + id;
-    anchor.innerHTML = "<span class=\"sr-only\">Permalink</span><i class=\"fa fa-link\"></i>";
-    anchor.title = "Permalink";
-    return anchor;
-  };
-
-  var linkifyAnchors = function (level, containingElement) {
-    var headers = containingElement.getElementsByTagName("h" + level);
-    for (var h = 0; h < headers.length; h++) {
-      var header = headers[h];
-
-      if (typeof header.id !== "undefined" && header.id !== "") {
-        header.appendChild(anchorForId(header.id));
-      }
-    }
-  };
-
-  document.onreadystatechange = function () {
-    if (this.readyState === "complete") {
-      var contentBlock = document.getElementsByClassName("docs")[0] || document.getElementsByClassName("news")[0];
-      if (!contentBlock) {
-        return;
-      }
-      for (var level = 1; level <= 6; level++) {
-        linkifyAnchors(level, contentBlock);
-      }
-    }
-  };
-</script>
-
-
-</body>
-</html>


[09/11] orc git commit: Deploy docs with CVE.

Posted by om...@apache.org.
http://git-wip-us.apache.org/repos/asf/orc/blob/e58d3fc4/docs/core-java.html
----------------------------------------------------------------------
diff --git a/docs/core-java.html b/docs/core-java.html
index ca4e99b..4424fa9 100644
--- a/docs/core-java.html
+++ b/docs/core-java.html
@@ -4,7 +4,7 @@
   <meta charset="UTF-8">
   <title>Using Core Java</title>
   <meta name="viewport" content="width=device-width,initial-scale=1">
-  <meta name="generator" content="Jekyll v2.4.0">
+  <meta name="generator" content="Jekyll v3.7.3">
   <link rel="stylesheet" href="//fonts.googleapis.com/css?family=Lato:300,300italic,400,400italic,700,700italic,900">
   <link rel="stylesheet" href="/css/screen.css">
   <link rel="icon" type="image/x-icon" href="/favicon.ico">
@@ -109,7 +109,7 @@
     
   
     
-      <option value="/docs/index.html">Background</option>
+      <option value="http://localhost:4000/docs/index.html">Background</option>
     
   
     
@@ -131,7 +131,7 @@
     
   
     
-      <option value="/docs/adopters.html">ORC Adopters</option>
+      <option value="http://localhost:4000/docs/adopters.html">ORC Adopters</option>
     
   
     
@@ -193,7 +193,7 @@
     
   
     
-      <option value="/docs/types.html">Types</option>
+      <option value="http://localhost:4000/docs/types.html">Types</option>
     
   
 
@@ -219,7 +219,7 @@
     
   
     
-      <option value="/docs/indexes.html">Indexes</option>
+      <option value="http://localhost:4000/docs/indexes.html">Indexes</option>
     
   
     
@@ -237,7 +237,7 @@
 
   
     
-      <option value="/docs/acid.html">ACID support</option>
+      <option value="http://localhost:4000/docs/acid.html">ACID support</option>
     
   
     
@@ -284,7 +284,7 @@
     
   
     
-      <option value="/docs/building.html">Building ORC</option>
+      <option value="http://localhost:4000/docs/building.html">Building ORC</option>
     
   
     
@@ -342,7 +342,7 @@
     
   
     
-      <option value="/docs/releases.html">Releases</option>
+      <option value="http://localhost:4000/docs/releases.html">Releases</option>
     
   
     
@@ -373,7 +373,7 @@
     
   
     
-      <option value="/docs/hive-ddl.html">Hive DDL</option>
+      <option value="http://localhost:4000/docs/hive-ddl.html">Hive DDL</option>
     
   
     
@@ -407,7 +407,7 @@
     
   
     
-      <option value="/docs/hive-config.html">Hive Configuration</option>
+      <option value="http://localhost:4000/docs/hive-config.html">Hive Configuration</option>
     
   
     
@@ -460,7 +460,7 @@
     
   
     
-      <option value="/docs/mapred.html">Using in MapRed</option>
+      <option value="http://localhost:4000/docs/mapred.html">Using in MapRed</option>
     
   
     
@@ -498,7 +498,7 @@
     
   
     
-      <option value="/docs/mapreduce.html">Using in MapReduce</option>
+      <option value="http://localhost:4000/docs/mapreduce.html">Using in MapReduce</option>
     
   
     
@@ -525,7 +525,7 @@
     
   
     
-      <option value="/docs/core-java.html">Using Core Java</option>
+      <option value="http://localhost:4000/docs/core-java.html">Using Core Java</option>
     
   
     
@@ -559,7 +559,7 @@
     
   
     
-      <option value="/docs/core-cpp.html">Using Core C++</option>
+      <option value="http://localhost:4000/docs/core-cpp.html">Using Core C++</option>
     
   
     
@@ -606,7 +606,7 @@
     
   
     
-      <option value="/docs/cpp-tools.html">C++ Tools</option>
+      <option value="http://localhost:4000/docs/cpp-tools.html">C++ Tools</option>
     
   
     
@@ -652,7 +652,7 @@
     
   
     
-      <option value="/docs/java-tools.html">Java Tools</option>
+      <option value="http://localhost:4000/docs/java-tools.html">Java Tools</option>
     
   
     
@@ -681,46 +681,46 @@ read and write the data.</p>
 <h2 id="vectorized-row-batch">Vectorized Row Batch</h2>
 
 <p>Data is passed to ORC as instances of
-<a href="/api/hive-storage-api/index.html?org/apache/hadoop/hive/ql/exec/vector/VectorizedRowBatch.html">VectorizedRowBatch</a>
+<a href="http://localhost:4000/api/hive-storage-api/index.html?org/apache/hadoop/hive/ql/exec/vector/VectorizedRowBatch.html">VectorizedRowBatch</a>
 that contain the data for 1024 rows. The focus is on speed and
-accessing the data fields directly. <code>cols</code> is an array of
-<a href="/api/hive-storage-api/index.html?org/apache/hadoop/hive/ql/exec/vector/ColumnVector.html">ColumnVector</a>
-and <code>size</code> is the number of rows.</p>
+accessing the data fields directly. <code class="highlighter-rouge">cols</code> is an array of
+<a href="http://localhost:4000/api/hive-storage-api/index.html?org/apache/hadoop/hive/ql/exec/vector/ColumnVector.html">ColumnVector</a>
+and <code class="highlighter-rouge">size</code> is the number of rows.</p>
 
-<pre><code class="language-java">package org.apache.hadoop.hive.ql.exec.vector;
+<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">package</span> <span class="n">org</span><span class="o">.</span><span class="na">apache</span><span class="o">.</span><span class="na">hadoop</span><span class="o">.</span><span class="na">hive</span><span class="o">.</span><span class="na">ql</span><span class="o">.</span><span class="na">exec</span><span class="o">.</span><span class="na">vector</span><span class="o">;</span>
 
-public class VectorizedRowBatch {
-  public ColumnVector[] cols;
-  public int size;
-  ...
-}
-</code></pre>
+<span class="kd">public</span> <span class="kd">class</span> <span class="nc">VectorizedRowBatch</span> <span class="o">{</span>
+  <span class="kd">public</span> <span class="n">ColumnVector</span><span class="o">[]</span> <span class="n">cols</span><span class="o">;</span>
+  <span class="kd">public</span> <span class="kt">int</span> <span class="n">size</span><span class="o">;</span>
+  <span class="o">...</span>
+<span class="o">}</span>
+</code></pre></div></div>
 
-<p><a href="/api/hive-storage-api/index.html?org/apache/hadoop/hive/ql/exec/vector/ColumnVector.html">ColumnVector</a>
+<p><a href="http://localhost:4000/api/hive-storage-api/index.html?org/apache/hadoop/hive/ql/exec/vector/ColumnVector.html">ColumnVector</a>
 is the parent type of the different kinds of columns and has some
 fields that are shared across all of the column types. In particular,
-the <code>noNulls</code> flag if there are no nulls in this column for this batch
-and the <code>isRepeating</code> flag for columns were the entire batch is the
-same value. For columns where <code>noNulls == false</code> the <code>isNull</code> array is
+the <code class="highlighter-rouge">noNulls</code> flag if there are no nulls in this column for this batch
+and the <code class="highlighter-rouge">isRepeating</code> flag for columns were the entire batch is the
+same value. For columns where <code class="highlighter-rouge">noNulls == false</code> the <code class="highlighter-rouge">isNull</code> array is
 true if that value is null.</p>
 
-<pre><code class="language-java">public abstract class ColumnVector {
+<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">public</span> <span class="kd">abstract</span> <span class="kd">class</span> <span class="nc">ColumnVector</span> <span class="o">{</span>
 
-  // If the whole column vector has no nulls, this is true, otherwise false.
-  public boolean noNulls;
+  <span class="c1">// If the whole column vector has no nulls, this is true, otherwise false.</span>
+  <span class="kd">public</span> <span class="kt">boolean</span> <span class="n">noNulls</span><span class="o">;</span>
 
-  // If hasNulls is true, then this array contains true if the value is
-  // is null, otherwise false.
-  public boolean[] isNull;
+  <span class="c1">// If hasNulls is true, then this array contains true if the value is</span>
+  <span class="c1">// is null, otherwise false.</span>
+  <span class="kd">public</span> <span class="kt">boolean</span><span class="o">[]</span> <span class="n">isNull</span><span class="o">;</span>
 
-  /*
+  <span class="cm">/*
    * True if same value repeats for whole column vector.
    * If so, vector[0] holds the repeating value.
-   */
-  public boolean isRepeating;
-  ...
-}
-</code></pre>
+   */</span>
+  <span class="kd">public</span> <span class="kt">boolean</span> <span class="n">isRepeating</span><span class="o">;</span>
+  <span class="o">...</span>
+<span class="o">}</span>
+</code></pre></div></div>
 
 <p>The subtypes of ColumnVector are:</p>
 
@@ -734,240 +734,240 @@ true if that value is null.</p>
   <tbody>
     <tr>
       <td>array</td>
-      <td><a href="/api/hive-storage-api/index.html?org/apache/hadoop/hive/ql/exec/vector/ListColumnVector.html">ListColumnVector</a></td>
+      <td><a href="http://localhost:4000/api/hive-storage-api/index.html?org/apache/hadoop/hive/ql/exec/vector/ListColumnVector.html">ListColumnVector</a></td>
     </tr>
     <tr>
       <td>binary</td>
-      <td><a href="/api/hive-storage-api/index.html?org/apache/hadoop/hive/ql/exec/vector/BytesColumnVector.html">BytesColumnVector</a></td>
+      <td><a href="http://localhost:4000/api/hive-storage-api/index.html?org/apache/hadoop/hive/ql/exec/vector/BytesColumnVector.html">BytesColumnVector</a></td>
     </tr>
     <tr>
       <td>bigint</td>
-      <td><a href="/api/hive-storage-api/index.html?org/apache/hadoop/hive/ql/exec/vector/LongColumnVector.html">LongColumnVector</a></td>
+      <td><a href="http://localhost:4000/api/hive-storage-api/index.html?org/apache/hadoop/hive/ql/exec/vector/LongColumnVector.html">LongColumnVector</a></td>
     </tr>
     <tr>
       <td>boolean</td>
-      <td><a href="/api/hive-storage-api/index.html?org/apache/hadoop/hive/ql/exec/vector/LongColumnVector.html">LongColumnVector</a></td>
+      <td><a href="http://localhost:4000/api/hive-storage-api/index.html?org/apache/hadoop/hive/ql/exec/vector/LongColumnVector.html">LongColumnVector</a></td>
     </tr>
     <tr>
       <td>char</td>
-      <td><a href="/api/hive-storage-api/index.html?org/apache/hadoop/hive/ql/exec/vector/BytesColumnVector.html">BytesColumnVector</a></td>
+      <td><a href="http://localhost:4000/api/hive-storage-api/index.html?org/apache/hadoop/hive/ql/exec/vector/BytesColumnVector.html">BytesColumnVector</a></td>
     </tr>
     <tr>
       <td>date</td>
-      <td><a href="/api/hive-storage-api/index.html?org/apache/hadoop/hive/ql/exec/vector/LongColumnVector.html">LongColumnVector</a></td>
+      <td><a href="http://localhost:4000/api/hive-storage-api/index.html?org/apache/hadoop/hive/ql/exec/vector/LongColumnVector.html">LongColumnVector</a></td>
     </tr>
     <tr>
       <td>decimal</td>
-      <td><a href="/api/hive-storage-api/index.html?org/apache/hadoop/hive/ql/exec/vector/DecimalColumnVector.html">DecimalColumnVector</a></td>
+      <td><a href="http://localhost:4000/api/hive-storage-api/index.html?org/apache/hadoop/hive/ql/exec/vector/DecimalColumnVector.html">DecimalColumnVector</a></td>
     </tr>
     <tr>
       <td>double</td>
-      <td><a href="/api/hive-storage-api/index.html?org/apache/hadoop/hive/ql/exec/vector/DoubleColumnVector.html">DoubleColumnVector</a></td>
+      <td><a href="http://localhost:4000/api/hive-storage-api/index.html?org/apache/hadoop/hive/ql/exec/vector/DoubleColumnVector.html">DoubleColumnVector</a></td>
     </tr>
     <tr>
       <td>float</td>
-      <td><a href="/api/hive-storage-api/index.html?org/apache/hadoop/hive/ql/exec/vector/DoubleColumnVector.html">DoubleColumnVector</a></td>
+      <td><a href="http://localhost:4000/api/hive-storage-api/index.html?org/apache/hadoop/hive/ql/exec/vector/DoubleColumnVector.html">DoubleColumnVector</a></td>
     </tr>
     <tr>
       <td>int</td>
-      <td><a href="/api/hive-storage-api/index.html?org/apache/hadoop/hive/ql/exec/vector/LongColumnVector.html">LongColumnVector</a></td>
+      <td><a href="http://localhost:4000/api/hive-storage-api/index.html?org/apache/hadoop/hive/ql/exec/vector/LongColumnVector.html">LongColumnVector</a></td>
     </tr>
     <tr>
       <td>map</td>
-      <td><a href="/api/hive-storage-api/index.html?org/apache/hadoop/hive/ql/exec/vector/MapColumnVector.html">MapColumnVector</a></td>
+      <td><a href="http://localhost:4000/api/hive-storage-api/index.html?org/apache/hadoop/hive/ql/exec/vector/MapColumnVector.html">MapColumnVector</a></td>
     </tr>
     <tr>
       <td>smallint</td>
-      <td><a href="/api/hive-storage-api/index.html?org/apache/hadoop/hive/ql/exec/vector/LongColumnVector.html">LongColumnVector</a></td>
+      <td><a href="http://localhost:4000/api/hive-storage-api/index.html?org/apache/hadoop/hive/ql/exec/vector/LongColumnVector.html">LongColumnVector</a></td>
     </tr>
     <tr>
       <td>string</td>
-      <td><a href="/api/hive-storage-api/index.html?org/apache/hadoop/hive/ql/exec/vector/BytesColumnVector.html">BytesColumnVector</a></td>
+      <td><a href="http://localhost:4000/api/hive-storage-api/index.html?org/apache/hadoop/hive/ql/exec/vector/BytesColumnVector.html">BytesColumnVector</a></td>
     </tr>
     <tr>
       <td>struct</td>
-      <td><a href="/api/hive-storage-api/index.html?org/apache/hadoop/hive/ql/exec/vector/StructColumnVector.html">StructColumnVector</a></td>
+      <td><a href="http://localhost:4000/api/hive-storage-api/index.html?org/apache/hadoop/hive/ql/exec/vector/StructColumnVector.html">StructColumnVector</a></td>
     </tr>
     <tr>
       <td>timestamp</td>
-      <td><a href="/api/hive-storage-api/index.html?org/apache/hadoop/hive/ql/exec/vector/TimestampColumnVector.html">TimestampColumnVector</a></td>
+      <td><a href="http://localhost:4000/api/hive-storage-api/index.html?org/apache/hadoop/hive/ql/exec/vector/TimestampColumnVector.html">TimestampColumnVector</a></td>
     </tr>
     <tr>
       <td>tinyint</td>
-      <td><a href="/api/hive-storage-api/index.html?org/apache/hadoop/hive/ql/exec/vector/LongColumnVector.html">LongColumnVector</a></td>
+      <td><a href="http://localhost:4000/api/hive-storage-api/index.html?org/apache/hadoop/hive/ql/exec/vector/LongColumnVector.html">LongColumnVector</a></td>
     </tr>
     <tr>
       <td>uniontype</td>
-      <td><a href="/api/hive-storage-api/index.html?org/apache/hadoop/hive/ql/exec/vector/UnionColumnVector.html">UnionColumnVector</a></td>
+      <td><a href="http://localhost:4000/api/hive-storage-api/index.html?org/apache/hadoop/hive/ql/exec/vector/UnionColumnVector.html">UnionColumnVector</a></td>
     </tr>
     <tr>
       <td>varchar</td>
-      <td><a href="/api/hive-storage-api/index.html?org/apache/hadoop/hive/ql/exec/vector/BytesColumnVector.html">BytesColumnVector</a></td>
+      <td><a href="http://localhost:4000/api/hive-storage-api/index.html?org/apache/hadoop/hive/ql/exec/vector/BytesColumnVector.html">BytesColumnVector</a></td>
     </tr>
   </tbody>
 </table>
 
-<p><a href="/api/hive-storage-api/index.html?org/apache/hadoop/hive/ql/exec/vector/LongColumnVector.html">LongColumnVector</a> handles all of the integer types (boolean, bigint,
+<p><a href="http://localhost:4000/api/hive-storage-api/index.html?org/apache/hadoop/hive/ql/exec/vector/LongColumnVector.html">LongColumnVector</a> handles all of the integer types (boolean, bigint,
 date, int, smallint, and tinyint). The data is represented as an array of
 longs where each value is sign-extended as necessary.</p>
 
-<pre><code class="language-java">public class LongColumnVector extends ColumnVector {
-  public long[] vector;
-  ...
-}
-</code></pre>
+<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">public</span> <span class="kd">class</span> <span class="nc">LongColumnVector</span> <span class="kd">extends</span> <span class="n">ColumnVector</span> <span class="o">{</span>
+  <span class="kd">public</span> <span class="kt">long</span><span class="o">[]</span> <span class="n">vector</span><span class="o">;</span>
+  <span class="o">...</span>
+<span class="o">}</span>
+</code></pre></div></div>
 
-<p><a href="/api/hive-storage-api/index.html?org/apache/hadoop/hive/ql/exec/vector/TimestampColumnVector.html">TimestampColumnVector</a>
+<p><a href="http://localhost:4000/api/hive-storage-api/index.html?org/apache/hadoop/hive/ql/exec/vector/TimestampColumnVector.html">TimestampColumnVector</a>
 handles timestamp values. The data is represented as an array of longs
 and an array of ints.</p>
 
-<pre><code class="language-java">public class TimestampColumnVector extends ColumnVector {
+<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">public</span> <span class="kd">class</span> <span class="nc">TimestampColumnVector</span> <span class="kd">extends</span> <span class="n">ColumnVector</span> <span class="o">{</span>
 
-  // the number of milliseconds since 1 Jan 1970 00:00 GMT
-  public long[] time;
+  <span class="c1">// the number of milliseconds since 1 Jan 1970 00:00 GMT</span>
+  <span class="kd">public</span> <span class="kt">long</span><span class="o">[]</span> <span class="n">time</span><span class="o">;</span>
 
-  // the number of nanoseconds within the second
-  public int[] nanos
-  ...
-}
-</code></pre>
+  <span class="c1">// the number of nanoseconds within the second</span>
+  <span class="kd">public</span> <span class="kt">int</span><span class="o">[]</span> <span class="n">nanos</span>
+  <span class="o">...</span>
+<span class="o">}</span>
+</code></pre></div></div>
 
-<p><a href="/api/hive-storage-api/index.html?org/apache/hadoop/hive/ql/exec/vector/DoubleColumnVector.html">DoubleColumnVector</a>
+<p><a href="http://localhost:4000/api/hive-storage-api/index.html?org/apache/hadoop/hive/ql/exec/vector/DoubleColumnVector.html">DoubleColumnVector</a>
 handles all of the floating point types (double, and float). The data
 is represented as an array of doubles.</p>
 
-<pre><code class="language-java">public class DoubleColumnVector extends ColumnVector {
-  public double[] vector;
-  ...
-}
-</code></pre>
+<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">public</span> <span class="kd">class</span> <span class="nc">DoubleColumnVector</span> <span class="kd">extends</span> <span class="n">ColumnVector</span> <span class="o">{</span>
+  <span class="kd">public</span> <span class="kt">double</span><span class="o">[]</span> <span class="n">vector</span><span class="o">;</span>
+  <span class="o">...</span>
+<span class="o">}</span>
+</code></pre></div></div>
 
-<p><a href="/api/hive-storage-api/index.html?org/apache/hadoop/hive/ql/exec/vector/DecimalColumnVector.html">DecimalColumnVector</a>
+<p><a href="http://localhost:4000/api/hive-storage-api/index.html?org/apache/hadoop/hive/ql/exec/vector/DecimalColumnVector.html">DecimalColumnVector</a>
 handles decimal columns. The data is represented as an array of
 HiveDecimalWritable. Note that this implementation is not performant
 and will likely be replaced.</p>
 
-<pre><code class="language-java">public class DecimalColumnVector extends ColumnVector {
-  public HiveDecimalWritable[] vector;
-  ...
-}
-</code></pre>
+<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">public</span> <span class="kd">class</span> <span class="nc">DecimalColumnVector</span> <span class="kd">extends</span> <span class="n">ColumnVector</span> <span class="o">{</span>
+  <span class="kd">public</span> <span class="n">HiveDecimalWritable</span><span class="o">[]</span> <span class="n">vector</span><span class="o">;</span>
+  <span class="o">...</span>
+<span class="o">}</span>
+</code></pre></div></div>
 
-<p><a href="/api/hive-storage-api/index.html?org/apache/hadoop/hive/ql/exec/vector/BytesColumnVector.html">BytesColumnVector</a>
+<p><a href="http://localhost:4000/api/hive-storage-api/index.html?org/apache/hadoop/hive/ql/exec/vector/BytesColumnVector.html">BytesColumnVector</a>
 handles all of the binary types (binary, char, string, and
 varchar). The data is represented as a byte array, offset, and
 length. The byte arrays may or may not be shared between values.</p>
 
-<pre><code class="language-java">public class BytesColumnVector extends ColumnVector {
-  public byte[][] vector;
-  public int[] start;
-  public int[] length;
-  ...
-}
-</code></pre>
+<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">public</span> <span class="kd">class</span> <span class="nc">BytesColumnVector</span> <span class="kd">extends</span> <span class="n">ColumnVector</span> <span class="o">{</span>
+  <span class="kd">public</span> <span class="kt">byte</span><span class="o">[][]</span> <span class="n">vector</span><span class="o">;</span>
+  <span class="kd">public</span> <span class="kt">int</span><span class="o">[]</span> <span class="n">start</span><span class="o">;</span>
+  <span class="kd">public</span> <span class="kt">int</span><span class="o">[]</span> <span class="n">length</span><span class="o">;</span>
+  <span class="o">...</span>
+<span class="o">}</span>
+</code></pre></div></div>
 
-<p><a href="/api/hive-storage-api/index.html?org/apache/hadoop/hive/ql/exec/vector/StructColumnVector.html">StructColumnVector</a>
+<p><a href="http://localhost:4000/api/hive-storage-api/index.html?org/apache/hadoop/hive/ql/exec/vector/StructColumnVector.html">StructColumnVector</a>
 handles the struct columns and represents the data as an array of
-<code>ColumnVector</code>. The value for row 5 consists of the fifth value from
-each of the <code>fields</code> values.</p>
+<code class="highlighter-rouge">ColumnVector</code>. The value for row 5 consists of the fifth value from
+each of the <code class="highlighter-rouge">fields</code> values.</p>
 
-<pre><code class="language-java">public class StructColumnVector extends ColumnVector {
-  public ColumnVector[] fields;
-  ...
-}
-</code></pre>
+<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">public</span> <span class="kd">class</span> <span class="nc">StructColumnVector</span> <span class="kd">extends</span> <span class="n">ColumnVector</span> <span class="o">{</span>
+  <span class="kd">public</span> <span class="n">ColumnVector</span><span class="o">[]</span> <span class="n">fields</span><span class="o">;</span>
+  <span class="o">...</span>
+<span class="o">}</span>
+</code></pre></div></div>
 
-<p><a href="/api/hive-storage-api/index.html?org/apache/hadoop/hive/ql/exec/vector/UnionColumnVector.html">UnionColumnVector</a>
+<p><a href="http://localhost:4000/api/hive-storage-api/index.html?org/apache/hadoop/hive/ql/exec/vector/UnionColumnVector.html">UnionColumnVector</a>
 handles the union columns and represents the data as an array of
-integers that pick the subtype and a <code>fields</code> array one per a
-subtype. Only the value of the <code>fields</code> that corresponds to
-<code>tags[row]</code> is set.</p>
-
-<pre><code class="language-java">public class UnionColumnVector extends ColumnVector {
-  public int[] tags;
-  public ColumnVector[] fields;
-  ...
-}
-</code></pre>
-
-<p><a href="/api/hive-storage-api/index.html?org/apache/hadoop/hive/ql/exec/vector/ListColumnVector.html">ListColumnVector</a>
+integers that pick the subtype and a <code class="highlighter-rouge">fields</code> array one per a
+subtype. Only the value of the <code class="highlighter-rouge">fields</code> that corresponds to
+<code class="highlighter-rouge">tags[row]</code> is set.</p>
+
+<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">public</span> <span class="kd">class</span> <span class="nc">UnionColumnVector</span> <span class="kd">extends</span> <span class="n">ColumnVector</span> <span class="o">{</span>
+  <span class="kd">public</span> <span class="kt">int</span><span class="o">[]</span> <span class="n">tags</span><span class="o">;</span>
+  <span class="kd">public</span> <span class="n">ColumnVector</span><span class="o">[]</span> <span class="n">fields</span><span class="o">;</span>
+  <span class="o">...</span>
+<span class="o">}</span>
+</code></pre></div></div>
+
+<p><a href="http://localhost:4000/api/hive-storage-api/index.html?org/apache/hadoop/hive/ql/exec/vector/ListColumnVector.html">ListColumnVector</a>
 handles the array columns and represents the data as two arrays of
-integers for the offset and lengths and a <code>ColumnVector</code> for the
+integers for the offset and lengths and a <code class="highlighter-rouge">ColumnVector</code> for the
 children values.</p>
 
-<pre><code class="language-java">public class ListColumnVector extends ColumnVector {
-  // for each row, the first offset of the child
-  public long[] offsets;
-  // for each row, the number of elements in the array
-  public long[] lengths;
-  // the offset in the child that should be used for new values
-  public int childCount;
-
-  // the values of the children
-  public ColumnVector child;
-  ...
-}
-</code></pre>
-
-<p><a href="/api/hive-storage-api/index.html?org/apache/hadoop/hive/ql/exec/vector/MapColumnVector.html">MapColumnVector</a>
+<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">public</span> <span class="kd">class</span> <span class="nc">ListColumnVector</span> <span class="kd">extends</span> <span class="n">ColumnVector</span> <span class="o">{</span>
+  <span class="c1">// for each row, the first offset of the child</span>
+  <span class="kd">public</span> <span class="kt">long</span><span class="o">[]</span> <span class="n">offsets</span><span class="o">;</span>
+  <span class="c1">// for each row, the number of elements in the array</span>
+  <span class="kd">public</span> <span class="kt">long</span><span class="o">[]</span> <span class="n">lengths</span><span class="o">;</span>
+  <span class="c1">// the offset in the child that should be used for new values</span>
+  <span class="kd">public</span> <span class="kt">int</span> <span class="n">childCount</span><span class="o">;</span>
+
+  <span class="c1">// the values of the children</span>
+  <span class="kd">public</span> <span class="n">ColumnVector</span> <span class="n">child</span><span class="o">;</span>
+  <span class="o">...</span>
+<span class="o">}</span>
+</code></pre></div></div>
+
+<p><a href="http://localhost:4000/api/hive-storage-api/index.html?org/apache/hadoop/hive/ql/exec/vector/MapColumnVector.html">MapColumnVector</a>
 handles the map columns and represents the data as two arrays of
-integers for the offset and lengths and two <code>ColumnVector</code>s for the
+integers for the offset and lengths and two <code class="highlighter-rouge">ColumnVector</code>s for the
 keys and values.</p>
 
-<pre><code class="language-java">public class MapColumnVector extends ColumnVector {
-  // for each row, the first offset of the child
-  public long[] offsets;
-  // for each row, the number of elements in the array
-  public long[] lengths;
-  // the offset in the child that should be used for new values
-  public int childCount;
-
-  // the values of the keys and values
-  public ColumnVector keys;
-  public ColumnVector values;
-  ...
-}
-</code></pre>
+<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">public</span> <span class="kd">class</span> <span class="nc">MapColumnVector</span> <span class="kd">extends</span> <span class="n">ColumnVector</span> <span class="o">{</span>
+  <span class="c1">// for each row, the first offset of the child</span>
+  <span class="kd">public</span> <span class="kt">long</span><span class="o">[]</span> <span class="n">offsets</span><span class="o">;</span>
+  <span class="c1">// for each row, the number of elements in the array</span>
+  <span class="kd">public</span> <span class="kt">long</span><span class="o">[]</span> <span class="n">lengths</span><span class="o">;</span>
+  <span class="c1">// the offset in the child that should be used for new values</span>
+  <span class="kd">public</span> <span class="kt">int</span> <span class="n">childCount</span><span class="o">;</span>
+
+  <span class="c1">// the values of the keys and values</span>
+  <span class="kd">public</span> <span class="n">ColumnVector</span> <span class="n">keys</span><span class="o">;</span>
+  <span class="kd">public</span> <span class="n">ColumnVector</span> <span class="n">values</span><span class="o">;</span>
+  <span class="o">...</span>
+<span class="o">}</span>
+</code></pre></div></div>
 
 <h2 id="writing-orc-files">Writing ORC Files</h2>
 
 <h3 id="simple-example">Simple Example</h3>
 <p>To write an ORC file, you need to define the schema and use the
-<a href="/api/orc-core/index.html?org/apache/orc/OrcFile.html">OrcFile</a>
+<a href="http://localhost:4000/api/orc-core/index.html?org/apache/orc/OrcFile.html">OrcFile</a>
 class to create a
-<a href="/api/orc-core/index.html?org/apache/orc/Writer.html">Writer</a>
+<a href="http://localhost:4000/api/orc-core/index.html?org/apache/orc/Writer.html">Writer</a>
 with the desired filename. This example sets the required schema
 parameter, but there are many other options to control the ORC writer.</p>
 
-<pre><code class="language-java">Configuration conf = new Configuration();
-TypeDescription schema = TypeDescription.fromString("struct&lt;x:int,y:int&gt;");
-Writer writer = OrcFile.createWriter(new Path("my-file.orc"),
-                  OrcFile.writerOptions(conf)
-                         .setSchema(schema));
-</code></pre>
+<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">Configuration</span> <span class="n">conf</span> <span class="o">=</span> <span class="k">new</span> <span class="n">Configuration</span><span class="o">();</span>
+<span class="n">TypeDescription</span> <span class="n">schema</span> <span class="o">=</span> <span class="n">TypeDescription</span><span class="o">.</span><span class="na">fromString</span><span class="o">(</span><span class="s">"struct&lt;x:int,y:int&gt;"</span><span class="o">);</span>
+<span class="n">Writer</span> <span class="n">writer</span> <span class="o">=</span> <span class="n">OrcFile</span><span class="o">.</span><span class="na">createWriter</span><span class="o">(</span><span class="k">new</span> <span class="n">Path</span><span class="o">(</span><span class="s">"my-file.orc"</span><span class="o">),</span>
+                  <span class="n">OrcFile</span><span class="o">.</span><span class="na">writerOptions</span><span class="o">(</span><span class="n">conf</span><span class="o">)</span>
+                         <span class="o">.</span><span class="na">setSchema</span><span class="o">(</span><span class="n">schema</span><span class="o">));</span>
+</code></pre></div></div>
 
 <p>Now you need to create a row batch, set the data, and write it to the file
-as the batch fills up. When the file is done, close the <code>Writer</code>.</p>
-
-<pre><code class="language-java">VectorizedRowBatch batch = schema.createRowBatch();
-LongColumnVector x = (LongColumnVector) batch.cols[0];
-LongColumnVector y = (LongColumnVector) batch.cols[1];
-for(int r=0; r &lt; 10000; ++r) {
-  int row = batch.size++;
-  x.vector[row] = r;
-  y.vector[row] = r * 3;
-  // If the batch is full, write it out and start over.
-  if (batch.size == batch.getMaxSize()) {
-    writer.addRowBatch(batch);
-    batch.reset();
-  }
-}
-if (batch.size != 0) {
-  writer.addRowBatch(batch);
-  batch.reset();
-}
-writer.close();
-</code></pre>
+as the batch fills up. When the file is done, close the <code class="highlighter-rouge">Writer</code>.</p>
+
+<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">VectorizedRowBatch</span> <span class="n">batch</span> <span class="o">=</span> <span class="n">schema</span><span class="o">.</span><span class="na">createRowBatch</span><span class="o">();</span>
+<span class="n">LongColumnVector</span> <span class="n">x</span> <span class="o">=</span> <span class="o">(</span><span class="n">LongColumnVector</span><span class="o">)</span> <span class="n">batch</span><span class="o">.</span><span class="na">cols</span><span class="o">[</span><span class="mi">0</span><span class="o">];</span>
+<span class="n">LongColumnVector</span> <span class="n">y</span> <span class="o">=</span> <span class="o">(</span><span class="n">LongColumnVector</span><span class="o">)</span> <span class="n">batch</span><span class="o">.</span><span class="na">cols</span><span class="o">[</span><span class="mi">1</span><span class="o">];</span>
+<span class="k">for</span><span class="o">(</span><span class="kt">int</span> <span class="n">r</span><span class="o">=</span><span class="mi">0</span><span class="o">;</span> <span class="n">r</span> <span class="o">&lt;</span> <span class="mi">10000</span><span class="o">;</span> <span class="o">++</span><span class="n">r</span><span class="o">)</span> <span class="o">{</span>
+  <span class="kt">int</span> <span class="n">row</span> <span class="o">=</span> <span class="n">batch</span><span class="o">.</span><span class="na">size</span><span class="o">++;</span>
+  <span class="n">x</span><span class="o">.</span><span class="na">vector</span><span class="o">[</span><span class="n">row</span><span class="o">]</span> <span class="o">=</span> <span class="n">r</span><span class="o">;</span>
+  <span class="n">y</span><span class="o">.</span><span class="na">vector</span><span class="o">[</span><span class="n">row</span><span class="o">]</span> <span class="o">=</span> <span class="n">r</span> <span class="o">*</span> <span class="mi">3</span><span class="o">;</span>
+  <span class="c1">// If the batch is full, write it out and start over.</span>
+  <span class="k">if</span> <span class="o">(</span><span class="n">batch</span><span class="o">.</span><span class="na">size</span> <span class="o">==</span> <span class="n">batch</span><span class="o">.</span><span class="na">getMaxSize</span><span class="o">())</span> <span class="o">{</span>
+    <span class="n">writer</span><span class="o">.</span><span class="na">addRowBatch</span><span class="o">(</span><span class="n">batch</span><span class="o">);</span>
+    <span class="n">batch</span><span class="o">.</span><span class="na">reset</span><span class="o">();</span>
+  <span class="o">}</span>
+<span class="o">}</span>
+<span class="k">if</span> <span class="o">(</span><span class="n">batch</span><span class="o">.</span><span class="na">size</span> <span class="o">!=</span> <span class="mi">0</span><span class="o">)</span> <span class="o">{</span>
+  <span class="n">writer</span><span class="o">.</span><span class="na">addRowBatch</span><span class="o">(</span><span class="n">batch</span><span class="o">);</span>
+  <span class="n">batch</span><span class="o">.</span><span class="na">reset</span><span class="o">();</span>
+<span class="o">}</span>
+<span class="n">writer</span><span class="o">.</span><span class="na">close</span><span class="o">();</span>
+</code></pre></div></div>
 
 <h3 id="advanced-example">Advanced Example</h3>
 
@@ -975,99 +975,99 @@ writer.close();
 columns and a map column. Each row’s map has 5 elements with keys
 ranging from “&lt;row&gt;.0” to “&lt;row&gt;.4”.</p>
 
-<pre><code class="language-java">Path testFilePath = new Path("advanced-example.orc");
-Configuration conf = new Configuration();
-
-TypeDescription schema =
-    TypeDescription.fromString("struct&lt;first:int," +
-                               "second:int,third:map&lt;string,int&gt;&gt;");
-
-Writer writer =
-    OrcFile.createWriter(testFilePath,
-        OrcFile.writerOptions(conf).setSchema(schema));
-
-VectorizedRowBatch batch = schema.createRowBatch();
-LongColumnVector first = (LongColumnVector) batch.cols[0];
-LongColumnVector second = (LongColumnVector) batch.cols[1];
-
-//Define map. You need also to cast the key and value vectors
-MapColumnVector map = (MapColumnVector) batch.cols[2];
-BytesColumnVector mapKey = (BytesColumnVector) map.keys;
-LongColumnVector mapValue = (LongColumnVector) map.values;
-
-// Each map has 5 elements
-final int MAP_SIZE = 5;
-final int BATCH_SIZE = batch.getMaxSize();
-
-// Ensure the map is big enough
-mapKey.ensureSize(BATCH_SIZE * MAP_SIZE, false);
-mapValue.ensureSize(BATCH_SIZE * MAP_SIZE, false);
-
-// add 1500 rows to file
-for(int r=0; r &lt; 1500; ++r) {
-  int row = batch.size++;
-
-  first.vector[row] = r;
-  second.vector[row] = r * 3;
-
-  map.offsets[row] = map.childCount;
-  map.lengths[row] = MAP_SIZE;
-  map.childCount += MAP_SIZE;
-
-  for (int mapElem = (int) map.offsets[row];
-       mapElem &lt; map.offsets[row] + MAP_SIZE; ++mapElem) {
-    String key = "row " + r + "." + (mapElem - map.offsets[row]);
-    mapKey.setVal(mapElem, key.getBytes(StandardCharsets.UTF_8));
-    mapValue.vector[mapElem] = mapElem;
-  }
-  if (row == BATCH_SIZE - 1) {
-    writer.addRowBatch(batch);
-    batch.reset();
-  }
-}
-if (batch.size != 0) {
-  writer.addRowBatch(batch);
-  batch.reset();
-}
-writer.close();
-
-</code></pre>
+<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">Path</span> <span class="n">testFilePath</span> <span class="o">=</span> <span class="k">new</span> <span class="n">Path</span><span class="o">(</span><span class="s">"advanced-example.orc"</span><span class="o">);</span>
+<span class="n">Configuration</span> <span class="n">conf</span> <span class="o">=</span> <span class="k">new</span> <span class="n">Configuration</span><span class="o">();</span>
+
+<span class="n">TypeDescription</span> <span class="n">schema</span> <span class="o">=</span>
+    <span class="n">TypeDescription</span><span class="o">.</span><span class="na">fromString</span><span class="o">(</span><span class="s">"struct&lt;first:int,"</span> <span class="o">+</span>
+                               <span class="s">"second:int,third:map&lt;string,int&gt;&gt;"</span><span class="o">);</span>
+
+<span class="n">Writer</span> <span class="n">writer</span> <span class="o">=</span>
+    <span class="n">OrcFile</span><span class="o">.</span><span class="na">createWriter</span><span class="o">(</span><span class="n">testFilePath</span><span class="o">,</span>
+        <span class="n">OrcFile</span><span class="o">.</span><span class="na">writerOptions</span><span class="o">(</span><span class="n">conf</span><span class="o">).</span><span class="na">setSchema</span><span class="o">(</span><span class="n">schema</span><span class="o">));</span>
+
+<span class="n">VectorizedRowBatch</span> <span class="n">batch</span> <span class="o">=</span> <span class="n">schema</span><span class="o">.</span><span class="na">createRowBatch</span><span class="o">();</span>
+<span class="n">LongColumnVector</span> <span class="n">first</span> <span class="o">=</span> <span class="o">(</span><span class="n">LongColumnVector</span><span class="o">)</span> <span class="n">batch</span><span class="o">.</span><span class="na">cols</span><span class="o">[</span><span class="mi">0</span><span class="o">];</span>
+<span class="n">LongColumnVector</span> <span class="n">second</span> <span class="o">=</span> <span class="o">(</span><span class="n">LongColumnVector</span><span class="o">)</span> <span class="n">batch</span><span class="o">.</span><span class="na">cols</span><span class="o">[</span><span class="mi">1</span><span class="o">];</span>
+
+<span class="c1">//Define map. You need also to cast the key and value vectors</span>
+<span class="n">MapColumnVector</span> <span class="n">map</span> <span class="o">=</span> <span class="o">(</span><span class="n">MapColumnVector</span><span class="o">)</span> <span class="n">batch</span><span class="o">.</span><span class="na">cols</span><span class="o">[</span><span class="mi">2</span><span class="o">];</span>
+<span class="n">BytesColumnVector</span> <span class="n">mapKey</span> <span class="o">=</span> <span class="o">(</span><span class="n">BytesColumnVector</span><span class="o">)</span> <span class="n">map</span><span class="o">.</span><span class="na">keys</span><span class="o">;</span>
+<span class="n">LongColumnVector</span> <span class="n">mapValue</span> <span class="o">=</span> <span class="o">(</span><span class="n">LongColumnVector</span><span class="o">)</span> <span class="n">map</span><span class="o">.</span><span class="na">values</span><span class="o">;</span>
+
+<span class="c1">// Each map has 5 elements</span>
+<span class="kd">final</span> <span class="kt">int</span> <span class="n">MAP_SIZE</span> <span class="o">=</span> <span class="mi">5</span><span class="o">;</span>
+<span class="kd">final</span> <span class="kt">int</span> <span class="n">BATCH_SIZE</span> <span class="o">=</span> <span class="n">batch</span><span class="o">.</span><span class="na">getMaxSize</span><span class="o">();</span>
+
+<span class="c1">// Ensure the map is big enough</span>
+<span class="n">mapKey</span><span class="o">.</span><span class="na">ensureSize</span><span class="o">(</span><span class="n">BATCH_SIZE</span> <span class="o">*</span> <span class="n">MAP_SIZE</span><span class="o">,</span> <span class="kc">false</span><span class="o">);</span>
+<span class="n">mapValue</span><span class="o">.</span><span class="na">ensureSize</span><span class="o">(</span><span class="n">BATCH_SIZE</span> <span class="o">*</span> <span class="n">MAP_SIZE</span><span class="o">,</span> <span class="kc">false</span><span class="o">);</span>
+
+<span class="c1">// add 1500 rows to file</span>
+<span class="k">for</span><span class="o">(</span><span class="kt">int</span> <span class="n">r</span><span class="o">=</span><span class="mi">0</span><span class="o">;</span> <span class="n">r</span> <span class="o">&lt;</span> <span class="mi">1500</span><span class="o">;</span> <span class="o">++</span><span class="n">r</span><span class="o">)</span> <span class="o">{</span>
+  <span class="kt">int</span> <span class="n">row</span> <span class="o">=</span> <span class="n">batch</span><span class="o">.</span><span class="na">size</span><span class="o">++;</span>
+
+  <span class="n">first</span><span class="o">.</span><span class="na">vector</span><span class="o">[</span><span class="n">row</span><span class="o">]</span> <span class="o">=</span> <span class="n">r</span><span class="o">;</span>
+  <span class="n">second</span><span class="o">.</span><span class="na">vector</span><span class="o">[</span><span class="n">row</span><span class="o">]</span> <span class="o">=</span> <span class="n">r</span> <span class="o">*</span> <span class="mi">3</span><span class="o">;</span>
+
+  <span class="n">map</span><span class="o">.</span><span class="na">offsets</span><span class="o">[</span><span class="n">row</span><span class="o">]</span> <span class="o">=</span> <span class="n">map</span><span class="o">.</span><span class="na">childCount</span><span class="o">;</span>
+  <span class="n">map</span><span class="o">.</span><span class="na">lengths</span><span class="o">[</span><span class="n">row</span><span class="o">]</span> <span class="o">=</span> <span class="n">MAP_SIZE</span><span class="o">;</span>
+  <span class="n">map</span><span class="o">.</span><span class="na">childCount</span> <span class="o">+=</span> <span class="n">MAP_SIZE</span><span class="o">;</span>
+
+  <span class="k">for</span> <span class="o">(</span><span class="kt">int</span> <span class="n">mapElem</span> <span class="o">=</span> <span class="o">(</span><span class="kt">int</span><span class="o">)</span> <span class="n">map</span><span class="o">.</span><span class="na">offsets</span><span class="o">[</span><span class="n">row</span><span class="o">];</span>
+       <span class="n">mapElem</span> <span class="o">&lt;</span> <span class="n">map</span><span class="o">.</span><span class="na">offsets</span><span class="o">[</span><span class="n">row</span><span class="o">]</span> <span class="o">+</span> <span class="n">MAP_SIZE</span><span class="o">;</span> <span class="o">++</span><span class="n">mapElem</span><span class="o">)</span> <span class="o">{</span>
+    <span class="n">String</span> <span class="n">key</span> <span class="o">=</span> <span class="s">"row "</span> <span class="o">+</span> <span class="n">r</span> <span class="o">+</span> <span class="s">"."</span> <span class="o">+</span> <span class="o">(</span><span class="n">mapElem</span> <span class="o">-</span> <span class="n">map</span><span class="o">.</span><span class="na">offsets</span><span class="o">[</span><span class="n">row</span><span class="o">]);</span>
+    <span class="n">mapKey</span><span class="o">.</span><span class="na">setVal</span><span class="o">(</span><span class="n">mapElem</span><span class="o">,</span> <span class="n">key</span><span class="o">.</span><span class="na">getBytes</span><span class="o">(</span><span class="n">StandardCharsets</span><span class="o">.</span><span class="na">UTF_8</span><span class="o">));</span>
+    <span class="n">mapValue</span><span class="o">.</span><span class="na">vector</span><span class="o">[</span><span class="n">mapElem</span><span class="o">]</span> <span class="o">=</span> <span class="n">mapElem</span><span class="o">;</span>
+  <span class="o">}</span>
+  <span class="k">if</span> <span class="o">(</span><span class="n">row</span> <span class="o">==</span> <span class="n">BATCH_SIZE</span> <span class="o">-</span> <span class="mi">1</span><span class="o">)</span> <span class="o">{</span>
+    <span class="n">writer</span><span class="o">.</span><span class="na">addRowBatch</span><span class="o">(</span><span class="n">batch</span><span class="o">);</span>
+    <span class="n">batch</span><span class="o">.</span><span class="na">reset</span><span class="o">();</span>
+  <span class="o">}</span>
+<span class="o">}</span>
+<span class="k">if</span> <span class="o">(</span><span class="n">batch</span><span class="o">.</span><span class="na">size</span> <span class="o">!=</span> <span class="mi">0</span><span class="o">)</span> <span class="o">{</span>
+  <span class="n">writer</span><span class="o">.</span><span class="na">addRowBatch</span><span class="o">(</span><span class="n">batch</span><span class="o">);</span>
+  <span class="n">batch</span><span class="o">.</span><span class="na">reset</span><span class="o">();</span>
+<span class="o">}</span>
+<span class="n">writer</span><span class="o">.</span><span class="na">close</span><span class="o">();</span>
+
+</code></pre></div></div>
 
 <h2 id="reading-orc-files">Reading ORC Files</h2>
 
 <p>To read ORC files, use the
-<a href="/api/orc-core/index.html?org/apache/orc/OrcFile.html">OrcFile</a>
+<a href="http://localhost:4000/api/orc-core/index.html?org/apache/orc/OrcFile.html">OrcFile</a>
 class to create a
-<a href="/api/orc-core/index.html?org/apache/orc/Reader.html">Reader</a>
+<a href="http://localhost:4000/api/orc-core/index.html?org/apache/orc/Reader.html">Reader</a>
 that contains the metadata about the file. There are a few options to
 the ORC reader, but far fewer than the writer and none of them are
 required. The reader has methods for getting the number of rows,
 schema, compression, etc. from the file.</p>
 
-<pre><code class="language-java">Reader reader = OrcFile.createReader(new Path("my-file.orc"),
-                  OrcFile.readerOptions(conf));
-</code></pre>
+<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">Reader</span> <span class="n">reader</span> <span class="o">=</span> <span class="n">OrcFile</span><span class="o">.</span><span class="na">createReader</span><span class="o">(</span><span class="k">new</span> <span class="n">Path</span><span class="o">(</span><span class="s">"my-file.orc"</span><span class="o">),</span>
+                  <span class="n">OrcFile</span><span class="o">.</span><span class="na">readerOptions</span><span class="o">(</span><span class="n">conf</span><span class="o">));</span>
+</code></pre></div></div>
 
 <p>To get the data, create a
-<a href="/api/orc-core/index.html?org/apache/orc/RecordReader.html">RecordReader</a>
+<a href="http://localhost:4000/api/orc-core/index.html?org/apache/orc/RecordReader.html">RecordReader</a>
 object. By default, the RecordReader reads all rows and all columns,
 but there are options to control the data that is read.</p>
 
-<pre><code class="language-java">RecordReader rows = reader.rows();
-VectorizedRowBatch batch = reader.getSchema().createRowBatch();
-</code></pre>
+<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">RecordReader</span> <span class="n">rows</span> <span class="o">=</span> <span class="n">reader</span><span class="o">.</span><span class="na">rows</span><span class="o">();</span>
+<span class="n">VectorizedRowBatch</span> <span class="n">batch</span> <span class="o">=</span> <span class="n">reader</span><span class="o">.</span><span class="na">getSchema</span><span class="o">().</span><span class="na">createRowBatch</span><span class="o">();</span>
+</code></pre></div></div>
 
-<p>With a <code>RecordReader</code> the user can ask for the next batch until there
+<p>With a <code class="highlighter-rouge">RecordReader</code> the user can ask for the next batch until there
 are no more left. The reader will stop the batch at certain boundaries, so the
 returned batch may not be full, but it will always contain some rows.</p>
 
-<pre><code class="language-java">while (rows.nextBatch(batch)) {
-  for(int r=0; r &lt; batch.size; ++r) {
-    ... process row r from batch
-  }
-}
-rows.close();
-</code></pre>
+<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">while</span> <span class="o">(</span><span class="n">rows</span><span class="o">.</span><span class="na">nextBatch</span><span class="o">(</span><span class="n">batch</span><span class="o">))</span> <span class="o">{</span>
+  <span class="k">for</span><span class="o">(</span><span class="kt">int</span> <span class="n">r</span><span class="o">=</span><span class="mi">0</span><span class="o">;</span> <span class="n">r</span> <span class="o">&lt;</span> <span class="n">batch</span><span class="o">.</span><span class="na">size</span><span class="o">;</span> <span class="o">++</span><span class="n">r</span><span class="o">)</span> <span class="o">{</span>
+    <span class="o">...</span> <span class="n">process</span> <span class="n">row</span> <span class="n">r</span> <span class="n">from</span> <span class="n">batch</span>
+  <span class="o">}</span>
+<span class="o">}</span>
+<span class="n">rows</span><span class="o">.</span><span class="na">close</span><span class="o">();</span>
+</code></pre></div></div>
 
           
 
@@ -1143,8 +1143,6 @@ rows.close();
   
 
   
-    
-  
 
   
     
@@ -1164,29 +1162,25 @@ rows.close();
     
   
     
-      <li class=""><a href="/docs/index.html">Background</a></li>
+      <li class=""><a href="http://localhost:4000/docs/index.html">Background</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
   
     
-      <li class=""><a href="/docs/adopters.html">ORC Adopters</a></li>
+      <li class=""><a href="http://localhost:4000/docs/adopters.html">ORC Adopters</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -1218,15 +1212,13 @@ rows.close();
     
   
     
-      <li class=""><a href="/docs/types.html">Types</a></li>
+      <li class=""><a href="http://localhost:4000/docs/types.html">Types</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -1248,19 +1240,17 @@ rows.close();
     
   
     
-      <li class=""><a href="/docs/indexes.html">Indexes</a></li>
+      <li class=""><a href="http://localhost:4000/docs/indexes.html">Indexes</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
-      <li class=""><a href="/docs/acid.html">ACID support</a></li>
+      <li class=""><a href="http://localhost:4000/docs/acid.html">ACID support</a></li>
       
 
 
@@ -1275,8 +1265,6 @@ rows.close();
   
 
   
-    
-  
 
   
     
@@ -1284,15 +1272,13 @@ rows.close();
     
   
     
-      <li class=""><a href="/docs/building.html">Building ORC</a></li>
+      <li class=""><a href="http://localhost:4000/docs/building.html">Building ORC</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -1322,7 +1308,7 @@ rows.close();
     
   
     
-      <li class=""><a href="/docs/releases.html">Releases</a></li>
+      <li class=""><a href="http://localhost:4000/docs/releases.html">Releases</a></li>
       
 
 
@@ -1337,8 +1323,6 @@ rows.close();
   
 
   
-    
-  
 
   
     
@@ -1356,15 +1340,13 @@ rows.close();
     
   
     
-      <li class=""><a href="/docs/hive-ddl.html">Hive DDL</a></li>
+      <li class=""><a href="http://localhost:4000/docs/hive-ddl.html">Hive DDL</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -1380,7 +1362,7 @@ rows.close();
     
   
     
-      <li class=""><a href="/docs/hive-config.html">Hive Configuration</a></li>
+      <li class=""><a href="http://localhost:4000/docs/hive-config.html">Hive Configuration</a></li>
       
 
 
@@ -1395,8 +1377,6 @@ rows.close();
   
 
   
-    
-  
 
   
     
@@ -1422,15 +1402,13 @@ rows.close();
     
   
     
-      <li class=""><a href="/docs/mapred.html">Using in MapRed</a></li>
+      <li class=""><a href="http://localhost:4000/docs/mapred.html">Using in MapRed</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -1458,7 +1436,7 @@ rows.close();
     
   
     
-      <li class=""><a href="/docs/mapreduce.html">Using in MapReduce</a></li>
+      <li class=""><a href="http://localhost:4000/docs/mapreduce.html">Using in MapReduce</a></li>
       
 
 
@@ -1473,8 +1451,6 @@ rows.close();
   
 
   
-    
-  
 
   
     
@@ -1486,15 +1462,13 @@ rows.close();
     
   
     
-      <li class="current"><a href="/docs/core-java.html">Using Core Java</a></li>
+      <li class="current"><a href="http://localhost:4000/docs/core-java.html">Using Core Java</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -1504,7 +1478,7 @@ rows.close();
     
   
     
-      <li class=""><a href="/docs/core-cpp.html">Using Core C++</a></li>
+      <li class=""><a href="http://localhost:4000/docs/core-cpp.html">Using Core C++</a></li>
       
 
 
@@ -1519,8 +1493,6 @@ rows.close();
   
 
   
-    
-  
 
   
     
@@ -1534,15 +1506,13 @@ rows.close();
     
   
     
-      <li class=""><a href="/docs/cpp-tools.html">C++ Tools</a></li>
+      <li class=""><a href="http://localhost:4000/docs/cpp-tools.html">C++ Tools</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -1566,7 +1536,7 @@ rows.close();
     
   
     
-      <li class=""><a href="/docs/java-tools.html">Java Tools</a></li>
+      <li class=""><a href="http://localhost:4000/docs/java-tools.html">Java Tools</a></li>
       
 
 

http://git-wip-us.apache.org/repos/asf/orc/blob/e58d3fc4/docs/cpp-tools.html
----------------------------------------------------------------------
diff --git a/docs/cpp-tools.html b/docs/cpp-tools.html
index abe6e2e..4b8d0d2 100644
--- a/docs/cpp-tools.html
+++ b/docs/cpp-tools.html
@@ -4,7 +4,7 @@
   <meta charset="UTF-8">
   <title>C++ Tools</title>
   <meta name="viewport" content="width=device-width,initial-scale=1">
-  <meta name="generator" content="Jekyll v2.4.0">
+  <meta name="generator" content="Jekyll v3.7.3">
   <link rel="stylesheet" href="//fonts.googleapis.com/css?family=Lato:300,300italic,400,400italic,700,700italic,900">
   <link rel="stylesheet" href="/css/screen.css">
   <link rel="icon" type="image/x-icon" href="/favicon.ico">
@@ -109,7 +109,7 @@
     
   
     
-      <option value="/docs/index.html">Background</option>
+      <option value="http://localhost:4000/docs/index.html">Background</option>
     
   
     
@@ -131,7 +131,7 @@
     
   
     
-      <option value="/docs/adopters.html">ORC Adopters</option>
+      <option value="http://localhost:4000/docs/adopters.html">ORC Adopters</option>
     
   
     
@@ -193,7 +193,7 @@
     
   
     
-      <option value="/docs/types.html">Types</option>
+      <option value="http://localhost:4000/docs/types.html">Types</option>
     
   
 
@@ -219,7 +219,7 @@
     
   
     
-      <option value="/docs/indexes.html">Indexes</option>
+      <option value="http://localhost:4000/docs/indexes.html">Indexes</option>
     
   
     
@@ -237,7 +237,7 @@
 
   
     
-      <option value="/docs/acid.html">ACID support</option>
+      <option value="http://localhost:4000/docs/acid.html">ACID support</option>
     
   
     
@@ -284,7 +284,7 @@
     
   
     
-      <option value="/docs/building.html">Building ORC</option>
+      <option value="http://localhost:4000/docs/building.html">Building ORC</option>
     
   
     
@@ -342,7 +342,7 @@
     
   
     
-      <option value="/docs/releases.html">Releases</option>
+      <option value="http://localhost:4000/docs/releases.html">Releases</option>
     
   
     
@@ -373,7 +373,7 @@
     
   
     
-      <option value="/docs/hive-ddl.html">Hive DDL</option>
+      <option value="http://localhost:4000/docs/hive-ddl.html">Hive DDL</option>
     
   
     
@@ -407,7 +407,7 @@
     
   
     
-      <option value="/docs/hive-config.html">Hive Configuration</option>
+      <option value="http://localhost:4000/docs/hive-config.html">Hive Configuration</option>
     
   
     
@@ -460,7 +460,7 @@
     
   
     
-      <option value="/docs/mapred.html">Using in MapRed</option>
+      <option value="http://localhost:4000/docs/mapred.html">Using in MapRed</option>
     
   
     
@@ -498,7 +498,7 @@
     
   
     
-      <option value="/docs/mapreduce.html">Using in MapReduce</option>
+      <option value="http://localhost:4000/docs/mapreduce.html">Using in MapReduce</option>
     
   
     
@@ -525,7 +525,7 @@
     
   
     
-      <option value="/docs/core-java.html">Using Core Java</option>
+      <option value="http://localhost:4000/docs/core-java.html">Using Core Java</option>
     
   
     
@@ -559,7 +559,7 @@
     
   
     
-      <option value="/docs/core-cpp.html">Using Core C++</option>
+      <option value="http://localhost:4000/docs/core-cpp.html">Using Core C++</option>
     
   
     
@@ -606,7 +606,7 @@
     
   
     
-      <option value="/docs/cpp-tools.html">C++ Tools</option>
+      <option value="http://localhost:4000/docs/cpp-tools.html">C++ Tools</option>
     
   
     
@@ -652,7 +652,7 @@
     
   
     
-      <option value="/docs/java-tools.html">Java Tools</option>
+      <option value="http://localhost:4000/docs/java-tools.html">Java Tools</option>
     
   
     
@@ -677,143 +677,143 @@
           <h2 id="orc-contents">orc-contents</h2>
 
 <p>Displays the contents of the ORC file as a JSON document. With the
-<code>columns</code> argument only the selected columns are printed.</p>
+<code class="highlighter-rouge">columns</code> argument only the selected columns are printed.</p>
 
-<pre><code class="language-shell">% orc-contents  [--columns=1,2,...] &lt;filename&gt;
-</code></pre>
+<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>% orc-contents  <span class="o">[</span><span class="nt">--columns</span><span class="o">=</span>1,2,...] &lt;filename&gt;
+</code></pre></div></div>
 
 <p>If you run it on the example file TestOrcFile.test1.orc, you’ll see (without
 the line breaks within each record):</p>
 
-<pre><code class="language-shell">% orc-contents examples/TestOrcFile.test1.orc
-{"boolean1": false, "byte1": 1, "short1": 1024, "int1": 65536, \\
- "long1": 9223372036854775807, "float1": 1, "double1": -15, \\
- "bytes1": [0, 1, 2, 3, 4], "string1": "hi", "middle": \\
-    {"list": [{"int1": 1, "string1": "bye"}, \\
-              {"int1": 2, "string1": "sigh"}]}, \\
- "list": [{"int1": 3, "string1": "good"}, \\
-          {"int1": 4, "string1": "bad"}], \\
- "map": []}
-{"boolean1": true, "byte1": 100, "short1": 2048, "int1": 65536,
- "long1": 9223372036854775807, "float1": 2, "double1": -5, \\
- "bytes1": [], "string1": "bye", \\
- "middle": {"list": [{"int1": 1, "string1": "bye"}, \\
-                     {"int1": 2, "string1": "sigh"}]}, \\
- "list": [{"int1": 100000000, "string1": "cat"}, \\
-          {"int1": -100000, "string1": "in"}, \\
-          {"int1": 1234, "string1": "hat"}], \\
- "map": [{"key": "chani", "value": {"int1": 5, "string1": "chani"}}, \\
-         {"key": "mauddib", \\
-          "value": {"int1": 1, "string1": "mauddib"}}]}
-</code></pre>
+<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>% orc-contents examples/TestOrcFile.test1.orc
+<span class="o">{</span><span class="s2">"boolean1"</span>: <span class="nb">false</span>, <span class="s2">"byte1"</span>: 1, <span class="s2">"short1"</span>: 1024, <span class="s2">"int1"</span>: 65536, <span class="se">\\</span>
+ <span class="s2">"long1"</span>: 9223372036854775807, <span class="s2">"float1"</span>: 1, <span class="s2">"double1"</span>: <span class="nt">-15</span>, <span class="se">\\</span>
+ <span class="s2">"bytes1"</span>: <span class="o">[</span>0, 1, 2, 3, 4], <span class="s2">"string1"</span>: <span class="s2">"hi"</span>, <span class="s2">"middle"</span>: <span class="se">\\</span>
+    <span class="o">{</span><span class="s2">"list"</span>: <span class="o">[{</span><span class="s2">"int1"</span>: 1, <span class="s2">"string1"</span>: <span class="s2">"bye"</span><span class="o">}</span>, <span class="se">\\</span>
+              <span class="o">{</span><span class="s2">"int1"</span>: 2, <span class="s2">"string1"</span>: <span class="s2">"sigh"</span><span class="o">}]}</span>, <span class="se">\\</span>
+ <span class="s2">"list"</span>: <span class="o">[{</span><span class="s2">"int1"</span>: 3, <span class="s2">"string1"</span>: <span class="s2">"good"</span><span class="o">}</span>, <span class="se">\\</span>
+          <span class="o">{</span><span class="s2">"int1"</span>: 4, <span class="s2">"string1"</span>: <span class="s2">"bad"</span><span class="o">}]</span>, <span class="se">\\</span>
+ <span class="s2">"map"</span>: <span class="o">[]}</span>
+<span class="o">{</span><span class="s2">"boolean1"</span>: <span class="nb">true</span>, <span class="s2">"byte1"</span>: 100, <span class="s2">"short1"</span>: 2048, <span class="s2">"int1"</span>: 65536,
+ <span class="s2">"long1"</span>: 9223372036854775807, <span class="s2">"float1"</span>: 2, <span class="s2">"double1"</span>: <span class="nt">-5</span>, <span class="se">\\</span>
+ <span class="s2">"bytes1"</span>: <span class="o">[]</span>, <span class="s2">"string1"</span>: <span class="s2">"bye"</span>, <span class="se">\\</span>
+ <span class="s2">"middle"</span>: <span class="o">{</span><span class="s2">"list"</span>: <span class="o">[{</span><span class="s2">"int1"</span>: 1, <span class="s2">"string1"</span>: <span class="s2">"bye"</span><span class="o">}</span>, <span class="se">\\</span>
+                     <span class="o">{</span><span class="s2">"int1"</span>: 2, <span class="s2">"string1"</span>: <span class="s2">"sigh"</span><span class="o">}]}</span>, <span class="se">\\</span>
+ <span class="s2">"list"</span>: <span class="o">[{</span><span class="s2">"int1"</span>: 100000000, <span class="s2">"string1"</span>: <span class="s2">"cat"</span><span class="o">}</span>, <span class="se">\\</span>
+          <span class="o">{</span><span class="s2">"int1"</span>: <span class="nt">-100000</span>, <span class="s2">"string1"</span>: <span class="s2">"in"</span><span class="o">}</span>, <span class="se">\\</span>
+          <span class="o">{</span><span class="s2">"int1"</span>: 1234, <span class="s2">"string1"</span>: <span class="s2">"hat"</span><span class="o">}]</span>, <span class="se">\\</span>
+ <span class="s2">"map"</span>: <span class="o">[{</span><span class="s2">"key"</span>: <span class="s2">"chani"</span>, <span class="s2">"value"</span>: <span class="o">{</span><span class="s2">"int1"</span>: 5, <span class="s2">"string1"</span>: <span class="s2">"chani"</span><span class="o">}}</span>, <span class="se">\\</span>
+         <span class="o">{</span><span class="s2">"key"</span>: <span class="s2">"mauddib"</span>, <span class="se">\\</span>
+          <span class="s2">"value"</span>: <span class="o">{</span><span class="s2">"int1"</span>: 1, <span class="s2">"string1"</span>: <span class="s2">"mauddib"</span><span class="o">}}]}</span>
+</code></pre></div></div>
 
 <h2 id="orc-metadata">orc-metadata</h2>
 
 <p>Displays the metadata of the ORC file as a JSON document. With the
-<code>verbose</code> option additional information about the layout of the file
+<code class="highlighter-rouge">verbose</code> option additional information about the layout of the file
 is also printed.</p>
 
 <p>For diagnosing problems, it is useful to use the ‘–raw’ option that
 prints the protocol buffers from the ORC file directly rather than
 interpreting them.</p>
 
-<pre><code class="language-shell">% orc-metadata [-v] [--raw] &lt;filename&gt;
-</code></pre>
+<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>% orc-metadata <span class="o">[</span><span class="nt">-v</span><span class="o">]</span> <span class="o">[</span><span class="nt">--raw</span><span class="o">]</span> &lt;filename&gt;
+</code></pre></div></div>
 
 <p>If you run it on the example file TestOrcFile.test1.orc, you’ll see:</p>
 
-<pre><code class="language-shell">% orc-metadata examples/TestOrcFile.test1.orc
-{ "name": "../examples/TestOrcFile.test1.orc",
-  "type": "struct&lt;boolean1:boolean,byte1:tinyint,short1:smallint,
+<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>% orc-metadata examples/TestOrcFile.test1.orc
+<span class="o">{</span> <span class="s2">"name"</span>: <span class="s2">"../examples/TestOrcFile.test1.orc"</span>,
+  <span class="s2">"type"</span>: <span class="s2">"struct&lt;boolean1:boolean,byte1:tinyint,short1:smallint,
 int1:int,long1:bigint,float1:float,double1:double,bytes1:binary,
 string1:string,middle:struct&lt;list:array&lt;struct&lt;int1:int,string1:
 string&gt;&gt;&gt;,list:array&lt;struct&lt;int1:int,string1:string&gt;&gt;,map:map&lt;
-string,struct&lt;int1:int,string1:string&gt;&gt;&gt;",
-  "rows": 2,
-  "stripe count": 1,
-  "format": "0.12", "writer version": "HIVE-8732",
-  "compression": "zlib", "compression block": 10000,
-  "file length": 1711,
-  "content": 1015, "stripe stats": 250, "footer": 421, "postscript": 24,
-  "row index stride": 10000,
-  "user metadata": {
-  },
-  "stripes": [
-    { "stripe": 0, "rows": 2,
-      "offset": 3, "length": 1012,
-      "index": 570, "data": 243, "footer": 199
-    }
-  ]
-}
-</code></pre>
+string,struct&lt;int1:int,string1:string&gt;&gt;&gt;"</span>,
+  <span class="s2">"rows"</span>: 2,
+  <span class="s2">"stripe count"</span>: 1,
+  <span class="s2">"format"</span>: <span class="s2">"0.12"</span>, <span class="s2">"writer version"</span>: <span class="s2">"HIVE-8732"</span>,
+  <span class="s2">"compression"</span>: <span class="s2">"zlib"</span>, <span class="s2">"compression block"</span>: 10000,
+  <span class="s2">"file length"</span>: 1711,
+  <span class="s2">"content"</span>: 1015, <span class="s2">"stripe stats"</span>: 250, <span class="s2">"footer"</span>: 421, <span class="s2">"postscript"</span>: 24,
+  <span class="s2">"row index stride"</span>: 10000,
+  <span class="s2">"user metadata"</span>: <span class="o">{</span>
+  <span class="o">}</span>,
+  <span class="s2">"stripes"</span>: <span class="o">[</span>
+    <span class="o">{</span> <span class="s2">"stripe"</span>: 0, <span class="s2">"rows"</span>: 2,
+      <span class="s2">"offset"</span>: 3, <span class="s2">"length"</span>: 1012,
+      <span class="s2">"index"</span>: 570, <span class="s2">"data"</span>: 243, <span class="s2">"footer"</span>: 199
+    <span class="o">}</span>
+  <span class="o">]</span>
+<span class="o">}</span>
+</code></pre></div></div>
 
 <h2 id="csv-import">csv-import</h2>
 
 <p>Imports CSV file into an Orc file using the specified schema.
-Compound types are not yet supported. <code>delimiter</code> option indicates
-the delimiter in the input CSV file and by default is <code>,</code>. <code>stripe</code>
-option means the stripe size and set to 128MB by default. <code>block</code>
-option is compression block size which is 64KB by default. <code>batch</code>
+Compound types are not yet supported. <code class="highlighter-rouge">delimiter</code> option indicates
+the delimiter in the input CSV file and by default is <code class="highlighter-rouge">,</code>. <code class="highlighter-rouge">stripe</code>
+option means the stripe size and set to 128MB by default. <code class="highlighter-rouge">block</code>
+option is compression block size which is 64KB by default. <code class="highlighter-rouge">batch</code>
 option is by default 1024 rows for one batch.</p>
 
-<pre><code class="language-shell">% csv-import [--delimiter=&lt;character&gt;] [--stripe=&lt;size&gt;]
-             [--block=&lt;size&gt;] [--batch=&lt;size&gt;]
+<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>% csv-import <span class="o">[</span><span class="nt">--delimiter</span><span class="o">=</span>&lt;character&gt;] <span class="o">[</span><span class="nt">--stripe</span><span class="o">=</span>&lt;size&gt;]
+             <span class="o">[</span><span class="nt">--block</span><span class="o">=</span>&lt;size&gt;] <span class="o">[</span><span class="nt">--batch</span><span class="o">=</span>&lt;size&gt;]
              &lt;schema&gt; &lt;inputCSVFile&gt; &lt;outputORCFile&gt;
-</code></pre>
+</code></pre></div></div>
 
 <p>If you run it on the example file TestCSVFileImport.test10rows.csv,
 you’ll see:</p>
 
-<pre><code class="language-shell">% csv-import "struct&lt;a:bigint,b:string,c:double&gt;"
+<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>% csv-import <span class="s2">"struct&lt;a:bigint,b:string,c:double&gt;"</span>
              examples/TestCSVFileImport.test10rows.csv /tmp/test.orc
-[2018-04-11 11:12:16] Start importing Orc file...
-[2018-04-11 11:12:16] Finish importing Orc file.
-[2018-04-11 11:12:16] Total writer elasped time: 0.001352s.
-[2018-04-11 11:12:16] Total writer CPU time: 0.001339s.
-</code></pre>
+<span class="o">[</span>2018-04-11 11:12:16] Start importing Orc file...
+<span class="o">[</span>2018-04-11 11:12:16] Finish importing Orc file.
+<span class="o">[</span>2018-04-11 11:12:16] Total writer elasped <span class="nb">time</span>: 0.001352s.
+<span class="o">[</span>2018-04-11 11:12:16] Total writer CPU <span class="nb">time</span>: 0.001339s.
+</code></pre></div></div>
 
 <h2 id="orc-scan">orc-scan</h2>
 
-<p>Scans and displays the row count of the ORC file. With the <code>batch</code> option
+<p>Scans and displays the row count of the ORC file. With the <code class="highlighter-rouge">batch</code> option
 to set the batch size which is 1024 rows by default. It is useful to check
 if the ORC file is damaged.</p>
 
-<pre><code class="language-shell">% orc-scan [--batch=&lt;size&gt;] &lt;filename&gt;
-</code></pre>
+<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>% orc-scan <span class="o">[</span><span class="nt">--batch</span><span class="o">=</span>&lt;size&gt;] &lt;filename&gt;
+</code></pre></div></div>
 
 <p>If you run it on the example file TestOrcFile.test1.orc, you’ll see:</p>
 
-<pre><code class="language-shell">% orc-scan examples/TestOrcFile.test1.orc
+<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>% orc-scan examples/TestOrcFile.test1.orc
 Rows: 2
 Batches: 1
-</code></pre>
+</code></pre></div></div>
 
 <h2 id="orc-statistics">orc-statistics</h2>
 
 <p>Displays the file-level and stripe-level column statistics of the ORC file.
-With the <code>withIndex</code> option to include column statistics in each row group.</p>
+With the <code class="highlighter-rouge">withIndex</code> option to include column statistics in each row group.</p>
 
-<pre><code class="language-shell">% orc-statistics [--withIndex] &lt;filename&gt;
-</code></pre>
+<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>% orc-statistics <span class="o">[</span><span class="nt">--withIndex</span><span class="o">]</span> &lt;filename&gt;
+</code></pre></div></div>
 
 <p>If you run it on the example file TestOrcFile.TestOrcFile.columnProjection.orc
 you’ll see:</p>
 
-<pre><code class="language-shell">% orc-statistics examples/TestOrcFile.columnProjection.orc
+<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>% orc-statistics examples/TestOrcFile.columnProjection.orc
 File examples/TestOrcFile.columnProjection.orc has 3 columns
-*** Column 0 ***
+<span class="k">***</span> Column 0 <span class="k">***</span>
 Column has 21000 values and has null value: no
 
-*** Column 1 ***
-Data type: Integer
+<span class="k">***</span> Column 1 <span class="k">***</span>
+Data <span class="nb">type</span>: Integer
 Values: 21000
 Has null: no
-Minimum: -2147439072
+Minimum: <span class="nt">-2147439072</span>
 Maximum: 2147257982
 Sum: 268482658568
 
-*** Column 2 ***
-Data type: String
+<span class="k">***</span> Column 2 <span class="k">***</span>
+Data <span class="nb">type</span>: String
 Values: 21000
 Has null: no
 Minimum: 100119c272d7db89
@@ -821,111 +821,111 @@ Maximum: fffe9f6f23b287f3
 Total length: 334559
 
 File examples/TestOrcFile.columnProjection.orc has 5 stripes
-*** Stripe 0 ***
+<span class="k">***</span> Stripe 0 <span class="k">***</span>
 
---- Column 0 ---
+<span class="nt">---</span> Column 0 <span class="nt">---</span>
 Column has 5000 values and has null value: no
 
---- Column 1 ---
-Data type: Integer
+<span class="nt">---</span> Column 1 <span class="nt">---</span>
+Data <span class="nb">type</span>: Integer
 Values: 5000
 Has null: no
-Minimum: -2145365268
+Minimum: <span class="nt">-2145365268</span>
 Maximum: 2147025027
-Sum: -29841423854
+Sum: <span class="nt">-29841423854</span>
 
---- Column 2 ---
-Data type: String
+<span class="nt">---</span> Column 2 <span class="nt">---</span>
+Data <span class="nb">type</span>: String
 Values: 5000
 Has null: no
 Minimum: 1005350489418be2
 Maximum: fffbb8718c92b09f
 Total length: 79644
 
-*** Stripe 1 ***
+<span class="k">***</span> Stripe 1 <span class="k">***</span>
 
---- Column 0 ---
+<span class="nt">---</span> Column 0 <span class="nt">---</span>
 Column has 5000 values and has null value: no
 
---- Column 1 ---
-Data type: Integer
+<span class="nt">---</span> Column 1 <span class="nt">---</span>
+Data <span class="nb">type</span>: Integer
 Values: 5000
 Has null: no
-Minimum: -2147115959
+Minimum: <span class="nt">-2147115959</span>
 Maximum: 2147257982
 Sum: 108604887785
 
---- Column 2 ---
-Data type: String
+<span class="nt">---</span> Column 2 <span class="nt">---</span>
+Data <span class="nb">type</span>: String
 Values: 5000
 Has null: no
 Minimum: 100119c272d7db89
 Maximum: fff0ae41d41e6afc
 Total length: 79640
 
-*** Stripe 2 ***
+<span class="k">***</span> Stripe 2 <span class="k">***</span>
 
---- Column 0 ---
+<span class="nt">---</span> Column 0 <span class="nt">---</span>
 Column has 5000 values and has null value: no
 
---- Column 1 ---
-Data type: Integer
+<span class="nt">---</span> Column 1 <span class="nt">---</span>
+Data <span class="nb">type</span>: Integer
 Values: 5000
 Has null: no
-Minimum: -2145932387
+Minimum: <span class="nt">-2145932387</span>
 Maximum: 2145877119
 Sum: 70064190848
 
---- Column 2 ---
-Data type: String
+<span class="nt">---</span> Column 2 <span class="nt">---</span>
+Data <span class="nb">type</span>: String
 Values: 5000
 Has null: no
 Minimum: 10130af874ae036c
 Maximum: fffe9f6f23b287f3
 Total length: 79645
 
-*** Stripe 3 ***
+<span class="k">***</span> Stripe 3 <span class="k">***</span>
 
---- Column 0 ---
+<span class="nt">---</span> Column 0 <span class="nt">---</span>
 Column has 5000 values and has null value: no
 
---- Column 1 ---
-Data type: Integer
+<span class="nt">---</span> Column 1 <span class="nt">---</span>
+Data <span class="nb">type</span>: Integer
 Values: 5000
 Has null: no
-Minimum: -2147439072
+Minimum: <span class="nt">-2147439072</span>
 Maximum: 2147074354
 Sum: 104681356482
 
---- Column 2 ---
-Data type: String
+<span class="nt">---</span> Column 2 <span class="nt">---</span>
+Data <span class="nb">type</span>: String
 Values: 5000
 Has null: no
 Minimum: 102547d48ed06518
 Maximum: fffa47c57dc7b69a
 Total length: 79689
 
-*** Stripe 4 ***
+<span class="k">***</span> Stripe 4 <span class="k">***</span>
 
---- Column 0 ---
+<span class="nt">---</span> Column 0 <span class="nt">---</span>
 Column has 1000 values and has null value: no
 
---- Column 1 ---
-Data type: Integer
+<span class="nt">---</span> Column 1 <span class="nt">---</span>
+Data <span class="nb">type</span>: Integer
 Values: 1000
 Has null: no
-Minimum: -2141222223
+Minimum: <span class="nt">-2141222223</span>
 Maximum: 2145816096
 Sum: 14973647307
 
---- Column 2 ---
-Data type: String
+<span class="nt">---</span> Column 2 <span class="nt">---</span>
+Data <span class="nb">type</span>: String
 Values: 1000
 Has null: no
 Minimum: 1059d81c9025a217
 Maximum: ffc17f0e35e1a6c0
 Total length: 15941
-</code></pre>
+</code></pre></div></div>
 
           
 
@@ -1007,8 +1007,6 @@ Total length: 15941
   
 
   
-    
-  
 
   
     
@@ -1028,29 +1026,25 @@ Total length: 15941
     
   
     
-      <li class=""><a href="/docs/index.html">Background</a></li>
+      <li class=""><a href="http://localhost:4000/docs/index.html">Background</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
   
     
-      <li class=""><a href="/docs/adopters.html">ORC Adopters</a></li>
+      <li class=""><a href="http://localhost:4000/docs/adopters.html">ORC Adopters</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -1082,15 +1076,13 @@ Total length: 15941
     
   
     
-      <li class=""><a href="/docs/types.html">Types</a></li>
+      <li class=""><a href="http://localhost:4000/docs/types.html">Types</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -1112,19 +1104,17 @@ Total length: 15941
     
   
     
-      <li class=""><a href="/docs/indexes.html">Indexes</a></li>
+      <li class=""><a href="http://localhost:4000/docs/indexes.html">Indexes</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
-      <li class=""><a href="/docs/acid.html">ACID support</a></li>
+      <li class=""><a href="http://localhost:4000/docs/acid.html">ACID support</a></li>
       
 
 
@@ -1139,8 +1129,6 @@ Total length: 15941
   
 
   
-    
-  
 
   
     
@@ -1148,15 +1136,13 @@ Total length: 15941
     
   
     
-      <li class=""><a href="/docs/building.html">Building ORC</a></li>
+      <li class=""><a href="http://localhost:4000/docs/building.html">Building ORC</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -1186,7 +1172,7 @@ Total length: 15941
     
   
     
-      <li class=""><a href="/docs/releases.html">Releases</a></li>
+      <li class=""><a href="http://localhost:4000/docs/releases.html">Releases</a></li>
       
 
 
@@ -1201,8 +1187,6 @@ Total length: 15941
   
 
   
-    
-  
 
   
     
@@ -1220,15 +1204,13 @@ Total length: 15941
     
   
     
-      <li class=""><a href="/docs/hive-ddl.html">Hive DDL</a></li>
+      <li class=""><a href="http://localhost:4000/docs/hive-ddl.html">Hive DDL</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -1244,7 +1226,7 @@ Total length: 15941
     
   
     
-      <li class=""><a href="/docs/hive-config.html">Hive Configuration</a></li>
+      <li class=""><a href="http://localhost:4000/docs/hive-config.html">Hive Configuration</a></li>
       
 
 
@@ -1259,8 +1241,6 @@ Total length: 15941
   
 
   
-    
-  
 
   
     
@@ -1286,15 +1266,13 @@ Total length: 15941
     
   
     
-      <li class=""><a href="/docs/mapred.html">Using in MapRed</a></li>
+      <li class=""><a href="http://localhost:4000/docs/mapred.html">Using in MapRed</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -1322,7 +1300,7 @@ Total length: 15941
     
   
     
-      <li class=""><a href="/docs/mapreduce.html">Using in MapReduce</a></li>
+      <li class=""><a href="http://localhost:4000/docs/mapreduce.html">Using in MapReduce</a></li>
       
 
 
@@ -1337,8 +1315,6 @@ Total length: 15941
   
 
   
-    
-  
 
   
     
@@ -1350,15 +1326,13 @@ Total length: 15941
     
   
     
-      <li class=""><a href="/docs/core-java.html">Using Core Java</a></li>
+      <li class=""><a href="http://localhost:4000/docs/core-java.html">Using Core Java</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -1368,7 +1342,7 @@ Total length: 15941
     
   
     
-      <li class=""><a href="/docs/core-cpp.html">Using Core C++</a></li>
+      <li class=""><a href="http://localhost:4000/docs/core-cpp.html">Using Core C++</a></li>
       
 
 
@@ -1383,8 +1357,6 @@ Total length: 15941
   
 
   
-    
-  
 
   
     
@@ -1398,15 +1370,13 @@ Total length: 15941
     
   
     
-      <li class="current"><a href="/docs/cpp-tools.html">C++ Tools</a></li>
+      <li class="current"><a href="http://localhost:4000/docs/cpp-tools.html">C++ Tools</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -1430,7 +1400,7 @@ Total length: 15941
     
   
     
-      <li class=""><a href="/docs/java-tools.html">Java Tools</a></li>
+      <li class=""><a href="http://localhost:4000/docs/java-tools.html">Java Tools</a></li>
       
 
 


[05/11] orc git commit: Deploy docs with CVE.

Posted by om...@apache.org.
http://git-wip-us.apache.org/repos/asf/orc/blob/e58d3fc4/specification/ORCv0/index.html
----------------------------------------------------------------------
diff --git a/specification/ORCv0/index.html b/specification/ORCv0/index.html
new file mode 100644
index 0000000..d6d5fde
--- /dev/null
+++ b/specification/ORCv0/index.html
@@ -0,0 +1,954 @@
+<!DOCTYPE HTML>
+<html lang="en-US">
+<head>
+  <meta charset="UTF-8">
+  <title>ORC Specification v0</title>
+  <meta name="viewport" content="width=device-width,initial-scale=1">
+  <meta name="generator" content="Jekyll v3.7.3">
+  <link rel="stylesheet" href="//fonts.googleapis.com/css?family=Lato:300,300italic,400,400italic,700,700italic,900">
+  <link rel="stylesheet" href="/css/screen.css">
+  <link rel="icon" type="image/x-icon" href="/favicon.ico">
+  <!--[if lt IE 9]>
+  <script src="/js/html5shiv.min.js"></script>
+  <script src="/js/respond.min.js"></script>
+  <![endif]-->
+</head>
+
+
+<body class="wrap">
+  <header role="banner">
+  <nav class="mobile-nav show-on-mobiles">
+    <ul>
+  <li class="">
+    <a href="/">Home</a>
+  </li>
+  <li class="">
+    <a href="/docs/"><span class="show-on-mobiles">Docs</span>
+                     <span class="hide-on-mobiles">Documentation</span></a>
+  </li>
+  <li class="">
+    <a href="/talks/">Talks</a>
+  </li>
+  <li class="">
+    <a href="/news/">News</a>
+  </li>
+  <li class="">
+    <a href="/help/">Help</a>
+  </li>
+  <li class="">
+    <a href="/develop/">Develop</a>
+  </li>
+</ul>
+
+  </nav>
+  <div class="grid">
+    <div class="unit one-third center-on-mobiles">
+      <h1>
+        <a href="/">
+          <span class="sr-only">Apache ORC</span>
+          <img src="/img/logo.png" width="249" height="101" alt="ORC Logo">
+        </a>
+      </h1>
+    </div>
+    <nav class="main-nav unit two-thirds hide-on-mobiles">
+      <ul>
+  <li class="">
+    <a href="/">Home</a>
+  </li>
+  <li class="">
+    <a href="/docs/"><span class="show-on-mobiles">Docs</span>
+                     <span class="hide-on-mobiles">Documentation</span></a>
+  </li>
+  <li class="">
+    <a href="/talks/">Talks</a>
+  </li>
+  <li class="">
+    <a href="/news/">News</a>
+  </li>
+  <li class="">
+    <a href="/help/">Help</a>
+  </li>
+  <li class="">
+    <a href="/develop/">Develop</a>
+  </li>
+</ul>
+
+    </nav>
+  </div>
+</header>
+
+
+  <section class="standalone">
+  <div class="grid">
+
+    <div class="unit whole">
+      <article>
+        <h1>ORC Specification v0</h1>
+        <p>This version of the file format was originally released as part of
+Hive 0.11.</p>
+
+<h1 id="motivation">Motivation</h1>
+
+<p>Hive’s RCFile was the standard format for storing tabular data in
+Hadoop for several years. However, RCFile has limitations because it
+treats each column as a binary blob without semantics. In Hive 0.11 we
+added a new file format named Optimized Row Columnar (ORC) file that
+uses and retains the type information from the table definition. ORC
+uses type specific readers and writers that provide light weight
+compression techniques such as dictionary encoding, bit packing, delta
+encoding, and run length encoding – resulting in dramatically smaller
+files. Additionally, ORC can apply generic compression using zlib, or
+Snappy on top of the lightweight compression for even smaller
+files. However, storage savings are only part of the gain. ORC
+supports projection, which selects subsets of the columns for reading,
+so that queries reading only one column read only the required
+bytes. Furthermore, ORC files include light weight indexes that
+include the minimum and maximum values for each column in each set of
+10,000 rows and the entire file. Using pushdown filters from Hive, the
+file reader can skip entire sets of rows that aren’t important for
+this query.</p>
+
+<p><img src="http://localhost:4000/img/OrcFileLayout.png" alt="ORC file structure" /></p>
+
+<h1 id="file-tail">File Tail</h1>
+
+<p>Since HDFS does not support changing the data in a file after it is
+written, ORC stores the top level index at the end of the file. The
+overall structure of the file is given in the figure above.  The
+file’s tail consists of 3 parts; the file metadata, file footer and
+postscript.</p>
+
+<p>The metadata for ORC is stored using
+<a href="https://s.apache.org/protobuf_encoding">Protocol Buffers</a>, which provides
+the ability to add new fields without breaking readers. This document
+incorporates the Protobuf definition from the
+<a href="https://s.apache.org/orc_proto">ORC source code</a> and the
+reader is encouraged to review the Protobuf encoding if they need to
+understand the byte-level encoding</p>
+
+<h2 id="postscript">Postscript</h2>
+
+<p>The Postscript section provides the necessary information to interpret
+the rest of the file including the length of the file’s Footer and
+Metadata sections, the version of the file, and the kind of general
+compression used (eg. none, zlib, or snappy). The Postscript is never
+compressed and ends one byte before the end of the file. The version
+stored in the Postscript is the lowest version of Hive that is
+guaranteed to be able to read the file and it stored as a sequence of
+the major and minor version. This version is stored as [0, 11].</p>
+
+<p>The process of reading an ORC file works backwards through the
+file. Rather than making multiple short reads, the ORC reader reads
+the last 16k bytes of the file with the hope that it will contain both
+the Footer and Postscript sections. The final byte of the file
+contains the serialized length of the Postscript, which must be less
+than 256 bytes. Once the Postscript is parsed, the compressed
+serialized length of the Footer is known and it can be decompressed
+and parsed.</p>
+
+<p>```message PostScript {
+ // the length of the footer section in bytes
+ optional uint64 footerLength = 1;
+ // the kind of generic compression used
+ optional CompressionKind compression = 2;
+ // the maximum size of each compression chunk
+ optional uint64 compressionBlockSize = 3;
+ // the version of the writer
+ repeated uint32 version = 4 [packed = true];
+ // the length of the metadata section in bytes
+ optional uint64 metadataLength = 5;
+ // the fixed string “ORC”
+ optional string magic = 8000;
+}</p>
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
+```enum CompressionKind {
+ NONE = 0;
+ ZLIB = 1;
+ SNAPPY = 2;
+ LZO = 3;
+ LZ4 = 4;
+ ZSTD = 5;
+}
+</code></pre></div></div>
+
+<h2 id="footer">Footer</h2>
+
+<p>The Footer section contains the layout of the body of the file, the
+type schema information, the number of rows, and the statistics about
+each of the columns.</p>
+
+<p>The file is broken in to three parts- Header, Body, and Tail. The
+Header consists of the bytes “ORC’’ to support tools that want to
+scan the front of the file to determine the type of the file. The Body
+contains the rows and indexes, and the Tail gives the file level
+information as described in this section.</p>
+
+<p>```message Footer {
+ // the length of the file header in bytes (always 3)
+ optional uint64 headerLength = 1;
+ // the length of the file header and body in bytes
+ optional uint64 contentLength = 2;
+ // the information about the stripes
+ repeated StripeInformation stripes = 3;
+ // the schema information
+ repeated Type types = 4;
+ // the user metadata that was added
+ repeated UserMetadataItem metadata = 5;
+ // the total number of rows in the file
+ optional uint64 numberOfRows = 6;
+ // the statistics of each column across the file
+ repeated ColumnStatistics statistics = 7;
+ // the maximum number of rows in each index entry
+ optional uint32 rowIndexStride = 8;
+}</p>
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
+### Stripe Information
+
+The body of the file is divided into stripes. Each stripe is self
+contained and may be read using only its own bytes combined with the
+file's Footer and Postscript. Each stripe contains only entire rows so
+that rows never straddle stripe boundaries. Stripes have three
+sections: a set of indexes for the rows within the stripe, the data
+itself, and a stripe footer. Both the indexes and the data sections
+are divided by columns so that only the data for the required columns
+needs to be read.
+
+```message StripeInformation {
+ // the start of the stripe within the file
+ optional uint64 offset = 1;
+ // the length of the indexes in bytes
+ optional uint64 indexLength = 2;
+ // the length of the data in bytes
+ optional uint64 dataLength = 3;
+ // the length of the footer in bytes
+ optional uint64 footerLength = 4;
+ // the number of rows in the stripe
+ optional uint64 numberOfRows = 5;
+}
+</code></pre></div></div>
+
+<h3 id="type-information">Type Information</h3>
+
+<p>All of the rows in an ORC file must have the same schema. Logically
+the schema is expressed as a tree as in the figure below, where
+the compound types have subcolumns under them.</p>
+
+<p><img src="http://localhost:4000/img/TreeWriters.png" alt="ORC column structure" /></p>
+
+<p>The equivalent Hive DDL would be:</p>
+
+<p>```create table Foobar (
+ myInt int,
+ myMap map&lt;string,
+ struct&lt;myString : string,
+ myDouble: double»,
+ myTime timestamp
+);</p>
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
+The type tree is flattened in to a list via a pre-order traversal
+where each type is assigned the next id. Clearly the root of the type
+tree is always type id 0. Compound types have a field named subtypes
+that contains the list of their children's type ids.
+
+```message Type {
+ enum Kind {
+ BOOLEAN = 0;
+ BYTE = 1;
+ SHORT = 2;
+ INT = 3;
+ LONG = 4;
+ FLOAT = 5;
+ DOUBLE = 6;
+ STRING = 7;
+ BINARY = 8;
+ TIMESTAMP = 9;
+ LIST = 10;
+ MAP = 11;
+ STRUCT = 12;
+ UNION = 13;
+ DECIMAL = 14;
+ DATE = 15;
+ VARCHAR = 16;
+ CHAR = 17;
+ }
+ // the kind of this type
+ required Kind kind = 1;
+ // the type ids of any subcolumns for list, map, struct, or union
+ repeated uint32 subtypes = 2 [packed=true];
+ // the list of field names for struct
+ repeated string fieldNames = 3;
+ // the maximum length of the type for varchar or char in UTF-8 characters
+ optional uint32 maximumLength = 4;
+ // the precision and scale for decimal
+ optional uint32 precision = 5;
+ optional uint32 scale = 6;
+}
+</code></pre></div></div>
+
+<h3 id="column-statistics">Column Statistics</h3>
+
+<p>The goal of the column statistics is that for each column, the writer
+records the count and depending on the type other useful fields. For
+most of the primitive types, it records the minimum and maximum
+values; and for numeric types it additionally stores the sum.
+From Hive 1.1.0 onwards, the column statistics will also record if
+there are any null values within the row group by setting the hasNull flag.
+The hasNull flag is used by ORC’s predicate pushdown to better answer
+‘IS NULL’ queries.</p>
+
+<p>```message ColumnStatistics {
+ // the number of values
+ optional uint64 numberOfValues = 1;
+ // At most one of these has a value for any column
+ optional IntegerStatistics intStatistics = 2;
+ optional DoubleStatistics doubleStatistics = 3;
+ optional StringStatistics stringStatistics = 4;
+ optional BucketStatistics bucketStatistics = 5;
+ optional DecimalStatistics decimalStatistics = 6;
+ optional DateStatistics dateStatistics = 7;
+ optional BinaryStatistics binaryStatistics = 8;
+ optional TimestampStatistics timestampStatistics = 9;
+ optional bool hasNull = 10;
+}</p>
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
+For integer types (tinyint, smallint, int, bigint), the column
+statistics includes the minimum, maximum, and sum. If the sum
+overflows long at any point during the calculation, no sum is
+recorded.
+
+```message IntegerStatistics {
+ optional sint64 minimum = 1;
+ optional sint64 maximum = 2;
+ optional sint64 sum = 3;
+}
+</code></pre></div></div>
+
+<p>For floating point types (float, double), the column statistics
+include the minimum, maximum, and sum. If the sum overflows a double,
+no sum is recorded.</p>
+
+<p>```message DoubleStatistics {
+ optional double minimum = 1;
+ optional double maximum = 2;
+ optional double sum = 3;
+}</p>
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
+For strings, the minimum value, maximum value, and the sum of the
+lengths of the values are recorded.
+
+```message StringStatistics {
+ optional string minimum = 1;
+ optional string maximum = 2;
+ // sum will store the total length of all strings
+ optional sint64 sum = 3;
+}
+</code></pre></div></div>
+
+<p>For booleans, the statistics include the count of false and true values.</p>
+
+<p>```message BucketStatistics {
+ repeated uint64 count = 1 [packed=true];
+}</p>
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
+For decimals, the minimum, maximum, and sum are stored.
+
+```message DecimalStatistics {
+ optional string minimum = 1;
+ optional string maximum = 2;
+ optional string sum = 3;
+}
+</code></pre></div></div>
+
+<p>Date columns record the minimum and maximum values as the number of
+days since the epoch (1/1/2015).</p>
+
+<p>```message DateStatistics {
+ // min,max values saved as days since epoch
+ optional sint32 minimum = 1;
+ optional sint32 maximum = 2;
+}</p>
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
+Timestamp columns record the minimum and maximum values as the number of
+milliseconds since the epoch (1/1/2015).
+
+```message TimestampStatistics {
+ // min,max values saved as milliseconds since epoch
+ optional sint64 minimum = 1;
+ optional sint64 maximum = 2;
+}
+</code></pre></div></div>
+
+<p>Binary columns store the aggregate number of bytes across all of the values.</p>
+
+<p>```message BinaryStatistics {
+ // sum will store the total binary blob length
+ optional sint64 sum = 1;
+}</p>
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
+### User Metadata
+
+The user can add arbitrary key/value pairs to an ORC file as it is
+written. The contents of the keys and values are completely
+application defined, but the key is a string and the value is
+binary. Care should be taken by applications to make sure that their
+keys are unique and in general should be prefixed with an organization
+code.
+
+```message UserMetadataItem {
+ // the user defined key
+ required string name = 1;
+ // the user defined binary value
+ required bytes value = 2;
+}
+</code></pre></div></div>
+
+<h3 id="file-metadata">File Metadata</h3>
+
+<p>The file Metadata section contains column statistics at the stripe
+level granularity. These statistics enable input split elimination
+based on the predicate push-down evaluated per a stripe.</p>
+
+<p>```message StripeStatistics {
+ repeated ColumnStatistics colStats = 1;
+}</p>
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
+```message Metadata {
+ repeated StripeStatistics stripeStats = 1;
+}
+</code></pre></div></div>
+
+<h1 id="compression">Compression</h1>
+
+<p>If the ORC file writer selects a generic compression codec (zlib or
+snappy), every part of the ORC file except for the Postscript is
+compressed with that codec. However, one of the requirements for ORC
+is that the reader be able to skip over compressed bytes without
+decompressing the entire stream. To manage this, ORC writes compressed
+streams in chunks with headers as in the figure below.
+To handle uncompressable data, if the compressed data is larger than
+the original, the original is stored and the isOriginal flag is
+set. Each header is 3 bytes long with (compressedLength * 2 +
+isOriginal) stored as a little endian value. For example, the header
+for a chunk that compressed to 100,000 bytes would be [0x40, 0x0d,
+0x03]. The header for 5 bytes that did not compress would be [0x0b,
+0x00, 0x00]. Each compression chunk is compressed independently so
+that as long as a decompressor starts at the top of a header, it can
+start decompressing without the previous bytes.</p>
+
+<p><img src="http://localhost:4000/img/CompressionStream.png" alt="compression streams" /></p>
+
+<p>The default compression chunk size is 256K, but writers can choose
+their own value. Larger chunks lead to better compression, but require
+more memory. The chunk size is recorded in the Postscript so that
+readers can allocate appropriately sized buffers. Readers are
+guaranteed that no chunk will expand to more than the compression chunk
+size.</p>
+
+<p>ORC files without generic compression write each stream directly
+with no headers.</p>
+
+<h1 id="run-length-encoding">Run Length Encoding</h1>
+
+<h2 id="base-128-varint">Base 128 Varint</h2>
+
+<p>Variable width integer encodings take advantage of the fact that most
+numbers are small and that having smaller encodings for small numbers
+shrinks the overall size of the data. ORC uses the varint format from
+Protocol Buffers, which writes data in little endian format using the
+low 7 bits of each byte. The high bit in each byte is set if the
+number continues into the next byte.</p>
+
+<table>
+  <thead>
+    <tr>
+      <th style="text-align: left">Unsigned Original</th>
+      <th style="text-align: left">Serialized</th>
+    </tr>
+  </thead>
+  <tbody>
+    <tr>
+      <td style="text-align: left">0</td>
+      <td style="text-align: left">0x00</td>
+    </tr>
+    <tr>
+      <td style="text-align: left">1</td>
+      <td style="text-align: left">0x01</td>
+    </tr>
+    <tr>
+      <td style="text-align: left">127</td>
+      <td style="text-align: left">0x7f</td>
+    </tr>
+    <tr>
+      <td style="text-align: left">128</td>
+      <td style="text-align: left">0x80, 0x01</td>
+    </tr>
+    <tr>
+      <td style="text-align: left">129</td>
+      <td style="text-align: left">0x81, 0x01</td>
+    </tr>
+    <tr>
+      <td style="text-align: left">16,383</td>
+      <td style="text-align: left">0xff, 0x7f</td>
+    </tr>
+    <tr>
+      <td style="text-align: left">16,384</td>
+      <td style="text-align: left">0x80, 0x80, 0x01</td>
+    </tr>
+    <tr>
+      <td style="text-align: left">16,385</td>
+      <td style="text-align: left">0x81, 0x80, 0x01</td>
+    </tr>
+  </tbody>
+</table>
+
+<p>For signed integer types, the number is converted into an unsigned
+number using a zigzag encoding. Zigzag encoding moves the sign bit to
+the least significant bit using the expression (val « 1) ^ (val »
+63) and derives its name from the fact that positive and negative
+numbers alternate once encoded. The unsigned number is then serialized
+as above.</p>
+
+<table>
+  <thead>
+    <tr>
+      <th style="text-align: left">Signed Original</th>
+      <th style="text-align: left">Unsigned</th>
+    </tr>
+  </thead>
+  <tbody>
+    <tr>
+      <td style="text-align: left">0</td>
+      <td style="text-align: left">0</td>
+    </tr>
+    <tr>
+      <td style="text-align: left">-1</td>
+      <td style="text-align: left">1</td>
+    </tr>
+    <tr>
+      <td style="text-align: left">1</td>
+      <td style="text-align: left">2</td>
+    </tr>
+    <tr>
+      <td style="text-align: left">-2</td>
+      <td style="text-align: left">3</td>
+    </tr>
+    <tr>
+      <td style="text-align: left">2</td>
+      <td style="text-align: left">4</td>
+    </tr>
+  </tbody>
+</table>
+
+<h2 id="byte-run-length-encoding">Byte Run Length Encoding</h2>
+
+<p>For byte streams, ORC uses a very light weight encoding of identical
+values.</p>
+
+<ul>
+  <li>Run - a sequence of at least 3 identical values</li>
+  <li>Literals - a sequence of non-identical values</li>
+</ul>
+
+<p>The first byte of each group of values is a header than determines
+whether it is a run (value between 0 to 127) or literal list (value
+between -128 to -1). For runs, the control byte is the length of the
+run minus the length of the minimal run (3) and the control byte for
+literal lists is the negative length of the list. For example, a
+hundred 0’s is encoded as [0x61, 0x00] and the sequence 0x44, 0x45
+would be encoded as [0xfe, 0x44, 0x45]. The next group can choose
+either of the encodings.</p>
+
+<h2 id="boolean-run-length-encoding">Boolean Run Length Encoding</h2>
+
+<p>For encoding boolean types, the bits are put in the bytes from most
+significant to least significant. The bytes are encoded using byte run
+length encoding as described in the previous section. For example,
+the byte sequence [0xff, 0x80] would be one true followed by
+seven false values.</p>
+
+<h2 id="integer-run-length-encoding-version-1">Integer Run Length Encoding, version 1</h2>
+
+<p>ORC v0 files use Run Length Encoding version 1 (RLEv1),
+which provides a lightweight compression of signed or unsigned integer
+sequences. RLEv1 has two sub-encodings:</p>
+
+<ul>
+  <li>Run - a sequence of values that differ by a small fixed delta</li>
+  <li>Literals - a sequence of varint encoded values</li>
+</ul>
+
+<p>Runs start with an initial byte of 0x00 to 0x7f, which encodes the
+length of the run - 3. A second byte provides the fixed delta in the
+range of -128 to 127. Finally, the first value of the run is encoded
+as a base 128 varint.</p>
+
+<p>For example, if the sequence is 100 instances of 7 the encoding would
+start with 100 - 3, followed by a delta of 0, and a varint of 7 for
+an encoding of [0x61, 0x00, 0x07]. To encode the sequence of numbers
+running from 100 to 1, the first byte is 100 - 3, the delta is -1,
+and the varint is 100 for an encoding of [0x61, 0xff, 0x64].</p>
+
+<p>Literals start with an initial byte of 0x80 to 0xff, which corresponds
+to the negative of number of literals in the sequence. Following the
+header byte, the list of N varints is encoded. Thus, if there are
+no runs, the overhead is 1 byte for each 128 integers. The first 5
+prime numbers [2, 3, 4, 7, 11] would encoded as [0xfb, 0x02, 0x03,
+0x04, 0x07, 0xb].</p>
+
+<h1 id="stripes">Stripes</h1>
+
+<p>The body of ORC files consists of a series of stripes. Stripes are
+large (typically ~200MB) and independent of each other and are often
+processed by different tasks. The defining characteristic for columnar
+storage formats is that the data for each column is stored separately
+and that reading data out of the file should be proportional to the
+number of columns read.</p>
+
+<p>In ORC files, each column is stored in several streams that are stored
+next to each other in the file. For example, an integer column is
+represented as two streams PRESENT, which uses one with a bit per
+value recording if the value is non-null, and DATA, which records the
+non-null values. If all of a column’s values in a stripe are non-null,
+the PRESENT stream is omitted from the stripe. For binary data, ORC
+uses three streams PRESENT, DATA, and LENGTH, which stores the length
+of each value. The details of each type will be presented in the
+following subsections.</p>
+
+<h2 id="stripe-footer">Stripe Footer</h2>
+
+<p>The stripe footer contains the encoding of each column and the
+directory of the streams including their location.</p>
+
+<p>```message StripeFooter {
+ // the location of each stream
+ repeated Stream streams = 1;
+ // the encoding of each column
+ repeated ColumnEncoding columns = 2;
+}</p>
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
+To describe each stream, ORC stores the kind of stream, the column id,
+and the stream's size in bytes. The details of what is stored in each stream
+depends on the type and encoding of the column.
+
+```message Stream {
+ enum Kind {
+ // boolean stream of whether the next value is non-null
+ PRESENT = 0;
+ // the primary data stream
+ DATA = 1;
+ // the length of each value for variable length data
+ LENGTH = 2;
+ // the dictionary blob
+ DICTIONARY\_DATA = 3;
+ // deprecated prior to Hive 0.11
+ // It was used to store the number of instances of each value in the
+ // dictionary
+ DICTIONARY_COUNT = 4;
+ // a secondary data stream
+ SECONDARY = 5;
+ // the index for seeking to particular row groups
+ ROW_INDEX = 6;
+ }
+ required Kind kind = 1;
+ // the column id
+ optional uint32 column = 2;
+ // the number of bytes in the file
+ optional uint64 length = 3;
+}
+</code></pre></div></div>
+
+<p>Depending on their type several options for encoding are possible. The
+encodings are divided into direct or dictionary-based categories and
+further refined as to whether they use RLE v1 or v2.</p>
+
+<p>```message ColumnEncoding {
+ enum Kind {
+ // the encoding is mapped directly to the stream using RLE v1
+ DIRECT = 0;
+ // the encoding uses a dictionary of unique values using RLE v1
+ DICTIONARY = 1;
+ // the encoding is direct using RLE v2
+ }
+ required Kind kind = 1;
+ // for dictionary encodings, record the size of the dictionary
+ optional uint32 dictionarySize = 2;
+}</p>
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
+# Column Encodings
+
+## SmallInt, Int, and BigInt Columns
+
+All of the 16, 32, and 64 bit integer column types use the same set of
+potential encodings, which is basically whether they use RLE v1 or
+v2. If the PRESENT stream is not included, all of the values are
+present. For values that have false bits in the present stream, no
+values are included in the data stream.
+
+Encoding  | Stream Kind | Optional | Contents
+:-------- | :---------- | :------- | :-------
+DIRECT    | PRESENT     | Yes      | Boolean RLE
+          | DATA        | No       | Signed Integer RLE v1
+
+## Float and Double Columns
+
+Floating point types are stored using IEEE 754 floating point bit
+layout. Float columns use 4 bytes per value and double columns use 8
+bytes.
+
+Encoding  | Stream Kind | Optional | Contents
+:-------- | :---------- | :------- | :-------
+DIRECT    | PRESENT     | Yes      | Boolean RLE
+          | DATA        | No       | IEEE 754 floating point representation
+
+## String, Char, and VarChar Columns
+
+String, char, and varchar columns may be encoded either using a
+dictionary encoding or a direct encoding. A direct encoding should be
+preferred when there are many distinct values. In all of the
+encodings, the PRESENT stream encodes whether the value is null. The
+Java ORC writer automatically picks the encoding after the first row
+group (10,000 rows).
+
+For direct encoding the UTF-8 bytes are saved in the DATA stream and
+the length of each value is written into the LENGTH stream. In direct
+encoding, if the values were ["Nevada", "California"]; the DATA
+would be "NevadaCalifornia" and the LENGTH would be [6, 10].
+
+For dictionary encodings the dictionary is sorted and UTF-8 bytes of
+each unique value are placed into DICTIONARY_DATA. The length of each
+item in the dictionary is put into the LENGTH stream. The DATA stream
+consists of the sequence of references to the dictionary elements.
+
+In dictionary encoding, if the values were ["Nevada",
+"California", "Nevada", "California", and "Florida"]; the
+DICTIONARY_DATA would be "CaliforniaFloridaNevada" and LENGTH would
+be [10, 7, 6]. The DATA would be [2, 0, 2, 0, 1].
+
+Encoding      | Stream Kind     | Optional | Contents
+:------------ | :-------------- | :------- | :-------
+DIRECT        | PRESENT         | Yes      | Boolean RLE
+              | DATA            | No       | String contents
+              | LENGTH          | No       | Unsigned Integer RLE v1
+DICTIONARY    | PRESENT         | Yes      | Boolean RLE
+              | DATA            | No       | Unsigned Integer RLE v1
+              | DICTIONARY_DATA | No       | String contents
+              | LENGTH          | No       | Unsigned Integer RLE v1
+
+## Boolean Columns
+
+Boolean columns are rare, but have a simple encoding.
+
+Encoding      | Stream Kind     | Optional | Contents
+:------------ | :-------------- | :------- | :-------
+DIRECT        | PRESENT         | Yes      | Boolean RLE
+              | DATA            | No       | Boolean RLE
+
+## TinyInt Columns
+
+TinyInt (byte) columns use byte run length encoding.
+
+Encoding      | Stream Kind     | Optional | Contents
+:------------ | :-------------- | :------- | :-------
+DIRECT        | PRESENT         | Yes      | Boolean RLE
+              | DATA            | No       | Byte RLE
+
+## Binary Columns
+
+Binary data is encoded with a PRESENT stream, a DATA stream that records
+the contents, and a LENGTH stream that records the number of bytes per a
+value.
+
+Encoding      | Stream Kind     | Optional | Contents
+:------------ | :-------------- | :------- | :-------
+DIRECT        | PRESENT         | Yes      | Boolean RLE
+              | DATA            | No       | String contents
+              | LENGTH          | No       | Unsigned Integer RLE v1
+
+## Decimal Columns
+
+Decimal was introduced in Hive 0.11 with infinite precision (the total
+number of digits). In Hive 0.13, the definition was change to limit
+the precision to a maximum of 38 digits, which conveniently uses 127
+bits plus a sign bit. The current encoding of decimal columns stores
+the integer representation of the value as an unbounded length zigzag
+encoded base 128 varint. The scale is stored in the SECONDARY stream
+as an signed integer.
+
+Encoding      | Stream Kind     | Optional | Contents
+:------------ | :-------------- | :------- | :-------
+DIRECT        | PRESENT         | Yes      | Boolean RLE
+              | DATA            | No       | Unbounded base 128 varints
+              | SECONDARY       | No       | Unsigned Integer RLE v1
+
+## Date Columns
+
+Date data is encoded with a PRESENT stream, a DATA stream that records
+the number of days after January 1, 1970 in UTC.
+
+Encoding      | Stream Kind     | Optional | Contents
+:------------ | :-------------- | :------- | :-------
+DIRECT        | PRESENT         | Yes      | Boolean RLE
+              | DATA            | No       | Signed Integer RLE v1
+
+## Timestamp Columns
+
+Timestamp records times down to nanoseconds as a PRESENT stream that
+records non-null values, a DATA stream that records the number of
+seconds after 1 January 2015, and a SECONDARY stream that records the
+number of nanoseconds.
+
+Because the number of nanoseconds often has a large number of trailing
+zeros, the number has trailing decimal zero digits removed and the
+last three bits are used to record how many zeros were removed. Thus
+1000 nanoseconds would be serialized as 0x0b and 100000 would be
+serialized as 0x0d.
+
+Encoding      | Stream Kind     | Optional | Contents
+:------------ | :-------------- | :------- | :-------
+DIRECT        | PRESENT         | Yes      | Boolean RLE
+              | DATA            | No       | Signed Integer RLE v1
+              | SECONDARY       | No       | Unsigned Integer RLE v1
+
+## Struct Columns
+
+Structs have no data themselves and delegate everything to their child
+columns except for their PRESENT stream. They have a child column
+for each of the fields.
+
+Encoding      | Stream Kind     | Optional | Contents
+:------------ | :-------------- | :------- | :-------
+DIRECT        | PRESENT         | Yes      | Boolean RLE
+
+## List Columns
+
+Lists are encoded as the PRESENT stream and a length stream with
+number of items in each list. They have a single child column for the
+element values.
+
+Encoding      | Stream Kind     | Optional | Contents
+:------------ | :-------------- | :------- | :-------
+DIRECT        | PRESENT         | Yes      | Boolean RLE
+              | LENGTH          | No       | Unsigned Integer RLE v1
+
+## Map Columns
+
+Maps are encoded as the PRESENT stream and a length stream with number
+of items in each list. They have a child column for the key and
+another child column for the value.
+
+Encoding      | Stream Kind     | Optional | Contents
+:------------ | :-------------- | :------- | :-------
+DIRECT        | PRESENT         | Yes      | Boolean RLE
+              | LENGTH          | No       | Unsigned Integer RLE v1
+
+## Union Columns
+
+Unions are encoded as the PRESENT stream and a tag stream that controls which
+potential variant is used. They have a child column for each variant of the
+union. Currently ORC union types are limited to 256 variants, which matches
+the Hive type model.
+
+Encoding      | Stream Kind     | Optional | Contents
+:------------ | :-------------- | :------- | :-------
+DIRECT        | PRESENT         | Yes      | Boolean RLE
+              | DIRECT          | No       | Byte RLE
+
+# Indexes
+
+## Row Group Index
+
+The row group indexes consist of a ROW_INDEX stream for each primitive
+column that has an entry for each row group. Row groups are controlled
+by the writer and default to 10,000 rows. Each RowIndexEntry gives the
+position of each stream for the column and the statistics for that row
+group.
+
+The index streams are placed at the front of the stripe, because in
+the default case of streaming they do not need to be read. They are
+only loaded when either predicate push down is being used or the
+reader seeks to a particular row.
+
+```message RowIndexEntry {
+ repeated uint64 positions = 1 [packed=true];
+ optional ColumnStatistics statistics = 2;
+}
+</code></pre></div></div>
+
+<p><code class="highlighter-rouge">message RowIndex {
+ repeated RowIndexEntry entry = 1;
+}
+</code></p>
+
+<p>To record positions, each stream needs a sequence of numbers. For
+uncompressed streams, the position is the byte offset of the RLE run’s
+start location followed by the number of values that need to be
+consumed from the run. In compressed streams, the first number is the
+start of the compression chunk in the stream, followed by the number
+of decompressed bytes that need to be consumed, and finally the number
+of values consumed in the RLE.</p>
+
+<p>For columns with multiple streams, the sequences of positions in each
+stream are concatenated. That was an unfortunate decision on my part
+that we should fix at some point, because it makes code that uses the
+indexes error-prone.</p>
+
+<p>Because dictionaries are accessed randomly, there is not a position to
+record for the dictionary and the entire dictionary must be read even
+if only part of a stripe is being read.</p>
+
+
+      </article>
+    </div>
+
+    <div class="clear"></div>
+
+  </div>
+</section>
+
+
+  <footer role="contentinfo">
+  <p>The contents of this website are &copy;&nbsp;2018
+     <a href="https://www.apache.org/">Apache Software Foundation</a>
+     under the terms of the <a
+      href="https://www.apache.org/licenses/LICENSE-2.0.html">
+      Apache&nbsp;License&nbsp;v2</a>. Apache ORC and its logo are trademarks
+      of the Apache Software Foundation.</p>
+</footer>
+
+  <script>
+  var anchorForId = function (id) {
+    var anchor = document.createElement("a");
+    anchor.className = "header-link";
+    anchor.href      = "#" + id;
+    anchor.innerHTML = "<span class=\"sr-only\">Permalink</span><i class=\"fa fa-link\"></i>";
+    anchor.title = "Permalink";
+    return anchor;
+  };
+
+  var linkifyAnchors = function (level, containingElement) {
+    var headers = containingElement.getElementsByTagName("h" + level);
+    for (var h = 0; h < headers.length; h++) {
+      var header = headers[h];
+
+      if (typeof header.id !== "undefined" && header.id !== "") {
+        header.appendChild(anchorForId(header.id));
+      }
+    }
+  };
+
+  document.onreadystatechange = function () {
+    if (this.readyState === "complete") {
+      var contentBlock = document.getElementsByClassName("docs")[0] || document.getElementsByClassName("news")[0];
+      if (!contentBlock) {
+        return;
+      }
+      for (var level = 1; level <= 6; level++) {
+        linkifyAnchors(level, contentBlock);
+      }
+    }
+  };
+</script>
+
+
+</body>
+</html>


[02/11] orc git commit: Deploy docs with CVE.

Posted by om...@apache.org.
http://git-wip-us.apache.org/repos/asf/orc/blob/e58d3fc4/specification/ORCv2.html
----------------------------------------------------------------------
diff --git a/specification/ORCv2.html b/specification/ORCv2.html
deleted file mode 100644
index db8aecf..0000000
--- a/specification/ORCv2.html
+++ /dev/null
@@ -1,1760 +0,0 @@
-<!DOCTYPE HTML>
-<html lang="en-US">
-<head>
-  <meta charset="UTF-8">
-  <title>Evolving Draft for ORC Specification v2</title>
-  <meta name="viewport" content="width=device-width,initial-scale=1">
-  <meta name="generator" content="Jekyll v2.4.0">
-  <link rel="stylesheet" href="//fonts.googleapis.com/css?family=Lato:300,300italic,400,400italic,700,700italic,900">
-  <link rel="stylesheet" href="/css/screen.css">
-  <link rel="icon" type="image/x-icon" href="/favicon.ico">
-  <!--[if lt IE 9]>
-  <script src="/js/html5shiv.min.js"></script>
-  <script src="/js/respond.min.js"></script>
-  <![endif]-->
-</head>
-
-
-<body class="wrap">
-  <header role="banner">
-  <nav class="mobile-nav show-on-mobiles">
-    <ul>
-  <li class="">
-    <a href="/">Home</a>
-  </li>
-  <li class="">
-    <a href="/docs/"><span class="show-on-mobiles">Docs</span>
-                     <span class="hide-on-mobiles">Documentation</span></a>
-  </li>
-  <li class="">
-    <a href="/talks/">Talks</a>
-  </li>
-  <li class="">
-    <a href="/news/">News</a>
-  </li>
-  <li class="">
-    <a href="/help/">Help</a>
-  </li>
-  <li class="">
-    <a href="/develop/">Develop</a>
-  </li>
-</ul>
-
-  </nav>
-  <div class="grid">
-    <div class="unit one-third center-on-mobiles">
-      <h1>
-        <a href="/">
-          <span class="sr-only">Apache ORC</span>
-          <img src="/img/logo.png" width="249" height="101" alt="ORC Logo">
-        </a>
-      </h1>
-    </div>
-    <nav class="main-nav unit two-thirds hide-on-mobiles">
-      <ul>
-  <li class="">
-    <a href="/">Home</a>
-  </li>
-  <li class="">
-    <a href="/docs/"><span class="show-on-mobiles">Docs</span>
-                     <span class="hide-on-mobiles">Documentation</span></a>
-  </li>
-  <li class="">
-    <a href="/talks/">Talks</a>
-  </li>
-  <li class="">
-    <a href="/news/">News</a>
-  </li>
-  <li class="">
-    <a href="/help/">Help</a>
-  </li>
-  <li class="">
-    <a href="/develop/">Develop</a>
-  </li>
-</ul>
-
-    </nav>
-  </div>
-</header>
-
-
-  <section class="standalone">
-  <div class="grid">
-
-    <div class="unit whole">
-      <article>
-        <h1>Evolving Draft for ORC Specification v2</h1>
-        <p>This specification is rapidly evolving and should only be used for
-developers on the project.</p>
-
-<h1 id="to-do-items">TO DO items</h1>
-
-<p>The list of things that we plan to change:</p>
-
-<ul>
-  <li>Move decimal encoding to RLEv3 and remove variable length encoding.</li>
-  <li>Create a better float/double encoding that splits mantissa and
-exponent.</li>
-  <li>Create a dictionary encoding for float, double, and decimal.</li>
-  <li>Create RLEv3:
-    <ul>
-      <li>64 and 128 bit variants</li>
-      <li>Zero suppression</li>
-      <li>Evaluate the rle subformats</li>
-    </ul>
-  </li>
-  <li>Group stripe data into stripelets to enable Async IO for reads.</li>
-  <li>Reorder stripe data into (stripe metadata, index, dictionary, data)</li>
-  <li>Stop sorting dictionaries and record the sort order separately in the index.</li>
-  <li>Remove use of RLEv1 and RLEv2.</li>
-  <li>Remove non-utf8 bloom filter.</li>
-  <li>Use numeric value for decimal statistics and bloom filter.</li>
-  <li>Add Zstd with dictionary.</li>
-</ul>
-
-<h1 id="motivation">Motivation</h1>
-
-<p>Hive’s RCFile was the standard format for storing tabular data in
-Hadoop for several years. However, RCFile has limitations because it
-treats each column as a binary blob without semantics. In Hive 0.11 we
-added a new file format named Optimized Row Columnar (ORC) file that
-uses and retains the type information from the table definition. ORC
-uses type specific readers and writers that provide light weight
-compression techniques such as dictionary encoding, bit packing, delta
-encoding, and run length encoding – resulting in dramatically smaller
-files. Additionally, ORC can apply generic compression using zlib, or
-Snappy on top of the lightweight compression for even smaller
-files. However, storage savings are only part of the gain. ORC
-supports projection, which selects subsets of the columns for reading,
-so that queries reading only one column read only the required
-bytes. Furthermore, ORC files include light weight indexes that
-include the minimum and maximum values for each column in each set of
-10,000 rows and the entire file. Using pushdown filters from Hive, the
-file reader can skip entire sets of rows that aren’t important for
-this query.</p>
-
-<p><img src="/img/OrcFileLayout.png" alt="ORC file structure" /></p>
-
-<h1 id="file-tail">File Tail</h1>
-
-<p>Since HDFS does not support changing the data in a file after it is
-written, ORC stores the top level index at the end of the file. The
-overall structure of the file is given in the figure above.  The
-file’s tail consists of 3 parts; the file metadata, file footer and
-postscript.</p>
-
-<p>The metadata for ORC is stored using
-<a href="https://s.apache.org/protobuf_encoding">Protocol Buffers</a>, which provides
-the ability to add new fields without breaking readers. This document
-incorporates the Protobuf definition from the
-<a href="https://s.apache.org/orc_proto">ORC source code</a> and the
-reader is encouraged to review the Protobuf encoding if they need to
-understand the byte-level encoding</p>
-
-<h2 id="postscript">Postscript</h2>
-
-<p>The Postscript section provides the necessary information to interpret
-the rest of the file including the length of the file’s Footer and
-Metadata sections, the version of the file, and the kind of general
-compression used (eg. none, zlib, or snappy). The Postscript is never
-compressed and ends one byte before the end of the file. The version
-stored in the Postscript is the lowest version of Hive that is
-guaranteed to be able to read the file and it stored as a sequence of
-the major and minor version. This file version is encoded as [0,12].</p>
-
-<p>The process of reading an ORC file works backwards through the
-file. Rather than making multiple short reads, the ORC reader reads
-the last 16k bytes of the file with the hope that it will contain both
-the Footer and Postscript sections. The final byte of the file
-contains the serialized length of the Postscript, which must be less
-than 256 bytes. Once the Postscript is parsed, the compressed
-serialized length of the Footer is known and it can be decompressed
-and parsed.</p>
-
-<p><code>message PostScript {
- // the length of the footer section in bytes
- optional uint64 footerLength = 1;
- // the kind of generic compression used
- optional CompressionKind compression = 2;
- // the maximum size of each compression chunk
- optional uint64 compressionBlockSize = 3;
- // the version of the writer
- repeated uint32 version = 4 [packed = true];
- // the length of the metadata section in bytes
- optional uint64 metadataLength = 5;
- // the fixed string "ORC"
- optional string magic = 8000;
-}
-</code></p>
-
-<p><code>enum CompressionKind {
- NONE = 0;
- ZLIB = 1;
- SNAPPY = 2;
- LZO = 3;
- LZ4 = 4;
- ZSTD = 5;
-}
-</code></p>
-
-<h2 id="footer">Footer</h2>
-
-<p>The Footer section contains the layout of the body of the file, the
-type schema information, the number of rows, and the statistics about
-each of the columns.</p>
-
-<p>The file is broken in to three parts- Header, Body, and Tail. The
-Header consists of the bytes “ORC’’ to support tools that want to
-scan the front of the file to determine the type of the file. The Body
-contains the rows and indexes, and the Tail gives the file level
-information as described in this section.</p>
-
-<p><code>message Footer {
- // the length of the file header in bytes (always 3)
- optional uint64 headerLength = 1;
- // the length of the file header and body in bytes
- optional uint64 contentLength = 2;
- // the information about the stripes
- repeated StripeInformation stripes = 3;
- // the schema information
- repeated Type types = 4;
- // the user metadata that was added
- repeated UserMetadataItem metadata = 5;
- // the total number of rows in the file
- optional uint64 numberOfRows = 6;
- // the statistics of each column across the file
- repeated ColumnStatistics statistics = 7;
- // the maximum number of rows in each index entry
- optional uint32 rowIndexStride = 8;
-}
-</code></p>
-
-<h3 id="stripe-information">Stripe Information</h3>
-
-<p>The body of the file is divided into stripes. Each stripe is self
-contained and may be read using only its own bytes combined with the
-file’s Footer and Postscript. Each stripe contains only entire rows so
-that rows never straddle stripe boundaries. Stripes have three
-sections: a set of indexes for the rows within the stripe, the data
-itself, and a stripe footer. Both the indexes and the data sections
-are divided by columns so that only the data for the required columns
-needs to be read.</p>
-
-<p><code>message StripeInformation {
- // the start of the stripe within the file
- optional uint64 offset = 1;
- // the length of the indexes in bytes
- optional uint64 indexLength = 2;
- // the length of the data in bytes
- optional uint64 dataLength = 3;
- // the length of the footer in bytes
- optional uint64 footerLength = 4;
- // the number of rows in the stripe
- optional uint64 numberOfRows = 5;
-}
-</code></p>
-
-<h3 id="type-information">Type Information</h3>
-
-<p>All of the rows in an ORC file must have the same schema. Logically
-the schema is expressed as a tree as in the figure below, where
-the compound types have subcolumns under them.</p>
-
-<p><img src="/img/TreeWriters.png" alt="ORC column structure" /></p>
-
-<p>The equivalent Hive DDL would be:</p>
-
-<p><code>create table Foobar (
- myInt int,
- myMap map&lt;string,
- struct&lt;myString : string,
- myDouble: double&gt;&gt;,
- myTime timestamp
-);
-</code></p>
-
-<p>The type tree is flattened in to a list via a pre-order traversal
-where each type is assigned the next id. Clearly the root of the type
-tree is always type id 0. Compound types have a field named subtypes
-that contains the list of their children’s type ids.</p>
-
-<p><code>message Type {
- enum Kind {
- BOOLEAN = 0;
- BYTE = 1;
- SHORT = 2;
- INT = 3;
- LONG = 4;
- FLOAT = 5;
- DOUBLE = 6;
- STRING = 7;
- BINARY = 8;
- TIMESTAMP = 9;
- LIST = 10;
- MAP = 11;
- STRUCT = 12;
- UNION = 13;
- DECIMAL = 14;
- DATE = 15;
- VARCHAR = 16;
- CHAR = 17;
- }
- // the kind of this type
- required Kind kind = 1;
- // the type ids of any subcolumns for list, map, struct, or union
- repeated uint32 subtypes = 2 [packed=true];
- // the list of field names for struct
- repeated string fieldNames = 3;
- // the maximum length of the type for varchar or char in UTF-8 characters
- optional uint32 maximumLength = 4;
- // the precision and scale for decimal
- optional uint32 precision = 5;
- optional uint32 scale = 6;
-}
-</code></p>
-
-<h3 id="column-statistics">Column Statistics</h3>
-
-<p>The goal of the column statistics is that for each column, the writer
-records the count and depending on the type other useful fields. For
-most of the primitive types, it records the minimum and maximum
-values; and for numeric types it additionally stores the sum.
-From Hive 1.1.0 onwards, the column statistics will also record if
-there are any null values within the row group by setting the hasNull flag.
-The hasNull flag is used by ORC’s predicate pushdown to better answer
-‘IS NULL’ queries.</p>
-
-<p><code>message ColumnStatistics {
- // the number of values
- optional uint64 numberOfValues = 1;
- // At most one of these has a value for any column
- optional IntegerStatistics intStatistics = 2;
- optional DoubleStatistics doubleStatistics = 3;
- optional StringStatistics stringStatistics = 4;
- optional BucketStatistics bucketStatistics = 5;
- optional DecimalStatistics decimalStatistics = 6;
- optional DateStatistics dateStatistics = 7;
- optional BinaryStatistics binaryStatistics = 8;
- optional TimestampStatistics timestampStatistics = 9;
- optional bool hasNull = 10;
-}
-</code></p>
-
-<p>For integer types (tinyint, smallint, int, bigint), the column
-statistics includes the minimum, maximum, and sum. If the sum
-overflows long at any point during the calculation, no sum is
-recorded.</p>
-
-<p><code>message IntegerStatistics {
- optional sint64 minimum = 1;
- optional sint64 maximum = 2;
- optional sint64 sum = 3;
-}
-</code></p>
-
-<p>For floating point types (float, double), the column statistics
-include the minimum, maximum, and sum. If the sum overflows a double,
-no sum is recorded.</p>
-
-<p><code>message DoubleStatistics {
- optional double minimum = 1;
- optional double maximum = 2;
- optional double sum = 3;
-}
-</code></p>
-
-<p>For strings, the minimum value, maximum value, and the sum of the
-lengths of the values are recorded.</p>
-
-<p><code>message StringStatistics {
- optional string minimum = 1;
- optional string maximum = 2;
- // sum will store the total length of all strings
- optional sint64 sum = 3;
-}
-</code></p>
-
-<p>For booleans, the statistics include the count of false and true values.</p>
-
-<p><code>message BucketStatistics {
- repeated uint64 count = 1 [packed=true];
-}
-</code></p>
-
-<p>For decimals, the minimum, maximum, and sum are stored.</p>
-
-<p><code>message DecimalStatistics {
- optional string minimum = 1;
- optional string maximum = 2;
- optional string sum = 3;
-}
-</code></p>
-
-<p>Date columns record the minimum and maximum values as the number of
-days since the epoch (1/1/2015).</p>
-
-<p><code>message DateStatistics {
- // min,max values saved as days since epoch
- optional sint32 minimum = 1;
- optional sint32 maximum = 2;
-}
-</code></p>
-
-<p>Timestamp columns record the minimum and maximum values as the number of
-milliseconds since the epoch (1/1/2015).</p>
-
-<p><code>message TimestampStatistics {
- // min,max values saved as milliseconds since epoch
- optional sint64 minimum = 1;
- optional sint64 maximum = 2;
-}
-</code></p>
-
-<p>Binary columns store the aggregate number of bytes across all of the values.</p>
-
-<p><code>message BinaryStatistics {
- // sum will store the total binary blob length
- optional sint64 sum = 1;
-}
-</code></p>
-
-<h3 id="user-metadata">User Metadata</h3>
-
-<p>The user can add arbitrary key/value pairs to an ORC file as it is
-written. The contents of the keys and values are completely
-application defined, but the key is a string and the value is
-binary. Care should be taken by applications to make sure that their
-keys are unique and in general should be prefixed with an organization
-code.</p>
-
-<p><code>message UserMetadataItem {
- // the user defined key
- required string name = 1;
- // the user defined binary value
- required bytes value = 2;
-}
-</code></p>
-
-<h3 id="file-metadata">File Metadata</h3>
-
-<p>The file Metadata section contains column statistics at the stripe
-level granularity. These statistics enable input split elimination
-based on the predicate push-down evaluated per a stripe.</p>
-
-<p><code>message StripeStatistics {
- repeated ColumnStatistics colStats = 1;
-}
-</code></p>
-
-<p><code>message Metadata {
- repeated StripeStatistics stripeStats = 1;
-}
-</code></p>
-
-<h1 id="compression">Compression</h1>
-
-<p>If the ORC file writer selects a generic compression codec (zlib or
-snappy), every part of the ORC file except for the Postscript is
-compressed with that codec. However, one of the requirements for ORC
-is that the reader be able to skip over compressed bytes without
-decompressing the entire stream. To manage this, ORC writes compressed
-streams in chunks with headers as in the figure below.
-To handle uncompressable data, if the compressed data is larger than
-the original, the original is stored and the isOriginal flag is
-set. Each header is 3 bytes long with (compressedLength * 2 +
-isOriginal) stored as a little endian value. For example, the header
-for a chunk that compressed to 100,000 bytes would be [0x40, 0x0d,
-0x03]. The header for 5 bytes that did not compress would be [0x0b,
-0x00, 0x00]. Each compression chunk is compressed independently so
-that as long as a decompressor starts at the top of a header, it can
-start decompressing without the previous bytes.</p>
-
-<p><img src="/img/CompressionStream.png" alt="compression streams" /></p>
-
-<p>The default compression chunk size is 256K, but writers can choose
-their own value. Larger chunks lead to better compression, but require
-more memory. The chunk size is recorded in the Postscript so that
-readers can allocate appropriately sized buffers. Readers are
-guaranteed that no chunk will expand to more than the compression chunk
-size.</p>
-
-<p>ORC files without generic compression write each stream directly
-with no headers.</p>
-
-<h1 id="run-length-encoding">Run Length Encoding</h1>
-
-<h2 id="base-128-varint">Base 128 Varint</h2>
-
-<p>Variable width integer encodings take advantage of the fact that most
-numbers are small and that having smaller encodings for small numbers
-shrinks the overall size of the data. ORC uses the varint format from
-Protocol Buffers, which writes data in little endian format using the
-low 7 bits of each byte. The high bit in each byte is set if the
-number continues into the next byte.</p>
-
-<table>
-  <thead>
-    <tr>
-      <th style="text-align: left">Unsigned Original</th>
-      <th style="text-align: left">Serialized</th>
-    </tr>
-  </thead>
-  <tbody>
-    <tr>
-      <td style="text-align: left">0</td>
-      <td style="text-align: left">0x00</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">1</td>
-      <td style="text-align: left">0x01</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">127</td>
-      <td style="text-align: left">0x7f</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">128</td>
-      <td style="text-align: left">0x80, 0x01</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">129</td>
-      <td style="text-align: left">0x81, 0x01</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">16,383</td>
-      <td style="text-align: left">0xff, 0x7f</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">16,384</td>
-      <td style="text-align: left">0x80, 0x80, 0x01</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">16,385</td>
-      <td style="text-align: left">0x81, 0x80, 0x01</td>
-    </tr>
-  </tbody>
-</table>
-
-<p>For signed integer types, the number is converted into an unsigned
-number using a zigzag encoding. Zigzag encoding moves the sign bit to
-the least significant bit using the expression (val « 1) ^ (val »
-63) and derives its name from the fact that positive and negative
-numbers alternate once encoded. The unsigned number is then serialized
-as above.</p>
-
-<table>
-  <thead>
-    <tr>
-      <th style="text-align: left">Signed Original</th>
-      <th style="text-align: left">Unsigned</th>
-    </tr>
-  </thead>
-  <tbody>
-    <tr>
-      <td style="text-align: left">0</td>
-      <td style="text-align: left">0</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">-1</td>
-      <td style="text-align: left">1</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">1</td>
-      <td style="text-align: left">2</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">-2</td>
-      <td style="text-align: left">3</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">2</td>
-      <td style="text-align: left">4</td>
-    </tr>
-  </tbody>
-</table>
-
-<h2 id="byte-run-length-encoding">Byte Run Length Encoding</h2>
-
-<p>For byte streams, ORC uses a very light weight encoding of identical
-values.</p>
-
-<ul>
-  <li>Run - a sequence of at least 3 identical values</li>
-  <li>Literals - a sequence of non-identical values</li>
-</ul>
-
-<p>The first byte of each group of values is a header than determines
-whether it is a run (value between 0 to 127) or literal list (value
-between -128 to -1). For runs, the control byte is the length of the
-run minus the length of the minimal run (3) and the control byte for
-literal lists is the negative length of the list. For example, a
-hundred 0’s is encoded as [0x61, 0x00] and the sequence 0x44, 0x45
-would be encoded as [0xfe, 0x44, 0x45]. The next group can choose
-either of the encodings.</p>
-
-<h2 id="boolean-run-length-encoding">Boolean Run Length Encoding</h2>
-
-<p>For encoding boolean types, the bits are put in the bytes from most
-significant to least significant. The bytes are encoded using byte run
-length encoding as described in the previous section. For example,
-the byte sequence [0xff, 0x80] would be one true followed by
-seven false values.</p>
-
-<h2 id="integer-run-length-encoding-version-1">Integer Run Length Encoding, version 1</h2>
-
-<p>In Hive 0.11 ORC files used Run Length Encoding version 1 (RLEv1),
-which provides a lightweight compression of signed or unsigned integer
-sequences. RLEv1 has two sub-encodings:</p>
-
-<ul>
-  <li>Run - a sequence of values that differ by a small fixed delta</li>
-  <li>Literals - a sequence of varint encoded values</li>
-</ul>
-
-<p>Runs start with an initial byte of 0x00 to 0x7f, which encodes the
-length of the run - 3. A second byte provides the fixed delta in the
-range of -128 to 127. Finally, the first value of the run is encoded
-as a base 128 varint.</p>
-
-<p>For example, if the sequence is 100 instances of 7 the encoding would
-start with 100 - 3, followed by a delta of 0, and a varint of 7 for
-an encoding of [0x61, 0x00, 0x07]. To encode the sequence of numbers
-running from 100 to 1, the first byte is 100 - 3, the delta is -1,
-and the varint is 100 for an encoding of [0x61, 0xff, 0x64].</p>
-
-<p>Literals start with an initial byte of 0x80 to 0xff, which corresponds
-to the negative of number of literals in the sequence. Following the
-header byte, the list of N varints is encoded. Thus, if there are
-no runs, the overhead is 1 byte for each 128 integers. The first 5
-prime numbers [2, 3, 4, 7, 11] would encoded as [0xfb, 0x02, 0x03,
-0x04, 0x07, 0xb].</p>
-
-<h2 id="integer-run-length-encoding-version-2">Integer Run Length Encoding, version 2</h2>
-
-<p>In Hive 0.12, ORC introduced Run Length Encoding version 2 (RLEv2),
-which has improved compression and fixed bit width encodings for
-faster expansion. RLEv2 uses four sub-encodings based on the data:</p>
-
-<ul>
-  <li>Short Repeat - used for short sequences with repeated values</li>
-  <li>Direct - used for random sequences with a fixed bit width</li>
-  <li>Patched Base - used for random sequences with a variable bit width</li>
-  <li>Delta - used for monotonically increasing or decreasing sequences</li>
-</ul>
-
-<h3 id="short-repeat">Short Repeat</h3>
-
-<p>The short repeat encoding is used for short repeating integer
-sequences with the goal of minimizing the overhead of the header. All
-of the bits listed in the header are from the first byte to the last
-and from most significant bit to least significant bit. If the type is
-signed, the value is zigzag encoded.</p>
-
-<ul>
-  <li>1 byte header
-    <ul>
-      <li>2 bits for encoding type (0)</li>
-      <li>3 bits for width (W) of repeating value (1 to 8 bytes)</li>
-      <li>3 bits for repeat count (3 to 10 values)</li>
-    </ul>
-  </li>
-  <li>W bytes in big endian format, which is zigzag encoded if they type
-is signed</li>
-</ul>
-
-<p>The unsigned sequence of [10000, 10000, 10000, 10000, 10000] would be
-serialized with short repeat encoding (0), a width of 2 bytes (1), and
-repeat count of 5 (2) as [0x0a, 0x27, 0x10].</p>
-
-<h3 id="direct">Direct</h3>
-
-<p>The direct encoding is used for integer sequences whose values have a
-relatively constant bit width. It encodes the values directly using a
-fixed width big endian encoding. The width of the values is encoded
-using the table below.</p>
-
-<p>The 5 bit width encoding table for RLEv2:</p>
-
-<table>
-  <thead>
-    <tr>
-      <th style="text-align: left">Width in Bits</th>
-      <th style="text-align: left">Encoded Value</th>
-      <th style="text-align: left">Notes</th>
-    </tr>
-  </thead>
-  <tbody>
-    <tr>
-      <td style="text-align: left">0</td>
-      <td style="text-align: left">0</td>
-      <td style="text-align: left">for delta encoding</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">1</td>
-      <td style="text-align: left">0</td>
-      <td style="text-align: left">for non-delta encoding</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">2</td>
-      <td style="text-align: left">1</td>
-      <td style="text-align: left"> </td>
-    </tr>
-    <tr>
-      <td style="text-align: left">4</td>
-      <td style="text-align: left">3</td>
-      <td style="text-align: left"> </td>
-    </tr>
-    <tr>
-      <td style="text-align: left">8</td>
-      <td style="text-align: left">7</td>
-      <td style="text-align: left"> </td>
-    </tr>
-    <tr>
-      <td style="text-align: left">16</td>
-      <td style="text-align: left">15</td>
-      <td style="text-align: left"> </td>
-    </tr>
-    <tr>
-      <td style="text-align: left">24</td>
-      <td style="text-align: left">23</td>
-      <td style="text-align: left"> </td>
-    </tr>
-    <tr>
-      <td style="text-align: left">32</td>
-      <td style="text-align: left">27</td>
-      <td style="text-align: left"> </td>
-    </tr>
-    <tr>
-      <td style="text-align: left">40</td>
-      <td style="text-align: left">28</td>
-      <td style="text-align: left"> </td>
-    </tr>
-    <tr>
-      <td style="text-align: left">48</td>
-      <td style="text-align: left">29</td>
-      <td style="text-align: left"> </td>
-    </tr>
-    <tr>
-      <td style="text-align: left">56</td>
-      <td style="text-align: left">30</td>
-      <td style="text-align: left"> </td>
-    </tr>
-    <tr>
-      <td style="text-align: left">64</td>
-      <td style="text-align: left">31</td>
-      <td style="text-align: left"> </td>
-    </tr>
-    <tr>
-      <td style="text-align: left">3</td>
-      <td style="text-align: left">2</td>
-      <td style="text-align: left">deprecated</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">5 &lt;= x &lt;= 7</td>
-      <td style="text-align: left">x - 1</td>
-      <td style="text-align: left">deprecated</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">9 &lt;= x &lt;= 15</td>
-      <td style="text-align: left">x - 1</td>
-      <td style="text-align: left">deprecated</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">17 &lt;= x &lt;= 21</td>
-      <td style="text-align: left">x - 1</td>
-      <td style="text-align: left">deprecated</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">26</td>
-      <td style="text-align: left">24</td>
-      <td style="text-align: left">deprecated</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">28</td>
-      <td style="text-align: left">25</td>
-      <td style="text-align: left">deprecated</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">30</td>
-      <td style="text-align: left">26</td>
-      <td style="text-align: left">deprecated</td>
-    </tr>
-  </tbody>
-</table>
-
-<ul>
-  <li>2 bytes header
-    <ul>
-      <li>2 bits for encoding type (1)</li>
-      <li>5 bits for encoded width (W) of values (1 to 64 bits) using the 5 bit
-width encoding table</li>
-      <li>9 bits for length (L) (1 to 512 values)</li>
-    </ul>
-  </li>
-  <li>W * L bits (padded to the next byte) encoded in big endian format, which is
-zigzag encoding if the type is signed</li>
-</ul>
-
-<p>The unsigned sequence of [23713, 43806, 57005, 48879] would be
-serialized with direct encoding (1), a width of 16 bits (15), and
-length of 4 (3) as [0x5e, 0x03, 0x5c, 0xa1, 0xab, 0x1e, 0xde, 0xad,
-0xbe, 0xef].</p>
-
-<h3 id="patched-base">Patched Base</h3>
-
-<p>The patched base encoding is used for integer sequences whose bit
-widths varies a lot. The minimum signed value of the sequence is found
-and subtracted from the other values. The bit width of those adjusted
-values is analyzed and the 90 percentile of the bit width is chosen
-as W. The 10\% of values larger than W use patches from a patch list
-to set the additional bits. Patches are encoded as a list of gaps in
-the index values and the additional value bits.</p>
-
-<ul>
-  <li>4 bytes header
-    <ul>
-      <li>2 bits for encoding type (2)</li>
-      <li>5 bits for encoded width (W) of values (1 to 64 bits) using the 5 bit
-  width encoding table</li>
-      <li>9 bits for length (L) (1 to 512 values)</li>
-      <li>3 bits for base value width (BW) (1 to 8 bytes)</li>
-      <li>5 bits for patch width (PW) (1 to 64 bits) using  the 5 bit width
-encoding table</li>
-      <li>3 bits for patch gap width (PGW) (1 to 8 bits)</li>
-      <li>5 bits for patch list length (PLL) (0 to 31 patches)</li>
-    </ul>
-  </li>
-  <li>Base value (BW bytes) - The base value is stored as a big endian value
-with negative values marked by the most significant bit set. If it that
-bit is set, the entire value is negated.</li>
-  <li>Data values (W * L bits padded to the byte) - A sequence of W bit positive
-values that are added to the base value.</li>
-  <li>Data values (W * L bits padded to the byte) - A sequence of W bit positive
-values that are added to the base value.</li>
-  <li>Patch list (PLL * (PGW + PW) bytes) - A list of patches for values
-that didn’t fit within W bits. Each entry in the list consists of a
-gap, which is the number of elements skipped from the previous
-patch, and a patch value. Patches are applied by logically or’ing
-the data values with the relevant patch shifted W bits left. If a
-patch is 0, it was introduced to skip over more than 255 items. The
-combined length of each patch (PGW + PW) must be less or equal to
-64.</li>
-</ul>
-
-<p>The unsigned sequence of [2030, 2000, 2020, 1000000, 2040, 2050, 2060, 2070,
-2080, 2090, 2100, 2110, 2120, 2130, 2140, 2150, 2160, 2170, 2180, 2190]
-has a minimum of 2000, which makes the adjusted
-sequence [30, 0, 20, 998000, 40, 50, 60, 70, 80, 90, 100, 110, 120, 130, 140,
-150, 160, 170, 180, 190]. It has an
-encoding of patched base (2), a bit width of 8 (7), a length of 20
-(19), a base value width of 2 bytes (1), a patch width of 12 bits (11),
-patch gap width of 2 bits (1), and a patch list length of 1 (1). The
-base value is 2000 and the combined result is [0x8e, 0x13, 0x2b, 0x21, 0x07,
-0xd0, 0x1e, 0x00, 0x14, 0x70, 0x28, 0x32, 0x3c, 0x46, 0x50, 0x5a, 0x64, 0x6e,
-0x78, 0x82, 0x8c, 0x96, 0xa0, 0xaa, 0xb4, 0xbe, 0xfc, 0xe8]</p>
-
-<h3 id="delta">Delta</h3>
-
-<p>The Delta encoding is used for monotonically increasing or decreasing
-sequences. The first two numbers in the sequence can not be identical,
-because the encoding is using the sign of the first delta to determine
-if the series is increasing or decreasing.</p>
-
-<ul>
-  <li>2 bytes header
-    <ul>
-      <li>2 bits for encoding type (3)</li>
-      <li>5 bits for encoded width (W) of deltas (0 to 64 bits) using the 5 bit
-width encoding table</li>
-      <li>9 bits for run length (L) (1 to 512 values)</li>
-    </ul>
-  </li>
-  <li>Base value - encoded as (signed or unsigned) varint</li>
-  <li>Delta base - encoded as signed varint</li>
-  <li>Delta values $W * (L - 2)$ bytes - encode each delta after the first
-one. If the delta base is positive, the sequence is increasing and if it is
-negative the sequence is decreasing.</li>
-</ul>
-
-<p>The unsigned sequence of [2, 3, 5, 7, 11, 13, 17, 19, 23, 29] would be
-serialized with delta encoding (3), a width of 4 bits (3), length of
-10 (9), a base of 2 (2), and first delta of 1 (2). The resulting
-sequence is [0xc6, 0x09, 0x02, 0x02, 0x22, 0x42, 0x42, 0x46].</p>
-
-<h1 id="stripes">Stripes</h1>
-
-<p>The body of ORC files consists of a series of stripes. Stripes are
-large (typically ~200MB) and independent of each other and are often
-processed by different tasks. The defining characteristic for columnar
-storage formats is that the data for each column is stored separately
-and that reading data out of the file should be proportional to the
-number of columns read.</p>
-
-<p>In ORC files, each column is stored in several streams that are stored
-next to each other in the file. For example, an integer column is
-represented as two streams PRESENT, which uses one with a bit per
-value recording if the value is non-null, and DATA, which records the
-non-null values. If all of a column’s values in a stripe are non-null,
-the PRESENT stream is omitted from the stripe. For binary data, ORC
-uses three streams PRESENT, DATA, and LENGTH, which stores the length
-of each value. The details of each type will be presented in the
-following subsections.</p>
-
-<h2 id="stripe-footer">Stripe Footer</h2>
-
-<p>The stripe footer contains the encoding of each column and the
-directory of the streams including their location.</p>
-
-<p><code>message StripeFooter {
- // the location of each stream
- repeated Stream streams = 1;
- // the encoding of each column
- repeated ColumnEncoding columns = 2;
-}
-</code></p>
-
-<p>To describe each stream, ORC stores the kind of stream, the column id,
-and the stream’s size in bytes. The details of what is stored in each stream
-depends on the type and encoding of the column.</p>
-
-<p><code>message Stream {
- enum Kind {
- // boolean stream of whether the next value is non-null
- PRESENT = 0;
- // the primary data stream
- DATA = 1;
- // the length of each value for variable length data
- LENGTH = 2;
- // the dictionary blob
- DICTIONARY\_DATA = 3;
- // deprecated prior to Hive 0.11
- // It was used to store the number of instances of each value in the
- // dictionary
- DICTIONARY_COUNT = 4;
- // a secondary data stream
- SECONDARY = 5;
- // the index for seeking to particular row groups
- ROW_INDEX = 6;
- // original bloom filters used before ORC-101
- BLOOM_FILTER = 7;
- // bloom filters that consistently use utf8
- BLOOM_FILTER_UTF8 = 8;
- }
- required Kind kind = 1;
- // the column id
- optional uint32 column = 2;
- // the number of bytes in the file
- optional uint64 length = 3;
-}
-</code></p>
-
-<p>Depending on their type several options for encoding are possible. The
-encodings are divided into direct or dictionary-based categories and
-further refined as to whether they use RLE v1 or v2.</p>
-
-<p><code>message ColumnEncoding {
- enum Kind {
- // the encoding is mapped directly to the stream using RLE v1
- DIRECT = 0;
- // the encoding uses a dictionary of unique values using RLE v1
- DICTIONARY = 1;
- // the encoding is direct using RLE v2
- DIRECT\_V2 = 2;
- // the encoding is dictionary-based using RLE v2
- DICTIONARY\_V2 = 3;
- }
- required Kind kind = 1;
- // for dictionary encodings, record the size of the dictionary
- optional uint32 dictionarySize = 2;
-}
-</code></p>
-
-<h1 id="column-encodings">Column Encodings</h1>
-
-<h2 id="smallint-int-and-bigint-columns">SmallInt, Int, and BigInt Columns</h2>
-
-<p>All of the 16, 32, and 64 bit integer column types use the same set of
-potential encodings, which is basically whether they use RLE v1 or
-v2. If the PRESENT stream is not included, all of the values are
-present. For values that have false bits in the present stream, no
-values are included in the data stream.</p>
-
-<table>
-  <thead>
-    <tr>
-      <th style="text-align: left">Encoding</th>
-      <th style="text-align: left">Stream Kind</th>
-      <th style="text-align: left">Optional</th>
-      <th style="text-align: left">Contents</th>
-    </tr>
-  </thead>
-  <tbody>
-    <tr>
-      <td style="text-align: left">DIRECT</td>
-      <td style="text-align: left">PRESENT</td>
-      <td style="text-align: left">Yes</td>
-      <td style="text-align: left">Boolean RLE</td>
-    </tr>
-    <tr>
-      <td style="text-align: left"> </td>
-      <td style="text-align: left">DATA</td>
-      <td style="text-align: left">No</td>
-      <td style="text-align: left">Signed Integer RLE v1</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">DIRECT_V2</td>
-      <td style="text-align: left">PRESENT</td>
-      <td style="text-align: left">Yes</td>
-      <td style="text-align: left">Boolean RLE</td>
-    </tr>
-    <tr>
-      <td style="text-align: left"> </td>
-      <td style="text-align: left">DATA</td>
-      <td style="text-align: left">No</td>
-      <td style="text-align: left">Signed Integer RLE v2</td>
-    </tr>
-  </tbody>
-</table>
-
-<h2 id="float-and-double-columns">Float and Double Columns</h2>
-
-<p>Floating point types are stored using IEEE 754 floating point bit
-layout. Float columns use 4 bytes per value and double columns use 8
-bytes.</p>
-
-<table>
-  <thead>
-    <tr>
-      <th style="text-align: left">Encoding</th>
-      <th style="text-align: left">Stream Kind</th>
-      <th style="text-align: left">Optional</th>
-      <th style="text-align: left">Contents</th>
-    </tr>
-  </thead>
-  <tbody>
-    <tr>
-      <td style="text-align: left">DIRECT</td>
-      <td style="text-align: left">PRESENT</td>
-      <td style="text-align: left">Yes</td>
-      <td style="text-align: left">Boolean RLE</td>
-    </tr>
-    <tr>
-      <td style="text-align: left"> </td>
-      <td style="text-align: left">DATA</td>
-      <td style="text-align: left">No</td>
-      <td style="text-align: left">IEEE 754 floating point representation</td>
-    </tr>
-  </tbody>
-</table>
-
-<h2 id="string-char-and-varchar-columns">String, Char, and VarChar Columns</h2>
-
-<p>String, char, and varchar columns may be encoded either using a
-dictionary encoding or a direct encoding. A direct encoding should be
-preferred when there are many distinct values. In all of the
-encodings, the PRESENT stream encodes whether the value is null. The
-Java ORC writer automatically picks the encoding after the first row
-group (10,000 rows).</p>
-
-<p>For direct encoding the UTF-8 bytes are saved in the DATA stream and
-the length of each value is written into the LENGTH stream. In direct
-encoding, if the values were [“Nevada”, “California”]; the DATA
-would be “NevadaCalifornia” and the LENGTH would be [6, 10].</p>
-
-<p>For dictionary encodings the dictionary is sorted and UTF-8 bytes of
-each unique value are placed into DICTIONARY_DATA. The length of each
-item in the dictionary is put into the LENGTH stream. The DATA stream
-consists of the sequence of references to the dictionary elements.</p>
-
-<p>In dictionary encoding, if the values were [“Nevada”,
-“California”, “Nevada”, “California”, and “Florida”]; the
-DICTIONARY_DATA would be “CaliforniaFloridaNevada” and LENGTH would
-be [10, 7, 6]. The DATA would be [2, 0, 2, 0, 1].</p>
-
-<table>
-  <thead>
-    <tr>
-      <th style="text-align: left">Encoding</th>
-      <th style="text-align: left">Stream Kind</th>
-      <th style="text-align: left">Optional</th>
-      <th style="text-align: left">Contents</th>
-    </tr>
-  </thead>
-  <tbody>
-    <tr>
-      <td style="text-align: left">DIRECT</td>
-      <td style="text-align: left">PRESENT</td>
-      <td style="text-align: left">Yes</td>
-      <td style="text-align: left">Boolean RLE</td>
-    </tr>
-    <tr>
-      <td style="text-align: left"> </td>
-      <td style="text-align: left">DATA</td>
-      <td style="text-align: left">No</td>
-      <td style="text-align: left">String contents</td>
-    </tr>
-    <tr>
-      <td style="text-align: left"> </td>
-      <td style="text-align: left">LENGTH</td>
-      <td style="text-align: left">No</td>
-      <td style="text-align: left">Unsigned Integer RLE v1</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">DICTIONARY</td>
-      <td style="text-align: left">PRESENT</td>
-      <td style="text-align: left">Yes</td>
-      <td style="text-align: left">Boolean RLE</td>
-    </tr>
-    <tr>
-      <td style="text-align: left"> </td>
-      <td style="text-align: left">DATA</td>
-      <td style="text-align: left">No</td>
-      <td style="text-align: left">Unsigned Integer RLE v1</td>
-    </tr>
-    <tr>
-      <td style="text-align: left"> </td>
-      <td style="text-align: left">DICTIONARY_DATA</td>
-      <td style="text-align: left">No</td>
-      <td style="text-align: left">String contents</td>
-    </tr>
-    <tr>
-      <td style="text-align: left"> </td>
-      <td style="text-align: left">LENGTH</td>
-      <td style="text-align: left">No</td>
-      <td style="text-align: left">Unsigned Integer RLE v1</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">DIRECT_V2</td>
-      <td style="text-align: left">PRESENT</td>
-      <td style="text-align: left">Yes</td>
-      <td style="text-align: left">Boolean RLE</td>
-    </tr>
-    <tr>
-      <td style="text-align: left"> </td>
-      <td style="text-align: left">DATA</td>
-      <td style="text-align: left">No</td>
-      <td style="text-align: left">String contents</td>
-    </tr>
-    <tr>
-      <td style="text-align: left"> </td>
-      <td style="text-align: left">LENGTH</td>
-      <td style="text-align: left">No</td>
-      <td style="text-align: left">Unsigned Integer RLE v2</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">DICTIONARY_V2</td>
-      <td style="text-align: left">PRESENT</td>
-      <td style="text-align: left">Yes</td>
-      <td style="text-align: left">Boolean RLE</td>
-    </tr>
-    <tr>
-      <td style="text-align: left"> </td>
-      <td style="text-align: left">DATA</td>
-      <td style="text-align: left">No</td>
-      <td style="text-align: left">Unsigned Integer RLE v2</td>
-    </tr>
-    <tr>
-      <td style="text-align: left"> </td>
-      <td style="text-align: left">DICTIONARY_DATA</td>
-      <td style="text-align: left">No</td>
-      <td style="text-align: left">String contents</td>
-    </tr>
-    <tr>
-      <td style="text-align: left"> </td>
-      <td style="text-align: left">LENGTH</td>
-      <td style="text-align: left">No</td>
-      <td style="text-align: left">Unsigned Integer RLE v2</td>
-    </tr>
-  </tbody>
-</table>
-
-<h2 id="boolean-columns">Boolean Columns</h2>
-
-<p>Boolean columns are rare, but have a simple encoding.</p>
-
-<table>
-  <thead>
-    <tr>
-      <th style="text-align: left">Encoding</th>
-      <th style="text-align: left">Stream Kind</th>
-      <th style="text-align: left">Optional</th>
-      <th style="text-align: left">Contents</th>
-    </tr>
-  </thead>
-  <tbody>
-    <tr>
-      <td style="text-align: left">DIRECT</td>
-      <td style="text-align: left">PRESENT</td>
-      <td style="text-align: left">Yes</td>
-      <td style="text-align: left">Boolean RLE</td>
-    </tr>
-    <tr>
-      <td style="text-align: left"> </td>
-      <td style="text-align: left">DATA</td>
-      <td style="text-align: left">No</td>
-      <td style="text-align: left">Boolean RLE</td>
-    </tr>
-  </tbody>
-</table>
-
-<h2 id="tinyint-columns">TinyInt Columns</h2>
-
-<p>TinyInt (byte) columns use byte run length encoding.</p>
-
-<table>
-  <thead>
-    <tr>
-      <th style="text-align: left">Encoding</th>
-      <th style="text-align: left">Stream Kind</th>
-      <th style="text-align: left">Optional</th>
-      <th style="text-align: left">Contents</th>
-    </tr>
-  </thead>
-  <tbody>
-    <tr>
-      <td style="text-align: left">DIRECT</td>
-      <td style="text-align: left">PRESENT</td>
-      <td style="text-align: left">Yes</td>
-      <td style="text-align: left">Boolean RLE</td>
-    </tr>
-    <tr>
-      <td style="text-align: left"> </td>
-      <td style="text-align: left">DATA</td>
-      <td style="text-align: left">No</td>
-      <td style="text-align: left">Byte RLE</td>
-    </tr>
-  </tbody>
-</table>
-
-<h2 id="binary-columns">Binary Columns</h2>
-
-<p>Binary data is encoded with a PRESENT stream, a DATA stream that records
-the contents, and a LENGTH stream that records the number of bytes per a
-value.</p>
-
-<table>
-  <thead>
-    <tr>
-      <th style="text-align: left">Encoding</th>
-      <th style="text-align: left">Stream Kind</th>
-      <th style="text-align: left">Optional</th>
-      <th style="text-align: left">Contents</th>
-    </tr>
-  </thead>
-  <tbody>
-    <tr>
-      <td style="text-align: left">DIRECT</td>
-      <td style="text-align: left">PRESENT</td>
-      <td style="text-align: left">Yes</td>
-      <td style="text-align: left">Boolean RLE</td>
-    </tr>
-    <tr>
-      <td style="text-align: left"> </td>
-      <td style="text-align: left">DATA</td>
-      <td style="text-align: left">No</td>
-      <td style="text-align: left">String contents</td>
-    </tr>
-    <tr>
-      <td style="text-align: left"> </td>
-      <td style="text-align: left">LENGTH</td>
-      <td style="text-align: left">No</td>
-      <td style="text-align: left">Unsigned Integer RLE v1</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">DIRECT_V2</td>
-      <td style="text-align: left">PRESENT</td>
-      <td style="text-align: left">Yes</td>
-      <td style="text-align: left">Boolean RLE</td>
-    </tr>
-    <tr>
-      <td style="text-align: left"> </td>
-      <td style="text-align: left">DATA</td>
-      <td style="text-align: left">No</td>
-      <td style="text-align: left">String contents</td>
-    </tr>
-    <tr>
-      <td style="text-align: left"> </td>
-      <td style="text-align: left">LENGTH</td>
-      <td style="text-align: left">No</td>
-      <td style="text-align: left">Unsigned Integer RLE v2</td>
-    </tr>
-  </tbody>
-</table>
-
-<h2 id="decimal-columns">Decimal Columns</h2>
-
-<p>Since Hive 0.13, all decimals have had fixed precision and scale.
-The goal is to use RLEv3 for the value and use the fixed scale from
-the type. As an interim solution, we are using RLE v2 for short decimals
-(precision &lt;= 18) and the old encoding for long decimals.</p>
-
-<table>
-  <thead>
-    <tr>
-      <th style="text-align: left">Encoding</th>
-      <th style="text-align: left">Stream Kind</th>
-      <th style="text-align: left">Optional</th>
-      <th style="text-align: left">Contents</th>
-    </tr>
-  </thead>
-  <tbody>
-    <tr>
-      <td style="text-align: left">DIRECT</td>
-      <td style="text-align: left">PRESENT</td>
-      <td style="text-align: left">Yes</td>
-      <td style="text-align: left">Boolean RLE</td>
-    </tr>
-    <tr>
-      <td style="text-align: left"> </td>
-      <td style="text-align: left">DATA</td>
-      <td style="text-align: left">No</td>
-      <td style="text-align: left">Signed Integer RLE v2</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">DIRECT_V2</td>
-      <td style="text-align: left">PRESENT</td>
-      <td style="text-align: left">Yes</td>
-      <td style="text-align: left">Boolean RLE</td>
-    </tr>
-    <tr>
-      <td style="text-align: left"> </td>
-      <td style="text-align: left">DATA</td>
-      <td style="text-align: left">No</td>
-      <td style="text-align: left">Unbounded base 128 varints</td>
-    </tr>
-    <tr>
-      <td style="text-align: left"> </td>
-      <td style="text-align: left">SECONDARY</td>
-      <td style="text-align: left">No</td>
-      <td style="text-align: left">Unsigned Integer RLE v2</td>
-    </tr>
-  </tbody>
-</table>
-
-<h2 id="date-columns">Date Columns</h2>
-
-<p>Date data is encoded with a PRESENT stream, a DATA stream that records
-the number of days after January 1, 1970 in UTC.</p>
-
-<table>
-  <thead>
-    <tr>
-      <th style="text-align: left">Encoding</th>
-      <th style="text-align: left">Stream Kind</th>
-      <th style="text-align: left">Optional</th>
-      <th style="text-align: left">Contents</th>
-    </tr>
-  </thead>
-  <tbody>
-    <tr>
-      <td style="text-align: left">DIRECT</td>
-      <td style="text-align: left">PRESENT</td>
-      <td style="text-align: left">Yes</td>
-      <td style="text-align: left">Boolean RLE</td>
-    </tr>
-    <tr>
-      <td style="text-align: left"> </td>
-      <td style="text-align: left">DATA</td>
-      <td style="text-align: left">No</td>
-      <td style="text-align: left">Signed Integer RLE v1</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">DIRECT_V2</td>
-      <td style="text-align: left">PRESENT</td>
-      <td style="text-align: left">Yes</td>
-      <td style="text-align: left">Boolean RLE</td>
-    </tr>
-    <tr>
-      <td style="text-align: left"> </td>
-      <td style="text-align: left">DATA</td>
-      <td style="text-align: left">No</td>
-      <td style="text-align: left">Signed Integer RLE v2</td>
-    </tr>
-  </tbody>
-</table>
-
-<h2 id="timestamp-columns">Timestamp Columns</h2>
-
-<p>Timestamp records times down to nanoseconds as a PRESENT stream that
-records non-null values, a DATA stream that records the number of
-seconds after 1 January 2015, and a SECONDARY stream that records the
-number of nanoseconds.</p>
-
-<p>Because the number of nanoseconds often has a large number of trailing
-zeros, the number has trailing decimal zero digits removed and the
-last three bits are used to record how many zeros were removed. Thus
-1000 nanoseconds would be serialized as 0x0b and 100000 would be
-serialized as 0x0d.</p>
-
-<table>
-  <thead>
-    <tr>
-      <th style="text-align: left">Encoding</th>
-      <th style="text-align: left">Stream Kind</th>
-      <th style="text-align: left">Optional</th>
-      <th style="text-align: left">Contents</th>
-    </tr>
-  </thead>
-  <tbody>
-    <tr>
-      <td style="text-align: left">DIRECT</td>
-      <td style="text-align: left">PRESENT</td>
-      <td style="text-align: left">Yes</td>
-      <td style="text-align: left">Boolean RLE</td>
-    </tr>
-    <tr>
-      <td style="text-align: left"> </td>
-      <td style="text-align: left">DATA</td>
-      <td style="text-align: left">No</td>
-      <td style="text-align: left">Signed Integer RLE v1</td>
-    </tr>
-    <tr>
-      <td style="text-align: left"> </td>
-      <td style="text-align: left">SECONDARY</td>
-      <td style="text-align: left">No</td>
-      <td style="text-align: left">Unsigned Integer RLE v1</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">DIRECT_V2</td>
-      <td style="text-align: left">PRESENT</td>
-      <td style="text-align: left">Yes</td>
-      <td style="text-align: left">Boolean RLE</td>
-    </tr>
-    <tr>
-      <td style="text-align: left"> </td>
-      <td style="text-align: left">DATA</td>
-      <td style="text-align: left">No</td>
-      <td style="text-align: left">Signed Integer RLE v2</td>
-    </tr>
-    <tr>
-      <td style="text-align: left"> </td>
-      <td style="text-align: left">SECONDARY</td>
-      <td style="text-align: left">No</td>
-      <td style="text-align: left">Unsigned Integer RLE v2</td>
-    </tr>
-  </tbody>
-</table>
-
-<h2 id="struct-columns">Struct Columns</h2>
-
-<p>Structs have no data themselves and delegate everything to their child
-columns except for their PRESENT stream. They have a child column
-for each of the fields.</p>
-
-<table>
-  <thead>
-    <tr>
-      <th style="text-align: left">Encoding</th>
-      <th style="text-align: left">Stream Kind</th>
-      <th style="text-align: left">Optional</th>
-      <th style="text-align: left">Contents</th>
-    </tr>
-  </thead>
-  <tbody>
-    <tr>
-      <td style="text-align: left">DIRECT</td>
-      <td style="text-align: left">PRESENT</td>
-      <td style="text-align: left">Yes</td>
-      <td style="text-align: left">Boolean RLE</td>
-    </tr>
-  </tbody>
-</table>
-
-<h2 id="list-columns">List Columns</h2>
-
-<p>Lists are encoded as the PRESENT stream and a length stream with
-number of items in each list. They have a single child column for the
-element values.</p>
-
-<table>
-  <thead>
-    <tr>
-      <th style="text-align: left">Encoding</th>
-      <th style="text-align: left">Stream Kind</th>
-      <th style="text-align: left">Optional</th>
-      <th style="text-align: left">Contents</th>
-    </tr>
-  </thead>
-  <tbody>
-    <tr>
-      <td style="text-align: left">DIRECT</td>
-      <td style="text-align: left">PRESENT</td>
-      <td style="text-align: left">Yes</td>
-      <td style="text-align: left">Boolean RLE</td>
-    </tr>
-    <tr>
-      <td style="text-align: left"> </td>
-      <td style="text-align: left">LENGTH</td>
-      <td style="text-align: left">No</td>
-      <td style="text-align: left">Unsigned Integer RLE v1</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">DIRECT_V2</td>
-      <td style="text-align: left">PRESENT</td>
-      <td style="text-align: left">Yes</td>
-      <td style="text-align: left">Boolean RLE</td>
-    </tr>
-    <tr>
-      <td style="text-align: left"> </td>
-      <td style="text-align: left">LENGTH</td>
-      <td style="text-align: left">No</td>
-      <td style="text-align: left">Unsigned Integer RLE v2</td>
-    </tr>
-  </tbody>
-</table>
-
-<h2 id="map-columns">Map Columns</h2>
-
-<p>Maps are encoded as the PRESENT stream and a length stream with number
-of items in each list. They have a child column for the key and
-another child column for the value.</p>
-
-<table>
-  <thead>
-    <tr>
-      <th style="text-align: left">Encoding</th>
-      <th style="text-align: left">Stream Kind</th>
-      <th style="text-align: left">Optional</th>
-      <th style="text-align: left">Contents</th>
-    </tr>
-  </thead>
-  <tbody>
-    <tr>
-      <td style="text-align: left">DIRECT</td>
-      <td style="text-align: left">PRESENT</td>
-      <td style="text-align: left">Yes</td>
-      <td style="text-align: left">Boolean RLE</td>
-    </tr>
-    <tr>
-      <td style="text-align: left"> </td>
-      <td style="text-align: left">LENGTH</td>
-      <td style="text-align: left">No</td>
-      <td style="text-align: left">Unsigned Integer RLE v1</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">DIRECT_V2</td>
-      <td style="text-align: left">PRESENT</td>
-      <td style="text-align: left">Yes</td>
-      <td style="text-align: left">Boolean RLE</td>
-    </tr>
-    <tr>
-      <td style="text-align: left"> </td>
-      <td style="text-align: left">LENGTH</td>
-      <td style="text-align: left">No</td>
-      <td style="text-align: left">Unsigned Integer RLE v2</td>
-    </tr>
-  </tbody>
-</table>
-
-<h2 id="union-columns">Union Columns</h2>
-
-<p>Unions are encoded as the PRESENT stream and a tag stream that controls which
-potential variant is used. They have a child column for each variant of the
-union. Currently ORC union types are limited to 256 variants, which matches
-the Hive type model.</p>
-
-<table>
-  <thead>
-    <tr>
-      <th style="text-align: left">Encoding</th>
-      <th style="text-align: left">Stream Kind</th>
-      <th style="text-align: left">Optional</th>
-      <th style="text-align: left">Contents</th>
-    </tr>
-  </thead>
-  <tbody>
-    <tr>
-      <td style="text-align: left">DIRECT</td>
-      <td style="text-align: left">PRESENT</td>
-      <td style="text-align: left">Yes</td>
-      <td style="text-align: left">Boolean RLE</td>
-    </tr>
-    <tr>
-      <td style="text-align: left"> </td>
-      <td style="text-align: left">DIRECT</td>
-      <td style="text-align: left">No</td>
-      <td style="text-align: left">Byte RLE</td>
-    </tr>
-  </tbody>
-</table>
-
-<h1 id="indexes">Indexes</h1>
-
-<h2 id="row-group-index">Row Group Index</h2>
-
-<p>The row group indexes consist of a ROW_INDEX stream for each primitive
-column that has an entry for each row group. Row groups are controlled
-by the writer and default to 10,000 rows. Each RowIndexEntry gives the
-position of each stream for the column and the statistics for that row
-group.</p>
-
-<p>The index streams are placed at the front of the stripe, because in
-the default case of streaming they do not need to be read. They are
-only loaded when either predicate push down is being used or the
-reader seeks to a particular row.</p>
-
-<p><code>message RowIndexEntry {
- repeated uint64 positions = 1 [packed=true];
- optional ColumnStatistics statistics = 2;
-}
-</code></p>
-
-<p><code>message RowIndex {
- repeated RowIndexEntry entry = 1;
-}
-</code></p>
-
-<p>To record positions, each stream needs a sequence of numbers. For
-uncompressed streams, the position is the byte offset of the RLE run’s
-start location followed by the number of values that need to be
-consumed from the run. In compressed streams, the first number is the
-start of the compression chunk in the stream, followed by the number
-of decompressed bytes that need to be consumed, and finally the number
-of values consumed in the RLE.</p>
-
-<p>For columns with multiple streams, the sequences of positions in each
-stream are concatenated. That was an unfortunate decision on my part
-that we should fix at some point, because it makes code that uses the
-indexes error-prone.</p>
-
-<p>Because dictionaries are accessed randomly, there is not a position to
-record for the dictionary and the entire dictionary must be read even
-if only part of a stripe is being read.</p>
-
-<h2 id="bloom-filter-index">Bloom Filter Index</h2>
-
-<p>Bloom Filters are added to ORC indexes from Hive 1.2.0 onwards.
-Predicate pushdown can make use of bloom filters to better prune
-the row groups that do not satisfy the filter condition.
-The bloom filter indexes consist of a BLOOM_FILTER stream for each
-column specified through ‘orc.bloom.filter.columns’ table properties.
-A BLOOM_FILTER stream records a bloom filter entry for each row
-group (default to 10,000 rows) in a column. Only the row groups that
-satisfy min/max row index evaluation will be evaluated against the
-bloom filter index.</p>
-
-<p>Each BloomFilterEntry stores the number of hash functions (‘k’) used
-and the bitset backing the bloom filter. The original encoding (pre
-ORC-101) of bloom filters used the bitset field encoded as a repeating
-sequence of longs in the bitset field with a little endian encoding
-(0x1 is bit 0 and 0x2 is bit 1.) After ORC-101, the encoding is a
-sequence of bytes with a little endian encoding in the utf8bitset field.</p>
-
-<p><code>message BloomFilter {
- optional uint32 numHashFunctions = 1;
- repeated fixed64 bitset = 2;
- optional bytes utf8bitset = 3;
-}
-</code></p>
-
-<p><code>message BloomFilterIndex {
- repeated BloomFilter bloomFilter = 1;
-}
-</code></p>
-
-<p>Bloom filter internally uses two different hash functions to map a key
-to a position in the bit set. For tinyint, smallint, int, bigint, float
-and double types, Thomas Wang’s 64-bit integer hash function is used.
-Floats are converted to IEEE-754 32 bit representation
-(using Java’s Float.floatToIntBits(float)). Similary, Doubles are
-converted to IEEE-754 64 bit representation (using Java’s
-Double.doubleToLongBits(double)). All these primitive types
-are cast to long base type before being passed on to the hash function.
-For strings and binary types, Murmur3 64 bit hash algorithm is used.
-The 64 bit variant of Murmur3 considers only the most significant
-8 bytes of Murmur3 128-bit algorithm. The 64 bit hashcode generated
-from the above algorithms is used as a base to derive ‘k’ different
-hash functions. We use the idea mentioned in the paper “Less Hashing,
-Same Performance: Building a Better Bloom Filter” by Kirsch et. al. to
-quickly compute the k hashcodes.</p>
-
-<p>The algorithm for computing k hashcodes and setting the bit position
-in a bloom filter is as follows:</p>
-
-<ol>
-  <li>Get 64 bit base hash code from Murmur3 or Thomas Wang’s hash algorithm.</li>
-  <li>Split the above hashcode into two 32-bit hashcodes (say hash1 and hash2).</li>
-  <li>k’th hashcode is obtained by (where k &gt; 0):
-    <ul>
-      <li>combinedHash = hash1 + (k * hash2)</li>
-    </ul>
-  </li>
-  <li>If combinedHash is negative flip all the bits:
-    <ul>
-      <li>combinedHash = ~combinedHash</li>
-    </ul>
-  </li>
-  <li>Bit set position is obtained by performing modulo with m:
-    <ul>
-      <li>position = combinedHash % m</li>
-    </ul>
-  </li>
-  <li>Set the position in bit set. The LSB 6 bits identifies the long index
-within bitset and bit position within the long uses little endian order.
-    <ul>
-      <li>bitset[position »&gt; 6] |= (1L « position);</li>
-    </ul>
-  </li>
-</ol>
-
-<p>Bloom filter streams are interlaced with row group indexes. This placement
-makes it convenient to read the bloom filter stream and row index stream
-together in single read operation.</p>
-
-<p><img src="/img/BloomFilter.png" alt="bloom filter" /></p>
-
-      </article>
-    </div>
-
-    <div class="clear"></div>
-
-  </div>
-</section>
-
-
-  <footer role="contentinfo">
-  <p>The contents of this website are &copy;&nbsp;2018
-     <a href="https://www.apache.org/">Apache Software Foundation</a>
-     under the terms of the <a
-      href="https://www.apache.org/licenses/LICENSE-2.0.html">
-      Apache&nbsp;License&nbsp;v2</a>. Apache ORC and its logo are trademarks
-      of the Apache Software Foundation.</p>
-</footer>
-
-  <script>
-  var anchorForId = function (id) {
-    var anchor = document.createElement("a");
-    anchor.className = "header-link";
-    anchor.href      = "#" + id;
-    anchor.innerHTML = "<span class=\"sr-only\">Permalink</span><i class=\"fa fa-link\"></i>";
-    anchor.title = "Permalink";
-    return anchor;
-  };
-
-  var linkifyAnchors = function (level, containingElement) {
-    var headers = containingElement.getElementsByTagName("h" + level);
-    for (var h = 0; h < headers.length; h++) {
-      var header = headers[h];
-
-      if (typeof header.id !== "undefined" && header.id !== "") {
-        header.appendChild(anchorForId(header.id));
-      }
-    }
-  };
-
-  document.onreadystatechange = function () {
-    if (this.readyState === "complete") {
-      var contentBlock = document.getElementsByClassName("docs")[0] || document.getElementsByClassName("news")[0];
-      if (!contentBlock) {
-        return;
-      }
-      for (var level = 1; level <= 6; level++) {
-        linkifyAnchors(level, contentBlock);
-      }
-    }
-  };
-</script>
-
-
-</body>
-</html>


[06/11] orc git commit: Deploy docs with CVE.

Posted by om...@apache.org.
http://git-wip-us.apache.org/repos/asf/orc/blob/e58d3fc4/news/index.html
----------------------------------------------------------------------
diff --git a/news/index.html b/news/index.html
index b1a2746..4c10d7a 100644
--- a/news/index.html
+++ b/news/index.html
@@ -4,7 +4,7 @@
   <meta charset="UTF-8">
   <title>News</title>
   <meta name="viewport" content="width=device-width,initial-scale=1">
-  <meta name="generator" content="Jekyll v2.4.0">
+  <meta name="generator" content="Jekyll v3.7.3">
   <link rel="stylesheet" href="//fonts.googleapis.com/css?family=Lato:300,300italic,400,400italic,700,700italic,900">
   <link rel="stylesheet" href="/css/screen.css">
   <link rel="icon" type="image/x-icon" href="/favicon.ico">
@@ -307,6 +307,10 @@ signed by <a href="https://dist.apache.org/repos/dist/release/orc/KEYS">Owen O
 
 <p>Known issues:</p>
 
+<ul>
+  <li><a href="http://localhost:4000/security/CVE-2018-8015">CVE-2018-8015</a> ORC files with malformed types cause stack overflow.</li>
+</ul>
+
 
   </div>
 </article>
@@ -361,7 +365,12 @@ signed by <a href="https://dist.apache.org/repos/dist/release/orc/KEYS">Owen O
 <p>Known issues:</p>
 
 <ul>
-  <li><a href="https://issues.apache.org/jira/browse/ORC-40">ORC-40</a> Predicate push down is not implemented in C++.</li>
+  <li>
+    <p><a href="http://localhost:4000/security/CVE-2018-8015">CVE-2018-8015</a> ORC files with malformed types cause stack overflow.</p>
+  </li>
+  <li>
+    <p><a href="http://localhost:4000/security/ORC-40">ORC-40</a> Predicate push down is not implemented in C++.</p>
+  </li>
 </ul>
 
 
@@ -418,7 +427,12 @@ signed by <a href="https://dist.apache.org/repos/dist/release/orc/KEYS">Prasanth
 <p>Known issues:</p>
 
 <ul>
-  <li><a href="https://issues.apache.org/jira/browse/ORC-40">ORC-40</a> Predicate push down is not implemented in C++.</li>
+  <li>
+    <p><a href="http://localhost:4000/security/CVE-2018-8015">CVE-2018-8015</a> ORC files with malformed types cause stack overflow.</p>
+  </li>
+  <li>
+    <p><a href="http://localhost:4000/security/ORC-40">ORC-40</a> Predicate push down is not implemented in C++.</p>
+  </li>
 </ul>
 
 
@@ -474,7 +488,7 @@ signed by <a href="https://dist.apache.org/repos/dist/release/orc/KEYS">Prasanth
 <p>Known issues:</p>
 
 <ul>
-  <li><a href="https://issues.apache.org/jira/browse/ORC-40">ORC-40</a> Predicate push down is not implemented in C++.</li>
+  <li><a href="http://localhost:4000/security/CVE-2018-8015">CVE-2018-8015</a> ORC files with malformed types cause stack overflow.</li>
 </ul>
 
 
@@ -597,7 +611,7 @@ signed by <a href="https://dist.apache.org/repos/dist/release/orc/KEYS">Owen O
 <p>Known issues:</p>
 
 <ul>
-  <li><a href="https://issues.apache.org/jira/browse/ORC-40">ORC-40</a> Predicate push down is not implemented in C++.</li>
+  <li><a href="http://localhost:4000/security/CVE-2018-8015">CVE-2018-8015</a> ORC files with malformed types cause stack overflow.</li>
 </ul>
 
 
@@ -653,7 +667,7 @@ signed by <a href="https://dist.apache.org/repos/dist/release/orc/KEYS">Owen O
 <p>Known issues:</p>
 
 <ul>
-  <li><a href="https://issues.apache.org/jira/browse/ORC-40">ORC-40</a> Predicate push down is not implemented in C++.</li>
+  <li><a href="http://localhost:4000/security/CVE-2018-8015">CVE-2018-8015</a> ORC files with malformed types cause stack overflow.</li>
 </ul>
 
 
@@ -709,7 +723,7 @@ signed by <a href="https://dist.apache.org/repos/dist/release/orc/KEYS">Owen O
 <p>Known issues:</p>
 
 <ul>
-  <li><a href="https://issues.apache.org/jira/browse/ORC-40">ORC-40</a> Predicate push down is not implemented in C++.</li>
+  <li><a href="http://localhost:4000/security/CVE-2018-8015">CVE-2018-8015</a> ORC files with malformed types cause stack overflow.</li>
 </ul>
 
 
@@ -766,10 +780,10 @@ signed by <a href="https://dist.apache.org/repos/dist/release/orc/KEYS">Owen O
 
 <ul>
   <li>
-    <p><a href="https://issues.apache.org/jira/browse/ORC-40">ORC-40</a> Predicate push down is not implemented in C++.</p>
+    <p><a href="http://localhost:4000/security/CVE-2018-8015">CVE-2018-8015</a> ORC files with malformed types cause stack overflow.</p>
   </li>
   <li>
-    <p><a href="https://issues.apache.org/jira/browse/ORC-135">ORC-135</a> Predicate push down is incorrect on timestamps when moved between timezones</p>
+    <p><a href="http://localhost:4000/security/ORC-135">ORC-135</a> Predicate push down is incorrect on timestamps when moved between timezones</p>
   </li>
 </ul>
 
@@ -826,12 +840,7 @@ signed by <a href="https://dist.apache.org/repos/dist/release/orc/KEYS">Owen O
 <p>Known issues:</p>
 
 <ul>
-  <li>
-    <p><a href="https://issues.apache.org/jira/browse/ORC-40">ORC-40</a> Predicate push down is not implemented in C++.</p>
-  </li>
-  <li>
-    <p><a href="https://issues.apache.org/jira/browse/ORC-135">ORC-135</a> Predicate push down is incorrect on timestamps when moved between timezones</p>
-  </li>
+  <li><a href="http://localhost:4000/security/ORC-135">ORC-135</a> Predicate push down is incorrect on timestamps when moved between timezones</li>
 </ul>
 
 
@@ -988,10 +997,10 @@ signed by <a href="https://dist.apache.org/repos/dist/release/orc/KEYS">Owen O
 
 <ul>
   <li>
-    <p><a href="https://issues.apache.org/jira/browse/ORC-40">ORC-40</a> Predicate push down is not implemented in C++.</p>
+    <p><a href="http://localhost:4000/security/CVE-2018-8015">CVE-2018-8015</a> ORC files with malformed types cause stack overflow.</p>
   </li>
   <li>
-    <p><a href="https://issues.apache.org/jira/browse/ORC-135">ORC-135</a> Predicate push down is incorrect on timestamps when moved between timezones</p>
+    <p><a href="http://localhost:4000/security/ORC-135">ORC-135</a> Predicate push down is incorrect on timestamps when moved between timezones</p>
   </li>
 </ul>
 
@@ -1056,10 +1065,10 @@ signed by <a href="https://dist.apache.org/repos/dist/release/orc/KEYS">Owen O
 
 <ul>
   <li>
-    <p><a href="https://issues.apache.org/jira/browse/ORC-40">ORC-40</a> Predicate push down is not implemented in C++.</p>
+    <p><a href="http://localhost:4000/security/CVE-2018-8015">CVE-2018-8015</a> ORC files with malformed types cause stack overflow.</p>
   </li>
   <li>
-    <p><a href="https://issues.apache.org/jira/browse/ORC-135">ORC-135</a> Predicate push down is incorrect on timestamps when moved between timezones</p>
+    <p><a href="http://localhost:4000/security/ORC-135">ORC-135</a> Predicate push down is incorrect on timestamps when moved between timezones</p>
   </li>
 </ul>
 
@@ -1124,10 +1133,10 @@ signed by <a href="https://dist.apache.org/repos/dist/release/orc/KEYS">Owen O
 
 <ul>
   <li>
-    <p><a href="https://issues.apache.org/jira/browse/ORC-40">ORC-40</a> Predicate push down is not implemented in C++.</p>
+    <p><a href="http://localhost:4000/security/CVE-2018-8015">CVE-2018-8015</a> ORC files with malformed types cause stack overflow.</p>
   </li>
   <li>
-    <p><a href="https://issues.apache.org/jira/browse/ORC-135">ORC-135</a> Predicate push down is incorrect on timestamps when moved between timezones</p>
+    <p><a href="http://localhost:4000/security/ORC-135">ORC-135</a> Predicate push down is incorrect on timestamps when moved between timezones</p>
   </li>
 </ul>
 
@@ -1192,13 +1201,13 @@ signed by <a href="https://dist.apache.org/repos/dist/release/orc/KEYS">Owen O
 
 <ul>
   <li>
-    <p><a href="https://issues.apache.org/jira/browse/ORC-40">ORC-40</a> Predicate push down is not implemented in C++.</p>
+    <p><a href="http://localhost:4000/security/CVE-2018-8015">CVE-2018-8015</a> ORC files with malformed types cause stack overflow.</p>
   </li>
   <li>
-    <p><a href="https://issues.apache.org/jira/browse/ORC-101">ORC-101</a> Bloom filters for string and decimal use inconsistent encoding</p>
+    <p><a href="http://localhost:4000/security/ORC-101">ORC-101</a> Bloom filters for string and decimal use inconsistent encoding</p>
   </li>
   <li>
-    <p><a href="https://issues.apache.org/jira/browse/ORC-135">ORC-135</a> Predicate push down is incorrect on timestamps when moved between timezones</p>
+    <p><a href="http://localhost:4000/security/ORC-135">ORC-135</a> Predicate push down is incorrect on timestamps when moved between timezones</p>
   </li>
 </ul>
 
@@ -1257,16 +1266,16 @@ timestamps from timezones with different daylight savings rules.</li>
 
 <ul>
   <li>
-    <p><a href="https://issues.apache.org/jira/browse/HIVE-14214">HIVE-14214</a> Schema evolution and predicate pushdown don’t work together.</p>
+    <p><a href="http://localhost:4000/security/CVE-2018-8015">CVE-2018-8015</a> ORC files with malformed types cause stack overflow.</p>
   </li>
   <li>
-    <p><a href="https://issues.apache.org/jira/browse/ORC-40">ORC-40</a> Predicate push down is not implemented in C++.</p>
+    <p><a href="http://localhost:4000/security/HIVE-14214">HIVE-14214</a> Schema evolution and predicate pushdown don’t work together.</p>
   </li>
   <li>
-    <p><a href="https://issues.apache.org/jira/browse/ORC-101">ORC-101</a> Bloom filters for string and decimal use inconsistent encoding</p>
+    <p><a href="http://localhost:4000/security/ORC-101">ORC-101</a> Bloom filters for string and decimal use inconsistent encoding</p>
   </li>
   <li>
-    <p><a href="https://issues.apache.org/jira/browse/ORC-135">ORC-135</a> Predicate push down is incorrect on timestamps when moved between timezones</p>
+    <p><a href="http://localhost:4000/security/ORC-135">ORC-135</a> Predicate push down is incorrect on timestamps when moved between timezones</p>
   </li>
 </ul>
 
@@ -1378,16 +1387,16 @@ timestamps from timezones with different daylight savings rules.</li>
 
 <ul>
   <li>
-    <p><a href="https://issues.apache.org/jira/browse/HIVE-14214">HIVE-14214</a> Schema evolution and predicate pushdown don’t work together.</p>
+    <p><a href="http://localhost:4000/security/CVE-2018-8015">CVE-2018-8015</a> ORC files with malformed types cause stack overflow.</p>
   </li>
   <li>
-    <p><a href="https://issues.apache.org/jira/browse/ORC-40">ORC-40</a> Predicate push down is not implemented in C++.</p>
+    <p><a href="http://localhost:4000/security/HIVE-14214">HIVE-14214</a> Schema evolution and predicate pushdown don’t work together.</p>
   </li>
   <li>
-    <p><a href="https://issues.apache.org/jira/browse/ORC-101">ORC-101</a> Bloom filters for string and decimal use inconsistent encoding</p>
+    <p><a href="http://localhost:4000/security/ORC-101">ORC-101</a> Bloom filters for string and decimal use inconsistent encoding</p>
   </li>
   <li>
-    <p><a href="https://issues.apache.org/jira/browse/ORC-135">ORC-135</a> Predicate push down is incorrect on timestamps when moved between timezones</p>
+    <p><a href="http://localhost:4000/security/ORC-135">ORC-135</a> Predicate push down is incorrect on timestamps when moved between timezones</p>
   </li>
 </ul>
 
@@ -1448,16 +1457,16 @@ timestamps from timezones with different daylight savings rules.</li>
 
 <ul>
   <li>
-    <p><a href="https://issues.apache.org/jira/browse/HIVE-14214">HIVE-14214</a> Schema evolution and predicate pushdown don’t work together.</p>
+    <p><a href="http://localhost:4000/security/CVE-2018-8015">CVE-2018-8015</a> ORC files with malformed types cause stack overflow.</p>
   </li>
   <li>
-    <p><a href="https://issues.apache.org/jira/browse/ORC-40">ORC-40</a> Predicate push down is not implemented in C++.</p>
+    <p><a href="http://localhost:4000/security/HIVE-14214">HIVE-14214</a> Schema evolution and predicate pushdown don’t work together.</p>
   </li>
   <li>
-    <p><a href="https://issues.apache.org/jira/browse/ORC-101">ORC-101</a> Bloom filters for string and decimal use inconsistent encoding</p>
+    <p><a href="http://localhost:4000/security/ORC-101">ORC-101</a> Bloom filters for string and decimal use inconsistent encoding</p>
   </li>
   <li>
-    <p><a href="https://issues.apache.org/jira/browse/ORC-135">ORC-135</a> Predicate push down is incorrect on timestamps when moved between timezones</p>
+    <p><a href="http://localhost:4000/security/ORC-135">ORC-135</a> Predicate push down is incorrect on timestamps when moved between timezones</p>
   </li>
 </ul>
 
@@ -1525,13 +1534,10 @@ signed by <a href="https://dist.apache.org/repos/dist/release/orc/KEYS">Owen O
 
 <ul>
   <li>
-    <p><a href="https://issues.apache.org/jira/browse/ORC-1">ORC-1</a> We are still working on moving the Java reader and writer out of Hive’s code base and thus it is not included here.</p>
-  </li>
-  <li>
-    <p><a href="https://issues.apache.org/jira/browse/ORC-10">ORC-10</a> When moving ORC files between timezones, different daylight savings rules will cause timestamps to shift in the C++ reader.</p>
+    <p><a href="http://localhost:4000/security/CVE-2018-8015">CVE-2018-8015</a> ORC files with malformed types cause stack overflow.</p>
   </li>
   <li>
-    <p><a href="https://issues.apache.org/jira/browse/ORC-40">ORC-40</a> Predicate push down is not implemented in C++.</p>
+    <p><a href="http://localhost:4000/security/ORC-10">ORC-10</a> When moving ORC files between timezones, different daylight savings rules will cause timestamps to shift in the C++ reader.</p>
   </li>
 </ul>
 
@@ -1601,7 +1607,7 @@ more.</p>
   <div class="post-content">
     <p>The ORC project has adopted a new logo. We hope you like it.</p>
 
-<p><img src="/img/logo.png" alt="orc logo" title="orc logo" /></p>
+<p><img src="http://localhost:4000/img/logo.png" alt="orc logo" title="orc logo" /></p>
 
 <p>Other great options included a big white hand on a black shield. <em>smile</em></p>
 

http://git-wip-us.apache.org/repos/asf/orc/blob/e58d3fc4/news/releases/index.html
----------------------------------------------------------------------
diff --git a/news/releases/index.html b/news/releases/index.html
index 3dc0454..b8c4b5b 100644
--- a/news/releases/index.html
+++ b/news/releases/index.html
@@ -4,7 +4,7 @@
   <meta charset="UTF-8">
   <title>Releases</title>
   <meta name="viewport" content="width=device-width,initial-scale=1">
-  <meta name="generator" content="Jekyll v2.4.0">
+  <meta name="generator" content="Jekyll v3.7.3">
   <link rel="stylesheet" href="//fonts.googleapis.com/css?family=Lato:300,300italic,400,400italic,700,700italic,900">
   <link rel="stylesheet" href="/css/screen.css">
   <link rel="icon" type="image/x-icon" href="/favicon.ico">
@@ -307,6 +307,10 @@ signed by <a href="https://dist.apache.org/repos/dist/release/orc/KEYS">Owen O
 
 <p>Known issues:</p>
 
+<ul>
+  <li><a href="http://localhost:4000/security/CVE-2018-8015">CVE-2018-8015</a> ORC files with malformed types cause stack overflow.</li>
+</ul>
+
 
   </div>
 </article>
@@ -361,7 +365,12 @@ signed by <a href="https://dist.apache.org/repos/dist/release/orc/KEYS">Owen O
 <p>Known issues:</p>
 
 <ul>
-  <li><a href="https://issues.apache.org/jira/browse/ORC-40">ORC-40</a> Predicate push down is not implemented in C++.</li>
+  <li>
+    <p><a href="http://localhost:4000/security/CVE-2018-8015">CVE-2018-8015</a> ORC files with malformed types cause stack overflow.</p>
+  </li>
+  <li>
+    <p><a href="http://localhost:4000/security/ORC-40">ORC-40</a> Predicate push down is not implemented in C++.</p>
+  </li>
 </ul>
 
 
@@ -418,7 +427,12 @@ signed by <a href="https://dist.apache.org/repos/dist/release/orc/KEYS">Prasanth
 <p>Known issues:</p>
 
 <ul>
-  <li><a href="https://issues.apache.org/jira/browse/ORC-40">ORC-40</a> Predicate push down is not implemented in C++.</li>
+  <li>
+    <p><a href="http://localhost:4000/security/CVE-2018-8015">CVE-2018-8015</a> ORC files with malformed types cause stack overflow.</p>
+  </li>
+  <li>
+    <p><a href="http://localhost:4000/security/ORC-40">ORC-40</a> Predicate push down is not implemented in C++.</p>
+  </li>
 </ul>
 
 
@@ -474,7 +488,7 @@ signed by <a href="https://dist.apache.org/repos/dist/release/orc/KEYS">Prasanth
 <p>Known issues:</p>
 
 <ul>
-  <li><a href="https://issues.apache.org/jira/browse/ORC-40">ORC-40</a> Predicate push down is not implemented in C++.</li>
+  <li><a href="http://localhost:4000/security/CVE-2018-8015">CVE-2018-8015</a> ORC files with malformed types cause stack overflow.</li>
 </ul>
 
 
@@ -531,7 +545,7 @@ signed by <a href="https://dist.apache.org/repos/dist/release/orc/KEYS">Owen O
 <p>Known issues:</p>
 
 <ul>
-  <li><a href="https://issues.apache.org/jira/browse/ORC-40">ORC-40</a> Predicate push down is not implemented in C++.</li>
+  <li><a href="http://localhost:4000/security/CVE-2018-8015">CVE-2018-8015</a> ORC files with malformed types cause stack overflow.</li>
 </ul>
 
 
@@ -587,7 +601,7 @@ signed by <a href="https://dist.apache.org/repos/dist/release/orc/KEYS">Owen O
 <p>Known issues:</p>
 
 <ul>
-  <li><a href="https://issues.apache.org/jira/browse/ORC-40">ORC-40</a> Predicate push down is not implemented in C++.</li>
+  <li><a href="http://localhost:4000/security/CVE-2018-8015">CVE-2018-8015</a> ORC files with malformed types cause stack overflow.</li>
 </ul>
 
 
@@ -643,7 +657,7 @@ signed by <a href="https://dist.apache.org/repos/dist/release/orc/KEYS">Owen O
 <p>Known issues:</p>
 
 <ul>
-  <li><a href="https://issues.apache.org/jira/browse/ORC-40">ORC-40</a> Predicate push down is not implemented in C++.</li>
+  <li><a href="http://localhost:4000/security/CVE-2018-8015">CVE-2018-8015</a> ORC files with malformed types cause stack overflow.</li>
 </ul>
 
 
@@ -700,10 +714,10 @@ signed by <a href="https://dist.apache.org/repos/dist/release/orc/KEYS">Owen O
 
 <ul>
   <li>
-    <p><a href="https://issues.apache.org/jira/browse/ORC-40">ORC-40</a> Predicate push down is not implemented in C++.</p>
+    <p><a href="http://localhost:4000/security/CVE-2018-8015">CVE-2018-8015</a> ORC files with malformed types cause stack overflow.</p>
   </li>
   <li>
-    <p><a href="https://issues.apache.org/jira/browse/ORC-135">ORC-135</a> Predicate push down is incorrect on timestamps when moved between timezones</p>
+    <p><a href="http://localhost:4000/security/ORC-135">ORC-135</a> Predicate push down is incorrect on timestamps when moved between timezones</p>
   </li>
 </ul>
 
@@ -760,12 +774,7 @@ signed by <a href="https://dist.apache.org/repos/dist/release/orc/KEYS">Owen O
 <p>Known issues:</p>
 
 <ul>
-  <li>
-    <p><a href="https://issues.apache.org/jira/browse/ORC-40">ORC-40</a> Predicate push down is not implemented in C++.</p>
-  </li>
-  <li>
-    <p><a href="https://issues.apache.org/jira/browse/ORC-135">ORC-135</a> Predicate push down is incorrect on timestamps when moved between timezones</p>
-  </li>
+  <li><a href="http://localhost:4000/security/ORC-135">ORC-135</a> Predicate push down is incorrect on timestamps when moved between timezones</li>
 </ul>
 
 
@@ -830,10 +839,10 @@ signed by <a href="https://dist.apache.org/repos/dist/release/orc/KEYS">Owen O
 
 <ul>
   <li>
-    <p><a href="https://issues.apache.org/jira/browse/ORC-40">ORC-40</a> Predicate push down is not implemented in C++.</p>
+    <p><a href="http://localhost:4000/security/CVE-2018-8015">CVE-2018-8015</a> ORC files with malformed types cause stack overflow.</p>
   </li>
   <li>
-    <p><a href="https://issues.apache.org/jira/browse/ORC-135">ORC-135</a> Predicate push down is incorrect on timestamps when moved between timezones</p>
+    <p><a href="http://localhost:4000/security/ORC-135">ORC-135</a> Predicate push down is incorrect on timestamps when moved between timezones</p>
   </li>
 </ul>
 
@@ -898,10 +907,10 @@ signed by <a href="https://dist.apache.org/repos/dist/release/orc/KEYS">Owen O
 
 <ul>
   <li>
-    <p><a href="https://issues.apache.org/jira/browse/ORC-40">ORC-40</a> Predicate push down is not implemented in C++.</p>
+    <p><a href="http://localhost:4000/security/CVE-2018-8015">CVE-2018-8015</a> ORC files with malformed types cause stack overflow.</p>
   </li>
   <li>
-    <p><a href="https://issues.apache.org/jira/browse/ORC-135">ORC-135</a> Predicate push down is incorrect on timestamps when moved between timezones</p>
+    <p><a href="http://localhost:4000/security/ORC-135">ORC-135</a> Predicate push down is incorrect on timestamps when moved between timezones</p>
   </li>
 </ul>
 
@@ -966,10 +975,10 @@ signed by <a href="https://dist.apache.org/repos/dist/release/orc/KEYS">Owen O
 
 <ul>
   <li>
-    <p><a href="https://issues.apache.org/jira/browse/ORC-40">ORC-40</a> Predicate push down is not implemented in C++.</p>
+    <p><a href="http://localhost:4000/security/CVE-2018-8015">CVE-2018-8015</a> ORC files with malformed types cause stack overflow.</p>
   </li>
   <li>
-    <p><a href="https://issues.apache.org/jira/browse/ORC-135">ORC-135</a> Predicate push down is incorrect on timestamps when moved between timezones</p>
+    <p><a href="http://localhost:4000/security/ORC-135">ORC-135</a> Predicate push down is incorrect on timestamps when moved between timezones</p>
   </li>
 </ul>
 
@@ -1034,13 +1043,13 @@ signed by <a href="https://dist.apache.org/repos/dist/release/orc/KEYS">Owen O
 
 <ul>
   <li>
-    <p><a href="https://issues.apache.org/jira/browse/ORC-40">ORC-40</a> Predicate push down is not implemented in C++.</p>
+    <p><a href="http://localhost:4000/security/CVE-2018-8015">CVE-2018-8015</a> ORC files with malformed types cause stack overflow.</p>
   </li>
   <li>
-    <p><a href="https://issues.apache.org/jira/browse/ORC-101">ORC-101</a> Bloom filters for string and decimal use inconsistent encoding</p>
+    <p><a href="http://localhost:4000/security/ORC-101">ORC-101</a> Bloom filters for string and decimal use inconsistent encoding</p>
   </li>
   <li>
-    <p><a href="https://issues.apache.org/jira/browse/ORC-135">ORC-135</a> Predicate push down is incorrect on timestamps when moved between timezones</p>
+    <p><a href="http://localhost:4000/security/ORC-135">ORC-135</a> Predicate push down is incorrect on timestamps when moved between timezones</p>
   </li>
 </ul>
 
@@ -1099,16 +1108,16 @@ timestamps from timezones with different daylight savings rules.</li>
 
 <ul>
   <li>
-    <p><a href="https://issues.apache.org/jira/browse/HIVE-14214">HIVE-14214</a> Schema evolution and predicate pushdown don’t work together.</p>
+    <p><a href="http://localhost:4000/security/CVE-2018-8015">CVE-2018-8015</a> ORC files with malformed types cause stack overflow.</p>
   </li>
   <li>
-    <p><a href="https://issues.apache.org/jira/browse/ORC-40">ORC-40</a> Predicate push down is not implemented in C++.</p>
+    <p><a href="http://localhost:4000/security/HIVE-14214">HIVE-14214</a> Schema evolution and predicate pushdown don’t work together.</p>
   </li>
   <li>
-    <p><a href="https://issues.apache.org/jira/browse/ORC-101">ORC-101</a> Bloom filters for string and decimal use inconsistent encoding</p>
+    <p><a href="http://localhost:4000/security/ORC-101">ORC-101</a> Bloom filters for string and decimal use inconsistent encoding</p>
   </li>
   <li>
-    <p><a href="https://issues.apache.org/jira/browse/ORC-135">ORC-135</a> Predicate push down is incorrect on timestamps when moved between timezones</p>
+    <p><a href="http://localhost:4000/security/ORC-135">ORC-135</a> Predicate push down is incorrect on timestamps when moved between timezones</p>
   </li>
 </ul>
 
@@ -1167,16 +1176,16 @@ timestamps from timezones with different daylight savings rules.</li>
 
 <ul>
   <li>
-    <p><a href="https://issues.apache.org/jira/browse/HIVE-14214">HIVE-14214</a> Schema evolution and predicate pushdown don’t work together.</p>
+    <p><a href="http://localhost:4000/security/CVE-2018-8015">CVE-2018-8015</a> ORC files with malformed types cause stack overflow.</p>
   </li>
   <li>
-    <p><a href="https://issues.apache.org/jira/browse/ORC-40">ORC-40</a> Predicate push down is not implemented in C++.</p>
+    <p><a href="http://localhost:4000/security/HIVE-14214">HIVE-14214</a> Schema evolution and predicate pushdown don’t work together.</p>
   </li>
   <li>
-    <p><a href="https://issues.apache.org/jira/browse/ORC-101">ORC-101</a> Bloom filters for string and decimal use inconsistent encoding</p>
+    <p><a href="http://localhost:4000/security/ORC-101">ORC-101</a> Bloom filters for string and decimal use inconsistent encoding</p>
   </li>
   <li>
-    <p><a href="https://issues.apache.org/jira/browse/ORC-135">ORC-135</a> Predicate push down is incorrect on timestamps when moved between timezones</p>
+    <p><a href="http://localhost:4000/security/ORC-135">ORC-135</a> Predicate push down is incorrect on timestamps when moved between timezones</p>
   </li>
 </ul>
 
@@ -1237,16 +1246,16 @@ timestamps from timezones with different daylight savings rules.</li>
 
 <ul>
   <li>
-    <p><a href="https://issues.apache.org/jira/browse/HIVE-14214">HIVE-14214</a> Schema evolution and predicate pushdown don’t work together.</p>
+    <p><a href="http://localhost:4000/security/CVE-2018-8015">CVE-2018-8015</a> ORC files with malformed types cause stack overflow.</p>
   </li>
   <li>
-    <p><a href="https://issues.apache.org/jira/browse/ORC-40">ORC-40</a> Predicate push down is not implemented in C++.</p>
+    <p><a href="http://localhost:4000/security/HIVE-14214">HIVE-14214</a> Schema evolution and predicate pushdown don’t work together.</p>
   </li>
   <li>
-    <p><a href="https://issues.apache.org/jira/browse/ORC-101">ORC-101</a> Bloom filters for string and decimal use inconsistent encoding</p>
+    <p><a href="http://localhost:4000/security/ORC-101">ORC-101</a> Bloom filters for string and decimal use inconsistent encoding</p>
   </li>
   <li>
-    <p><a href="https://issues.apache.org/jira/browse/ORC-135">ORC-135</a> Predicate push down is incorrect on timestamps when moved between timezones</p>
+    <p><a href="http://localhost:4000/security/ORC-135">ORC-135</a> Predicate push down is incorrect on timestamps when moved between timezones</p>
   </li>
 </ul>
 
@@ -1314,13 +1323,10 @@ signed by <a href="https://dist.apache.org/repos/dist/release/orc/KEYS">Owen O
 
 <ul>
   <li>
-    <p><a href="https://issues.apache.org/jira/browse/ORC-1">ORC-1</a> We are still working on moving the Java reader and writer out of Hive’s code base and thus it is not included here.</p>
-  </li>
-  <li>
-    <p><a href="https://issues.apache.org/jira/browse/ORC-10">ORC-10</a> When moving ORC files between timezones, different daylight savings rules will cause timestamps to shift in the C++ reader.</p>
+    <p><a href="http://localhost:4000/security/CVE-2018-8015">CVE-2018-8015</a> ORC files with malformed types cause stack overflow.</p>
   </li>
   <li>
-    <p><a href="https://issues.apache.org/jira/browse/ORC-40">ORC-40</a> Predicate push down is not implemented in C++.</p>
+    <p><a href="http://localhost:4000/security/ORC-10">ORC-10</a> When moving ORC files between timezones, different daylight savings rules will cause timestamps to shift in the C++ reader.</p>
   </li>
 </ul>
 

http://git-wip-us.apache.org/repos/asf/orc/blob/e58d3fc4/security/CVE-2018-8015/index.html
----------------------------------------------------------------------
diff --git a/security/CVE-2018-8015/index.html b/security/CVE-2018-8015/index.html
new file mode 100644
index 0000000..a51e319
--- /dev/null
+++ b/security/CVE-2018-8015/index.html
@@ -0,0 +1,187 @@
+<!DOCTYPE HTML>
+<html lang="en-US">
+<head>
+  <meta charset="UTF-8">
+  <title>CVE-2018-8015</title>
+  <meta name="viewport" content="width=device-width,initial-scale=1">
+  <meta name="generator" content="Jekyll v3.7.3">
+  <link rel="stylesheet" href="//fonts.googleapis.com/css?family=Lato:300,300italic,400,400italic,700,700italic,900">
+  <link rel="stylesheet" href="/css/screen.css">
+  <link rel="icon" type="image/x-icon" href="/favicon.ico">
+  <!--[if lt IE 9]>
+  <script src="/js/html5shiv.min.js"></script>
+  <script src="/js/respond.min.js"></script>
+  <![endif]-->
+</head>
+
+
+<body class="wrap">
+  <header role="banner">
+  <nav class="mobile-nav show-on-mobiles">
+    <ul>
+  <li class="">
+    <a href="/">Home</a>
+  </li>
+  <li class="">
+    <a href="/docs/"><span class="show-on-mobiles">Docs</span>
+                     <span class="hide-on-mobiles">Documentation</span></a>
+  </li>
+  <li class="">
+    <a href="/talks/">Talks</a>
+  </li>
+  <li class="">
+    <a href="/news/">News</a>
+  </li>
+  <li class="">
+    <a href="/help/">Help</a>
+  </li>
+  <li class="">
+    <a href="/develop/">Develop</a>
+  </li>
+</ul>
+
+  </nav>
+  <div class="grid">
+    <div class="unit one-third center-on-mobiles">
+      <h1>
+        <a href="/">
+          <span class="sr-only">Apache ORC</span>
+          <img src="/img/logo.png" width="249" height="101" alt="ORC Logo">
+        </a>
+      </h1>
+    </div>
+    <nav class="main-nav unit two-thirds hide-on-mobiles">
+      <ul>
+  <li class="">
+    <a href="/">Home</a>
+  </li>
+  <li class="">
+    <a href="/docs/"><span class="show-on-mobiles">Docs</span>
+                     <span class="hide-on-mobiles">Documentation</span></a>
+  </li>
+  <li class="">
+    <a href="/talks/">Talks</a>
+  </li>
+  <li class="">
+    <a href="/news/">News</a>
+  </li>
+  <li class="">
+    <a href="/help/">Help</a>
+  </li>
+  <li class="">
+    <a href="/develop/">Develop</a>
+  </li>
+</ul>
+
+    </nav>
+  </div>
+</header>
+
+
+  <section class="standalone">
+  <div class="grid">
+
+    <div class="unit whole">
+      <article>
+        <h1>CVE-2018-8015</h1>
+        <h1 id="apache-orc-denial-of-service-vulnerability">Apache ORC denial of service vulnerability</h1>
+
+<h2 id="date">Date:</h2>
+<p>2018-05-17</p>
+
+<h2 id="severity">Severity:</h2>
+
+<p>Medium</p>
+
+<h2 id="vendor">Vendor:</h2>
+
+<p><a href="https://apache.org">The Apache Software Foundation</a></p>
+
+<h2 id="versions-affected">Versions Affected:</h2>
+
+<ul>
+  <li>ORC 1.0.0 to 1.4.3</li>
+</ul>
+
+<h2 id="description">Description:</h2>
+
+<p>A malformed ORC file can trigger an endlessly recursive function call in the
+C++ or Java parser.</p>
+
+<p>The impact of this bug is most likely denial-of-service against software that
+uses the ORC file parser. With the C++ parser, the stack overflow might 
+possibly corrupt the stack.</p>
+
+<h2 id="mitigation">Mitigation:</h2>
+
+<ul>
+  <li>1.3.x and 1.4.x users should upgrade to 1.4.4.</li>
+  <li>1.0.x to 1.2.x users should apply ORC-360 (Java) and ORC-313 (C++).</li>
+</ul>
+
+<h2 id="example">Example:</h2>
+
+<p>An ORC file with a struct, union, array, or map type that includes itself as
+a child will cause the parser to infinitely recurse until the stack overflows.</p>
+
+<h2 id="credit">Credit:</h2>
+<p>This issue was discovered by Terry Chia.</p>
+
+<h2 id="references">References:</h2>
+<p><a href="http://localhost:4000/security/">Apache ORC security</a></p>
+
+      </article>
+    </div>
+
+    <div class="clear"></div>
+
+  </div>
+</section>
+
+
+  <footer role="contentinfo">
+  <p>The contents of this website are &copy;&nbsp;2018
+     <a href="https://www.apache.org/">Apache Software Foundation</a>
+     under the terms of the <a
+      href="https://www.apache.org/licenses/LICENSE-2.0.html">
+      Apache&nbsp;License&nbsp;v2</a>. Apache ORC and its logo are trademarks
+      of the Apache Software Foundation.</p>
+</footer>
+
+  <script>
+  var anchorForId = function (id) {
+    var anchor = document.createElement("a");
+    anchor.className = "header-link";
+    anchor.href      = "#" + id;
+    anchor.innerHTML = "<span class=\"sr-only\">Permalink</span><i class=\"fa fa-link\"></i>";
+    anchor.title = "Permalink";
+    return anchor;
+  };
+
+  var linkifyAnchors = function (level, containingElement) {
+    var headers = containingElement.getElementsByTagName("h" + level);
+    for (var h = 0; h < headers.length; h++) {
+      var header = headers[h];
+
+      if (typeof header.id !== "undefined" && header.id !== "") {
+        header.appendChild(anchorForId(header.id));
+      }
+    }
+  };
+
+  document.onreadystatechange = function () {
+    if (this.readyState === "complete") {
+      var contentBlock = document.getElementsByClassName("docs")[0] || document.getElementsByClassName("news")[0];
+      if (!contentBlock) {
+        return;
+      }
+      for (var level = 1; level <= 6; level++) {
+        linkifyAnchors(level, contentBlock);
+      }
+    }
+  };
+</script>
+
+
+</body>
+</html>

http://git-wip-us.apache.org/repos/asf/orc/blob/e58d3fc4/security/index.html
----------------------------------------------------------------------
diff --git a/security/index.html b/security/index.html
index 0db181b..7f038b1 100644
--- a/security/index.html
+++ b/security/index.html
@@ -4,7 +4,7 @@
   <meta charset="UTF-8">
   <title>Security</title>
   <meta name="viewport" content="width=device-width,initial-scale=1">
-  <meta name="generator" content="Jekyll v2.4.0">
+  <meta name="generator" content="Jekyll v3.7.3">
   <link rel="stylesheet" href="//fonts.googleapis.com/css?family=Lato:300,300italic,400,400italic,700,700italic,900">
   <link rel="stylesheet" href="/css/screen.css">
   <link rel="icon" type="image/x-icon" href="/favicon.ico">
@@ -125,7 +125,9 @@ the members of the
 
 <h2 id="fixed-cves">Fixed CVEs</h2>
 
-<p>None.</p>
+<ul>
+  <li><a href="CVE-2018-8015">CVE-2018-8015</a> - ORC files with malformed types cause stack overflow.</li>
+</ul>
 
       </article>
     </div>

http://git-wip-us.apache.org/repos/asf/orc/blob/e58d3fc4/specification/ORCv0.html
----------------------------------------------------------------------
diff --git a/specification/ORCv0.html b/specification/ORCv0.html
deleted file mode 100644
index ecf335a..0000000
--- a/specification/ORCv0.html
+++ /dev/null
@@ -1,1260 +0,0 @@
-<!DOCTYPE HTML>
-<html lang="en-US">
-<head>
-  <meta charset="UTF-8">
-  <title>ORC Specification v0</title>
-  <meta name="viewport" content="width=device-width,initial-scale=1">
-  <meta name="generator" content="Jekyll v2.4.0">
-  <link rel="stylesheet" href="//fonts.googleapis.com/css?family=Lato:300,300italic,400,400italic,700,700italic,900">
-  <link rel="stylesheet" href="/css/screen.css">
-  <link rel="icon" type="image/x-icon" href="/favicon.ico">
-  <!--[if lt IE 9]>
-  <script src="/js/html5shiv.min.js"></script>
-  <script src="/js/respond.min.js"></script>
-  <![endif]-->
-</head>
-
-
-<body class="wrap">
-  <header role="banner">
-  <nav class="mobile-nav show-on-mobiles">
-    <ul>
-  <li class="">
-    <a href="/">Home</a>
-  </li>
-  <li class="">
-    <a href="/docs/"><span class="show-on-mobiles">Docs</span>
-                     <span class="hide-on-mobiles">Documentation</span></a>
-  </li>
-  <li class="">
-    <a href="/talks/">Talks</a>
-  </li>
-  <li class="">
-    <a href="/news/">News</a>
-  </li>
-  <li class="">
-    <a href="/help/">Help</a>
-  </li>
-  <li class="">
-    <a href="/develop/">Develop</a>
-  </li>
-</ul>
-
-  </nav>
-  <div class="grid">
-    <div class="unit one-third center-on-mobiles">
-      <h1>
-        <a href="/">
-          <span class="sr-only">Apache ORC</span>
-          <img src="/img/logo.png" width="249" height="101" alt="ORC Logo">
-        </a>
-      </h1>
-    </div>
-    <nav class="main-nav unit two-thirds hide-on-mobiles">
-      <ul>
-  <li class="">
-    <a href="/">Home</a>
-  </li>
-  <li class="">
-    <a href="/docs/"><span class="show-on-mobiles">Docs</span>
-                     <span class="hide-on-mobiles">Documentation</span></a>
-  </li>
-  <li class="">
-    <a href="/talks/">Talks</a>
-  </li>
-  <li class="">
-    <a href="/news/">News</a>
-  </li>
-  <li class="">
-    <a href="/help/">Help</a>
-  </li>
-  <li class="">
-    <a href="/develop/">Develop</a>
-  </li>
-</ul>
-
-    </nav>
-  </div>
-</header>
-
-
-  <section class="standalone">
-  <div class="grid">
-
-    <div class="unit whole">
-      <article>
-        <h1>ORC Specification v0</h1>
-        <p>This version of the file format was originally released as part of
-Hive 0.11.</p>
-
-<h1 id="motivation">Motivation</h1>
-
-<p>Hive’s RCFile was the standard format for storing tabular data in
-Hadoop for several years. However, RCFile has limitations because it
-treats each column as a binary blob without semantics. In Hive 0.11 we
-added a new file format named Optimized Row Columnar (ORC) file that
-uses and retains the type information from the table definition. ORC
-uses type specific readers and writers that provide light weight
-compression techniques such as dictionary encoding, bit packing, delta
-encoding, and run length encoding – resulting in dramatically smaller
-files. Additionally, ORC can apply generic compression using zlib, or
-Snappy on top of the lightweight compression for even smaller
-files. However, storage savings are only part of the gain. ORC
-supports projection, which selects subsets of the columns for reading,
-so that queries reading only one column read only the required
-bytes. Furthermore, ORC files include light weight indexes that
-include the minimum and maximum values for each column in each set of
-10,000 rows and the entire file. Using pushdown filters from Hive, the
-file reader can skip entire sets of rows that aren’t important for
-this query.</p>
-
-<p><img src="/img/OrcFileLayout.png" alt="ORC file structure" /></p>
-
-<h1 id="file-tail">File Tail</h1>
-
-<p>Since HDFS does not support changing the data in a file after it is
-written, ORC stores the top level index at the end of the file. The
-overall structure of the file is given in the figure above.  The
-file’s tail consists of 3 parts; the file metadata, file footer and
-postscript.</p>
-
-<p>The metadata for ORC is stored using
-<a href="https://s.apache.org/protobuf_encoding">Protocol Buffers</a>, which provides
-the ability to add new fields without breaking readers. This document
-incorporates the Protobuf definition from the
-<a href="https://s.apache.org/orc_proto">ORC source code</a> and the
-reader is encouraged to review the Protobuf encoding if they need to
-understand the byte-level encoding</p>
-
-<h2 id="postscript">Postscript</h2>
-
-<p>The Postscript section provides the necessary information to interpret
-the rest of the file including the length of the file’s Footer and
-Metadata sections, the version of the file, and the kind of general
-compression used (eg. none, zlib, or snappy). The Postscript is never
-compressed and ends one byte before the end of the file. The version
-stored in the Postscript is the lowest version of Hive that is
-guaranteed to be able to read the file and it stored as a sequence of
-the major and minor version. This version is stored as [0, 11].</p>
-
-<p>The process of reading an ORC file works backwards through the
-file. Rather than making multiple short reads, the ORC reader reads
-the last 16k bytes of the file with the hope that it will contain both
-the Footer and Postscript sections. The final byte of the file
-contains the serialized length of the Postscript, which must be less
-than 256 bytes. Once the Postscript is parsed, the compressed
-serialized length of the Footer is known and it can be decompressed
-and parsed.</p>
-
-<p><code>message PostScript {
- // the length of the footer section in bytes
- optional uint64 footerLength = 1;
- // the kind of generic compression used
- optional CompressionKind compression = 2;
- // the maximum size of each compression chunk
- optional uint64 compressionBlockSize = 3;
- // the version of the writer
- repeated uint32 version = 4 [packed = true];
- // the length of the metadata section in bytes
- optional uint64 metadataLength = 5;
- // the fixed string "ORC"
- optional string magic = 8000;
-}
-</code></p>
-
-<p><code>enum CompressionKind {
- NONE = 0;
- ZLIB = 1;
- SNAPPY = 2;
- LZO = 3;
- LZ4 = 4;
- ZSTD = 5;
-}
-</code></p>
-
-<h2 id="footer">Footer</h2>
-
-<p>The Footer section contains the layout of the body of the file, the
-type schema information, the number of rows, and the statistics about
-each of the columns.</p>
-
-<p>The file is broken in to three parts- Header, Body, and Tail. The
-Header consists of the bytes “ORC’’ to support tools that want to
-scan the front of the file to determine the type of the file. The Body
-contains the rows and indexes, and the Tail gives the file level
-information as described in this section.</p>
-
-<p><code>message Footer {
- // the length of the file header in bytes (always 3)
- optional uint64 headerLength = 1;
- // the length of the file header and body in bytes
- optional uint64 contentLength = 2;
- // the information about the stripes
- repeated StripeInformation stripes = 3;
- // the schema information
- repeated Type types = 4;
- // the user metadata that was added
- repeated UserMetadataItem metadata = 5;
- // the total number of rows in the file
- optional uint64 numberOfRows = 6;
- // the statistics of each column across the file
- repeated ColumnStatistics statistics = 7;
- // the maximum number of rows in each index entry
- optional uint32 rowIndexStride = 8;
-}
-</code></p>
-
-<h3 id="stripe-information">Stripe Information</h3>
-
-<p>The body of the file is divided into stripes. Each stripe is self
-contained and may be read using only its own bytes combined with the
-file’s Footer and Postscript. Each stripe contains only entire rows so
-that rows never straddle stripe boundaries. Stripes have three
-sections: a set of indexes for the rows within the stripe, the data
-itself, and a stripe footer. Both the indexes and the data sections
-are divided by columns so that only the data for the required columns
-needs to be read.</p>
-
-<p><code>message StripeInformation {
- // the start of the stripe within the file
- optional uint64 offset = 1;
- // the length of the indexes in bytes
- optional uint64 indexLength = 2;
- // the length of the data in bytes
- optional uint64 dataLength = 3;
- // the length of the footer in bytes
- optional uint64 footerLength = 4;
- // the number of rows in the stripe
- optional uint64 numberOfRows = 5;
-}
-</code></p>
-
-<h3 id="type-information">Type Information</h3>
-
-<p>All of the rows in an ORC file must have the same schema. Logically
-the schema is expressed as a tree as in the figure below, where
-the compound types have subcolumns under them.</p>
-
-<p><img src="/img/TreeWriters.png" alt="ORC column structure" /></p>
-
-<p>The equivalent Hive DDL would be:</p>
-
-<p><code>create table Foobar (
- myInt int,
- myMap map&lt;string,
- struct&lt;myString : string,
- myDouble: double&gt;&gt;,
- myTime timestamp
-);
-</code></p>
-
-<p>The type tree is flattened in to a list via a pre-order traversal
-where each type is assigned the next id. Clearly the root of the type
-tree is always type id 0. Compound types have a field named subtypes
-that contains the list of their children’s type ids.</p>
-
-<p><code>message Type {
- enum Kind {
- BOOLEAN = 0;
- BYTE = 1;
- SHORT = 2;
- INT = 3;
- LONG = 4;
- FLOAT = 5;
- DOUBLE = 6;
- STRING = 7;
- BINARY = 8;
- TIMESTAMP = 9;
- LIST = 10;
- MAP = 11;
- STRUCT = 12;
- UNION = 13;
- DECIMAL = 14;
- DATE = 15;
- VARCHAR = 16;
- CHAR = 17;
- }
- // the kind of this type
- required Kind kind = 1;
- // the type ids of any subcolumns for list, map, struct, or union
- repeated uint32 subtypes = 2 [packed=true];
- // the list of field names for struct
- repeated string fieldNames = 3;
- // the maximum length of the type for varchar or char in UTF-8 characters
- optional uint32 maximumLength = 4;
- // the precision and scale for decimal
- optional uint32 precision = 5;
- optional uint32 scale = 6;
-}
-</code></p>
-
-<h3 id="column-statistics">Column Statistics</h3>
-
-<p>The goal of the column statistics is that for each column, the writer
-records the count and depending on the type other useful fields. For
-most of the primitive types, it records the minimum and maximum
-values; and for numeric types it additionally stores the sum.
-From Hive 1.1.0 onwards, the column statistics will also record if
-there are any null values within the row group by setting the hasNull flag.
-The hasNull flag is used by ORC’s predicate pushdown to better answer
-‘IS NULL’ queries.</p>
-
-<p><code>message ColumnStatistics {
- // the number of values
- optional uint64 numberOfValues = 1;
- // At most one of these has a value for any column
- optional IntegerStatistics intStatistics = 2;
- optional DoubleStatistics doubleStatistics = 3;
- optional StringStatistics stringStatistics = 4;
- optional BucketStatistics bucketStatistics = 5;
- optional DecimalStatistics decimalStatistics = 6;
- optional DateStatistics dateStatistics = 7;
- optional BinaryStatistics binaryStatistics = 8;
- optional TimestampStatistics timestampStatistics = 9;
- optional bool hasNull = 10;
-}
-</code></p>
-
-<p>For integer types (tinyint, smallint, int, bigint), the column
-statistics includes the minimum, maximum, and sum. If the sum
-overflows long at any point during the calculation, no sum is
-recorded.</p>
-
-<p><code>message IntegerStatistics {
- optional sint64 minimum = 1;
- optional sint64 maximum = 2;
- optional sint64 sum = 3;
-}
-</code></p>
-
-<p>For floating point types (float, double), the column statistics
-include the minimum, maximum, and sum. If the sum overflows a double,
-no sum is recorded.</p>
-
-<p><code>message DoubleStatistics {
- optional double minimum = 1;
- optional double maximum = 2;
- optional double sum = 3;
-}
-</code></p>
-
-<p>For strings, the minimum value, maximum value, and the sum of the
-lengths of the values are recorded.</p>
-
-<p><code>message StringStatistics {
- optional string minimum = 1;
- optional string maximum = 2;
- // sum will store the total length of all strings
- optional sint64 sum = 3;
-}
-</code></p>
-
-<p>For booleans, the statistics include the count of false and true values.</p>
-
-<p><code>message BucketStatistics {
- repeated uint64 count = 1 [packed=true];
-}
-</code></p>
-
-<p>For decimals, the minimum, maximum, and sum are stored.</p>
-
-<p><code>message DecimalStatistics {
- optional string minimum = 1;
- optional string maximum = 2;
- optional string sum = 3;
-}
-</code></p>
-
-<p>Date columns record the minimum and maximum values as the number of
-days since the epoch (1/1/2015).</p>
-
-<p><code>message DateStatistics {
- // min,max values saved as days since epoch
- optional sint32 minimum = 1;
- optional sint32 maximum = 2;
-}
-</code></p>
-
-<p>Timestamp columns record the minimum and maximum values as the number of
-milliseconds since the epoch (1/1/2015).</p>
-
-<p><code>message TimestampStatistics {
- // min,max values saved as milliseconds since epoch
- optional sint64 minimum = 1;
- optional sint64 maximum = 2;
-}
-</code></p>
-
-<p>Binary columns store the aggregate number of bytes across all of the values.</p>
-
-<p><code>message BinaryStatistics {
- // sum will store the total binary blob length
- optional sint64 sum = 1;
-}
-</code></p>
-
-<h3 id="user-metadata">User Metadata</h3>
-
-<p>The user can add arbitrary key/value pairs to an ORC file as it is
-written. The contents of the keys and values are completely
-application defined, but the key is a string and the value is
-binary. Care should be taken by applications to make sure that their
-keys are unique and in general should be prefixed with an organization
-code.</p>
-
-<p><code>message UserMetadataItem {
- // the user defined key
- required string name = 1;
- // the user defined binary value
- required bytes value = 2;
-}
-</code></p>
-
-<h3 id="file-metadata">File Metadata</h3>
-
-<p>The file Metadata section contains column statistics at the stripe
-level granularity. These statistics enable input split elimination
-based on the predicate push-down evaluated per a stripe.</p>
-
-<p><code>message StripeStatistics {
- repeated ColumnStatistics colStats = 1;
-}
-</code></p>
-
-<p><code>message Metadata {
- repeated StripeStatistics stripeStats = 1;
-}
-</code></p>
-
-<h1 id="compression">Compression</h1>
-
-<p>If the ORC file writer selects a generic compression codec (zlib or
-snappy), every part of the ORC file except for the Postscript is
-compressed with that codec. However, one of the requirements for ORC
-is that the reader be able to skip over compressed bytes without
-decompressing the entire stream. To manage this, ORC writes compressed
-streams in chunks with headers as in the figure below.
-To handle uncompressable data, if the compressed data is larger than
-the original, the original is stored and the isOriginal flag is
-set. Each header is 3 bytes long with (compressedLength * 2 +
-isOriginal) stored as a little endian value. For example, the header
-for a chunk that compressed to 100,000 bytes would be [0x40, 0x0d,
-0x03]. The header for 5 bytes that did not compress would be [0x0b,
-0x00, 0x00]. Each compression chunk is compressed independently so
-that as long as a decompressor starts at the top of a header, it can
-start decompressing without the previous bytes.</p>
-
-<p><img src="/img/CompressionStream.png" alt="compression streams" /></p>
-
-<p>The default compression chunk size is 256K, but writers can choose
-their own value. Larger chunks lead to better compression, but require
-more memory. The chunk size is recorded in the Postscript so that
-readers can allocate appropriately sized buffers. Readers are
-guaranteed that no chunk will expand to more than the compression chunk
-size.</p>
-
-<p>ORC files without generic compression write each stream directly
-with no headers.</p>
-
-<h1 id="run-length-encoding">Run Length Encoding</h1>
-
-<h2 id="base-128-varint">Base 128 Varint</h2>
-
-<p>Variable width integer encodings take advantage of the fact that most
-numbers are small and that having smaller encodings for small numbers
-shrinks the overall size of the data. ORC uses the varint format from
-Protocol Buffers, which writes data in little endian format using the
-low 7 bits of each byte. The high bit in each byte is set if the
-number continues into the next byte.</p>
-
-<table>
-  <thead>
-    <tr>
-      <th style="text-align: left">Unsigned Original</th>
-      <th style="text-align: left">Serialized</th>
-    </tr>
-  </thead>
-  <tbody>
-    <tr>
-      <td style="text-align: left">0</td>
-      <td style="text-align: left">0x00</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">1</td>
-      <td style="text-align: left">0x01</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">127</td>
-      <td style="text-align: left">0x7f</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">128</td>
-      <td style="text-align: left">0x80, 0x01</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">129</td>
-      <td style="text-align: left">0x81, 0x01</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">16,383</td>
-      <td style="text-align: left">0xff, 0x7f</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">16,384</td>
-      <td style="text-align: left">0x80, 0x80, 0x01</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">16,385</td>
-      <td style="text-align: left">0x81, 0x80, 0x01</td>
-    </tr>
-  </tbody>
-</table>
-
-<p>For signed integer types, the number is converted into an unsigned
-number using a zigzag encoding. Zigzag encoding moves the sign bit to
-the least significant bit using the expression (val « 1) ^ (val »
-63) and derives its name from the fact that positive and negative
-numbers alternate once encoded. The unsigned number is then serialized
-as above.</p>
-
-<table>
-  <thead>
-    <tr>
-      <th style="text-align: left">Signed Original</th>
-      <th style="text-align: left">Unsigned</th>
-    </tr>
-  </thead>
-  <tbody>
-    <tr>
-      <td style="text-align: left">0</td>
-      <td style="text-align: left">0</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">-1</td>
-      <td style="text-align: left">1</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">1</td>
-      <td style="text-align: left">2</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">-2</td>
-      <td style="text-align: left">3</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">2</td>
-      <td style="text-align: left">4</td>
-    </tr>
-  </tbody>
-</table>
-
-<h2 id="byte-run-length-encoding">Byte Run Length Encoding</h2>
-
-<p>For byte streams, ORC uses a very light weight encoding of identical
-values.</p>
-
-<ul>
-  <li>Run - a sequence of at least 3 identical values</li>
-  <li>Literals - a sequence of non-identical values</li>
-</ul>
-
-<p>The first byte of each group of values is a header than determines
-whether it is a run (value between 0 to 127) or literal list (value
-between -128 to -1). For runs, the control byte is the length of the
-run minus the length of the minimal run (3) and the control byte for
-literal lists is the negative length of the list. For example, a
-hundred 0’s is encoded as [0x61, 0x00] and the sequence 0x44, 0x45
-would be encoded as [0xfe, 0x44, 0x45]. The next group can choose
-either of the encodings.</p>
-
-<h2 id="boolean-run-length-encoding">Boolean Run Length Encoding</h2>
-
-<p>For encoding boolean types, the bits are put in the bytes from most
-significant to least significant. The bytes are encoded using byte run
-length encoding as described in the previous section. For example,
-the byte sequence [0xff, 0x80] would be one true followed by
-seven false values.</p>
-
-<h2 id="integer-run-length-encoding-version-1">Integer Run Length Encoding, version 1</h2>
-
-<p>ORC v0 files use Run Length Encoding version 1 (RLEv1),
-which provides a lightweight compression of signed or unsigned integer
-sequences. RLEv1 has two sub-encodings:</p>
-
-<ul>
-  <li>Run - a sequence of values that differ by a small fixed delta</li>
-  <li>Literals - a sequence of varint encoded values</li>
-</ul>
-
-<p>Runs start with an initial byte of 0x00 to 0x7f, which encodes the
-length of the run - 3. A second byte provides the fixed delta in the
-range of -128 to 127. Finally, the first value of the run is encoded
-as a base 128 varint.</p>
-
-<p>For example, if the sequence is 100 instances of 7 the encoding would
-start with 100 - 3, followed by a delta of 0, and a varint of 7 for
-an encoding of [0x61, 0x00, 0x07]. To encode the sequence of numbers
-running from 100 to 1, the first byte is 100 - 3, the delta is -1,
-and the varint is 100 for an encoding of [0x61, 0xff, 0x64].</p>
-
-<p>Literals start with an initial byte of 0x80 to 0xff, which corresponds
-to the negative of number of literals in the sequence. Following the
-header byte, the list of N varints is encoded. Thus, if there are
-no runs, the overhead is 1 byte for each 128 integers. The first 5
-prime numbers [2, 3, 4, 7, 11] would encoded as [0xfb, 0x02, 0x03,
-0x04, 0x07, 0xb].</p>
-
-<h1 id="stripes">Stripes</h1>
-
-<p>The body of ORC files consists of a series of stripes. Stripes are
-large (typically ~200MB) and independent of each other and are often
-processed by different tasks. The defining characteristic for columnar
-storage formats is that the data for each column is stored separately
-and that reading data out of the file should be proportional to the
-number of columns read.</p>
-
-<p>In ORC files, each column is stored in several streams that are stored
-next to each other in the file. For example, an integer column is
-represented as two streams PRESENT, which uses one with a bit per
-value recording if the value is non-null, and DATA, which records the
-non-null values. If all of a column’s values in a stripe are non-null,
-the PRESENT stream is omitted from the stripe. For binary data, ORC
-uses three streams PRESENT, DATA, and LENGTH, which stores the length
-of each value. The details of each type will be presented in the
-following subsections.</p>
-
-<h2 id="stripe-footer">Stripe Footer</h2>
-
-<p>The stripe footer contains the encoding of each column and the
-directory of the streams including their location.</p>
-
-<p><code>message StripeFooter {
- // the location of each stream
- repeated Stream streams = 1;
- // the encoding of each column
- repeated ColumnEncoding columns = 2;
-}
-</code></p>
-
-<p>To describe each stream, ORC stores the kind of stream, the column id,
-and the stream’s size in bytes. The details of what is stored in each stream
-depends on the type and encoding of the column.</p>
-
-<p><code>message Stream {
- enum Kind {
- // boolean stream of whether the next value is non-null
- PRESENT = 0;
- // the primary data stream
- DATA = 1;
- // the length of each value for variable length data
- LENGTH = 2;
- // the dictionary blob
- DICTIONARY\_DATA = 3;
- // deprecated prior to Hive 0.11
- // It was used to store the number of instances of each value in the
- // dictionary
- DICTIONARY_COUNT = 4;
- // a secondary data stream
- SECONDARY = 5;
- // the index for seeking to particular row groups
- ROW_INDEX = 6;
- }
- required Kind kind = 1;
- // the column id
- optional uint32 column = 2;
- // the number of bytes in the file
- optional uint64 length = 3;
-}
-</code></p>
-
-<p>Depending on their type several options for encoding are possible. The
-encodings are divided into direct or dictionary-based categories and
-further refined as to whether they use RLE v1 or v2.</p>
-
-<p><code>message ColumnEncoding {
- enum Kind {
- // the encoding is mapped directly to the stream using RLE v1
- DIRECT = 0;
- // the encoding uses a dictionary of unique values using RLE v1
- DICTIONARY = 1;
- // the encoding is direct using RLE v2
- }
- required Kind kind = 1;
- // for dictionary encodings, record the size of the dictionary
- optional uint32 dictionarySize = 2;
-}
-</code></p>
-
-<h1 id="column-encodings">Column Encodings</h1>
-
-<h2 id="smallint-int-and-bigint-columns">SmallInt, Int, and BigInt Columns</h2>
-
-<p>All of the 16, 32, and 64 bit integer column types use the same set of
-potential encodings, which is basically whether they use RLE v1 or
-v2. If the PRESENT stream is not included, all of the values are
-present. For values that have false bits in the present stream, no
-values are included in the data stream.</p>
-
-<table>
-  <thead>
-    <tr>
-      <th style="text-align: left">Encoding</th>
-      <th style="text-align: left">Stream Kind</th>
-      <th style="text-align: left">Optional</th>
-      <th style="text-align: left">Contents</th>
-    </tr>
-  </thead>
-  <tbody>
-    <tr>
-      <td style="text-align: left">DIRECT</td>
-      <td style="text-align: left">PRESENT</td>
-      <td style="text-align: left">Yes</td>
-      <td style="text-align: left">Boolean RLE</td>
-    </tr>
-    <tr>
-      <td style="text-align: left"> </td>
-      <td style="text-align: left">DATA</td>
-      <td style="text-align: left">No</td>
-      <td style="text-align: left">Signed Integer RLE v1</td>
-    </tr>
-  </tbody>
-</table>
-
-<h2 id="float-and-double-columns">Float and Double Columns</h2>
-
-<p>Floating point types are stored using IEEE 754 floating point bit
-layout. Float columns use 4 bytes per value and double columns use 8
-bytes.</p>
-
-<table>
-  <thead>
-    <tr>
-      <th style="text-align: left">Encoding</th>
-      <th style="text-align: left">Stream Kind</th>
-      <th style="text-align: left">Optional</th>
-      <th style="text-align: left">Contents</th>
-    </tr>
-  </thead>
-  <tbody>
-    <tr>
-      <td style="text-align: left">DIRECT</td>
-      <td style="text-align: left">PRESENT</td>
-      <td style="text-align: left">Yes</td>
-      <td style="text-align: left">Boolean RLE</td>
-    </tr>
-    <tr>
-      <td style="text-align: left"> </td>
-      <td style="text-align: left">DATA</td>
-      <td style="text-align: left">No</td>
-      <td style="text-align: left">IEEE 754 floating point representation</td>
-    </tr>
-  </tbody>
-</table>
-
-<h2 id="string-char-and-varchar-columns">String, Char, and VarChar Columns</h2>
-
-<p>String, char, and varchar columns may be encoded either using a
-dictionary encoding or a direct encoding. A direct encoding should be
-preferred when there are many distinct values. In all of the
-encodings, the PRESENT stream encodes whether the value is null. The
-Java ORC writer automatically picks the encoding after the first row
-group (10,000 rows).</p>
-
-<p>For direct encoding the UTF-8 bytes are saved in the DATA stream and
-the length of each value is written into the LENGTH stream. In direct
-encoding, if the values were [“Nevada”, “California”]; the DATA
-would be “NevadaCalifornia” and the LENGTH would be [6, 10].</p>
-
-<p>For dictionary encodings the dictionary is sorted and UTF-8 bytes of
-each unique value are placed into DICTIONARY_DATA. The length of each
-item in the dictionary is put into the LENGTH stream. The DATA stream
-consists of the sequence of references to the dictionary elements.</p>
-
-<p>In dictionary encoding, if the values were [“Nevada”,
-“California”, “Nevada”, “California”, and “Florida”]; the
-DICTIONARY_DATA would be “CaliforniaFloridaNevada” and LENGTH would
-be [10, 7, 6]. The DATA would be [2, 0, 2, 0, 1].</p>
-
-<table>
-  <thead>
-    <tr>
-      <th style="text-align: left">Encoding</th>
-      <th style="text-align: left">Stream Kind</th>
-      <th style="text-align: left">Optional</th>
-      <th style="text-align: left">Contents</th>
-    </tr>
-  </thead>
-  <tbody>
-    <tr>
-      <td style="text-align: left">DIRECT</td>
-      <td style="text-align: left">PRESENT</td>
-      <td style="text-align: left">Yes</td>
-      <td style="text-align: left">Boolean RLE</td>
-    </tr>
-    <tr>
-      <td style="text-align: left"> </td>
-      <td style="text-align: left">DATA</td>
-      <td style="text-align: left">No</td>
-      <td style="text-align: left">String contents</td>
-    </tr>
-    <tr>
-      <td style="text-align: left"> </td>
-      <td style="text-align: left">LENGTH</td>
-      <td style="text-align: left">No</td>
-      <td style="text-align: left">Unsigned Integer RLE v1</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">DICTIONARY</td>
-      <td style="text-align: left">PRESENT</td>
-      <td style="text-align: left">Yes</td>
-      <td style="text-align: left">Boolean RLE</td>
-    </tr>
-    <tr>
-      <td style="text-align: left"> </td>
-      <td style="text-align: left">DATA</td>
-      <td style="text-align: left">No</td>
-      <td style="text-align: left">Unsigned Integer RLE v1</td>
-    </tr>
-    <tr>
-      <td style="text-align: left"> </td>
-      <td style="text-align: left">DICTIONARY_DATA</td>
-      <td style="text-align: left">No</td>
-      <td style="text-align: left">String contents</td>
-    </tr>
-    <tr>
-      <td style="text-align: left"> </td>
-      <td style="text-align: left">LENGTH</td>
-      <td style="text-align: left">No</td>
-      <td style="text-align: left">Unsigned Integer RLE v1</td>
-    </tr>
-  </tbody>
-</table>
-
-<h2 id="boolean-columns">Boolean Columns</h2>
-
-<p>Boolean columns are rare, but have a simple encoding.</p>
-
-<table>
-  <thead>
-    <tr>
-      <th style="text-align: left">Encoding</th>
-      <th style="text-align: left">Stream Kind</th>
-      <th style="text-align: left">Optional</th>
-      <th style="text-align: left">Contents</th>
-    </tr>
-  </thead>
-  <tbody>
-    <tr>
-      <td style="text-align: left">DIRECT</td>
-      <td style="text-align: left">PRESENT</td>
-      <td style="text-align: left">Yes</td>
-      <td style="text-align: left">Boolean RLE</td>
-    </tr>
-    <tr>
-      <td style="text-align: left"> </td>
-      <td style="text-align: left">DATA</td>
-      <td style="text-align: left">No</td>
-      <td style="text-align: left">Boolean RLE</td>
-    </tr>
-  </tbody>
-</table>
-
-<h2 id="tinyint-columns">TinyInt Columns</h2>
-
-<p>TinyInt (byte) columns use byte run length encoding.</p>
-
-<table>
-  <thead>
-    <tr>
-      <th style="text-align: left">Encoding</th>
-      <th style="text-align: left">Stream Kind</th>
-      <th style="text-align: left">Optional</th>
-      <th style="text-align: left">Contents</th>
-    </tr>
-  </thead>
-  <tbody>
-    <tr>
-      <td style="text-align: left">DIRECT</td>
-      <td style="text-align: left">PRESENT</td>
-      <td style="text-align: left">Yes</td>
-      <td style="text-align: left">Boolean RLE</td>
-    </tr>
-    <tr>
-      <td style="text-align: left"> </td>
-      <td style="text-align: left">DATA</td>
-      <td style="text-align: left">No</td>
-      <td style="text-align: left">Byte RLE</td>
-    </tr>
-  </tbody>
-</table>
-
-<h2 id="binary-columns">Binary Columns</h2>
-
-<p>Binary data is encoded with a PRESENT stream, a DATA stream that records
-the contents, and a LENGTH stream that records the number of bytes per a
-value.</p>
-
-<table>
-  <thead>
-    <tr>
-      <th style="text-align: left">Encoding</th>
-      <th style="text-align: left">Stream Kind</th>
-      <th style="text-align: left">Optional</th>
-      <th style="text-align: left">Contents</th>
-    </tr>
-  </thead>
-  <tbody>
-    <tr>
-      <td style="text-align: left">DIRECT</td>
-      <td style="text-align: left">PRESENT</td>
-      <td style="text-align: left">Yes</td>
-      <td style="text-align: left">Boolean RLE</td>
-    </tr>
-    <tr>
-      <td style="text-align: left"> </td>
-      <td style="text-align: left">DATA</td>
-      <td style="text-align: left">No</td>
-      <td style="text-align: left">String contents</td>
-    </tr>
-    <tr>
-      <td style="text-align: left"> </td>
-      <td style="text-align: left">LENGTH</td>
-      <td style="text-align: left">No</td>
-      <td style="text-align: left">Unsigned Integer RLE v1</td>
-    </tr>
-  </tbody>
-</table>
-
-<h2 id="decimal-columns">Decimal Columns</h2>
-
-<p>Decimal was introduced in Hive 0.11 with infinite precision (the total
-number of digits). In Hive 0.13, the definition was change to limit
-the precision to a maximum of 38 digits, which conveniently uses 127
-bits plus a sign bit. The current encoding of decimal columns stores
-the integer representation of the value as an unbounded length zigzag
-encoded base 128 varint. The scale is stored in the SECONDARY stream
-as an signed integer.</p>
-
-<table>
-  <thead>
-    <tr>
-      <th style="text-align: left">Encoding</th>
-      <th style="text-align: left">Stream Kind</th>
-      <th style="text-align: left">Optional</th>
-      <th style="text-align: left">Contents</th>
-    </tr>
-  </thead>
-  <tbody>
-    <tr>
-      <td style="text-align: left">DIRECT</td>
-      <td style="text-align: left">PRESENT</td>
-      <td style="text-align: left">Yes</td>
-      <td style="text-align: left">Boolean RLE</td>
-    </tr>
-    <tr>
-      <td style="text-align: left"> </td>
-      <td style="text-align: left">DATA</td>
-      <td style="text-align: left">No</td>
-      <td style="text-align: left">Unbounded base 128 varints</td>
-    </tr>
-    <tr>
-      <td style="text-align: left"> </td>
-      <td style="text-align: left">SECONDARY</td>
-      <td style="text-align: left">No</td>
-      <td style="text-align: left">Unsigned Integer RLE v1</td>
-    </tr>
-  </tbody>
-</table>
-
-<h2 id="date-columns">Date Columns</h2>
-
-<p>Date data is encoded with a PRESENT stream, a DATA stream that records
-the number of days after January 1, 1970 in UTC.</p>
-
-<table>
-  <thead>
-    <tr>
-      <th style="text-align: left">Encoding</th>
-      <th style="text-align: left">Stream Kind</th>
-      <th style="text-align: left">Optional</th>
-      <th style="text-align: left">Contents</th>
-    </tr>
-  </thead>
-  <tbody>
-    <tr>
-      <td style="text-align: left">DIRECT</td>
-      <td style="text-align: left">PRESENT</td>
-      <td style="text-align: left">Yes</td>
-      <td style="text-align: left">Boolean RLE</td>
-    </tr>
-    <tr>
-      <td style="text-align: left"> </td>
-      <td style="text-align: left">DATA</td>
-      <td style="text-align: left">No</td>
-      <td style="text-align: left">Signed Integer RLE v1</td>
-    </tr>
-  </tbody>
-</table>
-
-<h2 id="timestamp-columns">Timestamp Columns</h2>
-
-<p>Timestamp records times down to nanoseconds as a PRESENT stream that
-records non-null values, a DATA stream that records the number of
-seconds after 1 January 2015, and a SECONDARY stream that records the
-number of nanoseconds.</p>
-
-<p>Because the number of nanoseconds often has a large number of trailing
-zeros, the number has trailing decimal zero digits removed and the
-last three bits are used to record how many zeros were removed. Thus
-1000 nanoseconds would be serialized as 0x0b and 100000 would be
-serialized as 0x0d.</p>
-
-<table>
-  <thead>
-    <tr>
-      <th style="text-align: left">Encoding</th>
-      <th style="text-align: left">Stream Kind</th>
-      <th style="text-align: left">Optional</th>
-      <th style="text-align: left">Contents</th>
-    </tr>
-  </thead>
-  <tbody>
-    <tr>
-      <td style="text-align: left">DIRECT</td>
-      <td style="text-align: left">PRESENT</td>
-      <td style="text-align: left">Yes</td>
-      <td style="text-align: left">Boolean RLE</td>
-    </tr>
-    <tr>
-      <td style="text-align: left"> </td>
-      <td style="text-align: left">DATA</td>
-      <td style="text-align: left">No</td>
-      <td style="text-align: left">Signed Integer RLE v1</td>
-    </tr>
-    <tr>
-      <td style="text-align: left"> </td>
-      <td style="text-align: left">SECONDARY</td>
-      <td style="text-align: left">No</td>
-      <td style="text-align: left">Unsigned Integer RLE v1</td>
-    </tr>
-  </tbody>
-</table>
-
-<h2 id="struct-columns">Struct Columns</h2>
-
-<p>Structs have no data themselves and delegate everything to their child
-columns except for their PRESENT stream. They have a child column
-for each of the fields.</p>
-
-<table>
-  <thead>
-    <tr>
-      <th style="text-align: left">Encoding</th>
-      <th style="text-align: left">Stream Kind</th>
-      <th style="text-align: left">Optional</th>
-      <th style="text-align: left">Contents</th>
-    </tr>
-  </thead>
-  <tbody>
-    <tr>
-      <td style="text-align: left">DIRECT</td>
-      <td style="text-align: left">PRESENT</td>
-      <td style="text-align: left">Yes</td>
-      <td style="text-align: left">Boolean RLE</td>
-    </tr>
-  </tbody>
-</table>
-
-<h2 id="list-columns">List Columns</h2>
-
-<p>Lists are encoded as the PRESENT stream and a length stream with
-number of items in each list. They have a single child column for the
-element values.</p>
-
-<table>
-  <thead>
-    <tr>
-      <th style="text-align: left">Encoding</th>
-      <th style="text-align: left">Stream Kind</th>
-      <th style="text-align: left">Optional</th>
-      <th style="text-align: left">Contents</th>
-    </tr>
-  </thead>
-  <tbody>
-    <tr>
-      <td style="text-align: left">DIRECT</td>
-      <td style="text-align: left">PRESENT</td>
-      <td style="text-align: left">Yes</td>
-      <td style="text-align: left">Boolean RLE</td>
-    </tr>
-    <tr>
-      <td style="text-align: left"> </td>
-      <td style="text-align: left">LENGTH</td>
-      <td style="text-align: left">No</td>
-      <td style="text-align: left">Unsigned Integer RLE v1</td>
-    </tr>
-  </tbody>
-</table>
-
-<h2 id="map-columns">Map Columns</h2>
-
-<p>Maps are encoded as the PRESENT stream and a length stream with number
-of items in each list. They have a child column for the key and
-another child column for the value.</p>
-
-<table>
-  <thead>
-    <tr>
-      <th style="text-align: left">Encoding</th>
-      <th style="text-align: left">Stream Kind</th>
-      <th style="text-align: left">Optional</th>
-      <th style="text-align: left">Contents</th>
-    </tr>
-  </thead>
-  <tbody>
-    <tr>
-      <td style="text-align: left">DIRECT</td>
-      <td style="text-align: left">PRESENT</td>
-      <td style="text-align: left">Yes</td>
-      <td style="text-align: left">Boolean RLE</td>
-    </tr>
-    <tr>
-      <td style="text-align: left"> </td>
-      <td style="text-align: left">LENGTH</td>
-      <td style="text-align: left">No</td>
-      <td style="text-align: left">Unsigned Integer RLE v1</td>
-    </tr>
-  </tbody>
-</table>
-
-<h2 id="union-columns">Union Columns</h2>
-
-<p>Unions are encoded as the PRESENT stream and a tag stream that controls which
-potential variant is used. They have a child column for each variant of the
-union. Currently ORC union types are limited to 256 variants, which matches
-the Hive type model.</p>
-
-<table>
-  <thead>
-    <tr>
-      <th style="text-align: left">Encoding</th>
-      <th style="text-align: left">Stream Kind</th>
-      <th style="text-align: left">Optional</th>
-      <th style="text-align: left">Contents</th>
-    </tr>
-  </thead>
-  <tbody>
-    <tr>
-      <td style="text-align: left">DIRECT</td>
-      <td style="text-align: left">PRESENT</td>
-      <td style="text-align: left">Yes</td>
-      <td style="text-align: left">Boolean RLE</td>
-    </tr>
-    <tr>
-      <td style="text-align: left"> </td>
-      <td style="text-align: left">DIRECT</td>
-      <td style="text-align: left">No</td>
-      <td style="text-align: left">Byte RLE</td>
-    </tr>
-  </tbody>
-</table>
-
-<h1 id="indexes">Indexes</h1>
-
-<h2 id="row-group-index">Row Group Index</h2>
-
-<p>The row group indexes consist of a ROW_INDEX stream for each primitive
-column that has an entry for each row group. Row groups are controlled
-by the writer and default to 10,000 rows. Each RowIndexEntry gives the
-position of each stream for the column and the statistics for that row
-group.</p>
-
-<p>The index streams are placed at the front of the stripe, because in
-the default case of streaming they do not need to be read. They are
-only loaded when either predicate push down is being used or the
-reader seeks to a particular row.</p>
-
-<p><code>message RowIndexEntry {
- repeated uint64 positions = 1 [packed=true];
- optional ColumnStatistics statistics = 2;
-}
-</code></p>
-
-<p><code>message RowIndex {
- repeated RowIndexEntry entry = 1;
-}
-</code></p>
-
-<p>To record positions, each stream needs a sequence of numbers. For
-uncompressed streams, the position is the byte offset of the RLE run’s
-start location followed by the number of values that need to be
-consumed from the run. In compressed streams, the first number is the
-start of the compression chunk in the stream, followed by the number
-of decompressed bytes that need to be consumed, and finally the number
-of values consumed in the RLE.</p>
-
-<p>For columns with multiple streams, the sequences of positions in each
-stream are concatenated. That was an unfortunate decision on my part
-that we should fix at some point, because it makes code that uses the
-indexes error-prone.</p>
-
-<p>Because dictionaries are accessed randomly, there is not a position to
-record for the dictionary and the entire dictionary must be read even
-if only part of a stripe is being read.</p>
-
-
-      </article>
-    </div>
-
-    <div class="clear"></div>
-
-  </div>
-</section>
-
-
-  <footer role="contentinfo">
-  <p>The contents of this website are &copy;&nbsp;2018
-     <a href="https://www.apache.org/">Apache Software Foundation</a>
-     under the terms of the <a
-      href="https://www.apache.org/licenses/LICENSE-2.0.html">
-      Apache&nbsp;License&nbsp;v2</a>. Apache ORC and its logo are trademarks
-      of the Apache Software Foundation.</p>
-</footer>
-
-  <script>
-  var anchorForId = function (id) {
-    var anchor = document.createElement("a");
-    anchor.className = "header-link";
-    anchor.href      = "#" + id;
-    anchor.innerHTML = "<span class=\"sr-only\">Permalink</span><i class=\"fa fa-link\"></i>";
-    anchor.title = "Permalink";
-    return anchor;
-  };
-
-  var linkifyAnchors = function (level, containingElement) {
-    var headers = containingElement.getElementsByTagName("h" + level);
-    for (var h = 0; h < headers.length; h++) {
-      var header = headers[h];
-
-      if (typeof header.id !== "undefined" && header.id !== "") {
-        header.appendChild(anchorForId(header.id));
-      }
-    }
-  };
-
-  document.onreadystatechange = function () {
-    if (this.readyState === "complete") {
-      var contentBlock = document.getElementsByClassName("docs")[0] || document.getElementsByClassName("news")[0];
-      if (!contentBlock) {
-        return;
-      }
-      for (var level = 1; level <= 6; level++) {
-        linkifyAnchors(level, contentBlock);
-      }
-    }
-  };
-</script>
-
-
-</body>
-</html>


[07/11] orc git commit: Deploy docs with CVE.

Posted by om...@apache.org.
http://git-wip-us.apache.org/repos/asf/orc/blob/e58d3fc4/docs/mapreduce.html
----------------------------------------------------------------------
diff --git a/docs/mapreduce.html b/docs/mapreduce.html
index 63fcd9c..162e3b7 100644
--- a/docs/mapreduce.html
+++ b/docs/mapreduce.html
@@ -4,7 +4,7 @@
   <meta charset="UTF-8">
   <title>Using in MapReduce</title>
   <meta name="viewport" content="width=device-width,initial-scale=1">
-  <meta name="generator" content="Jekyll v2.4.0">
+  <meta name="generator" content="Jekyll v3.7.3">
   <link rel="stylesheet" href="//fonts.googleapis.com/css?family=Lato:300,300italic,400,400italic,700,700italic,900">
   <link rel="stylesheet" href="/css/screen.css">
   <link rel="icon" type="image/x-icon" href="/favicon.ico">
@@ -109,7 +109,7 @@
     
   
     
-      <option value="/docs/index.html">Background</option>
+      <option value="http://localhost:4000/docs/index.html">Background</option>
     
   
     
@@ -131,7 +131,7 @@
     
   
     
-      <option value="/docs/adopters.html">ORC Adopters</option>
+      <option value="http://localhost:4000/docs/adopters.html">ORC Adopters</option>
     
   
     
@@ -193,7 +193,7 @@
     
   
     
-      <option value="/docs/types.html">Types</option>
+      <option value="http://localhost:4000/docs/types.html">Types</option>
     
   
 
@@ -219,7 +219,7 @@
     
   
     
-      <option value="/docs/indexes.html">Indexes</option>
+      <option value="http://localhost:4000/docs/indexes.html">Indexes</option>
     
   
     
@@ -237,7 +237,7 @@
 
   
     
-      <option value="/docs/acid.html">ACID support</option>
+      <option value="http://localhost:4000/docs/acid.html">ACID support</option>
     
   
     
@@ -284,7 +284,7 @@
     
   
     
-      <option value="/docs/building.html">Building ORC</option>
+      <option value="http://localhost:4000/docs/building.html">Building ORC</option>
     
   
     
@@ -342,7 +342,7 @@
     
   
     
-      <option value="/docs/releases.html">Releases</option>
+      <option value="http://localhost:4000/docs/releases.html">Releases</option>
     
   
     
@@ -373,7 +373,7 @@
     
   
     
-      <option value="/docs/hive-ddl.html">Hive DDL</option>
+      <option value="http://localhost:4000/docs/hive-ddl.html">Hive DDL</option>
     
   
     
@@ -407,7 +407,7 @@
     
   
     
-      <option value="/docs/hive-config.html">Hive Configuration</option>
+      <option value="http://localhost:4000/docs/hive-config.html">Hive Configuration</option>
     
   
     
@@ -460,7 +460,7 @@
     
   
     
-      <option value="/docs/mapred.html">Using in MapRed</option>
+      <option value="http://localhost:4000/docs/mapred.html">Using in MapRed</option>
     
   
     
@@ -498,7 +498,7 @@
     
   
     
-      <option value="/docs/mapreduce.html">Using in MapReduce</option>
+      <option value="http://localhost:4000/docs/mapreduce.html">Using in MapReduce</option>
     
   
     
@@ -525,7 +525,7 @@
     
   
     
-      <option value="/docs/core-java.html">Using Core Java</option>
+      <option value="http://localhost:4000/docs/core-java.html">Using Core Java</option>
     
   
     
@@ -559,7 +559,7 @@
     
   
     
-      <option value="/docs/core-cpp.html">Using Core C++</option>
+      <option value="http://localhost:4000/docs/core-cpp.html">Using Core C++</option>
     
   
     
@@ -606,7 +606,7 @@
     
   
     
-      <option value="/docs/cpp-tools.html">C++ Tools</option>
+      <option value="http://localhost:4000/docs/cpp-tools.html">C++ Tools</option>
     
   
     
@@ -652,7 +652,7 @@
     
   
     
-      <option value="/docs/java-tools.html">Java Tools</option>
+      <option value="http://localhost:4000/docs/java-tools.html">Java Tools</option>
     
   
     
@@ -676,31 +676,31 @@
           <h1>Using in MapReduce</h1>
           <p>This page describes how to read and write ORC files from Hadoop’s
 newer org.apache.hadoop.mapreduce MapReduce APIs. If you want to use the
-older org.apache.hadoop.mapred API, please look at the <a href="/docs/mapred.html">previous
+older org.apache.hadoop.mapred API, please look at the <a href="http://localhost:4000/docs/mapred.html">previous
 page</a>.</p>
 
 <h2 id="reading-orc-files">Reading ORC files</h2>
 
-<p>Add ORC and your desired version of Hadoop to your <code>pom.xml</code>:</p>
-
-<pre><code class="language-xml">&lt;dependencies&gt;
-  &lt;dependency&gt;
-    &lt;groupId&gt;org.apache.orc&lt;/groupId&gt;
-    &lt;artifactId&gt;orc-mapreduce&lt;/artifactId&gt;
-    &lt;version&gt;1.1.0&lt;/version&gt;
-  &lt;/dependency&gt;
-  &lt;dependency&gt;
-    &lt;groupId&gt;org.apache.hadoop&lt;/groupId&gt;
-    &lt;artifactId&gt;hadoop-mapreduce-client-core&lt;/artifactId&gt;
-    &lt;version&gt;2.7.0&lt;/version&gt;
-  &lt;/dependency&gt;
-&lt;/dependencies&gt;
-</code></pre>
+<p>Add ORC and your desired version of Hadoop to your <code class="highlighter-rouge">pom.xml</code>:</p>
+
+<div class="language-xml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nt">&lt;dependencies&gt;</span>
+  <span class="nt">&lt;dependency&gt;</span>
+    <span class="nt">&lt;groupId&gt;</span>org.apache.orc<span class="nt">&lt;/groupId&gt;</span>
+    <span class="nt">&lt;artifactId&gt;</span>orc-mapreduce<span class="nt">&lt;/artifactId&gt;</span>
+    <span class="nt">&lt;version&gt;</span>1.1.0<span class="nt">&lt;/version&gt;</span>
+  <span class="nt">&lt;/dependency&gt;</span>
+  <span class="nt">&lt;dependency&gt;</span>
+    <span class="nt">&lt;groupId&gt;</span>org.apache.hadoop<span class="nt">&lt;/groupId&gt;</span>
+    <span class="nt">&lt;artifactId&gt;</span>hadoop-mapreduce-client-core<span class="nt">&lt;/artifactId&gt;</span>
+    <span class="nt">&lt;version&gt;</span>2.7.0<span class="nt">&lt;/version&gt;</span>
+  <span class="nt">&lt;/dependency&gt;</span>
+<span class="nt">&lt;/dependencies&gt;</span>
+</code></pre></div></div>
 
 <p>Set the minimal properties in your JobConf:</p>
 
 <ul>
-  <li><strong>mapreduce.job.inputformat.class</strong> = <a href="/api/orc-mapreduce/index.html?org/apache/orc/mapreduce/OrcInputFormat.html">org.apache.orc.mapreduce.OrcInputFormat</a></li>
+  <li><strong>mapreduce.job.inputformat.class</strong> = <a href="http://localhost:4000/api/orc-mapreduce/index.html?org/apache/orc/mapreduce/OrcInputFormat.html">org.apache.orc.mapreduce.OrcInputFormat</a></li>
   <li><strong>mapreduce.input.fileinputformat.inputdir</strong> = your input directory</li>
 </ul>
 
@@ -723,7 +723,7 @@ the key and a value based on the table below expanded recursively.</p>
   <tbody>
     <tr>
       <td>array</td>
-      <td><a href="/api/orc-mapreduce/index.html?org/apache/orc/mapred/OrcStruct.html">org.apache.orc.mapred.OrcList</a></td>
+      <td><a href="http://localhost:4000/api/orc-mapreduce/index.html?org/apache/orc/mapred/OrcStruct.html">org.apache.orc.mapred.OrcList</a></td>
     </tr>
     <tr>
       <td>binary</td>
@@ -743,11 +743,11 @@ the key and a value based on the table below expanded recursively.</p>
     </tr>
     <tr>
       <td>date</td>
-      <td><a href="/api/hive-storage-api/index.html?org/apache/hadoop/hive/serde2/io/DateWritable.html">org.apache.hadoop.hive.serde2.io.DateWritable</a></td>
+      <td><a href="http://localhost:4000/api/hive-storage-api/index.html?org/apache/hadoop/hive/serde2/io/DateWritable.html">org.apache.hadoop.hive.serde2.io.DateWritable</a></td>
     </tr>
     <tr>
       <td>decimal</td>
-      <td><a href="/api/hive-storage-api/index.html?org/apache/hadoop/hive/serde2/io/HiveDecimalWritable.html">org.apache.hadoop.hive.serde2.io.HiveDecimalWritable</a></td>
+      <td><a href="http://localhost:4000/api/hive-storage-api/index.html?org/apache/hadoop/hive/serde2/io/HiveDecimalWritable.html">org.apache.hadoop.hive.serde2.io.HiveDecimalWritable</a></td>
     </tr>
     <tr>
       <td>double</td>
@@ -763,7 +763,7 @@ the key and a value based on the table below expanded recursively.</p>
     </tr>
     <tr>
       <td>map</td>
-      <td><a href="/api/orc-mapreduce/index.html?org/apache/orc/mapred/OrcMap.html">org.apache.orc.mapred.OrcMap</a></td>
+      <td><a href="http://localhost:4000/api/orc-mapreduce/index.html?org/apache/orc/mapred/OrcMap.html">org.apache.orc.mapred.OrcMap</a></td>
     </tr>
     <tr>
       <td>smallint</td>
@@ -775,11 +775,11 @@ the key and a value based on the table below expanded recursively.</p>
     </tr>
     <tr>
       <td>struct</td>
-      <td><a href="/api/orc-mapreduce/index.html?org/apache/orc/mapred/OrcStruct.html">org.apache.orc.mapred.OrcStruct</a></td>
+      <td><a href="http://localhost:4000/api/orc-mapreduce/index.html?org/apache/orc/mapred/OrcStruct.html">org.apache.orc.mapred.OrcStruct</a></td>
     </tr>
     <tr>
       <td>timestamp</td>
-      <td><a href="/api/orc-mapreduce/index.html?org/apache/orc/mapred/OrcTimestamp.html">org.apache.orc.mapred.OrcTimestamp</a></td>
+      <td><a href="http://localhost:4000/api/orc-mapreduce/index.html?org/apache/orc/mapred/OrcTimestamp.html">org.apache.orc.mapred.OrcTimestamp</a></td>
     </tr>
     <tr>
       <td>tinyint</td>
@@ -787,7 +787,7 @@ the key and a value based on the table below expanded recursively.</p>
     </tr>
     <tr>
       <td>uniontype</td>
-      <td><a href="/api/orc-mapreduce/index.html?org/apache/orc/mapred/OrcUnion.html">org.apache.orc.mapred.OrcUnion</a></td>
+      <td><a href="http://localhost:4000/api/orc-mapreduce/index.html?org/apache/orc/mapred/OrcUnion.html">org.apache.orc.mapred.OrcUnion</a></td>
     </tr>
     <tr>
       <td>varchar</td>
@@ -797,29 +797,29 @@ the key and a value based on the table below expanded recursively.</p>
 </table>
 
 <p>Let’s assume that your input directory contains ORC files with the
-schema <code>struct&lt;s:string,i:int&gt;</code> and you want to use the string field
+schema <code class="highlighter-rouge">struct&lt;s:string,i:int&gt;</code> and you want to use the string field
 as the key to the MapReduce shuffle and the integer as the value. The
 mapper code would look like:</p>
 
-<pre><code class="language-java">public static class MyMapper
-    extends Mapper&lt;NullWritable,OrcStruct,Text,IntWritable&gt; {
+<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">public</span> <span class="kd">static</span> <span class="kd">class</span> <span class="nc">MyMapper</span>
+    <span class="kd">extends</span> <span class="n">Mapper</span><span class="o">&lt;</span><span class="n">NullWritable</span><span class="o">,</span><span class="n">OrcStruct</span><span class="o">,</span><span class="n">Text</span><span class="o">,</span><span class="n">IntWritable</span><span class="o">&gt;</span> <span class="o">{</span>
 
-  // Assume the ORC file has type: struct&lt;s:string,i:int&gt;
-  public void map(NullWritable key, OrcStruct value,
-                  Context output) throws IOException, InterruptedException {
-    // take the first field as the key and the second field as the value
-    output.write((Text) value.getFieldValue(0),
-                 (IntWritable) value.getFieldValue(1));
-  }
-}
-</code></pre>
+  <span class="c1">// Assume the ORC file has type: struct&lt;s:string,i:int&gt;</span>
+  <span class="kd">public</span> <span class="kt">void</span> <span class="nf">map</span><span class="o">(</span><span class="n">NullWritable</span> <span class="n">key</span><span class="o">,</span> <span class="n">OrcStruct</span> <span class="n">value</span><span class="o">,</span>
+                  <span class="n">Context</span> <span class="n">output</span><span class="o">)</span> <span class="kd">throws</span> <span class="n">IOException</span><span class="o">,</span> <span class="n">InterruptedException</span> <span class="o">{</span>
+    <span class="c1">// take the first field as the key and the second field as the value</span>
+    <span class="n">output</span><span class="o">.</span><span class="na">write</span><span class="o">((</span><span class="n">Text</span><span class="o">)</span> <span class="n">value</span><span class="o">.</span><span class="na">getFieldValue</span><span class="o">(</span><span class="mi">0</span><span class="o">),</span>
+                 <span class="o">(</span><span class="n">IntWritable</span><span class="o">)</span> <span class="n">value</span><span class="o">.</span><span class="na">getFieldValue</span><span class="o">(</span><span class="mi">1</span><span class="o">));</span>
+  <span class="o">}</span>
+<span class="o">}</span>
+</code></pre></div></div>
 
 <h2 id="writing-orc-files">Writing ORC files</h2>
 
 <p>To write ORC files from your MapReduce job, you’ll need to set</p>
 
 <ul>
-  <li><strong>mapreduce.job.outputformat.class</strong> = <a href="/api/orc-mapreduce/index.html?org/apache/orc/mapreduce/OrcOutputFormat.html">org.apache.orc.mapreduce.OrcOutputFormat</a></li>
+  <li><strong>mapreduce.job.outputformat.class</strong> = <a href="http://localhost:4000/api/orc-mapreduce/index.html?org/apache/orc/mapreduce/OrcOutputFormat.html">org.apache.orc.mapreduce.OrcOutputFormat</a></li>
   <li><strong>mapreduce.output.fileoutputformat.outputdir</strong> = your output directory</li>
   <li><strong>orc.mapred.output.schema</strong> = the schema to write to the ORC file</li>
 </ul>
@@ -829,34 +829,34 @@ file and typically uses the OrcStruct.createValue(TypeDescription)
 function. For our example, let’s assume that the shuffle types are
 (Text, IntWritable) from the previous section and the reduce should
 gather the integer for each key together and write them as a list. The
-output schema would be <code>struct&lt;key:string,ints:array&lt;int&gt;&gt;</code>. As always
+output schema would be <code class="highlighter-rouge">struct&lt;key:string,ints:array&lt;int&gt;&gt;</code>. As always
 with MapReduce, if your method stores the values, you need to copy their
 value before getting the next.</p>
 
-<pre><code class="language-java">public static class MyReducer
-  extends Reducer&lt;Text,IntWritable,NullWritable,OrcStruct&gt; {
-
-  private TypeDescription schema =
-    TypeDescription.fromString("struct&lt;key:string,ints:array&lt;int&gt;&gt;");
-  // createValue creates the correct value type for the schema
-  private OrcStruct pair = (OrcStruct) OrcStruct.createValue(schema);
-  // get a handle to the list of ints
-  private OrcList&lt;IntWritable&gt; valueList =
-    (OrcList&lt;IntWritable&gt;) pair.getFieldValue(1);
-  private final NullWritable nada = NullWritable.get();
-
-  public void reduce(Text key, Iterable&lt;IntWritable&gt; values,
-                     Context output
-                     ) throws IOException, InterruptedException {
-    pair.setFieldValue(0, key);
-    valueList.clear();
-    for(IntWritable val: values) {
-      valueList.add(new IntWritable(val.get()));
-    }
-    output.write(nada, pair);
-  }
-}
-</code></pre>
+<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">public</span> <span class="kd">static</span> <span class="kd">class</span> <span class="nc">MyReducer</span>
+  <span class="kd">extends</span> <span class="n">Reducer</span><span class="o">&lt;</span><span class="n">Text</span><span class="o">,</span><span class="n">IntWritable</span><span class="o">,</span><span class="n">NullWritable</span><span class="o">,</span><span class="n">OrcStruct</span><span class="o">&gt;</span> <span class="o">{</span>
+
+  <span class="kd">private</span> <span class="n">TypeDescription</span> <span class="n">schema</span> <span class="o">=</span>
+    <span class="n">TypeDescription</span><span class="o">.</span><span class="na">fromString</span><span class="o">(</span><span class="s">"struct&lt;key:string,ints:array&lt;int&gt;&gt;"</span><span class="o">);</span>
+  <span class="c1">// createValue creates the correct value type for the schema</span>
+  <span class="kd">private</span> <span class="n">OrcStruct</span> <span class="n">pair</span> <span class="o">=</span> <span class="o">(</span><span class="n">OrcStruct</span><span class="o">)</span> <span class="n">OrcStruct</span><span class="o">.</span><span class="na">createValue</span><span class="o">(</span><span class="n">schema</span><span class="o">);</span>
+  <span class="c1">// get a handle to the list of ints</span>
+  <span class="kd">private</span> <span class="n">OrcList</span><span class="o">&lt;</span><span class="n">IntWritable</span><span class="o">&gt;</span> <span class="n">valueList</span> <span class="o">=</span>
+    <span class="o">(</span><span class="n">OrcList</span><span class="o">&lt;</span><span class="n">IntWritable</span><span class="o">&gt;)</span> <span class="n">pair</span><span class="o">.</span><span class="na">getFieldValue</span><span class="o">(</span><span class="mi">1</span><span class="o">);</span>
+  <span class="kd">private</span> <span class="kd">final</span> <span class="n">NullWritable</span> <span class="n">nada</span> <span class="o">=</span> <span class="n">NullWritable</span><span class="o">.</span><span class="na">get</span><span class="o">();</span>
+
+  <span class="kd">public</span> <span class="kt">void</span> <span class="nf">reduce</span><span class="o">(</span><span class="n">Text</span> <span class="n">key</span><span class="o">,</span> <span class="n">Iterable</span><span class="o">&lt;</span><span class="n">IntWritable</span><span class="o">&gt;</span> <span class="n">values</span><span class="o">,</span>
+                     <span class="n">Context</span> <span class="n">output</span>
+                     <span class="o">)</span> <span class="kd">throws</span> <span class="n">IOException</span><span class="o">,</span> <span class="n">InterruptedException</span> <span class="o">{</span>
+    <span class="n">pair</span><span class="o">.</span><span class="na">setFieldValue</span><span class="o">(</span><span class="mi">0</span><span class="o">,</span> <span class="n">key</span><span class="o">);</span>
+    <span class="n">valueList</span><span class="o">.</span><span class="na">clear</span><span class="o">();</span>
+    <span class="k">for</span><span class="o">(</span><span class="n">IntWritable</span> <span class="nl">val:</span> <span class="n">values</span><span class="o">)</span> <span class="o">{</span>
+      <span class="n">valueList</span><span class="o">.</span><span class="na">add</span><span class="o">(</span><span class="k">new</span> <span class="n">IntWritable</span><span class="o">(</span><span class="n">val</span><span class="o">.</span><span class="na">get</span><span class="o">()));</span>
+    <span class="o">}</span>
+    <span class="n">output</span><span class="o">.</span><span class="na">write</span><span class="o">(</span><span class="n">nada</span><span class="o">,</span> <span class="n">pair</span><span class="o">);</span>
+  <span class="o">}</span>
+<span class="o">}</span>
+</code></pre></div></div>
 
 <h2 id="sending-orcstruct-orclist-orcmap-or-orcunion-through-the-shuffle">Sending OrcStruct, OrcList, OrcMap, or OrcUnion through the Shuffle</h2>
 
@@ -865,9 +865,9 @@ MapReduce shuffle. The complex ORC types, since they are generic
 types, need to have their full type information provided to create the
 object. To enable MapReduce to properly instantiate the OrcStruct and
 other ORC types, we need to wrap it in either an
-<a href="/api/orc-mapreduce/index.html?org/apache/orc/mapred/OrcKey.html">OrcKey</a>
+<a href="http://localhost:4000/api/orc-mapreduce/index.html?org/apache/orc/mapred/OrcKey.html">OrcKey</a>
 for the shuffle key or
-<a href="/api/orc-mapreduce/index.html?org/apache/orc/mapred/OrcValue.html">OrcValue</a>
+<a href="http://localhost:4000/api/orc-mapreduce/index.html?org/apache/orc/mapred/OrcValue.html">OrcValue</a>
 for the shuffle value.</p>
 
 <p>To send two OrcStructs through the shuffle, define the following properties
@@ -884,27 +884,27 @@ in the JobConf:</p>
 respectively. These objects should be created once and reused as the mapper
 runs.</p>
 
-<pre><code class="language-java">public static class MyMapperShuffle
-    extends Mapper&lt;NullWritable,OrcStruct,OrcKey,OrcValue&gt; {
-  private OrcKey keyWrapper = new OrcKey();
-  private OrcValue valueWrapper = new OrcValue();
-  private OrcStruct outStruct = (OrcStruct) OrcStruct.createValue
-    (TypeDescription.fromString("struct&lt;i:int,j:int&gt;"));
-  private IntWritable i = (IntWritable) outStruct.getFieldValue("i");
-  private IntWritable j = (IntWritable) outStruct.getFieldValue("j");
-
-  // Assume the input has type: struct&lt;s:string,i:int&gt;
-  public void map(NullWritable key, OrcStruct value,
-                  Context output) throws IOException, InterruptedException {
-    keyWrapper.key = value;
-    valueWrapper.value = outStruct;
-    int val = ((IntWritable) value.getFieldValue("i")).get();
-    i.set(val * 2);
-    j.set(val * val);
-    output.write(keyWrapper, valueWrapper);
-  }
-}
-</code></pre>
+<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">public</span> <span class="kd">static</span> <span class="kd">class</span> <span class="nc">MyMapperShuffle</span>
+    <span class="kd">extends</span> <span class="n">Mapper</span><span class="o">&lt;</span><span class="n">NullWritable</span><span class="o">,</span><span class="n">OrcStruct</span><span class="o">,</span><span class="n">OrcKey</span><span class="o">,</span><span class="n">OrcValue</span><span class="o">&gt;</span> <span class="o">{</span>
+  <span class="kd">private</span> <span class="n">OrcKey</span> <span class="n">keyWrapper</span> <span class="o">=</span> <span class="k">new</span> <span class="n">OrcKey</span><span class="o">();</span>
+  <span class="kd">private</span> <span class="n">OrcValue</span> <span class="n">valueWrapper</span> <span class="o">=</span> <span class="k">new</span> <span class="n">OrcValue</span><span class="o">();</span>
+  <span class="kd">private</span> <span class="n">OrcStruct</span> <span class="n">outStruct</span> <span class="o">=</span> <span class="o">(</span><span class="n">OrcStruct</span><span class="o">)</span> <span class="n">OrcStruct</span><span class="o">.</span><span class="na">createValue</span>
+    <span class="o">(</span><span class="n">TypeDescription</span><span class="o">.</span><span class="na">fromString</span><span class="o">(</span><span class="s">"struct&lt;i:int,j:int&gt;"</span><span class="o">));</span>
+  <span class="kd">private</span> <span class="n">IntWritable</span> <span class="n">i</span> <span class="o">=</span> <span class="o">(</span><span class="n">IntWritable</span><span class="o">)</span> <span class="n">outStruct</span><span class="o">.</span><span class="na">getFieldValue</span><span class="o">(</span><span class="s">"i"</span><span class="o">);</span>
+  <span class="kd">private</span> <span class="n">IntWritable</span> <span class="n">j</span> <span class="o">=</span> <span class="o">(</span><span class="n">IntWritable</span><span class="o">)</span> <span class="n">outStruct</span><span class="o">.</span><span class="na">getFieldValue</span><span class="o">(</span><span class="s">"j"</span><span class="o">);</span>
+
+  <span class="c1">// Assume the input has type: struct&lt;s:string,i:int&gt;</span>
+  <span class="kd">public</span> <span class="kt">void</span> <span class="nf">map</span><span class="o">(</span><span class="n">NullWritable</span> <span class="n">key</span><span class="o">,</span> <span class="n">OrcStruct</span> <span class="n">value</span><span class="o">,</span>
+                  <span class="n">Context</span> <span class="n">output</span><span class="o">)</span> <span class="kd">throws</span> <span class="n">IOException</span><span class="o">,</span> <span class="n">InterruptedException</span> <span class="o">{</span>
+    <span class="n">keyWrapper</span><span class="o">.</span><span class="na">key</span> <span class="o">=</span> <span class="n">value</span><span class="o">;</span>
+    <span class="n">valueWrapper</span><span class="o">.</span><span class="na">value</span> <span class="o">=</span> <span class="n">outStruct</span><span class="o">;</span>
+    <span class="kt">int</span> <span class="n">val</span> <span class="o">=</span> <span class="o">((</span><span class="n">IntWritable</span><span class="o">)</span> <span class="n">value</span><span class="o">.</span><span class="na">getFieldValue</span><span class="o">(</span><span class="s">"i"</span><span class="o">)).</span><span class="na">get</span><span class="o">();</span>
+    <span class="n">i</span><span class="o">.</span><span class="na">set</span><span class="o">(</span><span class="n">val</span> <span class="o">*</span> <span class="mi">2</span><span class="o">);</span>
+    <span class="n">j</span><span class="o">.</span><span class="na">set</span><span class="o">(</span><span class="n">val</span> <span class="o">*</span> <span class="n">val</span><span class="o">);</span>
+    <span class="n">output</span><span class="o">.</span><span class="na">write</span><span class="o">(</span><span class="n">keyWrapper</span><span class="o">,</span> <span class="n">valueWrapper</span><span class="o">);</span>
+  <span class="o">}</span>
+<span class="o">}</span>
+</code></pre></div></div>
 
 <p>The reducer code accesses the underlying OrcStructs by using the
 OrcKey.key and OrcValue.value fields.</p>
@@ -980,8 +980,6 @@ OrcKey.key and OrcValue.value fields.</p>
   
 
   
-    
-  
 
   
     
@@ -1001,29 +999,25 @@ OrcKey.key and OrcValue.value fields.</p>
     
   
     
-      <li class=""><a href="/docs/index.html">Background</a></li>
+      <li class=""><a href="http://localhost:4000/docs/index.html">Background</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
   
     
-      <li class=""><a href="/docs/adopters.html">ORC Adopters</a></li>
+      <li class=""><a href="http://localhost:4000/docs/adopters.html">ORC Adopters</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -1055,15 +1049,13 @@ OrcKey.key and OrcValue.value fields.</p>
     
   
     
-      <li class=""><a href="/docs/types.html">Types</a></li>
+      <li class=""><a href="http://localhost:4000/docs/types.html">Types</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -1085,19 +1077,17 @@ OrcKey.key and OrcValue.value fields.</p>
     
   
     
-      <li class=""><a href="/docs/indexes.html">Indexes</a></li>
+      <li class=""><a href="http://localhost:4000/docs/indexes.html">Indexes</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
-      <li class=""><a href="/docs/acid.html">ACID support</a></li>
+      <li class=""><a href="http://localhost:4000/docs/acid.html">ACID support</a></li>
       
 
 
@@ -1112,8 +1102,6 @@ OrcKey.key and OrcValue.value fields.</p>
   
 
   
-    
-  
 
   
     
@@ -1121,15 +1109,13 @@ OrcKey.key and OrcValue.value fields.</p>
     
   
     
-      <li class=""><a href="/docs/building.html">Building ORC</a></li>
+      <li class=""><a href="http://localhost:4000/docs/building.html">Building ORC</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -1159,7 +1145,7 @@ OrcKey.key and OrcValue.value fields.</p>
     
   
     
-      <li class=""><a href="/docs/releases.html">Releases</a></li>
+      <li class=""><a href="http://localhost:4000/docs/releases.html">Releases</a></li>
       
 
 
@@ -1174,8 +1160,6 @@ OrcKey.key and OrcValue.value fields.</p>
   
 
   
-    
-  
 
   
     
@@ -1193,15 +1177,13 @@ OrcKey.key and OrcValue.value fields.</p>
     
   
     
-      <li class=""><a href="/docs/hive-ddl.html">Hive DDL</a></li>
+      <li class=""><a href="http://localhost:4000/docs/hive-ddl.html">Hive DDL</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -1217,7 +1199,7 @@ OrcKey.key and OrcValue.value fields.</p>
     
   
     
-      <li class=""><a href="/docs/hive-config.html">Hive Configuration</a></li>
+      <li class=""><a href="http://localhost:4000/docs/hive-config.html">Hive Configuration</a></li>
       
 
 
@@ -1232,8 +1214,6 @@ OrcKey.key and OrcValue.value fields.</p>
   
 
   
-    
-  
 
   
     
@@ -1259,15 +1239,13 @@ OrcKey.key and OrcValue.value fields.</p>
     
   
     
-      <li class=""><a href="/docs/mapred.html">Using in MapRed</a></li>
+      <li class=""><a href="http://localhost:4000/docs/mapred.html">Using in MapRed</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -1295,7 +1273,7 @@ OrcKey.key and OrcValue.value fields.</p>
     
   
     
-      <li class="current"><a href="/docs/mapreduce.html">Using in MapReduce</a></li>
+      <li class="current"><a href="http://localhost:4000/docs/mapreduce.html">Using in MapReduce</a></li>
       
 
 
@@ -1310,8 +1288,6 @@ OrcKey.key and OrcValue.value fields.</p>
   
 
   
-    
-  
 
   
     
@@ -1323,15 +1299,13 @@ OrcKey.key and OrcValue.value fields.</p>
     
   
     
-      <li class=""><a href="/docs/core-java.html">Using Core Java</a></li>
+      <li class=""><a href="http://localhost:4000/docs/core-java.html">Using Core Java</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -1341,7 +1315,7 @@ OrcKey.key and OrcValue.value fields.</p>
     
   
     
-      <li class=""><a href="/docs/core-cpp.html">Using Core C++</a></li>
+      <li class=""><a href="http://localhost:4000/docs/core-cpp.html">Using Core C++</a></li>
       
 
 
@@ -1356,8 +1330,6 @@ OrcKey.key and OrcValue.value fields.</p>
   
 
   
-    
-  
 
   
     
@@ -1371,15 +1343,13 @@ OrcKey.key and OrcValue.value fields.</p>
     
   
     
-      <li class=""><a href="/docs/cpp-tools.html">C++ Tools</a></li>
+      <li class=""><a href="http://localhost:4000/docs/cpp-tools.html">C++ Tools</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -1403,7 +1373,7 @@ OrcKey.key and OrcValue.value fields.</p>
     
   
     
-      <li class=""><a href="/docs/java-tools.html">Java Tools</a></li>
+      <li class=""><a href="http://localhost:4000/docs/java-tools.html">Java Tools</a></li>
       
 
 

http://git-wip-us.apache.org/repos/asf/orc/blob/e58d3fc4/docs/releases.html
----------------------------------------------------------------------
diff --git a/docs/releases.html b/docs/releases.html
index f168123..2c44da8 100644
--- a/docs/releases.html
+++ b/docs/releases.html
@@ -4,7 +4,7 @@
   <meta charset="UTF-8">
   <title>Releases</title>
   <meta name="viewport" content="width=device-width,initial-scale=1">
-  <meta name="generator" content="Jekyll v2.4.0">
+  <meta name="generator" content="Jekyll v3.7.3">
   <link rel="stylesheet" href="//fonts.googleapis.com/css?family=Lato:300,300italic,400,400italic,700,700italic,900">
   <link rel="stylesheet" href="/css/screen.css">
   <link rel="icon" type="image/x-icon" href="/favicon.ico">
@@ -109,7 +109,7 @@
     
   
     
-      <option value="/docs/index.html">Background</option>
+      <option value="http://localhost:4000/docs/index.html">Background</option>
     
   
     
@@ -131,7 +131,7 @@
     
   
     
-      <option value="/docs/adopters.html">ORC Adopters</option>
+      <option value="http://localhost:4000/docs/adopters.html">ORC Adopters</option>
     
   
     
@@ -193,7 +193,7 @@
     
   
     
-      <option value="/docs/types.html">Types</option>
+      <option value="http://localhost:4000/docs/types.html">Types</option>
     
   
 
@@ -219,7 +219,7 @@
     
   
     
-      <option value="/docs/indexes.html">Indexes</option>
+      <option value="http://localhost:4000/docs/indexes.html">Indexes</option>
     
   
     
@@ -237,7 +237,7 @@
 
   
     
-      <option value="/docs/acid.html">ACID support</option>
+      <option value="http://localhost:4000/docs/acid.html">ACID support</option>
     
   
     
@@ -284,7 +284,7 @@
     
   
     
-      <option value="/docs/building.html">Building ORC</option>
+      <option value="http://localhost:4000/docs/building.html">Building ORC</option>
     
   
     
@@ -342,7 +342,7 @@
     
   
     
-      <option value="/docs/releases.html">Releases</option>
+      <option value="http://localhost:4000/docs/releases.html">Releases</option>
     
   
     
@@ -373,7 +373,7 @@
     
   
     
-      <option value="/docs/hive-ddl.html">Hive DDL</option>
+      <option value="http://localhost:4000/docs/hive-ddl.html">Hive DDL</option>
     
   
     
@@ -407,7 +407,7 @@
     
   
     
-      <option value="/docs/hive-config.html">Hive Configuration</option>
+      <option value="http://localhost:4000/docs/hive-config.html">Hive Configuration</option>
     
   
     
@@ -460,7 +460,7 @@
     
   
     
-      <option value="/docs/mapred.html">Using in MapRed</option>
+      <option value="http://localhost:4000/docs/mapred.html">Using in MapRed</option>
     
   
     
@@ -498,7 +498,7 @@
     
   
     
-      <option value="/docs/mapreduce.html">Using in MapReduce</option>
+      <option value="http://localhost:4000/docs/mapreduce.html">Using in MapReduce</option>
     
   
     
@@ -525,7 +525,7 @@
     
   
     
-      <option value="/docs/core-java.html">Using Core Java</option>
+      <option value="http://localhost:4000/docs/core-java.html">Using Core Java</option>
     
   
     
@@ -559,7 +559,7 @@
     
   
     
-      <option value="/docs/core-cpp.html">Using Core C++</option>
+      <option value="http://localhost:4000/docs/core-cpp.html">Using Core C++</option>
     
   
     
@@ -606,7 +606,7 @@
     
   
     
-      <option value="/docs/cpp-tools.html">C++ Tools</option>
+      <option value="http://localhost:4000/docs/cpp-tools.html">C++ Tools</option>
     
   
     
@@ -652,7 +652,7 @@
     
   
     
-      <option value="/docs/java-tools.html">Java Tools</option>
+      <option value="http://localhost:4000/docs/java-tools.html">Java Tools</option>
     
   
     
@@ -712,10 +712,10 @@ signed by <a href="https://dist.apache.org/repos/dist/release/orc/KEYS">Owen O
 <p>All GPG signatures should be verified as matching one of the keys in ORC’s
 committers’ <a href="https://dist.apache.org/repos/dist/release/orc/KEYS">key list</a>.</p>
 
-<pre><code class="language-shell">% shasum -a 256 orc-X.Y.Z.tgz | diff - orc-X.Y.Z.tgz.sha256
-% gpg --import KEYS
-% gpg --verify orc-X.Y.Z.tgz.asc
-</code></pre>
+<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>% shasum <span class="nt">-a</span> 256 orc-X.Y.Z.tgz | diff - orc-X.Y.Z.tgz.sha256
+% gpg <span class="nt">--import</span> KEYS
+% gpg <span class="nt">--verify</span> orc-X.Y.Z.tgz.asc
+</code></pre></div></div>
 
 <h2 id="all-releases">All releases:</h2>
 
@@ -983,8 +983,6 @@ committers’ <a href="https://dist.apache.org/repos/dist/release/orc/KEYS">key
   
 
   
-    
-  
 
   
     
@@ -1004,29 +1002,25 @@ committers’ <a href="https://dist.apache.org/repos/dist/release/orc/KEYS">key
     
   
     
-      <li class=""><a href="/docs/index.html">Background</a></li>
+      <li class=""><a href="http://localhost:4000/docs/index.html">Background</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
   
     
-      <li class=""><a href="/docs/adopters.html">ORC Adopters</a></li>
+      <li class=""><a href="http://localhost:4000/docs/adopters.html">ORC Adopters</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -1058,15 +1052,13 @@ committers’ <a href="https://dist.apache.org/repos/dist/release/orc/KEYS">key
     
   
     
-      <li class=""><a href="/docs/types.html">Types</a></li>
+      <li class=""><a href="http://localhost:4000/docs/types.html">Types</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -1088,19 +1080,17 @@ committers’ <a href="https://dist.apache.org/repos/dist/release/orc/KEYS">key
     
   
     
-      <li class=""><a href="/docs/indexes.html">Indexes</a></li>
+      <li class=""><a href="http://localhost:4000/docs/indexes.html">Indexes</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
-      <li class=""><a href="/docs/acid.html">ACID support</a></li>
+      <li class=""><a href="http://localhost:4000/docs/acid.html">ACID support</a></li>
       
 
 
@@ -1115,8 +1105,6 @@ committers’ <a href="https://dist.apache.org/repos/dist/release/orc/KEYS">key
   
 
   
-    
-  
 
   
     
@@ -1124,15 +1112,13 @@ committers’ <a href="https://dist.apache.org/repos/dist/release/orc/KEYS">key
     
   
     
-      <li class=""><a href="/docs/building.html">Building ORC</a></li>
+      <li class=""><a href="http://localhost:4000/docs/building.html">Building ORC</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -1162,7 +1148,7 @@ committers’ <a href="https://dist.apache.org/repos/dist/release/orc/KEYS">key
     
   
     
-      <li class="current"><a href="/docs/releases.html">Releases</a></li>
+      <li class="current"><a href="http://localhost:4000/docs/releases.html">Releases</a></li>
       
 
 
@@ -1177,8 +1163,6 @@ committers’ <a href="https://dist.apache.org/repos/dist/release/orc/KEYS">key
   
 
   
-    
-  
 
   
     
@@ -1196,15 +1180,13 @@ committers’ <a href="https://dist.apache.org/repos/dist/release/orc/KEYS">key
     
   
     
-      <li class=""><a href="/docs/hive-ddl.html">Hive DDL</a></li>
+      <li class=""><a href="http://localhost:4000/docs/hive-ddl.html">Hive DDL</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -1220,7 +1202,7 @@ committers’ <a href="https://dist.apache.org/repos/dist/release/orc/KEYS">key
     
   
     
-      <li class=""><a href="/docs/hive-config.html">Hive Configuration</a></li>
+      <li class=""><a href="http://localhost:4000/docs/hive-config.html">Hive Configuration</a></li>
       
 
 
@@ -1235,8 +1217,6 @@ committers’ <a href="https://dist.apache.org/repos/dist/release/orc/KEYS">key
   
 
   
-    
-  
 
   
     
@@ -1262,15 +1242,13 @@ committers’ <a href="https://dist.apache.org/repos/dist/release/orc/KEYS">key
     
   
     
-      <li class=""><a href="/docs/mapred.html">Using in MapRed</a></li>
+      <li class=""><a href="http://localhost:4000/docs/mapred.html">Using in MapRed</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -1298,7 +1276,7 @@ committers’ <a href="https://dist.apache.org/repos/dist/release/orc/KEYS">key
     
   
     
-      <li class=""><a href="/docs/mapreduce.html">Using in MapReduce</a></li>
+      <li class=""><a href="http://localhost:4000/docs/mapreduce.html">Using in MapReduce</a></li>
       
 
 
@@ -1313,8 +1291,6 @@ committers’ <a href="https://dist.apache.org/repos/dist/release/orc/KEYS">key
   
 
   
-    
-  
 
   
     
@@ -1326,15 +1302,13 @@ committers’ <a href="https://dist.apache.org/repos/dist/release/orc/KEYS">key
     
   
     
-      <li class=""><a href="/docs/core-java.html">Using Core Java</a></li>
+      <li class=""><a href="http://localhost:4000/docs/core-java.html">Using Core Java</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -1344,7 +1318,7 @@ committers’ <a href="https://dist.apache.org/repos/dist/release/orc/KEYS">key
     
   
     
-      <li class=""><a href="/docs/core-cpp.html">Using Core C++</a></li>
+      <li class=""><a href="http://localhost:4000/docs/core-cpp.html">Using Core C++</a></li>
       
 
 
@@ -1359,8 +1333,6 @@ committers’ <a href="https://dist.apache.org/repos/dist/release/orc/KEYS">key
   
 
   
-    
-  
 
   
     
@@ -1374,15 +1346,13 @@ committers’ <a href="https://dist.apache.org/repos/dist/release/orc/KEYS">key
     
   
     
-      <li class=""><a href="/docs/cpp-tools.html">C++ Tools</a></li>
+      <li class=""><a href="http://localhost:4000/docs/cpp-tools.html">C++ Tools</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -1406,7 +1376,7 @@ committers’ <a href="https://dist.apache.org/repos/dist/release/orc/KEYS">key
     
   
     
-      <li class=""><a href="/docs/java-tools.html">Java Tools</a></li>
+      <li class=""><a href="http://localhost:4000/docs/java-tools.html">Java Tools</a></li>
       
 
 

http://git-wip-us.apache.org/repos/asf/orc/blob/e58d3fc4/docs/types.html
----------------------------------------------------------------------
diff --git a/docs/types.html b/docs/types.html
index 149fa88..9538981 100644
--- a/docs/types.html
+++ b/docs/types.html
@@ -4,7 +4,7 @@
   <meta charset="UTF-8">
   <title>Types</title>
   <meta name="viewport" content="width=device-width,initial-scale=1">
-  <meta name="generator" content="Jekyll v2.4.0">
+  <meta name="generator" content="Jekyll v3.7.3">
   <link rel="stylesheet" href="//fonts.googleapis.com/css?family=Lato:300,300italic,400,400italic,700,700italic,900">
   <link rel="stylesheet" href="/css/screen.css">
   <link rel="icon" type="image/x-icon" href="/favicon.ico">
@@ -109,7 +109,7 @@
     
   
     
-      <option value="/docs/index.html">Background</option>
+      <option value="http://localhost:4000/docs/index.html">Background</option>
     
   
     
@@ -131,7 +131,7 @@
     
   
     
-      <option value="/docs/adopters.html">ORC Adopters</option>
+      <option value="http://localhost:4000/docs/adopters.html">ORC Adopters</option>
     
   
     
@@ -193,7 +193,7 @@
     
   
     
-      <option value="/docs/types.html">Types</option>
+      <option value="http://localhost:4000/docs/types.html">Types</option>
     
   
 
@@ -219,7 +219,7 @@
     
   
     
-      <option value="/docs/indexes.html">Indexes</option>
+      <option value="http://localhost:4000/docs/indexes.html">Indexes</option>
     
   
     
@@ -237,7 +237,7 @@
 
   
     
-      <option value="/docs/acid.html">ACID support</option>
+      <option value="http://localhost:4000/docs/acid.html">ACID support</option>
     
   
     
@@ -284,7 +284,7 @@
     
   
     
-      <option value="/docs/building.html">Building ORC</option>
+      <option value="http://localhost:4000/docs/building.html">Building ORC</option>
     
   
     
@@ -342,7 +342,7 @@
     
   
     
-      <option value="/docs/releases.html">Releases</option>
+      <option value="http://localhost:4000/docs/releases.html">Releases</option>
     
   
     
@@ -373,7 +373,7 @@
     
   
     
-      <option value="/docs/hive-ddl.html">Hive DDL</option>
+      <option value="http://localhost:4000/docs/hive-ddl.html">Hive DDL</option>
     
   
     
@@ -407,7 +407,7 @@
     
   
     
-      <option value="/docs/hive-config.html">Hive Configuration</option>
+      <option value="http://localhost:4000/docs/hive-config.html">Hive Configuration</option>
     
   
     
@@ -460,7 +460,7 @@
     
   
     
-      <option value="/docs/mapred.html">Using in MapRed</option>
+      <option value="http://localhost:4000/docs/mapred.html">Using in MapRed</option>
     
   
     
@@ -498,7 +498,7 @@
     
   
     
-      <option value="/docs/mapreduce.html">Using in MapReduce</option>
+      <option value="http://localhost:4000/docs/mapreduce.html">Using in MapReduce</option>
     
   
     
@@ -525,7 +525,7 @@
     
   
     
-      <option value="/docs/core-java.html">Using Core Java</option>
+      <option value="http://localhost:4000/docs/core-java.html">Using Core Java</option>
     
   
     
@@ -559,7 +559,7 @@
     
   
     
-      <option value="/docs/core-cpp.html">Using Core C++</option>
+      <option value="http://localhost:4000/docs/core-cpp.html">Using Core C++</option>
     
   
     
@@ -606,7 +606,7 @@
     
   
     
-      <option value="/docs/cpp-tools.html">C++ Tools</option>
+      <option value="http://localhost:4000/docs/cpp-tools.html">C++ Tools</option>
     
   
     
@@ -652,7 +652,7 @@
     
   
     
-      <option value="/docs/java-tools.html">Java Tools</option>
+      <option value="http://localhost:4000/docs/java-tools.html">Java Tools</option>
     
   
     
@@ -740,7 +740,7 @@ columns have one child column for each of the variants.</p>
 <p>Given the following definition of the table Foobar, the columns in the
 file would form the given tree.</p>
 
-<p><code>create table Foobar (
+<p><code class="highlighter-rouge">create table Foobar (
  myInt int,
  myMap map&lt;string,
  struct&lt;myString : string,
@@ -749,7 +749,7 @@ file would form the given tree.</p>
 );
 </code></p>
 
-<p><img src="/img/TreeWriters.png" alt="ORC column structure" /></p>
+<p><img src="http://localhost:4000/img/TreeWriters.png" alt="ORC column structure" /></p>
 
 
           
@@ -799,8 +799,6 @@ file would form the given tree.</p>
   
 
   
-    
-  
 
   
     
@@ -820,29 +818,25 @@ file would form the given tree.</p>
     
   
     
-      <li class=""><a href="/docs/index.html">Background</a></li>
+      <li class=""><a href="http://localhost:4000/docs/index.html">Background</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
   
     
-      <li class=""><a href="/docs/adopters.html">ORC Adopters</a></li>
+      <li class=""><a href="http://localhost:4000/docs/adopters.html">ORC Adopters</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -874,15 +868,13 @@ file would form the given tree.</p>
     
   
     
-      <li class="current"><a href="/docs/types.html">Types</a></li>
+      <li class="current"><a href="http://localhost:4000/docs/types.html">Types</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -904,19 +896,17 @@ file would form the given tree.</p>
     
   
     
-      <li class=""><a href="/docs/indexes.html">Indexes</a></li>
+      <li class=""><a href="http://localhost:4000/docs/indexes.html">Indexes</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
-      <li class=""><a href="/docs/acid.html">ACID support</a></li>
+      <li class=""><a href="http://localhost:4000/docs/acid.html">ACID support</a></li>
       
 
 
@@ -931,8 +921,6 @@ file would form the given tree.</p>
   
 
   
-    
-  
 
   
     
@@ -940,15 +928,13 @@ file would form the given tree.</p>
     
   
     
-      <li class=""><a href="/docs/building.html">Building ORC</a></li>
+      <li class=""><a href="http://localhost:4000/docs/building.html">Building ORC</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -978,7 +964,7 @@ file would form the given tree.</p>
     
   
     
-      <li class=""><a href="/docs/releases.html">Releases</a></li>
+      <li class=""><a href="http://localhost:4000/docs/releases.html">Releases</a></li>
       
 
 
@@ -993,8 +979,6 @@ file would form the given tree.</p>
   
 
   
-    
-  
 
   
     
@@ -1012,15 +996,13 @@ file would form the given tree.</p>
     
   
     
-      <li class=""><a href="/docs/hive-ddl.html">Hive DDL</a></li>
+      <li class=""><a href="http://localhost:4000/docs/hive-ddl.html">Hive DDL</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -1036,7 +1018,7 @@ file would form the given tree.</p>
     
   
     
-      <li class=""><a href="/docs/hive-config.html">Hive Configuration</a></li>
+      <li class=""><a href="http://localhost:4000/docs/hive-config.html">Hive Configuration</a></li>
       
 
 
@@ -1051,8 +1033,6 @@ file would form the given tree.</p>
   
 
   
-    
-  
 
   
     
@@ -1078,15 +1058,13 @@ file would form the given tree.</p>
     
   
     
-      <li class=""><a href="/docs/mapred.html">Using in MapRed</a></li>
+      <li class=""><a href="http://localhost:4000/docs/mapred.html">Using in MapRed</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -1114,7 +1092,7 @@ file would form the given tree.</p>
     
   
     
-      <li class=""><a href="/docs/mapreduce.html">Using in MapReduce</a></li>
+      <li class=""><a href="http://localhost:4000/docs/mapreduce.html">Using in MapReduce</a></li>
       
 
 
@@ -1129,8 +1107,6 @@ file would form the given tree.</p>
   
 
   
-    
-  
 
   
     
@@ -1142,15 +1118,13 @@ file would form the given tree.</p>
     
   
     
-      <li class=""><a href="/docs/core-java.html">Using Core Java</a></li>
+      <li class=""><a href="http://localhost:4000/docs/core-java.html">Using Core Java</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -1160,7 +1134,7 @@ file would form the given tree.</p>
     
   
     
-      <li class=""><a href="/docs/core-cpp.html">Using Core C++</a></li>
+      <li class=""><a href="http://localhost:4000/docs/core-cpp.html">Using Core C++</a></li>
       
 
 
@@ -1175,8 +1149,6 @@ file would form the given tree.</p>
   
 
   
-    
-  
 
   
     
@@ -1190,15 +1162,13 @@ file would form the given tree.</p>
     
   
     
-      <li class=""><a href="/docs/cpp-tools.html">C++ Tools</a></li>
+      <li class=""><a href="http://localhost:4000/docs/cpp-tools.html">C++ Tools</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -1222,7 +1192,7 @@ file would form the given tree.</p>
     
   
     
-      <li class=""><a href="/docs/java-tools.html">Java Tools</a></li>
+      <li class=""><a href="http://localhost:4000/docs/java-tools.html">Java Tools</a></li>
       
 
 

http://git-wip-us.apache.org/repos/asf/orc/blob/e58d3fc4/help/index.html
----------------------------------------------------------------------
diff --git a/help/index.html b/help/index.html
index d0cffcb..5c5ee74 100644
--- a/help/index.html
+++ b/help/index.html
@@ -4,7 +4,7 @@
   <meta charset="UTF-8">
   <title>Getting Help</title>
   <meta name="viewport" content="width=device-width,initial-scale=1">
-  <meta name="generator" content="Jekyll v2.4.0">
+  <meta name="generator" content="Jekyll v3.7.3">
   <link rel="stylesheet" href="//fonts.googleapis.com/css?family=Lato:300,300italic,400,400italic,700,700italic,900">
   <link rel="stylesheet" href="/css/screen.css">
   <link rel="icon" type="image/x-icon" href="/favicon.ico">
@@ -89,11 +89,11 @@
 <h2 id="mailing-lists">Mailing Lists</h2>
 
 <p>The best option is to send email to the user list
-<a href="&#109;&#097;&#105;&#108;&#116;&#111;:&#117;&#115;&#101;&#114;&#064;&#111;&#114;&#099;&#046;&#097;&#112;&#097;&#099;&#104;&#101;&#046;&#111;&#114;&#103;">&#117;&#115;&#101;&#114;&#064;&#111;&#114;&#099;&#046;&#097;&#112;&#097;&#099;&#104;&#101;&#046;&#111;&#114;&#103;</a>, which is for user
+<a href="mailto:user@orc.apache.org">user@orc.apache.org</a>, which is for user
 discussions. All of the historic traffic is available in the
 <a href="https://mail-archives.apache.org/mod_mbox/orc-user/">archive</a>. To
 subscribe to the user list, please send email to
-<a href="&#109;&#097;&#105;&#108;&#116;&#111;:&#117;&#115;&#101;&#114;&#045;&#115;&#117;&#098;&#115;&#099;&#114;&#105;&#098;&#101;&#064;&#111;&#114;&#099;&#046;&#097;&#112;&#097;&#099;&#104;&#101;&#046;&#111;&#114;&#103;">&#117;&#115;&#101;&#114;&#045;&#115;&#117;&#098;&#115;&#099;&#114;&#105;&#098;&#101;&#064;&#111;&#114;&#099;&#046;&#097;&#112;&#097;&#099;&#104;&#101;&#046;&#111;&#114;&#103;</a>.</p>
+<a href="mailto:user-subscribe@orc.apache.org">user-subscribe@orc.apache.org</a>.</p>
 
 <h2 id="bug-reports">Bug Reports</h2>
 

http://git-wip-us.apache.org/repos/asf/orc/blob/e58d3fc4/index.html
----------------------------------------------------------------------
diff --git a/index.html b/index.html
index 4968736..1b368ed 100644
--- a/index.html
+++ b/index.html
@@ -4,7 +4,7 @@
   <meta charset="UTF-8">
   <title>Apache ORC &bull; High-Performance Columnar Storage for Hadoop</title>
   <meta name="viewport" content="width=device-width,initial-scale=1">
-  <meta name="generator" content="Jekyll v2.4.0">
+  <meta name="generator" content="Jekyll v3.7.3">
   <link rel="stylesheet" href="//fonts.googleapis.com/css?family=Lato:300,300italic,400,400italic,700,700italic,900">
   <link rel="stylesheet" href="/css/screen.css">
   <link rel="icon" type="image/x-icon" href="/favicon.ico">

http://git-wip-us.apache.org/repos/asf/orc/blob/e58d3fc4/news/2015/04/22/orc-as-tlp/index.html
----------------------------------------------------------------------
diff --git a/news/2015/04/22/orc-as-tlp/index.html b/news/2015/04/22/orc-as-tlp/index.html
index c1f0977..221122c 100644
--- a/news/2015/04/22/orc-as-tlp/index.html
+++ b/news/2015/04/22/orc-as-tlp/index.html
@@ -4,7 +4,7 @@
   <meta charset="UTF-8">
   <title>ORC becomes an Apache Top Level Project</title>
   <meta name="viewport" content="width=device-width,initial-scale=1">
-  <meta name="generator" content="Jekyll v2.4.0">
+  <meta name="generator" content="Jekyll v3.7.3">
   <link rel="stylesheet" href="//fonts.googleapis.com/css?family=Lato:300,300italic,400,400italic,700,700italic,900">
   <link rel="stylesheet" href="/css/screen.css">
   <link rel="icon" type="image/x-icon" href="/favicon.ico">

http://git-wip-us.apache.org/repos/asf/orc/blob/e58d3fc4/news/2015/05/11/new-committers/index.html
----------------------------------------------------------------------
diff --git a/news/2015/05/11/new-committers/index.html b/news/2015/05/11/new-committers/index.html
index c3fa6af..977b30f 100644
--- a/news/2015/05/11/new-committers/index.html
+++ b/news/2015/05/11/new-committers/index.html
@@ -4,7 +4,7 @@
   <meta charset="UTF-8">
   <title>ORC adds 7 committers</title>
   <meta name="viewport" content="width=device-width,initial-scale=1">
-  <meta name="generator" content="Jekyll v2.4.0">
+  <meta name="generator" content="Jekyll v3.7.3">
   <link rel="stylesheet" href="//fonts.googleapis.com/css?family=Lato:300,300italic,400,400italic,700,700italic,900">
   <link rel="stylesheet" href="/css/screen.css">
   <link rel="icon" type="image/x-icon" href="/favicon.ico">

http://git-wip-us.apache.org/repos/asf/orc/blob/e58d3fc4/news/2015/06/26/new-logo/index.html
----------------------------------------------------------------------
diff --git a/news/2015/06/26/new-logo/index.html b/news/2015/06/26/new-logo/index.html
index dca975d..e731556 100644
--- a/news/2015/06/26/new-logo/index.html
+++ b/news/2015/06/26/new-logo/index.html
@@ -4,7 +4,7 @@
   <meta charset="UTF-8">
   <title>ORC adopts new logo</title>
   <meta name="viewport" content="width=device-width,initial-scale=1">
-  <meta name="generator" content="Jekyll v2.4.0">
+  <meta name="generator" content="Jekyll v3.7.3">
   <link rel="stylesheet" href="//fonts.googleapis.com/css?family=Lato:300,300italic,400,400italic,700,700italic,900">
   <link rel="stylesheet" href="/css/screen.css">
   <link rel="icon" type="image/x-icon" href="/favicon.ico">
@@ -173,7 +173,7 @@
   <div class="post-content">
     <p>The ORC project has adopted a new logo. We hope you like it.</p>
 
-<p><img src="/img/logo.png" alt="orc logo" title="orc logo" /></p>
+<p><img src="http://localhost:4000/img/logo.png" alt="orc logo" title="orc logo" /></p>
 
 <p>Other great options included a big white hand on a black shield. <em>smile</em></p>
 

http://git-wip-us.apache.org/repos/asf/orc/blob/e58d3fc4/news/2015/11/19/aliaksei-on-pmc/index.html
----------------------------------------------------------------------
diff --git a/news/2015/11/19/aliaksei-on-pmc/index.html b/news/2015/11/19/aliaksei-on-pmc/index.html
index 22bfd17..ab9d6d7 100644
--- a/news/2015/11/19/aliaksei-on-pmc/index.html
+++ b/news/2015/11/19/aliaksei-on-pmc/index.html
@@ -4,7 +4,7 @@
   <meta charset="UTF-8">
   <title>ORC adds Aliaksei Sandryhaila to PMC</title>
   <meta name="viewport" content="width=device-width,initial-scale=1">
-  <meta name="generator" content="Jekyll v2.4.0">
+  <meta name="generator" content="Jekyll v3.7.3">
   <link rel="stylesheet" href="//fonts.googleapis.com/css?family=Lato:300,300italic,400,400italic,700,700italic,900">
   <link rel="stylesheet" href="/css/screen.css">
   <link rel="icon" type="image/x-icon" href="/favicon.ico">

http://git-wip-us.apache.org/repos/asf/orc/blob/e58d3fc4/news/2016/01/25/ORC-1.0.0/index.html
----------------------------------------------------------------------
diff --git a/news/2016/01/25/ORC-1.0.0/index.html b/news/2016/01/25/ORC-1.0.0/index.html
index c00459e..a01de7a 100644
--- a/news/2016/01/25/ORC-1.0.0/index.html
+++ b/news/2016/01/25/ORC-1.0.0/index.html
@@ -4,7 +4,7 @@
   <meta charset="UTF-8">
   <title>ORC 1.0.0 Released</title>
   <meta name="viewport" content="width=device-width,initial-scale=1">
-  <meta name="generator" content="Jekyll v2.4.0">
+  <meta name="generator" content="Jekyll v3.7.3">
   <link rel="stylesheet" href="//fonts.googleapis.com/css?family=Lato:300,300italic,400,400italic,700,700italic,900">
   <link rel="stylesheet" href="/css/screen.css">
   <link rel="icon" type="image/x-icon" href="/favicon.ico">
@@ -207,13 +207,10 @@ signed by <a href="https://dist.apache.org/repos/dist/release/orc/KEYS">Owen O
 
 <ul>
   <li>
-    <p><a href="https://issues.apache.org/jira/browse/ORC-1">ORC-1</a> We are still working on moving the Java reader and writer out of Hive’s code base and thus it is not included here.</p>
+    <p><a href="http://localhost:4000/security/CVE-2018-8015">CVE-2018-8015</a> ORC files with malformed types cause stack overflow.</p>
   </li>
   <li>
-    <p><a href="https://issues.apache.org/jira/browse/ORC-10">ORC-10</a> When moving ORC files between timezones, different daylight savings rules will cause timestamps to shift in the C++ reader.</p>
-  </li>
-  <li>
-    <p><a href="https://issues.apache.org/jira/browse/ORC-40">ORC-40</a> Predicate push down is not implemented in C++.</p>
+    <p><a href="http://localhost:4000/security/ORC-10">ORC-10</a> When moving ORC files between timezones, different daylight savings rules will cause timestamps to shift in the C++ reader.</p>
   </li>
 </ul>
 

http://git-wip-us.apache.org/repos/asf/orc/blob/e58d3fc4/news/2016/06/10/ORC-1.1.0/index.html
----------------------------------------------------------------------
diff --git a/news/2016/06/10/ORC-1.1.0/index.html b/news/2016/06/10/ORC-1.1.0/index.html
index ee22331..86fac0a 100644
--- a/news/2016/06/10/ORC-1.1.0/index.html
+++ b/news/2016/06/10/ORC-1.1.0/index.html
@@ -4,7 +4,7 @@
   <meta charset="UTF-8">
   <title>ORC 1.1.0 Released</title>
   <meta name="viewport" content="width=device-width,initial-scale=1">
-  <meta name="generator" content="Jekyll v2.4.0">
+  <meta name="generator" content="Jekyll v3.7.3">
   <link rel="stylesheet" href="//fonts.googleapis.com/css?family=Lato:300,300italic,400,400italic,700,700italic,900">
   <link rel="stylesheet" href="/css/screen.css">
   <link rel="icon" type="image/x-icon" href="/favicon.ico">
@@ -200,16 +200,16 @@ timestamps from timezones with different daylight savings rules.</li>
 
 <ul>
   <li>
-    <p><a href="https://issues.apache.org/jira/browse/HIVE-14214">HIVE-14214</a> Schema evolution and predicate pushdown don’t work together.</p>
+    <p><a href="http://localhost:4000/security/CVE-2018-8015">CVE-2018-8015</a> ORC files with malformed types cause stack overflow.</p>
   </li>
   <li>
-    <p><a href="https://issues.apache.org/jira/browse/ORC-40">ORC-40</a> Predicate push down is not implemented in C++.</p>
+    <p><a href="http://localhost:4000/security/HIVE-14214">HIVE-14214</a> Schema evolution and predicate pushdown don’t work together.</p>
   </li>
   <li>
-    <p><a href="https://issues.apache.org/jira/browse/ORC-101">ORC-101</a> Bloom filters for string and decimal use inconsistent encoding</p>
+    <p><a href="http://localhost:4000/security/ORC-101">ORC-101</a> Bloom filters for string and decimal use inconsistent encoding</p>
   </li>
   <li>
-    <p><a href="https://issues.apache.org/jira/browse/ORC-135">ORC-135</a> Predicate push down is incorrect on timestamps when moved between timezones</p>
+    <p><a href="http://localhost:4000/security/ORC-135">ORC-135</a> Predicate push down is incorrect on timestamps when moved between timezones</p>
   </li>
 </ul>
 

http://git-wip-us.apache.org/repos/asf/orc/blob/e58d3fc4/news/2016/06/13/ORC-1.1.1/index.html
----------------------------------------------------------------------
diff --git a/news/2016/06/13/ORC-1.1.1/index.html b/news/2016/06/13/ORC-1.1.1/index.html
index 91ab3cd..ccfbc21 100644
--- a/news/2016/06/13/ORC-1.1.1/index.html
+++ b/news/2016/06/13/ORC-1.1.1/index.html
@@ -4,7 +4,7 @@
   <meta charset="UTF-8">
   <title>ORC 1.1.1 Released</title>
   <meta name="viewport" content="width=device-width,initial-scale=1">
-  <meta name="generator" content="Jekyll v2.4.0">
+  <meta name="generator" content="Jekyll v3.7.3">
   <link rel="stylesheet" href="//fonts.googleapis.com/css?family=Lato:300,300italic,400,400italic,700,700italic,900">
   <link rel="stylesheet" href="/css/screen.css">
   <link rel="icon" type="image/x-icon" href="/favicon.ico">
@@ -198,16 +198,16 @@ timestamps from timezones with different daylight savings rules.</li>
 
 <ul>
   <li>
-    <p><a href="https://issues.apache.org/jira/browse/HIVE-14214">HIVE-14214</a> Schema evolution and predicate pushdown don’t work together.</p>
+    <p><a href="http://localhost:4000/security/CVE-2018-8015">CVE-2018-8015</a> ORC files with malformed types cause stack overflow.</p>
   </li>
   <li>
-    <p><a href="https://issues.apache.org/jira/browse/ORC-40">ORC-40</a> Predicate push down is not implemented in C++.</p>
+    <p><a href="http://localhost:4000/security/HIVE-14214">HIVE-14214</a> Schema evolution and predicate pushdown don’t work together.</p>
   </li>
   <li>
-    <p><a href="https://issues.apache.org/jira/browse/ORC-101">ORC-101</a> Bloom filters for string and decimal use inconsistent encoding</p>
+    <p><a href="http://localhost:4000/security/ORC-101">ORC-101</a> Bloom filters for string and decimal use inconsistent encoding</p>
   </li>
   <li>
-    <p><a href="https://issues.apache.org/jira/browse/ORC-135">ORC-135</a> Predicate push down is incorrect on timestamps when moved between timezones</p>
+    <p><a href="http://localhost:4000/security/ORC-135">ORC-135</a> Predicate push down is incorrect on timestamps when moved between timezones</p>
   </li>
 </ul>
 

http://git-wip-us.apache.org/repos/asf/orc/blob/e58d3fc4/news/2016/06/28/file-format-talk/index.html
----------------------------------------------------------------------
diff --git a/news/2016/06/28/file-format-talk/index.html b/news/2016/06/28/file-format-talk/index.html
index 5a3e823..5a2070a 100644
--- a/news/2016/06/28/file-format-talk/index.html
+++ b/news/2016/06/28/file-format-talk/index.html
@@ -4,7 +4,7 @@
   <meta charset="UTF-8">
   <title>File format benchmark</title>
   <meta name="viewport" content="width=device-width,initial-scale=1">
-  <meta name="generator" content="Jekyll v2.4.0">
+  <meta name="generator" content="Jekyll v3.7.3">
   <link rel="stylesheet" href="//fonts.googleapis.com/css?family=Lato:300,300italic,400,400italic,700,700italic,900">
   <link rel="stylesheet" href="/css/screen.css">
   <link rel="icon" type="image/x-icon" href="/favicon.ico">

http://git-wip-us.apache.org/repos/asf/orc/blob/e58d3fc4/news/2016/07/08/ORC-1.1.2/index.html
----------------------------------------------------------------------
diff --git a/news/2016/07/08/ORC-1.1.2/index.html b/news/2016/07/08/ORC-1.1.2/index.html
index be3db5a..87d78d5 100644
--- a/news/2016/07/08/ORC-1.1.2/index.html
+++ b/news/2016/07/08/ORC-1.1.2/index.html
@@ -4,7 +4,7 @@
   <meta charset="UTF-8">
   <title>ORC 1.1.2 Released</title>
   <meta name="viewport" content="width=device-width,initial-scale=1">
-  <meta name="generator" content="Jekyll v2.4.0">
+  <meta name="generator" content="Jekyll v3.7.3">
   <link rel="stylesheet" href="//fonts.googleapis.com/css?family=Lato:300,300italic,400,400italic,700,700italic,900">
   <link rel="stylesheet" href="/css/screen.css">
   <link rel="icon" type="image/x-icon" href="/favicon.ico">
@@ -198,16 +198,16 @@ timestamps from timezones with different daylight savings rules.</li>
 
 <ul>
   <li>
-    <p><a href="https://issues.apache.org/jira/browse/HIVE-14214">HIVE-14214</a> Schema evolution and predicate pushdown don’t work together.</p>
+    <p><a href="http://localhost:4000/security/CVE-2018-8015">CVE-2018-8015</a> ORC files with malformed types cause stack overflow.</p>
   </li>
   <li>
-    <p><a href="https://issues.apache.org/jira/browse/ORC-40">ORC-40</a> Predicate push down is not implemented in C++.</p>
+    <p><a href="http://localhost:4000/security/HIVE-14214">HIVE-14214</a> Schema evolution and predicate pushdown don’t work together.</p>
   </li>
   <li>
-    <p><a href="https://issues.apache.org/jira/browse/ORC-101">ORC-101</a> Bloom filters for string and decimal use inconsistent encoding</p>
+    <p><a href="http://localhost:4000/security/ORC-101">ORC-101</a> Bloom filters for string and decimal use inconsistent encoding</p>
   </li>
   <li>
-    <p><a href="https://issues.apache.org/jira/browse/ORC-135">ORC-135</a> Predicate push down is incorrect on timestamps when moved between timezones</p>
+    <p><a href="http://localhost:4000/security/ORC-135">ORC-135</a> Predicate push down is incorrect on timestamps when moved between timezones</p>
   </li>
 </ul>
 

http://git-wip-us.apache.org/repos/asf/orc/blob/e58d3fc4/news/2016/08/25/ORC-1.2.0/index.html
----------------------------------------------------------------------
diff --git a/news/2016/08/25/ORC-1.2.0/index.html b/news/2016/08/25/ORC-1.2.0/index.html
index 9dd0b12..84e2e77 100644
--- a/news/2016/08/25/ORC-1.2.0/index.html
+++ b/news/2016/08/25/ORC-1.2.0/index.html
@@ -4,7 +4,7 @@
   <meta charset="UTF-8">
   <title>ORC 1.2.0 Released</title>
   <meta name="viewport" content="width=device-width,initial-scale=1">
-  <meta name="generator" content="Jekyll v2.4.0">
+  <meta name="generator" content="Jekyll v3.7.3">
   <link rel="stylesheet" href="//fonts.googleapis.com/css?family=Lato:300,300italic,400,400italic,700,700italic,900">
   <link rel="stylesheet" href="/css/screen.css">
   <link rel="icon" type="image/x-icon" href="/favicon.ico">
@@ -204,13 +204,13 @@ signed by <a href="https://dist.apache.org/repos/dist/release/orc/KEYS">Owen O
 
 <ul>
   <li>
-    <p><a href="https://issues.apache.org/jira/browse/ORC-40">ORC-40</a> Predicate push down is not implemented in C++.</p>
+    <p><a href="http://localhost:4000/security/CVE-2018-8015">CVE-2018-8015</a> ORC files with malformed types cause stack overflow.</p>
   </li>
   <li>
-    <p><a href="https://issues.apache.org/jira/browse/ORC-101">ORC-101</a> Bloom filters for string and decimal use inconsistent encoding</p>
+    <p><a href="http://localhost:4000/security/ORC-101">ORC-101</a> Bloom filters for string and decimal use inconsistent encoding</p>
   </li>
   <li>
-    <p><a href="https://issues.apache.org/jira/browse/ORC-135">ORC-135</a> Predicate push down is incorrect on timestamps when moved between timezones</p>
+    <p><a href="http://localhost:4000/security/ORC-135">ORC-135</a> Predicate push down is incorrect on timestamps when moved between timezones</p>
   </li>
 </ul>
 

http://git-wip-us.apache.org/repos/asf/orc/blob/e58d3fc4/news/2016/10/05/ORC-1.2.1/index.html
----------------------------------------------------------------------
diff --git a/news/2016/10/05/ORC-1.2.1/index.html b/news/2016/10/05/ORC-1.2.1/index.html
index 26dca39..31d1827 100644
--- a/news/2016/10/05/ORC-1.2.1/index.html
+++ b/news/2016/10/05/ORC-1.2.1/index.html
@@ -4,7 +4,7 @@
   <meta charset="UTF-8">
   <title>ORC 1.2.1 Released</title>
   <meta name="viewport" content="width=device-width,initial-scale=1">
-  <meta name="generator" content="Jekyll v2.4.0">
+  <meta name="generator" content="Jekyll v3.7.3">
   <link rel="stylesheet" href="//fonts.googleapis.com/css?family=Lato:300,300italic,400,400italic,700,700italic,900">
   <link rel="stylesheet" href="/css/screen.css">
   <link rel="icon" type="image/x-icon" href="/favicon.ico">
@@ -204,10 +204,10 @@ signed by <a href="https://dist.apache.org/repos/dist/release/orc/KEYS">Owen O
 
 <ul>
   <li>
-    <p><a href="https://issues.apache.org/jira/browse/ORC-40">ORC-40</a> Predicate push down is not implemented in C++.</p>
+    <p><a href="http://localhost:4000/security/CVE-2018-8015">CVE-2018-8015</a> ORC files with malformed types cause stack overflow.</p>
   </li>
   <li>
-    <p><a href="https://issues.apache.org/jira/browse/ORC-135">ORC-135</a> Predicate push down is incorrect on timestamps when moved between timezones</p>
+    <p><a href="http://localhost:4000/security/ORC-135">ORC-135</a> Predicate push down is incorrect on timestamps when moved between timezones</p>
   </li>
 </ul>
 

http://git-wip-us.apache.org/repos/asf/orc/blob/e58d3fc4/news/2016/12/01/ORC-1.2.2/index.html
----------------------------------------------------------------------
diff --git a/news/2016/12/01/ORC-1.2.2/index.html b/news/2016/12/01/ORC-1.2.2/index.html
index 16701ee..50f8070 100644
--- a/news/2016/12/01/ORC-1.2.2/index.html
+++ b/news/2016/12/01/ORC-1.2.2/index.html
@@ -4,7 +4,7 @@
   <meta charset="UTF-8">
   <title>ORC 1.2.2 Released</title>
   <meta name="viewport" content="width=device-width,initial-scale=1">
-  <meta name="generator" content="Jekyll v2.4.0">
+  <meta name="generator" content="Jekyll v3.7.3">
   <link rel="stylesheet" href="//fonts.googleapis.com/css?family=Lato:300,300italic,400,400italic,700,700italic,900">
   <link rel="stylesheet" href="/css/screen.css">
   <link rel="icon" type="image/x-icon" href="/favicon.ico">
@@ -204,10 +204,10 @@ signed by <a href="https://dist.apache.org/repos/dist/release/orc/KEYS">Owen O
 
 <ul>
   <li>
-    <p><a href="https://issues.apache.org/jira/browse/ORC-40">ORC-40</a> Predicate push down is not implemented in C++.</p>
+    <p><a href="http://localhost:4000/security/CVE-2018-8015">CVE-2018-8015</a> ORC files with malformed types cause stack overflow.</p>
   </li>
   <li>
-    <p><a href="https://issues.apache.org/jira/browse/ORC-135">ORC-135</a> Predicate push down is incorrect on timestamps when moved between timezones</p>
+    <p><a href="http://localhost:4000/security/ORC-135">ORC-135</a> Predicate push down is incorrect on timestamps when moved between timezones</p>
   </li>
 </ul>
 

http://git-wip-us.apache.org/repos/asf/orc/blob/e58d3fc4/news/2016/12/12/ORC-1.2.3/index.html
----------------------------------------------------------------------
diff --git a/news/2016/12/12/ORC-1.2.3/index.html b/news/2016/12/12/ORC-1.2.3/index.html
index eca5539..7de6086 100644
--- a/news/2016/12/12/ORC-1.2.3/index.html
+++ b/news/2016/12/12/ORC-1.2.3/index.html
@@ -4,7 +4,7 @@
   <meta charset="UTF-8">
   <title>ORC 1.2.3 Released</title>
   <meta name="viewport" content="width=device-width,initial-scale=1">
-  <meta name="generator" content="Jekyll v2.4.0">
+  <meta name="generator" content="Jekyll v3.7.3">
   <link rel="stylesheet" href="//fonts.googleapis.com/css?family=Lato:300,300italic,400,400italic,700,700italic,900">
   <link rel="stylesheet" href="/css/screen.css">
   <link rel="icon" type="image/x-icon" href="/favicon.ico">
@@ -205,10 +205,10 @@ signed by <a href="https://dist.apache.org/repos/dist/release/orc/KEYS">Owen O
 
 <ul>
   <li>
-    <p><a href="https://issues.apache.org/jira/browse/ORC-40">ORC-40</a> Predicate push down is not implemented in C++.</p>
+    <p><a href="http://localhost:4000/security/CVE-2018-8015">CVE-2018-8015</a> ORC files with malformed types cause stack overflow.</p>
   </li>
   <li>
-    <p><a href="https://issues.apache.org/jira/browse/ORC-135">ORC-135</a> Predicate push down is incorrect on timestamps when moved between timezones</p>
+    <p><a href="http://localhost:4000/security/ORC-135">ORC-135</a> Predicate push down is incorrect on timestamps when moved between timezones</p>
   </li>
 </ul>
 

http://git-wip-us.apache.org/repos/asf/orc/blob/e58d3fc4/news/2016/12/15/new-committers/index.html
----------------------------------------------------------------------
diff --git a/news/2016/12/15/new-committers/index.html b/news/2016/12/15/new-committers/index.html
index a55b864..690e1f9 100644
--- a/news/2016/12/15/new-committers/index.html
+++ b/news/2016/12/15/new-committers/index.html
@@ -4,7 +4,7 @@
   <meta charset="UTF-8">
   <title>ORC adds new committers</title>
   <meta name="viewport" content="width=device-width,initial-scale=1">
-  <meta name="generator" content="Jekyll v2.4.0">
+  <meta name="generator" content="Jekyll v3.7.3">
   <link rel="stylesheet" href="//fonts.googleapis.com/css?family=Lato:300,300italic,400,400italic,700,700italic,900">
   <link rel="stylesheet" href="/css/screen.css">
   <link rel="icon" type="image/x-icon" href="/favicon.ico">

http://git-wip-us.apache.org/repos/asf/orc/blob/e58d3fc4/news/2017/01/04/gopal-on-pmc/index.html
----------------------------------------------------------------------
diff --git a/news/2017/01/04/gopal-on-pmc/index.html b/news/2017/01/04/gopal-on-pmc/index.html
index b20ab1e..0a505f4 100644
--- a/news/2017/01/04/gopal-on-pmc/index.html
+++ b/news/2017/01/04/gopal-on-pmc/index.html
@@ -4,7 +4,7 @@
   <meta charset="UTF-8">
   <title>ORC adds Gopal Vijayaraghavan to PMC</title>
   <meta name="viewport" content="width=device-width,initial-scale=1">
-  <meta name="generator" content="Jekyll v2.4.0">
+  <meta name="generator" content="Jekyll v3.7.3">
   <link rel="stylesheet" href="//fonts.googleapis.com/css?family=Lato:300,300italic,400,400italic,700,700italic,900">
   <link rel="stylesheet" href="/css/screen.css">
   <link rel="icon" type="image/x-icon" href="/favicon.ico">

http://git-wip-us.apache.org/repos/asf/orc/blob/e58d3fc4/news/2017/01/23/ORC-1.3.0/index.html
----------------------------------------------------------------------
diff --git a/news/2017/01/23/ORC-1.3.0/index.html b/news/2017/01/23/ORC-1.3.0/index.html
index 434c916..a42401c 100644
--- a/news/2017/01/23/ORC-1.3.0/index.html
+++ b/news/2017/01/23/ORC-1.3.0/index.html
@@ -4,7 +4,7 @@
   <meta charset="UTF-8">
   <title>ORC 1.3.0 Released</title>
   <meta name="viewport" content="width=device-width,initial-scale=1">
-  <meta name="generator" content="Jekyll v2.4.0">
+  <meta name="generator" content="Jekyll v3.7.3">
   <link rel="stylesheet" href="//fonts.googleapis.com/css?family=Lato:300,300italic,400,400italic,700,700italic,900">
   <link rel="stylesheet" href="/css/screen.css">
   <link rel="icon" type="image/x-icon" href="/favicon.ico">
@@ -196,12 +196,7 @@ signed by <a href="https://dist.apache.org/repos/dist/release/orc/KEYS">Owen O
 <p>Known issues:</p>
 
 <ul>
-  <li>
-    <p><a href="https://issues.apache.org/jira/browse/ORC-40">ORC-40</a> Predicate push down is not implemented in C++.</p>
-  </li>
-  <li>
-    <p><a href="https://issues.apache.org/jira/browse/ORC-135">ORC-135</a> Predicate push down is incorrect on timestamps when moved between timezones</p>
-  </li>
+  <li><a href="http://localhost:4000/security/ORC-135">ORC-135</a> Predicate push down is incorrect on timestamps when moved between timezones</li>
 </ul>
 
 

http://git-wip-us.apache.org/repos/asf/orc/blob/e58d3fc4/news/2017/02/03/ORC-1.3.1/index.html
----------------------------------------------------------------------
diff --git a/news/2017/02/03/ORC-1.3.1/index.html b/news/2017/02/03/ORC-1.3.1/index.html
index 0a2bd9f..22dc586 100644
--- a/news/2017/02/03/ORC-1.3.1/index.html
+++ b/news/2017/02/03/ORC-1.3.1/index.html
@@ -4,7 +4,7 @@
   <meta charset="UTF-8">
   <title>ORC 1.3.1 Released</title>
   <meta name="viewport" content="width=device-width,initial-scale=1">
-  <meta name="generator" content="Jekyll v2.4.0">
+  <meta name="generator" content="Jekyll v3.7.3">
   <link rel="stylesheet" href="//fonts.googleapis.com/css?family=Lato:300,300italic,400,400italic,700,700italic,900">
   <link rel="stylesheet" href="/css/screen.css">
   <link rel="icon" type="image/x-icon" href="/favicon.ico">
@@ -197,10 +197,10 @@ signed by <a href="https://dist.apache.org/repos/dist/release/orc/KEYS">Owen O
 
 <ul>
   <li>
-    <p><a href="https://issues.apache.org/jira/browse/ORC-40">ORC-40</a> Predicate push down is not implemented in C++.</p>
+    <p><a href="http://localhost:4000/security/CVE-2018-8015">CVE-2018-8015</a> ORC files with malformed types cause stack overflow.</p>
   </li>
   <li>
-    <p><a href="https://issues.apache.org/jira/browse/ORC-135">ORC-135</a> Predicate push down is incorrect on timestamps when moved between timezones</p>
+    <p><a href="http://localhost:4000/security/ORC-135">ORC-135</a> Predicate push down is incorrect on timestamps when moved between timezones</p>
   </li>
 </ul>
 

http://git-wip-us.apache.org/repos/asf/orc/blob/e58d3fc4/news/2017/02/13/ORC-1.3.2/index.html
----------------------------------------------------------------------
diff --git a/news/2017/02/13/ORC-1.3.2/index.html b/news/2017/02/13/ORC-1.3.2/index.html
index c8ea65c..e8c0434 100644
--- a/news/2017/02/13/ORC-1.3.2/index.html
+++ b/news/2017/02/13/ORC-1.3.2/index.html
@@ -4,7 +4,7 @@
   <meta charset="UTF-8">
   <title>ORC 1.3.2 Released</title>
   <meta name="viewport" content="width=device-width,initial-scale=1">
-  <meta name="generator" content="Jekyll v2.4.0">
+  <meta name="generator" content="Jekyll v3.7.3">
   <link rel="stylesheet" href="//fonts.googleapis.com/css?family=Lato:300,300italic,400,400italic,700,700italic,900">
   <link rel="stylesheet" href="/css/screen.css">
   <link rel="icon" type="image/x-icon" href="/favicon.ico">
@@ -196,7 +196,7 @@ signed by <a href="https://dist.apache.org/repos/dist/release/orc/KEYS">Owen O
 <p>Known issues:</p>
 
 <ul>
-  <li><a href="https://issues.apache.org/jira/browse/ORC-40">ORC-40</a> Predicate push down is not implemented in C++.</li>
+  <li><a href="http://localhost:4000/security/CVE-2018-8015">CVE-2018-8015</a> ORC files with malformed types cause stack overflow.</li>
 </ul>
 
 

http://git-wip-us.apache.org/repos/asf/orc/blob/e58d3fc4/news/2017/02/21/ORC-1.3.3/index.html
----------------------------------------------------------------------
diff --git a/news/2017/02/21/ORC-1.3.3/index.html b/news/2017/02/21/ORC-1.3.3/index.html
index dbcffd0..f36460a 100644
--- a/news/2017/02/21/ORC-1.3.3/index.html
+++ b/news/2017/02/21/ORC-1.3.3/index.html
@@ -4,7 +4,7 @@
   <meta charset="UTF-8">
   <title>ORC 1.3.3 Released</title>
   <meta name="viewport" content="width=device-width,initial-scale=1">
-  <meta name="generator" content="Jekyll v2.4.0">
+  <meta name="generator" content="Jekyll v3.7.3">
   <link rel="stylesheet" href="//fonts.googleapis.com/css?family=Lato:300,300italic,400,400italic,700,700italic,900">
   <link rel="stylesheet" href="/css/screen.css">
   <link rel="icon" type="image/x-icon" href="/favicon.ico">
@@ -196,7 +196,7 @@ signed by <a href="https://dist.apache.org/repos/dist/release/orc/KEYS">Owen O
 <p>Known issues:</p>
 
 <ul>
-  <li><a href="https://issues.apache.org/jira/browse/ORC-40">ORC-40</a> Predicate push down is not implemented in C++.</li>
+  <li><a href="http://localhost:4000/security/CVE-2018-8015">CVE-2018-8015</a> ORC files with malformed types cause stack overflow.</li>
 </ul>
 
 

http://git-wip-us.apache.org/repos/asf/orc/blob/e58d3fc4/news/2017/05/08/ORC-1.4.0/index.html
----------------------------------------------------------------------
diff --git a/news/2017/05/08/ORC-1.4.0/index.html b/news/2017/05/08/ORC-1.4.0/index.html
index a631aa7..1973299 100644
--- a/news/2017/05/08/ORC-1.4.0/index.html
+++ b/news/2017/05/08/ORC-1.4.0/index.html
@@ -4,7 +4,7 @@
   <meta charset="UTF-8">
   <title>ORC 1.4.0 Released</title>
   <meta name="viewport" content="width=device-width,initial-scale=1">
-  <meta name="generator" content="Jekyll v2.4.0">
+  <meta name="generator" content="Jekyll v3.7.3">
   <link rel="stylesheet" href="//fonts.googleapis.com/css?family=Lato:300,300italic,400,400italic,700,700italic,900">
   <link rel="stylesheet" href="/css/screen.css">
   <link rel="icon" type="image/x-icon" href="/favicon.ico">
@@ -197,7 +197,7 @@ signed by <a href="https://dist.apache.org/repos/dist/release/orc/KEYS">Owen O
 <p>Known issues:</p>
 
 <ul>
-  <li><a href="https://issues.apache.org/jira/browse/ORC-40">ORC-40</a> Predicate push down is not implemented in C++.</li>
+  <li><a href="http://localhost:4000/security/CVE-2018-8015">CVE-2018-8015</a> ORC files with malformed types cause stack overflow.</li>
 </ul>
 
 

http://git-wip-us.apache.org/repos/asf/orc/blob/e58d3fc4/news/2017/05/16/new-committer/index.html
----------------------------------------------------------------------
diff --git a/news/2017/05/16/new-committer/index.html b/news/2017/05/16/new-committer/index.html
index 5b10ac0..d3c9c7e 100644
--- a/news/2017/05/16/new-committer/index.html
+++ b/news/2017/05/16/new-committer/index.html
@@ -4,7 +4,7 @@
   <meta charset="UTF-8">
   <title>Deepak Majeti added as committer</title>
   <meta name="viewport" content="width=device-width,initial-scale=1">
-  <meta name="generator" content="Jekyll v2.4.0">
+  <meta name="generator" content="Jekyll v3.7.3">
   <link rel="stylesheet" href="//fonts.googleapis.com/css?family=Lato:300,300italic,400,400italic,700,700italic,900">
   <link rel="stylesheet" href="/css/screen.css">
   <link rel="icon" type="image/x-icon" href="/favicon.ico">

http://git-wip-us.apache.org/repos/asf/orc/blob/e58d3fc4/news/2017/09/06/new-pmc/index.html
----------------------------------------------------------------------
diff --git a/news/2017/09/06/new-pmc/index.html b/news/2017/09/06/new-pmc/index.html
index 3e1ee38..0c73eed 100644
--- a/news/2017/09/06/new-pmc/index.html
+++ b/news/2017/09/06/new-pmc/index.html
@@ -4,7 +4,7 @@
   <meta charset="UTF-8">
   <title>ORC adds Eugene and Deepak to PMC</title>
   <meta name="viewport" content="width=device-width,initial-scale=1">
-  <meta name="generator" content="Jekyll v2.4.0">
+  <meta name="generator" content="Jekyll v3.7.3">
   <link rel="stylesheet" href="//fonts.googleapis.com/css?family=Lato:300,300italic,400,400italic,700,700italic,900">
   <link rel="stylesheet" href="/css/screen.css">
   <link rel="icon" type="image/x-icon" href="/favicon.ico">

http://git-wip-us.apache.org/repos/asf/orc/blob/e58d3fc4/news/2017/10/16/ORC-1.3.4/index.html
----------------------------------------------------------------------
diff --git a/news/2017/10/16/ORC-1.3.4/index.html b/news/2017/10/16/ORC-1.3.4/index.html
index f5b4bb3..cbe7eba 100644
--- a/news/2017/10/16/ORC-1.3.4/index.html
+++ b/news/2017/10/16/ORC-1.3.4/index.html
@@ -4,7 +4,7 @@
   <meta charset="UTF-8">
   <title>ORC 1.3.4 Released</title>
   <meta name="viewport" content="width=device-width,initial-scale=1">
-  <meta name="generator" content="Jekyll v2.4.0">
+  <meta name="generator" content="Jekyll v3.7.3">
   <link rel="stylesheet" href="//fonts.googleapis.com/css?family=Lato:300,300italic,400,400italic,700,700italic,900">
   <link rel="stylesheet" href="/css/screen.css">
   <link rel="icon" type="image/x-icon" href="/favicon.ico">
@@ -196,7 +196,7 @@ signed by <a href="https://dist.apache.org/repos/dist/release/orc/KEYS">Prasanth
 <p>Known issues:</p>
 
 <ul>
-  <li><a href="https://issues.apache.org/jira/browse/ORC-40">ORC-40</a> Predicate push down is not implemented in C++.</li>
+  <li><a href="http://localhost:4000/security/CVE-2018-8015">CVE-2018-8015</a> ORC files with malformed types cause stack overflow.</li>
 </ul>
 
 

http://git-wip-us.apache.org/repos/asf/orc/blob/e58d3fc4/news/2017/10/16/ORC-1.4.1/index.html
----------------------------------------------------------------------
diff --git a/news/2017/10/16/ORC-1.4.1/index.html b/news/2017/10/16/ORC-1.4.1/index.html
index 6a3d30b..db56e97 100644
--- a/news/2017/10/16/ORC-1.4.1/index.html
+++ b/news/2017/10/16/ORC-1.4.1/index.html
@@ -4,7 +4,7 @@
   <meta charset="UTF-8">
   <title>ORC 1.4.1 Released</title>
   <meta name="viewport" content="width=device-width,initial-scale=1">
-  <meta name="generator" content="Jekyll v2.4.0">
+  <meta name="generator" content="Jekyll v3.7.3">
   <link rel="stylesheet" href="//fonts.googleapis.com/css?family=Lato:300,300italic,400,400italic,700,700italic,900">
   <link rel="stylesheet" href="/css/screen.css">
   <link rel="icon" type="image/x-icon" href="/favicon.ico">
@@ -197,7 +197,12 @@ signed by <a href="https://dist.apache.org/repos/dist/release/orc/KEYS">Prasanth
 <p>Known issues:</p>
 
 <ul>
-  <li><a href="https://issues.apache.org/jira/browse/ORC-40">ORC-40</a> Predicate push down is not implemented in C++.</li>
+  <li>
+    <p><a href="http://localhost:4000/security/CVE-2018-8015">CVE-2018-8015</a> ORC files with malformed types cause stack overflow.</p>
+  </li>
+  <li>
+    <p><a href="http://localhost:4000/security/ORC-40">ORC-40</a> Predicate push down is not implemented in C++.</p>
+  </li>
 </ul>
 
 

http://git-wip-us.apache.org/repos/asf/orc/blob/e58d3fc4/news/2018/01/23/ORC-1.4.2/index.html
----------------------------------------------------------------------
diff --git a/news/2018/01/23/ORC-1.4.2/index.html b/news/2018/01/23/ORC-1.4.2/index.html
index 8edd212..492d684 100644
--- a/news/2018/01/23/ORC-1.4.2/index.html
+++ b/news/2018/01/23/ORC-1.4.2/index.html
@@ -4,7 +4,7 @@
   <meta charset="UTF-8">
   <title>ORC 1.4.2 Released</title>
   <meta name="viewport" content="width=device-width,initial-scale=1">
-  <meta name="generator" content="Jekyll v2.4.0">
+  <meta name="generator" content="Jekyll v3.7.3">
   <link rel="stylesheet" href="//fonts.googleapis.com/css?family=Lato:300,300italic,400,400italic,700,700italic,900">
   <link rel="stylesheet" href="/css/screen.css">
   <link rel="icon" type="image/x-icon" href="/favicon.ico">
@@ -197,7 +197,12 @@ signed by <a href="https://dist.apache.org/repos/dist/release/orc/KEYS">Owen O
 <p>Known issues:</p>
 
 <ul>
-  <li><a href="https://issues.apache.org/jira/browse/ORC-40">ORC-40</a> Predicate push down is not implemented in C++.</li>
+  <li>
+    <p><a href="http://localhost:4000/security/CVE-2018-8015">CVE-2018-8015</a> ORC files with malformed types cause stack overflow.</p>
+  </li>
+  <li>
+    <p><a href="http://localhost:4000/security/ORC-40">ORC-40</a> Predicate push down is not implemented in C++.</p>
+  </li>
 </ul>
 
 

http://git-wip-us.apache.org/repos/asf/orc/blob/e58d3fc4/news/2018/02/09/ORC-1.4.3/index.html
----------------------------------------------------------------------
diff --git a/news/2018/02/09/ORC-1.4.3/index.html b/news/2018/02/09/ORC-1.4.3/index.html
index 167d3c2..d466222 100644
--- a/news/2018/02/09/ORC-1.4.3/index.html
+++ b/news/2018/02/09/ORC-1.4.3/index.html
@@ -4,7 +4,7 @@
   <meta charset="UTF-8">
   <title>ORC 1.4.3 Released</title>
   <meta name="viewport" content="width=device-width,initial-scale=1">
-  <meta name="generator" content="Jekyll v2.4.0">
+  <meta name="generator" content="Jekyll v3.7.3">
   <link rel="stylesheet" href="//fonts.googleapis.com/css?family=Lato:300,300italic,400,400italic,700,700italic,900">
   <link rel="stylesheet" href="/css/screen.css">
   <link rel="icon" type="image/x-icon" href="/favicon.ico">
@@ -196,6 +196,10 @@ signed by <a href="https://dist.apache.org/repos/dist/release/orc/KEYS">Owen O
 
 <p>Known issues:</p>
 
+<ul>
+  <li><a href="http://localhost:4000/security/CVE-2018-8015">CVE-2018-8015</a> ORC files with malformed types cause stack overflow.</li>
+</ul>
+
 
   </div>
 </article>

http://git-wip-us.apache.org/repos/asf/orc/blob/e58d3fc4/news/2018/05/14/ORC-1.4.4/index.html
----------------------------------------------------------------------
diff --git a/news/2018/05/14/ORC-1.4.4/index.html b/news/2018/05/14/ORC-1.4.4/index.html
index 3a7f829..e61fdca 100644
--- a/news/2018/05/14/ORC-1.4.4/index.html
+++ b/news/2018/05/14/ORC-1.4.4/index.html
@@ -4,7 +4,7 @@
   <meta charset="UTF-8">
   <title>ORC 1.4.4 Released</title>
   <meta name="viewport" content="width=device-width,initial-scale=1">
-  <meta name="generator" content="Jekyll v2.4.0">
+  <meta name="generator" content="Jekyll v3.7.3">
   <link rel="stylesheet" href="//fonts.googleapis.com/css?family=Lato:300,300italic,400,400italic,700,700italic,900">
   <link rel="stylesheet" href="/css/screen.css">
   <link rel="icon" type="image/x-icon" href="/favicon.ico">

http://git-wip-us.apache.org/repos/asf/orc/blob/e58d3fc4/news/2018/05/14/ORC-1.5.0/index.html
----------------------------------------------------------------------
diff --git a/news/2018/05/14/ORC-1.5.0/index.html b/news/2018/05/14/ORC-1.5.0/index.html
index 16033df..bb9c91d 100644
--- a/news/2018/05/14/ORC-1.5.0/index.html
+++ b/news/2018/05/14/ORC-1.5.0/index.html
@@ -4,7 +4,7 @@
   <meta charset="UTF-8">
   <title>ORC 1.5.0 Released</title>
   <meta name="viewport" content="width=device-width,initial-scale=1">
-  <meta name="generator" content="Jekyll v2.4.0">
+  <meta name="generator" content="Jekyll v3.7.3">
   <link rel="stylesheet" href="//fonts.googleapis.com/css?family=Lato:300,300italic,400,400italic,700,700italic,900">
   <link rel="stylesheet" href="/css/screen.css">
   <link rel="icon" type="image/x-icon" href="/favicon.ico">


[04/11] orc git commit: Deploy docs with CVE.

Posted by om...@apache.org.
http://git-wip-us.apache.org/repos/asf/orc/blob/e58d3fc4/specification/ORCv1.html
----------------------------------------------------------------------
diff --git a/specification/ORCv1.html b/specification/ORCv1.html
deleted file mode 100644
index e3cad2e..0000000
--- a/specification/ORCv1.html
+++ /dev/null
@@ -1,1744 +0,0 @@
-<!DOCTYPE HTML>
-<html lang="en-US">
-<head>
-  <meta charset="UTF-8">
-  <title>ORC Specification v1</title>
-  <meta name="viewport" content="width=device-width,initial-scale=1">
-  <meta name="generator" content="Jekyll v2.4.0">
-  <link rel="stylesheet" href="//fonts.googleapis.com/css?family=Lato:300,300italic,400,400italic,700,700italic,900">
-  <link rel="stylesheet" href="/css/screen.css">
-  <link rel="icon" type="image/x-icon" href="/favicon.ico">
-  <!--[if lt IE 9]>
-  <script src="/js/html5shiv.min.js"></script>
-  <script src="/js/respond.min.js"></script>
-  <![endif]-->
-</head>
-
-
-<body class="wrap">
-  <header role="banner">
-  <nav class="mobile-nav show-on-mobiles">
-    <ul>
-  <li class="">
-    <a href="/">Home</a>
-  </li>
-  <li class="">
-    <a href="/docs/"><span class="show-on-mobiles">Docs</span>
-                     <span class="hide-on-mobiles">Documentation</span></a>
-  </li>
-  <li class="">
-    <a href="/talks/">Talks</a>
-  </li>
-  <li class="">
-    <a href="/news/">News</a>
-  </li>
-  <li class="">
-    <a href="/help/">Help</a>
-  </li>
-  <li class="">
-    <a href="/develop/">Develop</a>
-  </li>
-</ul>
-
-  </nav>
-  <div class="grid">
-    <div class="unit one-third center-on-mobiles">
-      <h1>
-        <a href="/">
-          <span class="sr-only">Apache ORC</span>
-          <img src="/img/logo.png" width="249" height="101" alt="ORC Logo">
-        </a>
-      </h1>
-    </div>
-    <nav class="main-nav unit two-thirds hide-on-mobiles">
-      <ul>
-  <li class="">
-    <a href="/">Home</a>
-  </li>
-  <li class="">
-    <a href="/docs/"><span class="show-on-mobiles">Docs</span>
-                     <span class="hide-on-mobiles">Documentation</span></a>
-  </li>
-  <li class="">
-    <a href="/talks/">Talks</a>
-  </li>
-  <li class="">
-    <a href="/news/">News</a>
-  </li>
-  <li class="">
-    <a href="/help/">Help</a>
-  </li>
-  <li class="">
-    <a href="/develop/">Develop</a>
-  </li>
-</ul>
-
-    </nav>
-  </div>
-</header>
-
-
-  <section class="standalone">
-  <div class="grid">
-
-    <div class="unit whole">
-      <article>
-        <h1>ORC Specification v1</h1>
-        <p>This version of the file format was originally released as part of
-Hive 0.12.</p>
-
-<h1 id="motivation">Motivation</h1>
-
-<p>Hive’s RCFile was the standard format for storing tabular data in
-Hadoop for several years. However, RCFile has limitations because it
-treats each column as a binary blob without semantics. In Hive 0.11 we
-added a new file format named Optimized Row Columnar (ORC) file that
-uses and retains the type information from the table definition. ORC
-uses type specific readers and writers that provide light weight
-compression techniques such as dictionary encoding, bit packing, delta
-encoding, and run length encoding – resulting in dramatically smaller
-files. Additionally, ORC can apply generic compression using zlib, or
-Snappy on top of the lightweight compression for even smaller
-files. However, storage savings are only part of the gain. ORC
-supports projection, which selects subsets of the columns for reading,
-so that queries reading only one column read only the required
-bytes. Furthermore, ORC files include light weight indexes that
-include the minimum and maximum values for each column in each set of
-10,000 rows and the entire file. Using pushdown filters from Hive, the
-file reader can skip entire sets of rows that aren’t important for
-this query.</p>
-
-<p><img src="/img/OrcFileLayout.png" alt="ORC file structure" /></p>
-
-<h1 id="file-tail">File Tail</h1>
-
-<p>Since HDFS does not support changing the data in a file after it is
-written, ORC stores the top level index at the end of the file. The
-overall structure of the file is given in the figure above.  The
-file’s tail consists of 3 parts; the file metadata, file footer and
-postscript.</p>
-
-<p>The metadata for ORC is stored using
-<a href="https://s.apache.org/protobuf_encoding">Protocol Buffers</a>, which provides
-the ability to add new fields without breaking readers. This document
-incorporates the Protobuf definition from the
-<a href="https://s.apache.org/orc_proto">ORC source code</a> and the
-reader is encouraged to review the Protobuf encoding if they need to
-understand the byte-level encoding</p>
-
-<h2 id="postscript">Postscript</h2>
-
-<p>The Postscript section provides the necessary information to interpret
-the rest of the file including the length of the file’s Footer and
-Metadata sections, the version of the file, and the kind of general
-compression used (eg. none, zlib, or snappy). The Postscript is never
-compressed and ends one byte before the end of the file. The version
-stored in the Postscript is the lowest version of Hive that is
-guaranteed to be able to read the file and it stored as a sequence of
-the major and minor version. This file version is encoded as [0,12].</p>
-
-<p>The process of reading an ORC file works backwards through the
-file. Rather than making multiple short reads, the ORC reader reads
-the last 16k bytes of the file with the hope that it will contain both
-the Footer and Postscript sections. The final byte of the file
-contains the serialized length of the Postscript, which must be less
-than 256 bytes. Once the Postscript is parsed, the compressed
-serialized length of the Footer is known and it can be decompressed
-and parsed.</p>
-
-<p><code>message PostScript {
- // the length of the footer section in bytes
- optional uint64 footerLength = 1;
- // the kind of generic compression used
- optional CompressionKind compression = 2;
- // the maximum size of each compression chunk
- optional uint64 compressionBlockSize = 3;
- // the version of the writer
- repeated uint32 version = 4 [packed = true];
- // the length of the metadata section in bytes
- optional uint64 metadataLength = 5;
- // the fixed string "ORC"
- optional string magic = 8000;
-}
-</code></p>
-
-<p><code>enum CompressionKind {
- NONE = 0;
- ZLIB = 1;
- SNAPPY = 2;
- LZO = 3;
- LZ4 = 4;
- ZSTD = 5;
-}
-</code></p>
-
-<h2 id="footer">Footer</h2>
-
-<p>The Footer section contains the layout of the body of the file, the
-type schema information, the number of rows, and the statistics about
-each of the columns.</p>
-
-<p>The file is broken in to three parts- Header, Body, and Tail. The
-Header consists of the bytes “ORC’’ to support tools that want to
-scan the front of the file to determine the type of the file. The Body
-contains the rows and indexes, and the Tail gives the file level
-information as described in this section.</p>
-
-<p><code>message Footer {
- // the length of the file header in bytes (always 3)
- optional uint64 headerLength = 1;
- // the length of the file header and body in bytes
- optional uint64 contentLength = 2;
- // the information about the stripes
- repeated StripeInformation stripes = 3;
- // the schema information
- repeated Type types = 4;
- // the user metadata that was added
- repeated UserMetadataItem metadata = 5;
- // the total number of rows in the file
- optional uint64 numberOfRows = 6;
- // the statistics of each column across the file
- repeated ColumnStatistics statistics = 7;
- // the maximum number of rows in each index entry
- optional uint32 rowIndexStride = 8;
-}
-</code></p>
-
-<h3 id="stripe-information">Stripe Information</h3>
-
-<p>The body of the file is divided into stripes. Each stripe is self
-contained and may be read using only its own bytes combined with the
-file’s Footer and Postscript. Each stripe contains only entire rows so
-that rows never straddle stripe boundaries. Stripes have three
-sections: a set of indexes for the rows within the stripe, the data
-itself, and a stripe footer. Both the indexes and the data sections
-are divided by columns so that only the data for the required columns
-needs to be read.</p>
-
-<p><code>message StripeInformation {
- // the start of the stripe within the file
- optional uint64 offset = 1;
- // the length of the indexes in bytes
- optional uint64 indexLength = 2;
- // the length of the data in bytes
- optional uint64 dataLength = 3;
- // the length of the footer in bytes
- optional uint64 footerLength = 4;
- // the number of rows in the stripe
- optional uint64 numberOfRows = 5;
-}
-</code></p>
-
-<h3 id="type-information">Type Information</h3>
-
-<p>All of the rows in an ORC file must have the same schema. Logically
-the schema is expressed as a tree as in the figure below, where
-the compound types have subcolumns under them.</p>
-
-<p><img src="/img/TreeWriters.png" alt="ORC column structure" /></p>
-
-<p>The equivalent Hive DDL would be:</p>
-
-<p><code>create table Foobar (
- myInt int,
- myMap map&lt;string,
- struct&lt;myString : string,
- myDouble: double&gt;&gt;,
- myTime timestamp
-);
-</code></p>
-
-<p>The type tree is flattened in to a list via a pre-order traversal
-where each type is assigned the next id. Clearly the root of the type
-tree is always type id 0. Compound types have a field named subtypes
-that contains the list of their children’s type ids.</p>
-
-<p><code>message Type {
- enum Kind {
- BOOLEAN = 0;
- BYTE = 1;
- SHORT = 2;
- INT = 3;
- LONG = 4;
- FLOAT = 5;
- DOUBLE = 6;
- STRING = 7;
- BINARY = 8;
- TIMESTAMP = 9;
- LIST = 10;
- MAP = 11;
- STRUCT = 12;
- UNION = 13;
- DECIMAL = 14;
- DATE = 15;
- VARCHAR = 16;
- CHAR = 17;
- }
- // the kind of this type
- required Kind kind = 1;
- // the type ids of any subcolumns for list, map, struct, or union
- repeated uint32 subtypes = 2 [packed=true];
- // the list of field names for struct
- repeated string fieldNames = 3;
- // the maximum length of the type for varchar or char in UTF-8 characters
- optional uint32 maximumLength = 4;
- // the precision and scale for decimal
- optional uint32 precision = 5;
- optional uint32 scale = 6;
-}
-</code></p>
-
-<h3 id="column-statistics">Column Statistics</h3>
-
-<p>The goal of the column statistics is that for each column, the writer
-records the count and depending on the type other useful fields. For
-most of the primitive types, it records the minimum and maximum
-values; and for numeric types it additionally stores the sum.
-From Hive 1.1.0 onwards, the column statistics will also record if
-there are any null values within the row group by setting the hasNull flag.
-The hasNull flag is used by ORC’s predicate pushdown to better answer
-‘IS NULL’ queries.</p>
-
-<p><code>message ColumnStatistics {
- // the number of values
- optional uint64 numberOfValues = 1;
- // At most one of these has a value for any column
- optional IntegerStatistics intStatistics = 2;
- optional DoubleStatistics doubleStatistics = 3;
- optional StringStatistics stringStatistics = 4;
- optional BucketStatistics bucketStatistics = 5;
- optional DecimalStatistics decimalStatistics = 6;
- optional DateStatistics dateStatistics = 7;
- optional BinaryStatistics binaryStatistics = 8;
- optional TimestampStatistics timestampStatistics = 9;
- optional bool hasNull = 10;
-}
-</code></p>
-
-<p>For integer types (tinyint, smallint, int, bigint), the column
-statistics includes the minimum, maximum, and sum. If the sum
-overflows long at any point during the calculation, no sum is
-recorded.</p>
-
-<p><code>message IntegerStatistics {
- optional sint64 minimum = 1;
- optional sint64 maximum = 2;
- optional sint64 sum = 3;
-}
-</code></p>
-
-<p>For floating point types (float, double), the column statistics
-include the minimum, maximum, and sum. If the sum overflows a double,
-no sum is recorded.</p>
-
-<p><code>message DoubleStatistics {
- optional double minimum = 1;
- optional double maximum = 2;
- optional double sum = 3;
-}
-</code></p>
-
-<p>For strings, the minimum value, maximum value, and the sum of the
-lengths of the values are recorded.</p>
-
-<p><code>message StringStatistics {
- optional string minimum = 1;
- optional string maximum = 2;
- // sum will store the total length of all strings
- optional sint64 sum = 3;
-}
-</code></p>
-
-<p>For booleans, the statistics include the count of false and true values.</p>
-
-<p><code>message BucketStatistics {
- repeated uint64 count = 1 [packed=true];
-}
-</code></p>
-
-<p>For decimals, the minimum, maximum, and sum are stored.</p>
-
-<p><code>message DecimalStatistics {
- optional string minimum = 1;
- optional string maximum = 2;
- optional string sum = 3;
-}
-</code></p>
-
-<p>Date columns record the minimum and maximum values as the number of
-days since the epoch (1/1/2015).</p>
-
-<p><code>message DateStatistics {
- // min,max values saved as days since epoch
- optional sint32 minimum = 1;
- optional sint32 maximum = 2;
-}
-</code></p>
-
-<p>Timestamp columns record the minimum and maximum values as the number of
-milliseconds since the epoch (1/1/2015).</p>
-
-<p><code>message TimestampStatistics {
- // min,max values saved as milliseconds since epoch
- optional sint64 minimum = 1;
- optional sint64 maximum = 2;
-}
-</code></p>
-
-<p>Binary columns store the aggregate number of bytes across all of the values.</p>
-
-<p><code>message BinaryStatistics {
- // sum will store the total binary blob length
- optional sint64 sum = 1;
-}
-</code></p>
-
-<h3 id="user-metadata">User Metadata</h3>
-
-<p>The user can add arbitrary key/value pairs to an ORC file as it is
-written. The contents of the keys and values are completely
-application defined, but the key is a string and the value is
-binary. Care should be taken by applications to make sure that their
-keys are unique and in general should be prefixed with an organization
-code.</p>
-
-<p><code>message UserMetadataItem {
- // the user defined key
- required string name = 1;
- // the user defined binary value
- required bytes value = 2;
-}
-</code></p>
-
-<h3 id="file-metadata">File Metadata</h3>
-
-<p>The file Metadata section contains column statistics at the stripe
-level granularity. These statistics enable input split elimination
-based on the predicate push-down evaluated per a stripe.</p>
-
-<p><code>message StripeStatistics {
- repeated ColumnStatistics colStats = 1;
-}
-</code></p>
-
-<p><code>message Metadata {
- repeated StripeStatistics stripeStats = 1;
-}
-</code></p>
-
-<h1 id="compression">Compression</h1>
-
-<p>If the ORC file writer selects a generic compression codec (zlib or
-snappy), every part of the ORC file except for the Postscript is
-compressed with that codec. However, one of the requirements for ORC
-is that the reader be able to skip over compressed bytes without
-decompressing the entire stream. To manage this, ORC writes compressed
-streams in chunks with headers as in the figure below.
-To handle uncompressable data, if the compressed data is larger than
-the original, the original is stored and the isOriginal flag is
-set. Each header is 3 bytes long with (compressedLength * 2 +
-isOriginal) stored as a little endian value. For example, the header
-for a chunk that compressed to 100,000 bytes would be [0x40, 0x0d,
-0x03]. The header for 5 bytes that did not compress would be [0x0b,
-0x00, 0x00]. Each compression chunk is compressed independently so
-that as long as a decompressor starts at the top of a header, it can
-start decompressing without the previous bytes.</p>
-
-<p><img src="/img/CompressionStream.png" alt="compression streams" /></p>
-
-<p>The default compression chunk size is 256K, but writers can choose
-their own value. Larger chunks lead to better compression, but require
-more memory. The chunk size is recorded in the Postscript so that
-readers can allocate appropriately sized buffers. Readers are
-guaranteed that no chunk will expand to more than the compression chunk
-size.</p>
-
-<p>ORC files without generic compression write each stream directly
-with no headers.</p>
-
-<h1 id="run-length-encoding">Run Length Encoding</h1>
-
-<h2 id="base-128-varint">Base 128 Varint</h2>
-
-<p>Variable width integer encodings take advantage of the fact that most
-numbers are small and that having smaller encodings for small numbers
-shrinks the overall size of the data. ORC uses the varint format from
-Protocol Buffers, which writes data in little endian format using the
-low 7 bits of each byte. The high bit in each byte is set if the
-number continues into the next byte.</p>
-
-<table>
-  <thead>
-    <tr>
-      <th style="text-align: left">Unsigned Original</th>
-      <th style="text-align: left">Serialized</th>
-    </tr>
-  </thead>
-  <tbody>
-    <tr>
-      <td style="text-align: left">0</td>
-      <td style="text-align: left">0x00</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">1</td>
-      <td style="text-align: left">0x01</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">127</td>
-      <td style="text-align: left">0x7f</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">128</td>
-      <td style="text-align: left">0x80, 0x01</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">129</td>
-      <td style="text-align: left">0x81, 0x01</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">16,383</td>
-      <td style="text-align: left">0xff, 0x7f</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">16,384</td>
-      <td style="text-align: left">0x80, 0x80, 0x01</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">16,385</td>
-      <td style="text-align: left">0x81, 0x80, 0x01</td>
-    </tr>
-  </tbody>
-</table>
-
-<p>For signed integer types, the number is converted into an unsigned
-number using a zigzag encoding. Zigzag encoding moves the sign bit to
-the least significant bit using the expression (val « 1) ^ (val »
-63) and derives its name from the fact that positive and negative
-numbers alternate once encoded. The unsigned number is then serialized
-as above.</p>
-
-<table>
-  <thead>
-    <tr>
-      <th style="text-align: left">Signed Original</th>
-      <th style="text-align: left">Unsigned</th>
-    </tr>
-  </thead>
-  <tbody>
-    <tr>
-      <td style="text-align: left">0</td>
-      <td style="text-align: left">0</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">-1</td>
-      <td style="text-align: left">1</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">1</td>
-      <td style="text-align: left">2</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">-2</td>
-      <td style="text-align: left">3</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">2</td>
-      <td style="text-align: left">4</td>
-    </tr>
-  </tbody>
-</table>
-
-<h2 id="byte-run-length-encoding">Byte Run Length Encoding</h2>
-
-<p>For byte streams, ORC uses a very light weight encoding of identical
-values.</p>
-
-<ul>
-  <li>Run - a sequence of at least 3 identical values</li>
-  <li>Literals - a sequence of non-identical values</li>
-</ul>
-
-<p>The first byte of each group of values is a header than determines
-whether it is a run (value between 0 to 127) or literal list (value
-between -128 to -1). For runs, the control byte is the length of the
-run minus the length of the minimal run (3) and the control byte for
-literal lists is the negative length of the list. For example, a
-hundred 0’s is encoded as [0x61, 0x00] and the sequence 0x44, 0x45
-would be encoded as [0xfe, 0x44, 0x45]. The next group can choose
-either of the encodings.</p>
-
-<h2 id="boolean-run-length-encoding">Boolean Run Length Encoding</h2>
-
-<p>For encoding boolean types, the bits are put in the bytes from most
-significant to least significant. The bytes are encoded using byte run
-length encoding as described in the previous section. For example,
-the byte sequence [0xff, 0x80] would be one true followed by
-seven false values.</p>
-
-<h2 id="integer-run-length-encoding-version-1">Integer Run Length Encoding, version 1</h2>
-
-<p>In Hive 0.11 ORC files used Run Length Encoding version 1 (RLEv1),
-which provides a lightweight compression of signed or unsigned integer
-sequences. RLEv1 has two sub-encodings:</p>
-
-<ul>
-  <li>Run - a sequence of values that differ by a small fixed delta</li>
-  <li>Literals - a sequence of varint encoded values</li>
-</ul>
-
-<p>Runs start with an initial byte of 0x00 to 0x7f, which encodes the
-length of the run - 3. A second byte provides the fixed delta in the
-range of -128 to 127. Finally, the first value of the run is encoded
-as a base 128 varint.</p>
-
-<p>For example, if the sequence is 100 instances of 7 the encoding would
-start with 100 - 3, followed by a delta of 0, and a varint of 7 for
-an encoding of [0x61, 0x00, 0x07]. To encode the sequence of numbers
-running from 100 to 1, the first byte is 100 - 3, the delta is -1,
-and the varint is 100 for an encoding of [0x61, 0xff, 0x64].</p>
-
-<p>Literals start with an initial byte of 0x80 to 0xff, which corresponds
-to the negative of number of literals in the sequence. Following the
-header byte, the list of N varints is encoded. Thus, if there are
-no runs, the overhead is 1 byte for each 128 integers. The first 5
-prime numbers [2, 3, 4, 7, 11] would encoded as [0xfb, 0x02, 0x03,
-0x04, 0x07, 0xb].</p>
-
-<h2 id="integer-run-length-encoding-version-2">Integer Run Length Encoding, version 2</h2>
-
-<p>In Hive 0.12, ORC introduced Run Length Encoding version 2 (RLEv2),
-which has improved compression and fixed bit width encodings for
-faster expansion. RLEv2 uses four sub-encodings based on the data:</p>
-
-<ul>
-  <li>Short Repeat - used for short sequences with repeated values</li>
-  <li>Direct - used for random sequences with a fixed bit width</li>
-  <li>Patched Base - used for random sequences with a variable bit width</li>
-  <li>Delta - used for monotonically increasing or decreasing sequences</li>
-</ul>
-
-<h3 id="short-repeat">Short Repeat</h3>
-
-<p>The short repeat encoding is used for short repeating integer
-sequences with the goal of minimizing the overhead of the header. All
-of the bits listed in the header are from the first byte to the last
-and from most significant bit to least significant bit. If the type is
-signed, the value is zigzag encoded.</p>
-
-<ul>
-  <li>1 byte header
-    <ul>
-      <li>2 bits for encoding type (0)</li>
-      <li>3 bits for width (W) of repeating value (1 to 8 bytes)</li>
-      <li>3 bits for repeat count (3 to 10 values)</li>
-    </ul>
-  </li>
-  <li>W bytes in big endian format, which is zigzag encoded if they type
-is signed</li>
-</ul>
-
-<p>The unsigned sequence of [10000, 10000, 10000, 10000, 10000] would be
-serialized with short repeat encoding (0), a width of 2 bytes (1), and
-repeat count of 5 (2) as [0x0a, 0x27, 0x10].</p>
-
-<h3 id="direct">Direct</h3>
-
-<p>The direct encoding is used for integer sequences whose values have a
-relatively constant bit width. It encodes the values directly using a
-fixed width big endian encoding. The width of the values is encoded
-using the table below.</p>
-
-<p>The 5 bit width encoding table for RLEv2:</p>
-
-<table>
-  <thead>
-    <tr>
-      <th style="text-align: left">Width in Bits</th>
-      <th style="text-align: left">Encoded Value</th>
-      <th style="text-align: left">Notes</th>
-    </tr>
-  </thead>
-  <tbody>
-    <tr>
-      <td style="text-align: left">0</td>
-      <td style="text-align: left">0</td>
-      <td style="text-align: left">for delta encoding</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">1</td>
-      <td style="text-align: left">0</td>
-      <td style="text-align: left">for non-delta encoding</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">2</td>
-      <td style="text-align: left">1</td>
-      <td style="text-align: left"> </td>
-    </tr>
-    <tr>
-      <td style="text-align: left">4</td>
-      <td style="text-align: left">3</td>
-      <td style="text-align: left"> </td>
-    </tr>
-    <tr>
-      <td style="text-align: left">8</td>
-      <td style="text-align: left">7</td>
-      <td style="text-align: left"> </td>
-    </tr>
-    <tr>
-      <td style="text-align: left">16</td>
-      <td style="text-align: left">15</td>
-      <td style="text-align: left"> </td>
-    </tr>
-    <tr>
-      <td style="text-align: left">24</td>
-      <td style="text-align: left">23</td>
-      <td style="text-align: left"> </td>
-    </tr>
-    <tr>
-      <td style="text-align: left">32</td>
-      <td style="text-align: left">27</td>
-      <td style="text-align: left"> </td>
-    </tr>
-    <tr>
-      <td style="text-align: left">40</td>
-      <td style="text-align: left">28</td>
-      <td style="text-align: left"> </td>
-    </tr>
-    <tr>
-      <td style="text-align: left">48</td>
-      <td style="text-align: left">29</td>
-      <td style="text-align: left"> </td>
-    </tr>
-    <tr>
-      <td style="text-align: left">56</td>
-      <td style="text-align: left">30</td>
-      <td style="text-align: left"> </td>
-    </tr>
-    <tr>
-      <td style="text-align: left">64</td>
-      <td style="text-align: left">31</td>
-      <td style="text-align: left"> </td>
-    </tr>
-    <tr>
-      <td style="text-align: left">3</td>
-      <td style="text-align: left">2</td>
-      <td style="text-align: left">deprecated</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">5 &lt;= x &lt;= 7</td>
-      <td style="text-align: left">x - 1</td>
-      <td style="text-align: left">deprecated</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">9 &lt;= x &lt;= 15</td>
-      <td style="text-align: left">x - 1</td>
-      <td style="text-align: left">deprecated</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">17 &lt;= x &lt;= 21</td>
-      <td style="text-align: left">x - 1</td>
-      <td style="text-align: left">deprecated</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">26</td>
-      <td style="text-align: left">24</td>
-      <td style="text-align: left">deprecated</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">28</td>
-      <td style="text-align: left">25</td>
-      <td style="text-align: left">deprecated</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">30</td>
-      <td style="text-align: left">26</td>
-      <td style="text-align: left">deprecated</td>
-    </tr>
-  </tbody>
-</table>
-
-<ul>
-  <li>2 bytes header
-    <ul>
-      <li>2 bits for encoding type (1)</li>
-      <li>5 bits for encoded width (W) of values (1 to 64 bits) using the 5 bit
-width encoding table</li>
-      <li>9 bits for length (L) (1 to 512 values)</li>
-    </ul>
-  </li>
-  <li>W * L bits (padded to the next byte) encoded in big endian format, which is
-zigzag encoding if the type is signed</li>
-</ul>
-
-<p>The unsigned sequence of [23713, 43806, 57005, 48879] would be
-serialized with direct encoding (1), a width of 16 bits (15), and
-length of 4 (3) as [0x5e, 0x03, 0x5c, 0xa1, 0xab, 0x1e, 0xde, 0xad,
-0xbe, 0xef].</p>
-
-<h3 id="patched-base">Patched Base</h3>
-
-<p>The patched base encoding is used for integer sequences whose bit
-widths varies a lot. The minimum signed value of the sequence is found
-and subtracted from the other values. The bit width of those adjusted
-values is analyzed and the 90 percentile of the bit width is chosen
-as W. The 10\% of values larger than W use patches from a patch list
-to set the additional bits. Patches are encoded as a list of gaps in
-the index values and the additional value bits.</p>
-
-<ul>
-  <li>4 bytes header
-    <ul>
-      <li>2 bits for encoding type (2)</li>
-      <li>5 bits for encoded width (W) of values (1 to 64 bits) using the 5 bit
-  width encoding table</li>
-      <li>9 bits for length (L) (1 to 512 values)</li>
-      <li>3 bits for base value width (BW) (1 to 8 bytes)</li>
-      <li>5 bits for patch width (PW) (1 to 64 bits) using  the 5 bit width
-encoding table</li>
-      <li>3 bits for patch gap width (PGW) (1 to 8 bits)</li>
-      <li>5 bits for patch list length (PLL) (0 to 31 patches)</li>
-    </ul>
-  </li>
-  <li>Base value (BW bytes) - The base value is stored as a big endian value
-with negative values marked by the most significant bit set. If it that
-bit is set, the entire value is negated.</li>
-  <li>Data values (W * L bits padded to the byte) - A sequence of W bit positive
-values that are added to the base value.</li>
-  <li>Data values (W * L bits padded to the byte) - A sequence of W bit positive
-values that are added to the base value.</li>
-  <li>Patch list (PLL * (PGW + PW) bytes) - A list of patches for values
-that didn’t fit within W bits. Each entry in the list consists of a
-gap, which is the number of elements skipped from the previous
-patch, and a patch value. Patches are applied by logically or’ing
-the data values with the relevant patch shifted W bits left. If a
-patch is 0, it was introduced to skip over more than 255 items. The
-combined length of each patch (PGW + PW) must be less or equal to
-64.</li>
-</ul>
-
-<p>The unsigned sequence of [2030, 2000, 2020, 1000000, 2040, 2050, 2060, 2070,
-2080, 2090, 2100, 2110, 2120, 2130, 2140, 2150, 2160, 2170, 2180, 2190]
-has a minimum of 2000, which makes the adjusted
-sequence [30, 0, 20, 998000, 40, 50, 60, 70, 80, 90, 100, 110, 120, 130, 140,
-150, 160, 170, 180, 190]. It has an
-encoding of patched base (2), a bit width of 8 (7), a length of 20
-(19), a base value width of 2 bytes (1), a patch width of 12 bits (11),
-patch gap width of 2 bits (1), and a patch list length of 1 (1). The
-base value is 2000 and the combined result is [0x8e, 0x13, 0x2b, 0x21, 0x07,
-0xd0, 0x1e, 0x00, 0x14, 0x70, 0x28, 0x32, 0x3c, 0x46, 0x50, 0x5a, 0x64, 0x6e,
-0x78, 0x82, 0x8c, 0x96, 0xa0, 0xaa, 0xb4, 0xbe, 0xfc, 0xe8]</p>
-
-<h3 id="delta">Delta</h3>
-
-<p>The Delta encoding is used for monotonically increasing or decreasing
-sequences. The first two numbers in the sequence can not be identical,
-because the encoding is using the sign of the first delta to determine
-if the series is increasing or decreasing.</p>
-
-<ul>
-  <li>2 bytes header
-    <ul>
-      <li>2 bits for encoding type (3)</li>
-      <li>5 bits for encoded width (W) of deltas (0 to 64 bits) using the 5 bit
-width encoding table</li>
-      <li>9 bits for run length (L) (1 to 512 values)</li>
-    </ul>
-  </li>
-  <li>Base value - encoded as (signed or unsigned) varint</li>
-  <li>Delta base - encoded as signed varint</li>
-  <li>Delta values $W * (L - 2)$ bytes - encode each delta after the first
-one. If the delta base is positive, the sequence is increasing and if it is
-negative the sequence is decreasing.</li>
-</ul>
-
-<p>The unsigned sequence of [2, 3, 5, 7, 11, 13, 17, 19, 23, 29] would be
-serialized with delta encoding (3), a width of 4 bits (3), length of
-10 (9), a base of 2 (2), and first delta of 1 (2). The resulting
-sequence is [0xc6, 0x09, 0x02, 0x02, 0x22, 0x42, 0x42, 0x46].</p>
-
-<h1 id="stripes">Stripes</h1>
-
-<p>The body of ORC files consists of a series of stripes. Stripes are
-large (typically ~200MB) and independent of each other and are often
-processed by different tasks. The defining characteristic for columnar
-storage formats is that the data for each column is stored separately
-and that reading data out of the file should be proportional to the
-number of columns read.</p>
-
-<p>In ORC files, each column is stored in several streams that are stored
-next to each other in the file. For example, an integer column is
-represented as two streams PRESENT, which uses one with a bit per
-value recording if the value is non-null, and DATA, which records the
-non-null values. If all of a column’s values in a stripe are non-null,
-the PRESENT stream is omitted from the stripe. For binary data, ORC
-uses three streams PRESENT, DATA, and LENGTH, which stores the length
-of each value. The details of each type will be presented in the
-following subsections.</p>
-
-<h2 id="stripe-footer">Stripe Footer</h2>
-
-<p>The stripe footer contains the encoding of each column and the
-directory of the streams including their location.</p>
-
-<p><code>message StripeFooter {
- // the location of each stream
- repeated Stream streams = 1;
- // the encoding of each column
- repeated ColumnEncoding columns = 2;
-}
-</code></p>
-
-<p>To describe each stream, ORC stores the kind of stream, the column id,
-and the stream’s size in bytes. The details of what is stored in each stream
-depends on the type and encoding of the column.</p>
-
-<p><code>message Stream {
- enum Kind {
- // boolean stream of whether the next value is non-null
- PRESENT = 0;
- // the primary data stream
- DATA = 1;
- // the length of each value for variable length data
- LENGTH = 2;
- // the dictionary blob
- DICTIONARY\_DATA = 3;
- // deprecated prior to Hive 0.11
- // It was used to store the number of instances of each value in the
- // dictionary
- DICTIONARY_COUNT = 4;
- // a secondary data stream
- SECONDARY = 5;
- // the index for seeking to particular row groups
- ROW_INDEX = 6;
- // original bloom filters used before ORC-101
- BLOOM_FILTER = 7;
- // bloom filters that consistently use utf8
- BLOOM_FILTER_UTF8 = 8;
- }
- required Kind kind = 1;
- // the column id
- optional uint32 column = 2;
- // the number of bytes in the file
- optional uint64 length = 3;
-}
-</code></p>
-
-<p>Depending on their type several options for encoding are possible. The
-encodings are divided into direct or dictionary-based categories and
-further refined as to whether they use RLE v1 or v2.</p>
-
-<p><code>message ColumnEncoding {
- enum Kind {
- // the encoding is mapped directly to the stream using RLE v1
- DIRECT = 0;
- // the encoding uses a dictionary of unique values using RLE v1
- DICTIONARY = 1;
- // the encoding is direct using RLE v2
- DIRECT\_V2 = 2;
- // the encoding is dictionary-based using RLE v2
- DICTIONARY\_V2 = 3;
- }
- required Kind kind = 1;
- // for dictionary encodings, record the size of the dictionary
- optional uint32 dictionarySize = 2;
-}
-</code></p>
-
-<h1 id="column-encodings">Column Encodings</h1>
-
-<h2 id="smallint-int-and-bigint-columns">SmallInt, Int, and BigInt Columns</h2>
-
-<p>All of the 16, 32, and 64 bit integer column types use the same set of
-potential encodings, which is basically whether they use RLE v1 or
-v2. If the PRESENT stream is not included, all of the values are
-present. For values that have false bits in the present stream, no
-values are included in the data stream.</p>
-
-<table>
-  <thead>
-    <tr>
-      <th style="text-align: left">Encoding</th>
-      <th style="text-align: left">Stream Kind</th>
-      <th style="text-align: left">Optional</th>
-      <th style="text-align: left">Contents</th>
-    </tr>
-  </thead>
-  <tbody>
-    <tr>
-      <td style="text-align: left">DIRECT</td>
-      <td style="text-align: left">PRESENT</td>
-      <td style="text-align: left">Yes</td>
-      <td style="text-align: left">Boolean RLE</td>
-    </tr>
-    <tr>
-      <td style="text-align: left"> </td>
-      <td style="text-align: left">DATA</td>
-      <td style="text-align: left">No</td>
-      <td style="text-align: left">Signed Integer RLE v1</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">DIRECT_V2</td>
-      <td style="text-align: left">PRESENT</td>
-      <td style="text-align: left">Yes</td>
-      <td style="text-align: left">Boolean RLE</td>
-    </tr>
-    <tr>
-      <td style="text-align: left"> </td>
-      <td style="text-align: left">DATA</td>
-      <td style="text-align: left">No</td>
-      <td style="text-align: left">Signed Integer RLE v2</td>
-    </tr>
-  </tbody>
-</table>
-
-<h2 id="float-and-double-columns">Float and Double Columns</h2>
-
-<p>Floating point types are stored using IEEE 754 floating point bit
-layout. Float columns use 4 bytes per value and double columns use 8
-bytes.</p>
-
-<table>
-  <thead>
-    <tr>
-      <th style="text-align: left">Encoding</th>
-      <th style="text-align: left">Stream Kind</th>
-      <th style="text-align: left">Optional</th>
-      <th style="text-align: left">Contents</th>
-    </tr>
-  </thead>
-  <tbody>
-    <tr>
-      <td style="text-align: left">DIRECT</td>
-      <td style="text-align: left">PRESENT</td>
-      <td style="text-align: left">Yes</td>
-      <td style="text-align: left">Boolean RLE</td>
-    </tr>
-    <tr>
-      <td style="text-align: left"> </td>
-      <td style="text-align: left">DATA</td>
-      <td style="text-align: left">No</td>
-      <td style="text-align: left">IEEE 754 floating point representation</td>
-    </tr>
-  </tbody>
-</table>
-
-<h2 id="string-char-and-varchar-columns">String, Char, and VarChar Columns</h2>
-
-<p>String, char, and varchar columns may be encoded either using a
-dictionary encoding or a direct encoding. A direct encoding should be
-preferred when there are many distinct values. In all of the
-encodings, the PRESENT stream encodes whether the value is null. The
-Java ORC writer automatically picks the encoding after the first row
-group (10,000 rows).</p>
-
-<p>For direct encoding the UTF-8 bytes are saved in the DATA stream and
-the length of each value is written into the LENGTH stream. In direct
-encoding, if the values were [“Nevada”, “California”]; the DATA
-would be “NevadaCalifornia” and the LENGTH would be [6, 10].</p>
-
-<p>For dictionary encodings the dictionary is sorted and UTF-8 bytes of
-each unique value are placed into DICTIONARY_DATA. The length of each
-item in the dictionary is put into the LENGTH stream. The DATA stream
-consists of the sequence of references to the dictionary elements.</p>
-
-<p>In dictionary encoding, if the values were [“Nevada”,
-“California”, “Nevada”, “California”, and “Florida”]; the
-DICTIONARY_DATA would be “CaliforniaFloridaNevada” and LENGTH would
-be [10, 7, 6]. The DATA would be [2, 0, 2, 0, 1].</p>
-
-<table>
-  <thead>
-    <tr>
-      <th style="text-align: left">Encoding</th>
-      <th style="text-align: left">Stream Kind</th>
-      <th style="text-align: left">Optional</th>
-      <th style="text-align: left">Contents</th>
-    </tr>
-  </thead>
-  <tbody>
-    <tr>
-      <td style="text-align: left">DIRECT</td>
-      <td style="text-align: left">PRESENT</td>
-      <td style="text-align: left">Yes</td>
-      <td style="text-align: left">Boolean RLE</td>
-    </tr>
-    <tr>
-      <td style="text-align: left"> </td>
-      <td style="text-align: left">DATA</td>
-      <td style="text-align: left">No</td>
-      <td style="text-align: left">String contents</td>
-    </tr>
-    <tr>
-      <td style="text-align: left"> </td>
-      <td style="text-align: left">LENGTH</td>
-      <td style="text-align: left">No</td>
-      <td style="text-align: left">Unsigned Integer RLE v1</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">DICTIONARY</td>
-      <td style="text-align: left">PRESENT</td>
-      <td style="text-align: left">Yes</td>
-      <td style="text-align: left">Boolean RLE</td>
-    </tr>
-    <tr>
-      <td style="text-align: left"> </td>
-      <td style="text-align: left">DATA</td>
-      <td style="text-align: left">No</td>
-      <td style="text-align: left">Unsigned Integer RLE v1</td>
-    </tr>
-    <tr>
-      <td style="text-align: left"> </td>
-      <td style="text-align: left">DICTIONARY_DATA</td>
-      <td style="text-align: left">No</td>
-      <td style="text-align: left">String contents</td>
-    </tr>
-    <tr>
-      <td style="text-align: left"> </td>
-      <td style="text-align: left">LENGTH</td>
-      <td style="text-align: left">No</td>
-      <td style="text-align: left">Unsigned Integer RLE v1</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">DIRECT_V2</td>
-      <td style="text-align: left">PRESENT</td>
-      <td style="text-align: left">Yes</td>
-      <td style="text-align: left">Boolean RLE</td>
-    </tr>
-    <tr>
-      <td style="text-align: left"> </td>
-      <td style="text-align: left">DATA</td>
-      <td style="text-align: left">No</td>
-      <td style="text-align: left">String contents</td>
-    </tr>
-    <tr>
-      <td style="text-align: left"> </td>
-      <td style="text-align: left">LENGTH</td>
-      <td style="text-align: left">No</td>
-      <td style="text-align: left">Unsigned Integer RLE v2</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">DICTIONARY_V2</td>
-      <td style="text-align: left">PRESENT</td>
-      <td style="text-align: left">Yes</td>
-      <td style="text-align: left">Boolean RLE</td>
-    </tr>
-    <tr>
-      <td style="text-align: left"> </td>
-      <td style="text-align: left">DATA</td>
-      <td style="text-align: left">No</td>
-      <td style="text-align: left">Unsigned Integer RLE v2</td>
-    </tr>
-    <tr>
-      <td style="text-align: left"> </td>
-      <td style="text-align: left">DICTIONARY_DATA</td>
-      <td style="text-align: left">No</td>
-      <td style="text-align: left">String contents</td>
-    </tr>
-    <tr>
-      <td style="text-align: left"> </td>
-      <td style="text-align: left">LENGTH</td>
-      <td style="text-align: left">No</td>
-      <td style="text-align: left">Unsigned Integer RLE v2</td>
-    </tr>
-  </tbody>
-</table>
-
-<h2 id="boolean-columns">Boolean Columns</h2>
-
-<p>Boolean columns are rare, but have a simple encoding.</p>
-
-<table>
-  <thead>
-    <tr>
-      <th style="text-align: left">Encoding</th>
-      <th style="text-align: left">Stream Kind</th>
-      <th style="text-align: left">Optional</th>
-      <th style="text-align: left">Contents</th>
-    </tr>
-  </thead>
-  <tbody>
-    <tr>
-      <td style="text-align: left">DIRECT</td>
-      <td style="text-align: left">PRESENT</td>
-      <td style="text-align: left">Yes</td>
-      <td style="text-align: left">Boolean RLE</td>
-    </tr>
-    <tr>
-      <td style="text-align: left"> </td>
-      <td style="text-align: left">DATA</td>
-      <td style="text-align: left">No</td>
-      <td style="text-align: left">Boolean RLE</td>
-    </tr>
-  </tbody>
-</table>
-
-<h2 id="tinyint-columns">TinyInt Columns</h2>
-
-<p>TinyInt (byte) columns use byte run length encoding.</p>
-
-<table>
-  <thead>
-    <tr>
-      <th style="text-align: left">Encoding</th>
-      <th style="text-align: left">Stream Kind</th>
-      <th style="text-align: left">Optional</th>
-      <th style="text-align: left">Contents</th>
-    </tr>
-  </thead>
-  <tbody>
-    <tr>
-      <td style="text-align: left">DIRECT</td>
-      <td style="text-align: left">PRESENT</td>
-      <td style="text-align: left">Yes</td>
-      <td style="text-align: left">Boolean RLE</td>
-    </tr>
-    <tr>
-      <td style="text-align: left"> </td>
-      <td style="text-align: left">DATA</td>
-      <td style="text-align: left">No</td>
-      <td style="text-align: left">Byte RLE</td>
-    </tr>
-  </tbody>
-</table>
-
-<h2 id="binary-columns">Binary Columns</h2>
-
-<p>Binary data is encoded with a PRESENT stream, a DATA stream that records
-the contents, and a LENGTH stream that records the number of bytes per a
-value.</p>
-
-<table>
-  <thead>
-    <tr>
-      <th style="text-align: left">Encoding</th>
-      <th style="text-align: left">Stream Kind</th>
-      <th style="text-align: left">Optional</th>
-      <th style="text-align: left">Contents</th>
-    </tr>
-  </thead>
-  <tbody>
-    <tr>
-      <td style="text-align: left">DIRECT</td>
-      <td style="text-align: left">PRESENT</td>
-      <td style="text-align: left">Yes</td>
-      <td style="text-align: left">Boolean RLE</td>
-    </tr>
-    <tr>
-      <td style="text-align: left"> </td>
-      <td style="text-align: left">DATA</td>
-      <td style="text-align: left">No</td>
-      <td style="text-align: left">String contents</td>
-    </tr>
-    <tr>
-      <td style="text-align: left"> </td>
-      <td style="text-align: left">LENGTH</td>
-      <td style="text-align: left">No</td>
-      <td style="text-align: left">Unsigned Integer RLE v1</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">DIRECT_V2</td>
-      <td style="text-align: left">PRESENT</td>
-      <td style="text-align: left">Yes</td>
-      <td style="text-align: left">Boolean RLE</td>
-    </tr>
-    <tr>
-      <td style="text-align: left"> </td>
-      <td style="text-align: left">DATA</td>
-      <td style="text-align: left">No</td>
-      <td style="text-align: left">String contents</td>
-    </tr>
-    <tr>
-      <td style="text-align: left"> </td>
-      <td style="text-align: left">LENGTH</td>
-      <td style="text-align: left">No</td>
-      <td style="text-align: left">Unsigned Integer RLE v2</td>
-    </tr>
-  </tbody>
-</table>
-
-<h2 id="decimal-columns">Decimal Columns</h2>
-
-<p>Decimal was introduced in Hive 0.11 with infinite precision (the total
-number of digits). In Hive 0.13, the definition was change to limit
-the precision to a maximum of 38 digits, which conveniently uses 127
-bits plus a sign bit. The current encoding of decimal columns stores
-the integer representation of the value as an unbounded length zigzag
-encoded base 128 varint. The scale is stored in the SECONDARY stream
-as an signed integer.</p>
-
-<table>
-  <thead>
-    <tr>
-      <th style="text-align: left">Encoding</th>
-      <th style="text-align: left">Stream Kind</th>
-      <th style="text-align: left">Optional</th>
-      <th style="text-align: left">Contents</th>
-    </tr>
-  </thead>
-  <tbody>
-    <tr>
-      <td style="text-align: left">DIRECT</td>
-      <td style="text-align: left">PRESENT</td>
-      <td style="text-align: left">Yes</td>
-      <td style="text-align: left">Boolean RLE</td>
-    </tr>
-    <tr>
-      <td style="text-align: left"> </td>
-      <td style="text-align: left">DATA</td>
-      <td style="text-align: left">No</td>
-      <td style="text-align: left">Unbounded base 128 varints</td>
-    </tr>
-    <tr>
-      <td style="text-align: left"> </td>
-      <td style="text-align: left">SECONDARY</td>
-      <td style="text-align: left">No</td>
-      <td style="text-align: left">Unsigned Integer RLE v1</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">DIRECT_V2</td>
-      <td style="text-align: left">PRESENT</td>
-      <td style="text-align: left">Yes</td>
-      <td style="text-align: left">Boolean RLE</td>
-    </tr>
-    <tr>
-      <td style="text-align: left"> </td>
-      <td style="text-align: left">DATA</td>
-      <td style="text-align: left">No</td>
-      <td style="text-align: left">Unbounded base 128 varints</td>
-    </tr>
-    <tr>
-      <td style="text-align: left"> </td>
-      <td style="text-align: left">SECONDARY</td>
-      <td style="text-align: left">No</td>
-      <td style="text-align: left">Unsigned Integer RLE v2</td>
-    </tr>
-  </tbody>
-</table>
-
-<h2 id="date-columns">Date Columns</h2>
-
-<p>Date data is encoded with a PRESENT stream, a DATA stream that records
-the number of days after January 1, 1970 in UTC.</p>
-
-<table>
-  <thead>
-    <tr>
-      <th style="text-align: left">Encoding</th>
-      <th style="text-align: left">Stream Kind</th>
-      <th style="text-align: left">Optional</th>
-      <th style="text-align: left">Contents</th>
-    </tr>
-  </thead>
-  <tbody>
-    <tr>
-      <td style="text-align: left">DIRECT</td>
-      <td style="text-align: left">PRESENT</td>
-      <td style="text-align: left">Yes</td>
-      <td style="text-align: left">Boolean RLE</td>
-    </tr>
-    <tr>
-      <td style="text-align: left"> </td>
-      <td style="text-align: left">DATA</td>
-      <td style="text-align: left">No</td>
-      <td style="text-align: left">Signed Integer RLE v1</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">DIRECT_V2</td>
-      <td style="text-align: left">PRESENT</td>
-      <td style="text-align: left">Yes</td>
-      <td style="text-align: left">Boolean RLE</td>
-    </tr>
-    <tr>
-      <td style="text-align: left"> </td>
-      <td style="text-align: left">DATA</td>
-      <td style="text-align: left">No</td>
-      <td style="text-align: left">Signed Integer RLE v2</td>
-    </tr>
-  </tbody>
-</table>
-
-<h2 id="timestamp-columns">Timestamp Columns</h2>
-
-<p>Timestamp records times down to nanoseconds as a PRESENT stream that
-records non-null values, a DATA stream that records the number of
-seconds after 1 January 2015, and a SECONDARY stream that records the
-number of nanoseconds.</p>
-
-<p>Because the number of nanoseconds often has a large number of trailing
-zeros, the number has trailing decimal zero digits removed and the
-last three bits are used to record how many zeros were removed. Thus
-1000 nanoseconds would be serialized as 0x0b and 100000 would be
-serialized as 0x0d.</p>
-
-<table>
-  <thead>
-    <tr>
-      <th style="text-align: left">Encoding</th>
-      <th style="text-align: left">Stream Kind</th>
-      <th style="text-align: left">Optional</th>
-      <th style="text-align: left">Contents</th>
-    </tr>
-  </thead>
-  <tbody>
-    <tr>
-      <td style="text-align: left">DIRECT</td>
-      <td style="text-align: left">PRESENT</td>
-      <td style="text-align: left">Yes</td>
-      <td style="text-align: left">Boolean RLE</td>
-    </tr>
-    <tr>
-      <td style="text-align: left"> </td>
-      <td style="text-align: left">DATA</td>
-      <td style="text-align: left">No</td>
-      <td style="text-align: left">Signed Integer RLE v1</td>
-    </tr>
-    <tr>
-      <td style="text-align: left"> </td>
-      <td style="text-align: left">SECONDARY</td>
-      <td style="text-align: left">No</td>
-      <td style="text-align: left">Unsigned Integer RLE v1</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">DIRECT_V2</td>
-      <td style="text-align: left">PRESENT</td>
-      <td style="text-align: left">Yes</td>
-      <td style="text-align: left">Boolean RLE</td>
-    </tr>
-    <tr>
-      <td style="text-align: left"> </td>
-      <td style="text-align: left">DATA</td>
-      <td style="text-align: left">No</td>
-      <td style="text-align: left">Signed Integer RLE v2</td>
-    </tr>
-    <tr>
-      <td style="text-align: left"> </td>
-      <td style="text-align: left">SECONDARY</td>
-      <td style="text-align: left">No</td>
-      <td style="text-align: left">Unsigned Integer RLE v2</td>
-    </tr>
-  </tbody>
-</table>
-
-<h2 id="struct-columns">Struct Columns</h2>
-
-<p>Structs have no data themselves and delegate everything to their child
-columns except for their PRESENT stream. They have a child column
-for each of the fields.</p>
-
-<table>
-  <thead>
-    <tr>
-      <th style="text-align: left">Encoding</th>
-      <th style="text-align: left">Stream Kind</th>
-      <th style="text-align: left">Optional</th>
-      <th style="text-align: left">Contents</th>
-    </tr>
-  </thead>
-  <tbody>
-    <tr>
-      <td style="text-align: left">DIRECT</td>
-      <td style="text-align: left">PRESENT</td>
-      <td style="text-align: left">Yes</td>
-      <td style="text-align: left">Boolean RLE</td>
-    </tr>
-  </tbody>
-</table>
-
-<h2 id="list-columns">List Columns</h2>
-
-<p>Lists are encoded as the PRESENT stream and a length stream with
-number of items in each list. They have a single child column for the
-element values.</p>
-
-<table>
-  <thead>
-    <tr>
-      <th style="text-align: left">Encoding</th>
-      <th style="text-align: left">Stream Kind</th>
-      <th style="text-align: left">Optional</th>
-      <th style="text-align: left">Contents</th>
-    </tr>
-  </thead>
-  <tbody>
-    <tr>
-      <td style="text-align: left">DIRECT</td>
-      <td style="text-align: left">PRESENT</td>
-      <td style="text-align: left">Yes</td>
-      <td style="text-align: left">Boolean RLE</td>
-    </tr>
-    <tr>
-      <td style="text-align: left"> </td>
-      <td style="text-align: left">LENGTH</td>
-      <td style="text-align: left">No</td>
-      <td style="text-align: left">Unsigned Integer RLE v1</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">DIRECT_V2</td>
-      <td style="text-align: left">PRESENT</td>
-      <td style="text-align: left">Yes</td>
-      <td style="text-align: left">Boolean RLE</td>
-    </tr>
-    <tr>
-      <td style="text-align: left"> </td>
-      <td style="text-align: left">LENGTH</td>
-      <td style="text-align: left">No</td>
-      <td style="text-align: left">Unsigned Integer RLE v2</td>
-    </tr>
-  </tbody>
-</table>
-
-<h2 id="map-columns">Map Columns</h2>
-
-<p>Maps are encoded as the PRESENT stream and a length stream with number
-of items in each list. They have a child column for the key and
-another child column for the value.</p>
-
-<table>
-  <thead>
-    <tr>
-      <th style="text-align: left">Encoding</th>
-      <th style="text-align: left">Stream Kind</th>
-      <th style="text-align: left">Optional</th>
-      <th style="text-align: left">Contents</th>
-    </tr>
-  </thead>
-  <tbody>
-    <tr>
-      <td style="text-align: left">DIRECT</td>
-      <td style="text-align: left">PRESENT</td>
-      <td style="text-align: left">Yes</td>
-      <td style="text-align: left">Boolean RLE</td>
-    </tr>
-    <tr>
-      <td style="text-align: left"> </td>
-      <td style="text-align: left">LENGTH</td>
-      <td style="text-align: left">No</td>
-      <td style="text-align: left">Unsigned Integer RLE v1</td>
-    </tr>
-    <tr>
-      <td style="text-align: left">DIRECT_V2</td>
-      <td style="text-align: left">PRESENT</td>
-      <td style="text-align: left">Yes</td>
-      <td style="text-align: left">Boolean RLE</td>
-    </tr>
-    <tr>
-      <td style="text-align: left"> </td>
-      <td style="text-align: left">LENGTH</td>
-      <td style="text-align: left">No</td>
-      <td style="text-align: left">Unsigned Integer RLE v2</td>
-    </tr>
-  </tbody>
-</table>
-
-<h2 id="union-columns">Union Columns</h2>
-
-<p>Unions are encoded as the PRESENT stream and a tag stream that controls which
-potential variant is used. They have a child column for each variant of the
-union. Currently ORC union types are limited to 256 variants, which matches
-the Hive type model.</p>
-
-<table>
-  <thead>
-    <tr>
-      <th style="text-align: left">Encoding</th>
-      <th style="text-align: left">Stream Kind</th>
-      <th style="text-align: left">Optional</th>
-      <th style="text-align: left">Contents</th>
-    </tr>
-  </thead>
-  <tbody>
-    <tr>
-      <td style="text-align: left">DIRECT</td>
-      <td style="text-align: left">PRESENT</td>
-      <td style="text-align: left">Yes</td>
-      <td style="text-align: left">Boolean RLE</td>
-    </tr>
-    <tr>
-      <td style="text-align: left"> </td>
-      <td style="text-align: left">DIRECT</td>
-      <td style="text-align: left">No</td>
-      <td style="text-align: left">Byte RLE</td>
-    </tr>
-  </tbody>
-</table>
-
-<h1 id="indexes">Indexes</h1>
-
-<h2 id="row-group-index">Row Group Index</h2>
-
-<p>The row group indexes consist of a ROW_INDEX stream for each primitive
-column that has an entry for each row group. Row groups are controlled
-by the writer and default to 10,000 rows. Each RowIndexEntry gives the
-position of each stream for the column and the statistics for that row
-group.</p>
-
-<p>The index streams are placed at the front of the stripe, because in
-the default case of streaming they do not need to be read. They are
-only loaded when either predicate push down is being used or the
-reader seeks to a particular row.</p>
-
-<p><code>message RowIndexEntry {
- repeated uint64 positions = 1 [packed=true];
- optional ColumnStatistics statistics = 2;
-}
-</code></p>
-
-<p><code>message RowIndex {
- repeated RowIndexEntry entry = 1;
-}
-</code></p>
-
-<p>To record positions, each stream needs a sequence of numbers. For
-uncompressed streams, the position is the byte offset of the RLE run’s
-start location followed by the number of values that need to be
-consumed from the run. In compressed streams, the first number is the
-start of the compression chunk in the stream, followed by the number
-of decompressed bytes that need to be consumed, and finally the number
-of values consumed in the RLE.</p>
-
-<p>For columns with multiple streams, the sequences of positions in each
-stream are concatenated. That was an unfortunate decision on my part
-that we should fix at some point, because it makes code that uses the
-indexes error-prone.</p>
-
-<p>Because dictionaries are accessed randomly, there is not a position to
-record for the dictionary and the entire dictionary must be read even
-if only part of a stripe is being read.</p>
-
-<h2 id="bloom-filter-index">Bloom Filter Index</h2>
-
-<p>Bloom Filters are added to ORC indexes from Hive 1.2.0 onwards.
-Predicate pushdown can make use of bloom filters to better prune
-the row groups that do not satisfy the filter condition.
-The bloom filter indexes consist of a BLOOM_FILTER stream for each
-column specified through ‘orc.bloom.filter.columns’ table properties.
-A BLOOM_FILTER stream records a bloom filter entry for each row
-group (default to 10,000 rows) in a column. Only the row groups that
-satisfy min/max row index evaluation will be evaluated against the
-bloom filter index.</p>
-
-<p>Each BloomFilterEntry stores the number of hash functions (‘k’) used
-and the bitset backing the bloom filter. The original encoding (pre
-ORC-101) of bloom filters used the bitset field encoded as a repeating
-sequence of longs in the bitset field with a little endian encoding
-(0x1 is bit 0 and 0x2 is bit 1.) After ORC-101, the encoding is a
-sequence of bytes with a little endian encoding in the utf8bitset field.</p>
-
-<p><code>message BloomFilter {
- optional uint32 numHashFunctions = 1;
- repeated fixed64 bitset = 2;
- optional bytes utf8bitset = 3;
-}
-</code></p>
-
-<p><code>message BloomFilterIndex {
- repeated BloomFilter bloomFilter = 1;
-}
-</code></p>
-
-<p>Bloom filter internally uses two different hash functions to map a key
-to a position in the bit set. For tinyint, smallint, int, bigint, float
-and double types, Thomas Wang’s 64-bit integer hash function is used.
-Floats are converted to IEEE-754 32 bit representation
-(using Java’s Float.floatToIntBits(float)). Similary, Doubles are
-converted to IEEE-754 64 bit representation (using Java’s
-Double.doubleToLongBits(double)). All these primitive types
-are cast to long base type before being passed on to the hash function.
-For strings and binary types, Murmur3 64 bit hash algorithm is used.
-The 64 bit variant of Murmur3 considers only the most significant
-8 bytes of Murmur3 128-bit algorithm. The 64 bit hashcode generated
-from the above algorithms is used as a base to derive ‘k’ different
-hash functions. We use the idea mentioned in the paper “Less Hashing,
-Same Performance: Building a Better Bloom Filter” by Kirsch et. al. to
-quickly compute the k hashcodes.</p>
-
-<p>The algorithm for computing k hashcodes and setting the bit position
-in a bloom filter is as follows:</p>
-
-<ol>
-  <li>Get 64 bit base hash code from Murmur3 or Thomas Wang’s hash algorithm.</li>
-  <li>Split the above hashcode into two 32-bit hashcodes (say hash1 and hash2).</li>
-  <li>k’th hashcode is obtained by (where k &gt; 0):
-    <ul>
-      <li>combinedHash = hash1 + (k * hash2)</li>
-    </ul>
-  </li>
-  <li>If combinedHash is negative flip all the bits:
-    <ul>
-      <li>combinedHash = ~combinedHash</li>
-    </ul>
-  </li>
-  <li>Bit set position is obtained by performing modulo with m:
-    <ul>
-      <li>position = combinedHash % m</li>
-    </ul>
-  </li>
-  <li>Set the position in bit set. The LSB 6 bits identifies the long index
-within bitset and bit position within the long uses little endian order.
-    <ul>
-      <li>bitset[position »&gt; 6] |= (1L « position);</li>
-    </ul>
-  </li>
-</ol>
-
-<p>Bloom filter streams are interlaced with row group indexes. This placement
-makes it convenient to read the bloom filter stream and row index stream
-together in single read operation.</p>
-
-<p><img src="/img/BloomFilter.png" alt="bloom filter" /></p>
-
-      </article>
-    </div>
-
-    <div class="clear"></div>
-
-  </div>
-</section>
-
-
-  <footer role="contentinfo">
-  <p>The contents of this website are &copy;&nbsp;2018
-     <a href="https://www.apache.org/">Apache Software Foundation</a>
-     under the terms of the <a
-      href="https://www.apache.org/licenses/LICENSE-2.0.html">
-      Apache&nbsp;License&nbsp;v2</a>. Apache ORC and its logo are trademarks
-      of the Apache Software Foundation.</p>
-</footer>
-
-  <script>
-  var anchorForId = function (id) {
-    var anchor = document.createElement("a");
-    anchor.className = "header-link";
-    anchor.href      = "#" + id;
-    anchor.innerHTML = "<span class=\"sr-only\">Permalink</span><i class=\"fa fa-link\"></i>";
-    anchor.title = "Permalink";
-    return anchor;
-  };
-
-  var linkifyAnchors = function (level, containingElement) {
-    var headers = containingElement.getElementsByTagName("h" + level);
-    for (var h = 0; h < headers.length; h++) {
-      var header = headers[h];
-
-      if (typeof header.id !== "undefined" && header.id !== "") {
-        header.appendChild(anchorForId(header.id));
-      }
-    }
-  };
-
-  document.onreadystatechange = function () {
-    if (this.readyState === "complete") {
-      var contentBlock = document.getElementsByClassName("docs")[0] || document.getElementsByClassName("news")[0];
-      if (!contentBlock) {
-        return;
-      }
-      for (var level = 1; level <= 6; level++) {
-        linkifyAnchors(level, contentBlock);
-      }
-    }
-  };
-</script>
-
-
-</body>
-</html>


[03/11] orc git commit: Deploy docs with CVE.

Posted by om...@apache.org.
http://git-wip-us.apache.org/repos/asf/orc/blob/e58d3fc4/specification/ORCv1/index.html
----------------------------------------------------------------------
diff --git a/specification/ORCv1/index.html b/specification/ORCv1/index.html
new file mode 100644
index 0000000..08c07eb
--- /dev/null
+++ b/specification/ORCv1/index.html
@@ -0,0 +1,1317 @@
+<!DOCTYPE HTML>
+<html lang="en-US">
+<head>
+  <meta charset="UTF-8">
+  <title>ORC Specification v1</title>
+  <meta name="viewport" content="width=device-width,initial-scale=1">
+  <meta name="generator" content="Jekyll v3.7.3">
+  <link rel="stylesheet" href="//fonts.googleapis.com/css?family=Lato:300,300italic,400,400italic,700,700italic,900">
+  <link rel="stylesheet" href="/css/screen.css">
+  <link rel="icon" type="image/x-icon" href="/favicon.ico">
+  <!--[if lt IE 9]>
+  <script src="/js/html5shiv.min.js"></script>
+  <script src="/js/respond.min.js"></script>
+  <![endif]-->
+</head>
+
+
+<body class="wrap">
+  <header role="banner">
+  <nav class="mobile-nav show-on-mobiles">
+    <ul>
+  <li class="">
+    <a href="/">Home</a>
+  </li>
+  <li class="">
+    <a href="/docs/"><span class="show-on-mobiles">Docs</span>
+                     <span class="hide-on-mobiles">Documentation</span></a>
+  </li>
+  <li class="">
+    <a href="/talks/">Talks</a>
+  </li>
+  <li class="">
+    <a href="/news/">News</a>
+  </li>
+  <li class="">
+    <a href="/help/">Help</a>
+  </li>
+  <li class="">
+    <a href="/develop/">Develop</a>
+  </li>
+</ul>
+
+  </nav>
+  <div class="grid">
+    <div class="unit one-third center-on-mobiles">
+      <h1>
+        <a href="/">
+          <span class="sr-only">Apache ORC</span>
+          <img src="/img/logo.png" width="249" height="101" alt="ORC Logo">
+        </a>
+      </h1>
+    </div>
+    <nav class="main-nav unit two-thirds hide-on-mobiles">
+      <ul>
+  <li class="">
+    <a href="/">Home</a>
+  </li>
+  <li class="">
+    <a href="/docs/"><span class="show-on-mobiles">Docs</span>
+                     <span class="hide-on-mobiles">Documentation</span></a>
+  </li>
+  <li class="">
+    <a href="/talks/">Talks</a>
+  </li>
+  <li class="">
+    <a href="/news/">News</a>
+  </li>
+  <li class="">
+    <a href="/help/">Help</a>
+  </li>
+  <li class="">
+    <a href="/develop/">Develop</a>
+  </li>
+</ul>
+
+    </nav>
+  </div>
+</header>
+
+
+  <section class="standalone">
+  <div class="grid">
+
+    <div class="unit whole">
+      <article>
+        <h1>ORC Specification v1</h1>
+        <p>This version of the file format was originally released as part of
+Hive 0.12.</p>
+
+<h1 id="motivation">Motivation</h1>
+
+<p>Hive’s RCFile was the standard format for storing tabular data in
+Hadoop for several years. However, RCFile has limitations because it
+treats each column as a binary blob without semantics. In Hive 0.11 we
+added a new file format named Optimized Row Columnar (ORC) file that
+uses and retains the type information from the table definition. ORC
+uses type specific readers and writers that provide light weight
+compression techniques such as dictionary encoding, bit packing, delta
+encoding, and run length encoding – resulting in dramatically smaller
+files. Additionally, ORC can apply generic compression using zlib, or
+Snappy on top of the lightweight compression for even smaller
+files. However, storage savings are only part of the gain. ORC
+supports projection, which selects subsets of the columns for reading,
+so that queries reading only one column read only the required
+bytes. Furthermore, ORC files include light weight indexes that
+include the minimum and maximum values for each column in each set of
+10,000 rows and the entire file. Using pushdown filters from Hive, the
+file reader can skip entire sets of rows that aren’t important for
+this query.</p>
+
+<p><img src="http://localhost:4000/img/OrcFileLayout.png" alt="ORC file structure" /></p>
+
+<h1 id="file-tail">File Tail</h1>
+
+<p>Since HDFS does not support changing the data in a file after it is
+written, ORC stores the top level index at the end of the file. The
+overall structure of the file is given in the figure above.  The
+file’s tail consists of 3 parts; the file metadata, file footer and
+postscript.</p>
+
+<p>The metadata for ORC is stored using
+<a href="https://s.apache.org/protobuf_encoding">Protocol Buffers</a>, which provides
+the ability to add new fields without breaking readers. This document
+incorporates the Protobuf definition from the
+<a href="https://s.apache.org/orc_proto">ORC source code</a> and the
+reader is encouraged to review the Protobuf encoding if they need to
+understand the byte-level encoding</p>
+
+<h2 id="postscript">Postscript</h2>
+
+<p>The Postscript section provides the necessary information to interpret
+the rest of the file including the length of the file’s Footer and
+Metadata sections, the version of the file, and the kind of general
+compression used (eg. none, zlib, or snappy). The Postscript is never
+compressed and ends one byte before the end of the file. The version
+stored in the Postscript is the lowest version of Hive that is
+guaranteed to be able to read the file and it stored as a sequence of
+the major and minor version. This file version is encoded as [0,12].</p>
+
+<p>The process of reading an ORC file works backwards through the
+file. Rather than making multiple short reads, the ORC reader reads
+the last 16k bytes of the file with the hope that it will contain both
+the Footer and Postscript sections. The final byte of the file
+contains the serialized length of the Postscript, which must be less
+than 256 bytes. Once the Postscript is parsed, the compressed
+serialized length of the Footer is known and it can be decompressed
+and parsed.</p>
+
+<p>```message PostScript {
+ // the length of the footer section in bytes
+ optional uint64 footerLength = 1;
+ // the kind of generic compression used
+ optional CompressionKind compression = 2;
+ // the maximum size of each compression chunk
+ optional uint64 compressionBlockSize = 3;
+ // the version of the writer
+ repeated uint32 version = 4 [packed = true];
+ // the length of the metadata section in bytes
+ optional uint64 metadataLength = 5;
+ // the fixed string “ORC”
+ optional string magic = 8000;
+}</p>
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
+```enum CompressionKind {
+ NONE = 0;
+ ZLIB = 1;
+ SNAPPY = 2;
+ LZO = 3;
+ LZ4 = 4;
+ ZSTD = 5;
+}
+</code></pre></div></div>
+
+<h2 id="footer">Footer</h2>
+
+<p>The Footer section contains the layout of the body of the file, the
+type schema information, the number of rows, and the statistics about
+each of the columns.</p>
+
+<p>The file is broken in to three parts- Header, Body, and Tail. The
+Header consists of the bytes “ORC’’ to support tools that want to
+scan the front of the file to determine the type of the file. The Body
+contains the rows and indexes, and the Tail gives the file level
+information as described in this section.</p>
+
+<p>```message Footer {
+ // the length of the file header in bytes (always 3)
+ optional uint64 headerLength = 1;
+ // the length of the file header and body in bytes
+ optional uint64 contentLength = 2;
+ // the information about the stripes
+ repeated StripeInformation stripes = 3;
+ // the schema information
+ repeated Type types = 4;
+ // the user metadata that was added
+ repeated UserMetadataItem metadata = 5;
+ // the total number of rows in the file
+ optional uint64 numberOfRows = 6;
+ // the statistics of each column across the file
+ repeated ColumnStatistics statistics = 7;
+ // the maximum number of rows in each index entry
+ optional uint32 rowIndexStride = 8;
+}</p>
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
+### Stripe Information
+
+The body of the file is divided into stripes. Each stripe is self
+contained and may be read using only its own bytes combined with the
+file's Footer and Postscript. Each stripe contains only entire rows so
+that rows never straddle stripe boundaries. Stripes have three
+sections: a set of indexes for the rows within the stripe, the data
+itself, and a stripe footer. Both the indexes and the data sections
+are divided by columns so that only the data for the required columns
+needs to be read.
+
+```message StripeInformation {
+ // the start of the stripe within the file
+ optional uint64 offset = 1;
+ // the length of the indexes in bytes
+ optional uint64 indexLength = 2;
+ // the length of the data in bytes
+ optional uint64 dataLength = 3;
+ // the length of the footer in bytes
+ optional uint64 footerLength = 4;
+ // the number of rows in the stripe
+ optional uint64 numberOfRows = 5;
+}
+</code></pre></div></div>
+
+<h3 id="type-information">Type Information</h3>
+
+<p>All of the rows in an ORC file must have the same schema. Logically
+the schema is expressed as a tree as in the figure below, where
+the compound types have subcolumns under them.</p>
+
+<p><img src="http://localhost:4000/img/TreeWriters.png" alt="ORC column structure" /></p>
+
+<p>The equivalent Hive DDL would be:</p>
+
+<p>```create table Foobar (
+ myInt int,
+ myMap map&lt;string,
+ struct&lt;myString : string,
+ myDouble: double»,
+ myTime timestamp
+);</p>
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
+The type tree is flattened in to a list via a pre-order traversal
+where each type is assigned the next id. Clearly the root of the type
+tree is always type id 0. Compound types have a field named subtypes
+that contains the list of their children's type ids.
+
+```message Type {
+ enum Kind {
+ BOOLEAN = 0;
+ BYTE = 1;
+ SHORT = 2;
+ INT = 3;
+ LONG = 4;
+ FLOAT = 5;
+ DOUBLE = 6;
+ STRING = 7;
+ BINARY = 8;
+ TIMESTAMP = 9;
+ LIST = 10;
+ MAP = 11;
+ STRUCT = 12;
+ UNION = 13;
+ DECIMAL = 14;
+ DATE = 15;
+ VARCHAR = 16;
+ CHAR = 17;
+ }
+ // the kind of this type
+ required Kind kind = 1;
+ // the type ids of any subcolumns for list, map, struct, or union
+ repeated uint32 subtypes = 2 [packed=true];
+ // the list of field names for struct
+ repeated string fieldNames = 3;
+ // the maximum length of the type for varchar or char in UTF-8 characters
+ optional uint32 maximumLength = 4;
+ // the precision and scale for decimal
+ optional uint32 precision = 5;
+ optional uint32 scale = 6;
+}
+</code></pre></div></div>
+
+<h3 id="column-statistics">Column Statistics</h3>
+
+<p>The goal of the column statistics is that for each column, the writer
+records the count and depending on the type other useful fields. For
+most of the primitive types, it records the minimum and maximum
+values; and for numeric types it additionally stores the sum.
+From Hive 1.1.0 onwards, the column statistics will also record if
+there are any null values within the row group by setting the hasNull flag.
+The hasNull flag is used by ORC’s predicate pushdown to better answer
+‘IS NULL’ queries.</p>
+
+<p>```message ColumnStatistics {
+ // the number of values
+ optional uint64 numberOfValues = 1;
+ // At most one of these has a value for any column
+ optional IntegerStatistics intStatistics = 2;
+ optional DoubleStatistics doubleStatistics = 3;
+ optional StringStatistics stringStatistics = 4;
+ optional BucketStatistics bucketStatistics = 5;
+ optional DecimalStatistics decimalStatistics = 6;
+ optional DateStatistics dateStatistics = 7;
+ optional BinaryStatistics binaryStatistics = 8;
+ optional TimestampStatistics timestampStatistics = 9;
+ optional bool hasNull = 10;
+}</p>
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
+For integer types (tinyint, smallint, int, bigint), the column
+statistics includes the minimum, maximum, and sum. If the sum
+overflows long at any point during the calculation, no sum is
+recorded.
+
+```message IntegerStatistics {
+ optional sint64 minimum = 1;
+ optional sint64 maximum = 2;
+ optional sint64 sum = 3;
+}
+</code></pre></div></div>
+
+<p>For floating point types (float, double), the column statistics
+include the minimum, maximum, and sum. If the sum overflows a double,
+no sum is recorded.</p>
+
+<p>```message DoubleStatistics {
+ optional double minimum = 1;
+ optional double maximum = 2;
+ optional double sum = 3;
+}</p>
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
+For strings, the minimum value, maximum value, and the sum of the
+lengths of the values are recorded.
+
+```message StringStatistics {
+ optional string minimum = 1;
+ optional string maximum = 2;
+ // sum will store the total length of all strings
+ optional sint64 sum = 3;
+}
+</code></pre></div></div>
+
+<p>For booleans, the statistics include the count of false and true values.</p>
+
+<p>```message BucketStatistics {
+ repeated uint64 count = 1 [packed=true];
+}</p>
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
+For decimals, the minimum, maximum, and sum are stored.
+
+```message DecimalStatistics {
+ optional string minimum = 1;
+ optional string maximum = 2;
+ optional string sum = 3;
+}
+</code></pre></div></div>
+
+<p>Date columns record the minimum and maximum values as the number of
+days since the epoch (1/1/2015).</p>
+
+<p>```message DateStatistics {
+ // min,max values saved as days since epoch
+ optional sint32 minimum = 1;
+ optional sint32 maximum = 2;
+}</p>
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
+Timestamp columns record the minimum and maximum values as the number of
+milliseconds since the epoch (1/1/2015).
+
+```message TimestampStatistics {
+ // min,max values saved as milliseconds since epoch
+ optional sint64 minimum = 1;
+ optional sint64 maximum = 2;
+}
+</code></pre></div></div>
+
+<p>Binary columns store the aggregate number of bytes across all of the values.</p>
+
+<p>```message BinaryStatistics {
+ // sum will store the total binary blob length
+ optional sint64 sum = 1;
+}</p>
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
+### User Metadata
+
+The user can add arbitrary key/value pairs to an ORC file as it is
+written. The contents of the keys and values are completely
+application defined, but the key is a string and the value is
+binary. Care should be taken by applications to make sure that their
+keys are unique and in general should be prefixed with an organization
+code.
+
+```message UserMetadataItem {
+ // the user defined key
+ required string name = 1;
+ // the user defined binary value
+ required bytes value = 2;
+}
+</code></pre></div></div>
+
+<h3 id="file-metadata">File Metadata</h3>
+
+<p>The file Metadata section contains column statistics at the stripe
+level granularity. These statistics enable input split elimination
+based on the predicate push-down evaluated per a stripe.</p>
+
+<p>```message StripeStatistics {
+ repeated ColumnStatistics colStats = 1;
+}</p>
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
+```message Metadata {
+ repeated StripeStatistics stripeStats = 1;
+}
+</code></pre></div></div>
+
+<h1 id="compression">Compression</h1>
+
+<p>If the ORC file writer selects a generic compression codec (zlib or
+snappy), every part of the ORC file except for the Postscript is
+compressed with that codec. However, one of the requirements for ORC
+is that the reader be able to skip over compressed bytes without
+decompressing the entire stream. To manage this, ORC writes compressed
+streams in chunks with headers as in the figure below.
+To handle uncompressable data, if the compressed data is larger than
+the original, the original is stored and the isOriginal flag is
+set. Each header is 3 bytes long with (compressedLength * 2 +
+isOriginal) stored as a little endian value. For example, the header
+for a chunk that compressed to 100,000 bytes would be [0x40, 0x0d,
+0x03]. The header for 5 bytes that did not compress would be [0x0b,
+0x00, 0x00]. Each compression chunk is compressed independently so
+that as long as a decompressor starts at the top of a header, it can
+start decompressing without the previous bytes.</p>
+
+<p><img src="http://localhost:4000/img/CompressionStream.png" alt="compression streams" /></p>
+
+<p>The default compression chunk size is 256K, but writers can choose
+their own value. Larger chunks lead to better compression, but require
+more memory. The chunk size is recorded in the Postscript so that
+readers can allocate appropriately sized buffers. Readers are
+guaranteed that no chunk will expand to more than the compression chunk
+size.</p>
+
+<p>ORC files without generic compression write each stream directly
+with no headers.</p>
+
+<h1 id="run-length-encoding">Run Length Encoding</h1>
+
+<h2 id="base-128-varint">Base 128 Varint</h2>
+
+<p>Variable width integer encodings take advantage of the fact that most
+numbers are small and that having smaller encodings for small numbers
+shrinks the overall size of the data. ORC uses the varint format from
+Protocol Buffers, which writes data in little endian format using the
+low 7 bits of each byte. The high bit in each byte is set if the
+number continues into the next byte.</p>
+
+<table>
+  <thead>
+    <tr>
+      <th style="text-align: left">Unsigned Original</th>
+      <th style="text-align: left">Serialized</th>
+    </tr>
+  </thead>
+  <tbody>
+    <tr>
+      <td style="text-align: left">0</td>
+      <td style="text-align: left">0x00</td>
+    </tr>
+    <tr>
+      <td style="text-align: left">1</td>
+      <td style="text-align: left">0x01</td>
+    </tr>
+    <tr>
+      <td style="text-align: left">127</td>
+      <td style="text-align: left">0x7f</td>
+    </tr>
+    <tr>
+      <td style="text-align: left">128</td>
+      <td style="text-align: left">0x80, 0x01</td>
+    </tr>
+    <tr>
+      <td style="text-align: left">129</td>
+      <td style="text-align: left">0x81, 0x01</td>
+    </tr>
+    <tr>
+      <td style="text-align: left">16,383</td>
+      <td style="text-align: left">0xff, 0x7f</td>
+    </tr>
+    <tr>
+      <td style="text-align: left">16,384</td>
+      <td style="text-align: left">0x80, 0x80, 0x01</td>
+    </tr>
+    <tr>
+      <td style="text-align: left">16,385</td>
+      <td style="text-align: left">0x81, 0x80, 0x01</td>
+    </tr>
+  </tbody>
+</table>
+
+<p>For signed integer types, the number is converted into an unsigned
+number using a zigzag encoding. Zigzag encoding moves the sign bit to
+the least significant bit using the expression (val « 1) ^ (val »
+63) and derives its name from the fact that positive and negative
+numbers alternate once encoded. The unsigned number is then serialized
+as above.</p>
+
+<table>
+  <thead>
+    <tr>
+      <th style="text-align: left">Signed Original</th>
+      <th style="text-align: left">Unsigned</th>
+    </tr>
+  </thead>
+  <tbody>
+    <tr>
+      <td style="text-align: left">0</td>
+      <td style="text-align: left">0</td>
+    </tr>
+    <tr>
+      <td style="text-align: left">-1</td>
+      <td style="text-align: left">1</td>
+    </tr>
+    <tr>
+      <td style="text-align: left">1</td>
+      <td style="text-align: left">2</td>
+    </tr>
+    <tr>
+      <td style="text-align: left">-2</td>
+      <td style="text-align: left">3</td>
+    </tr>
+    <tr>
+      <td style="text-align: left">2</td>
+      <td style="text-align: left">4</td>
+    </tr>
+  </tbody>
+</table>
+
+<h2 id="byte-run-length-encoding">Byte Run Length Encoding</h2>
+
+<p>For byte streams, ORC uses a very light weight encoding of identical
+values.</p>
+
+<ul>
+  <li>Run - a sequence of at least 3 identical values</li>
+  <li>Literals - a sequence of non-identical values</li>
+</ul>
+
+<p>The first byte of each group of values is a header than determines
+whether it is a run (value between 0 to 127) or literal list (value
+between -128 to -1). For runs, the control byte is the length of the
+run minus the length of the minimal run (3) and the control byte for
+literal lists is the negative length of the list. For example, a
+hundred 0’s is encoded as [0x61, 0x00] and the sequence 0x44, 0x45
+would be encoded as [0xfe, 0x44, 0x45]. The next group can choose
+either of the encodings.</p>
+
+<h2 id="boolean-run-length-encoding">Boolean Run Length Encoding</h2>
+
+<p>For encoding boolean types, the bits are put in the bytes from most
+significant to least significant. The bytes are encoded using byte run
+length encoding as described in the previous section. For example,
+the byte sequence [0xff, 0x80] would be one true followed by
+seven false values.</p>
+
+<h2 id="integer-run-length-encoding-version-1">Integer Run Length Encoding, version 1</h2>
+
+<p>In Hive 0.11 ORC files used Run Length Encoding version 1 (RLEv1),
+which provides a lightweight compression of signed or unsigned integer
+sequences. RLEv1 has two sub-encodings:</p>
+
+<ul>
+  <li>Run - a sequence of values that differ by a small fixed delta</li>
+  <li>Literals - a sequence of varint encoded values</li>
+</ul>
+
+<p>Runs start with an initial byte of 0x00 to 0x7f, which encodes the
+length of the run - 3. A second byte provides the fixed delta in the
+range of -128 to 127. Finally, the first value of the run is encoded
+as a base 128 varint.</p>
+
+<p>For example, if the sequence is 100 instances of 7 the encoding would
+start with 100 - 3, followed by a delta of 0, and a varint of 7 for
+an encoding of [0x61, 0x00, 0x07]. To encode the sequence of numbers
+running from 100 to 1, the first byte is 100 - 3, the delta is -1,
+and the varint is 100 for an encoding of [0x61, 0xff, 0x64].</p>
+
+<p>Literals start with an initial byte of 0x80 to 0xff, which corresponds
+to the negative of number of literals in the sequence. Following the
+header byte, the list of N varints is encoded. Thus, if there are
+no runs, the overhead is 1 byte for each 128 integers. The first 5
+prime numbers [2, 3, 4, 7, 11] would encoded as [0xfb, 0x02, 0x03,
+0x04, 0x07, 0xb].</p>
+
+<h2 id="integer-run-length-encoding-version-2">Integer Run Length Encoding, version 2</h2>
+
+<p>In Hive 0.12, ORC introduced Run Length Encoding version 2 (RLEv2),
+which has improved compression and fixed bit width encodings for
+faster expansion. RLEv2 uses four sub-encodings based on the data:</p>
+
+<ul>
+  <li>Short Repeat - used for short sequences with repeated values</li>
+  <li>Direct - used for random sequences with a fixed bit width</li>
+  <li>Patched Base - used for random sequences with a variable bit width</li>
+  <li>Delta - used for monotonically increasing or decreasing sequences</li>
+</ul>
+
+<h3 id="short-repeat">Short Repeat</h3>
+
+<p>The short repeat encoding is used for short repeating integer
+sequences with the goal of minimizing the overhead of the header. All
+of the bits listed in the header are from the first byte to the last
+and from most significant bit to least significant bit. If the type is
+signed, the value is zigzag encoded.</p>
+
+<ul>
+  <li>1 byte header
+    <ul>
+      <li>2 bits for encoding type (0)</li>
+      <li>3 bits for width (W) of repeating value (1 to 8 bytes)</li>
+      <li>3 bits for repeat count (3 to 10 values)</li>
+    </ul>
+  </li>
+  <li>W bytes in big endian format, which is zigzag encoded if they type
+is signed</li>
+</ul>
+
+<p>The unsigned sequence of [10000, 10000, 10000, 10000, 10000] would be
+serialized with short repeat encoding (0), a width of 2 bytes (1), and
+repeat count of 5 (2) as [0x0a, 0x27, 0x10].</p>
+
+<h3 id="direct">Direct</h3>
+
+<p>The direct encoding is used for integer sequences whose values have a
+relatively constant bit width. It encodes the values directly using a
+fixed width big endian encoding. The width of the values is encoded
+using the table below.</p>
+
+<p>The 5 bit width encoding table for RLEv2:</p>
+
+<table>
+  <thead>
+    <tr>
+      <th style="text-align: left">Width in Bits</th>
+      <th style="text-align: left">Encoded Value</th>
+      <th style="text-align: left">Notes</th>
+    </tr>
+  </thead>
+  <tbody>
+    <tr>
+      <td style="text-align: left">0</td>
+      <td style="text-align: left">0</td>
+      <td style="text-align: left">for delta encoding</td>
+    </tr>
+    <tr>
+      <td style="text-align: left">1</td>
+      <td style="text-align: left">0</td>
+      <td style="text-align: left">for non-delta encoding</td>
+    </tr>
+    <tr>
+      <td style="text-align: left">2</td>
+      <td style="text-align: left">1</td>
+      <td style="text-align: left"> </td>
+    </tr>
+    <tr>
+      <td style="text-align: left">4</td>
+      <td style="text-align: left">3</td>
+      <td style="text-align: left"> </td>
+    </tr>
+    <tr>
+      <td style="text-align: left">8</td>
+      <td style="text-align: left">7</td>
+      <td style="text-align: left"> </td>
+    </tr>
+    <tr>
+      <td style="text-align: left">16</td>
+      <td style="text-align: left">15</td>
+      <td style="text-align: left"> </td>
+    </tr>
+    <tr>
+      <td style="text-align: left">24</td>
+      <td style="text-align: left">23</td>
+      <td style="text-align: left"> </td>
+    </tr>
+    <tr>
+      <td style="text-align: left">32</td>
+      <td style="text-align: left">27</td>
+      <td style="text-align: left"> </td>
+    </tr>
+    <tr>
+      <td style="text-align: left">40</td>
+      <td style="text-align: left">28</td>
+      <td style="text-align: left"> </td>
+    </tr>
+    <tr>
+      <td style="text-align: left">48</td>
+      <td style="text-align: left">29</td>
+      <td style="text-align: left"> </td>
+    </tr>
+    <tr>
+      <td style="text-align: left">56</td>
+      <td style="text-align: left">30</td>
+      <td style="text-align: left"> </td>
+    </tr>
+    <tr>
+      <td style="text-align: left">64</td>
+      <td style="text-align: left">31</td>
+      <td style="text-align: left"> </td>
+    </tr>
+    <tr>
+      <td style="text-align: left">3</td>
+      <td style="text-align: left">2</td>
+      <td style="text-align: left">deprecated</td>
+    </tr>
+    <tr>
+      <td style="text-align: left">5 &lt;= x &lt;= 7</td>
+      <td style="text-align: left">x - 1</td>
+      <td style="text-align: left">deprecated</td>
+    </tr>
+    <tr>
+      <td style="text-align: left">9 &lt;= x &lt;= 15</td>
+      <td style="text-align: left">x - 1</td>
+      <td style="text-align: left">deprecated</td>
+    </tr>
+    <tr>
+      <td style="text-align: left">17 &lt;= x &lt;= 21</td>
+      <td style="text-align: left">x - 1</td>
+      <td style="text-align: left">deprecated</td>
+    </tr>
+    <tr>
+      <td style="text-align: left">26</td>
+      <td style="text-align: left">24</td>
+      <td style="text-align: left">deprecated</td>
+    </tr>
+    <tr>
+      <td style="text-align: left">28</td>
+      <td style="text-align: left">25</td>
+      <td style="text-align: left">deprecated</td>
+    </tr>
+    <tr>
+      <td style="text-align: left">30</td>
+      <td style="text-align: left">26</td>
+      <td style="text-align: left">deprecated</td>
+    </tr>
+  </tbody>
+</table>
+
+<ul>
+  <li>2 bytes header
+    <ul>
+      <li>2 bits for encoding type (1)</li>
+      <li>5 bits for encoded width (W) of values (1 to 64 bits) using the 5 bit
+width encoding table</li>
+      <li>9 bits for length (L) (1 to 512 values)</li>
+    </ul>
+  </li>
+  <li>W * L bits (padded to the next byte) encoded in big endian format, which is
+zigzag encoding if the type is signed</li>
+</ul>
+
+<p>The unsigned sequence of [23713, 43806, 57005, 48879] would be
+serialized with direct encoding (1), a width of 16 bits (15), and
+length of 4 (3) as [0x5e, 0x03, 0x5c, 0xa1, 0xab, 0x1e, 0xde, 0xad,
+0xbe, 0xef].</p>
+
+<h3 id="patched-base">Patched Base</h3>
+
+<p>The patched base encoding is used for integer sequences whose bit
+widths varies a lot. The minimum signed value of the sequence is found
+and subtracted from the other values. The bit width of those adjusted
+values is analyzed and the 90 percentile of the bit width is chosen
+as W. The 10\% of values larger than W use patches from a patch list
+to set the additional bits. Patches are encoded as a list of gaps in
+the index values and the additional value bits.</p>
+
+<ul>
+  <li>4 bytes header
+    <ul>
+      <li>2 bits for encoding type (2)</li>
+      <li>5 bits for encoded width (W) of values (1 to 64 bits) using the 5 bit
+  width encoding table</li>
+      <li>9 bits for length (L) (1 to 512 values)</li>
+      <li>3 bits for base value width (BW) (1 to 8 bytes)</li>
+      <li>5 bits for patch width (PW) (1 to 64 bits) using  the 5 bit width
+encoding table</li>
+      <li>3 bits for patch gap width (PGW) (1 to 8 bits)</li>
+      <li>5 bits for patch list length (PLL) (0 to 31 patches)</li>
+    </ul>
+  </li>
+  <li>Base value (BW bytes) - The base value is stored as a big endian value
+with negative values marked by the most significant bit set. If it that
+bit is set, the entire value is negated.</li>
+  <li>Data values (W * L bits padded to the byte) - A sequence of W bit positive
+values that are added to the base value.</li>
+  <li>Data values (W * L bits padded to the byte) - A sequence of W bit positive
+values that are added to the base value.</li>
+  <li>Patch list (PLL * (PGW + PW) bytes) - A list of patches for values
+that didn’t fit within W bits. Each entry in the list consists of a
+gap, which is the number of elements skipped from the previous
+patch, and a patch value. Patches are applied by logically or’ing
+the data values with the relevant patch shifted W bits left. If a
+patch is 0, it was introduced to skip over more than 255 items. The
+combined length of each patch (PGW + PW) must be less or equal to
+64.</li>
+</ul>
+
+<p>The unsigned sequence of [2030, 2000, 2020, 1000000, 2040, 2050, 2060, 2070,
+2080, 2090, 2100, 2110, 2120, 2130, 2140, 2150, 2160, 2170, 2180, 2190]
+has a minimum of 2000, which makes the adjusted
+sequence [30, 0, 20, 998000, 40, 50, 60, 70, 80, 90, 100, 110, 120, 130, 140,
+150, 160, 170, 180, 190]. It has an
+encoding of patched base (2), a bit width of 8 (7), a length of 20
+(19), a base value width of 2 bytes (1), a patch width of 12 bits (11),
+patch gap width of 2 bits (1), and a patch list length of 1 (1). The
+base value is 2000 and the combined result is [0x8e, 0x13, 0x2b, 0x21, 0x07,
+0xd0, 0x1e, 0x00, 0x14, 0x70, 0x28, 0x32, 0x3c, 0x46, 0x50, 0x5a, 0x64, 0x6e,
+0x78, 0x82, 0x8c, 0x96, 0xa0, 0xaa, 0xb4, 0xbe, 0xfc, 0xe8]</p>
+
+<h3 id="delta">Delta</h3>
+
+<p>The Delta encoding is used for monotonically increasing or decreasing
+sequences. The first two numbers in the sequence can not be identical,
+because the encoding is using the sign of the first delta to determine
+if the series is increasing or decreasing.</p>
+
+<ul>
+  <li>2 bytes header
+    <ul>
+      <li>2 bits for encoding type (3)</li>
+      <li>5 bits for encoded width (W) of deltas (0 to 64 bits) using the 5 bit
+width encoding table</li>
+      <li>9 bits for run length (L) (1 to 512 values)</li>
+    </ul>
+  </li>
+  <li>Base value - encoded as (signed or unsigned) varint</li>
+  <li>Delta base - encoded as signed varint</li>
+  <li>Delta values $W * (L - 2)$ bytes - encode each delta after the first
+one. If the delta base is positive, the sequence is increasing and if it is
+negative the sequence is decreasing.</li>
+</ul>
+
+<p>The unsigned sequence of [2, 3, 5, 7, 11, 13, 17, 19, 23, 29] would be
+serialized with delta encoding (3), a width of 4 bits (3), length of
+10 (9), a base of 2 (2), and first delta of 1 (2). The resulting
+sequence is [0xc6, 0x09, 0x02, 0x02, 0x22, 0x42, 0x42, 0x46].</p>
+
+<h1 id="stripes">Stripes</h1>
+
+<p>The body of ORC files consists of a series of stripes. Stripes are
+large (typically ~200MB) and independent of each other and are often
+processed by different tasks. The defining characteristic for columnar
+storage formats is that the data for each column is stored separately
+and that reading data out of the file should be proportional to the
+number of columns read.</p>
+
+<p>In ORC files, each column is stored in several streams that are stored
+next to each other in the file. For example, an integer column is
+represented as two streams PRESENT, which uses one with a bit per
+value recording if the value is non-null, and DATA, which records the
+non-null values. If all of a column’s values in a stripe are non-null,
+the PRESENT stream is omitted from the stripe. For binary data, ORC
+uses three streams PRESENT, DATA, and LENGTH, which stores the length
+of each value. The details of each type will be presented in the
+following subsections.</p>
+
+<h2 id="stripe-footer">Stripe Footer</h2>
+
+<p>The stripe footer contains the encoding of each column and the
+directory of the streams including their location.</p>
+
+<p>```message StripeFooter {
+ // the location of each stream
+ repeated Stream streams = 1;
+ // the encoding of each column
+ repeated ColumnEncoding columns = 2;
+}</p>
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
+To describe each stream, ORC stores the kind of stream, the column id,
+and the stream's size in bytes. The details of what is stored in each stream
+depends on the type and encoding of the column.
+
+```message Stream {
+ enum Kind {
+ // boolean stream of whether the next value is non-null
+ PRESENT = 0;
+ // the primary data stream
+ DATA = 1;
+ // the length of each value for variable length data
+ LENGTH = 2;
+ // the dictionary blob
+ DICTIONARY\_DATA = 3;
+ // deprecated prior to Hive 0.11
+ // It was used to store the number of instances of each value in the
+ // dictionary
+ DICTIONARY_COUNT = 4;
+ // a secondary data stream
+ SECONDARY = 5;
+ // the index for seeking to particular row groups
+ ROW_INDEX = 6;
+ // original bloom filters used before ORC-101
+ BLOOM_FILTER = 7;
+ // bloom filters that consistently use utf8
+ BLOOM_FILTER_UTF8 = 8;
+ }
+ required Kind kind = 1;
+ // the column id
+ optional uint32 column = 2;
+ // the number of bytes in the file
+ optional uint64 length = 3;
+}
+</code></pre></div></div>
+
+<p>Depending on their type several options for encoding are possible. The
+encodings are divided into direct or dictionary-based categories and
+further refined as to whether they use RLE v1 or v2.</p>
+
+<p>```message ColumnEncoding {
+ enum Kind {
+ // the encoding is mapped directly to the stream using RLE v1
+ DIRECT = 0;
+ // the encoding uses a dictionary of unique values using RLE v1
+ DICTIONARY = 1;
+ // the encoding is direct using RLE v2
+ DIRECT_V2 = 2;
+ // the encoding is dictionary-based using RLE v2
+ DICTIONARY_V2 = 3;
+ }
+ required Kind kind = 1;
+ // for dictionary encodings, record the size of the dictionary
+ optional uint32 dictionarySize = 2;
+}</p>
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
+# Column Encodings
+
+## SmallInt, Int, and BigInt Columns
+
+All of the 16, 32, and 64 bit integer column types use the same set of
+potential encodings, which is basically whether they use RLE v1 or
+v2. If the PRESENT stream is not included, all of the values are
+present. For values that have false bits in the present stream, no
+values are included in the data stream.
+
+Encoding  | Stream Kind | Optional | Contents
+:-------- | :---------- | :------- | :-------
+DIRECT    | PRESENT     | Yes      | Boolean RLE
+          | DATA        | No       | Signed Integer RLE v1
+DIRECT_V2 | PRESENT     | Yes      | Boolean RLE
+          | DATA        | No       | Signed Integer RLE v2
+
+## Float and Double Columns
+
+Floating point types are stored using IEEE 754 floating point bit
+layout. Float columns use 4 bytes per value and double columns use 8
+bytes.
+
+Encoding  | Stream Kind | Optional | Contents
+:-------- | :---------- | :------- | :-------
+DIRECT    | PRESENT     | Yes      | Boolean RLE
+          | DATA        | No       | IEEE 754 floating point representation
+
+## String, Char, and VarChar Columns
+
+String, char, and varchar columns may be encoded either using a
+dictionary encoding or a direct encoding. A direct encoding should be
+preferred when there are many distinct values. In all of the
+encodings, the PRESENT stream encodes whether the value is null. The
+Java ORC writer automatically picks the encoding after the first row
+group (10,000 rows).
+
+For direct encoding the UTF-8 bytes are saved in the DATA stream and
+the length of each value is written into the LENGTH stream. In direct
+encoding, if the values were ["Nevada", "California"]; the DATA
+would be "NevadaCalifornia" and the LENGTH would be [6, 10].
+
+For dictionary encodings the dictionary is sorted and UTF-8 bytes of
+each unique value are placed into DICTIONARY_DATA. The length of each
+item in the dictionary is put into the LENGTH stream. The DATA stream
+consists of the sequence of references to the dictionary elements.
+
+In dictionary encoding, if the values were ["Nevada",
+"California", "Nevada", "California", and "Florida"]; the
+DICTIONARY_DATA would be "CaliforniaFloridaNevada" and LENGTH would
+be [10, 7, 6]. The DATA would be [2, 0, 2, 0, 1].
+
+Encoding      | Stream Kind     | Optional | Contents
+:------------ | :-------------- | :------- | :-------
+DIRECT        | PRESENT         | Yes      | Boolean RLE
+              | DATA            | No       | String contents
+              | LENGTH          | No       | Unsigned Integer RLE v1
+DICTIONARY    | PRESENT         | Yes      | Boolean RLE
+              | DATA            | No       | Unsigned Integer RLE v1
+              | DICTIONARY_DATA | No       | String contents
+              | LENGTH          | No       | Unsigned Integer RLE v1
+DIRECT_V2     | PRESENT         | Yes      | Boolean RLE
+              | DATA            | No       | String contents
+              | LENGTH          | No       | Unsigned Integer RLE v2
+DICTIONARY_V2 | PRESENT         | Yes      | Boolean RLE
+              | DATA            | No       | Unsigned Integer RLE v2
+              | DICTIONARY_DATA | No       | String contents
+              | LENGTH          | No       | Unsigned Integer RLE v2
+
+## Boolean Columns
+
+Boolean columns are rare, but have a simple encoding.
+
+Encoding      | Stream Kind     | Optional | Contents
+:------------ | :-------------- | :------- | :-------
+DIRECT        | PRESENT         | Yes      | Boolean RLE
+              | DATA            | No       | Boolean RLE
+
+## TinyInt Columns
+
+TinyInt (byte) columns use byte run length encoding.
+
+Encoding      | Stream Kind     | Optional | Contents
+:------------ | :-------------- | :------- | :-------
+DIRECT        | PRESENT         | Yes      | Boolean RLE
+              | DATA            | No       | Byte RLE
+
+## Binary Columns
+
+Binary data is encoded with a PRESENT stream, a DATA stream that records
+the contents, and a LENGTH stream that records the number of bytes per a
+value.
+
+Encoding      | Stream Kind     | Optional | Contents
+:------------ | :-------------- | :------- | :-------
+DIRECT        | PRESENT         | Yes      | Boolean RLE
+              | DATA            | No       | String contents
+              | LENGTH          | No       | Unsigned Integer RLE v1
+DIRECT_V2     | PRESENT         | Yes      | Boolean RLE
+              | DATA            | No       | String contents
+              | LENGTH          | No       | Unsigned Integer RLE v2
+
+## Decimal Columns
+
+Decimal was introduced in Hive 0.11 with infinite precision (the total
+number of digits). In Hive 0.13, the definition was change to limit
+the precision to a maximum of 38 digits, which conveniently uses 127
+bits plus a sign bit. The current encoding of decimal columns stores
+the integer representation of the value as an unbounded length zigzag
+encoded base 128 varint. The scale is stored in the SECONDARY stream
+as an signed integer.
+
+Encoding      | Stream Kind     | Optional | Contents
+:------------ | :-------------- | :------- | :-------
+DIRECT        | PRESENT         | Yes      | Boolean RLE
+              | DATA            | No       | Unbounded base 128 varints
+              | SECONDARY       | No       | Unsigned Integer RLE v1
+DIRECT_V2     | PRESENT         | Yes      | Boolean RLE
+              | DATA            | No       | Unbounded base 128 varints
+              | SECONDARY       | No       | Unsigned Integer RLE v2
+
+## Date Columns
+
+Date data is encoded with a PRESENT stream, a DATA stream that records
+the number of days after January 1, 1970 in UTC.
+
+Encoding      | Stream Kind     | Optional | Contents
+:------------ | :-------------- | :------- | :-------
+DIRECT        | PRESENT         | Yes      | Boolean RLE
+              | DATA            | No       | Signed Integer RLE v1
+DIRECT_V2     | PRESENT         | Yes      | Boolean RLE
+              | DATA            | No       | Signed Integer RLE v2
+
+## Timestamp Columns
+
+Timestamp records times down to nanoseconds as a PRESENT stream that
+records non-null values, a DATA stream that records the number of
+seconds after 1 January 2015, and a SECONDARY stream that records the
+number of nanoseconds.
+
+Because the number of nanoseconds often has a large number of trailing
+zeros, the number has trailing decimal zero digits removed and the
+last three bits are used to record how many zeros were removed. Thus
+1000 nanoseconds would be serialized as 0x0b and 100000 would be
+serialized as 0x0d.
+
+Encoding      | Stream Kind     | Optional | Contents
+:------------ | :-------------- | :------- | :-------
+DIRECT        | PRESENT         | Yes      | Boolean RLE
+              | DATA            | No       | Signed Integer RLE v1
+              | SECONDARY       | No       | Unsigned Integer RLE v1
+DIRECT_V2     | PRESENT         | Yes      | Boolean RLE
+              | DATA            | No       | Signed Integer RLE v2
+              | SECONDARY       | No       | Unsigned Integer RLE v2
+
+## Struct Columns
+
+Structs have no data themselves and delegate everything to their child
+columns except for their PRESENT stream. They have a child column
+for each of the fields.
+
+Encoding      | Stream Kind     | Optional | Contents
+:------------ | :-------------- | :------- | :-------
+DIRECT        | PRESENT         | Yes      | Boolean RLE
+
+## List Columns
+
+Lists are encoded as the PRESENT stream and a length stream with
+number of items in each list. They have a single child column for the
+element values.
+
+Encoding      | Stream Kind     | Optional | Contents
+:------------ | :-------------- | :------- | :-------
+DIRECT        | PRESENT         | Yes      | Boolean RLE
+              | LENGTH          | No       | Unsigned Integer RLE v1
+DIRECT_V2     | PRESENT         | Yes      | Boolean RLE
+              | LENGTH          | No       | Unsigned Integer RLE v2
+
+## Map Columns
+
+Maps are encoded as the PRESENT stream and a length stream with number
+of items in each list. They have a child column for the key and
+another child column for the value.
+
+Encoding      | Stream Kind     | Optional | Contents
+:------------ | :-------------- | :------- | :-------
+DIRECT        | PRESENT         | Yes      | Boolean RLE
+              | LENGTH          | No       | Unsigned Integer RLE v1
+DIRECT_V2     | PRESENT         | Yes      | Boolean RLE
+              | LENGTH          | No       | Unsigned Integer RLE v2
+
+## Union Columns
+
+Unions are encoded as the PRESENT stream and a tag stream that controls which
+potential variant is used. They have a child column for each variant of the
+union. Currently ORC union types are limited to 256 variants, which matches
+the Hive type model.
+
+Encoding      | Stream Kind     | Optional | Contents
+:------------ | :-------------- | :------- | :-------
+DIRECT        | PRESENT         | Yes      | Boolean RLE
+              | DIRECT          | No       | Byte RLE
+
+# Indexes
+
+## Row Group Index
+
+The row group indexes consist of a ROW_INDEX stream for each primitive
+column that has an entry for each row group. Row groups are controlled
+by the writer and default to 10,000 rows. Each RowIndexEntry gives the
+position of each stream for the column and the statistics for that row
+group.
+
+The index streams are placed at the front of the stripe, because in
+the default case of streaming they do not need to be read. They are
+only loaded when either predicate push down is being used or the
+reader seeks to a particular row.
+
+```message RowIndexEntry {
+ repeated uint64 positions = 1 [packed=true];
+ optional ColumnStatistics statistics = 2;
+}
+</code></pre></div></div>
+
+<p>```message RowIndex {
+ repeated RowIndexEntry entry = 1;
+}</p>
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
+To record positions, each stream needs a sequence of numbers. For
+uncompressed streams, the position is the byte offset of the RLE run's
+start location followed by the number of values that need to be
+consumed from the run. In compressed streams, the first number is the
+start of the compression chunk in the stream, followed by the number
+of decompressed bytes that need to be consumed, and finally the number
+of values consumed in the RLE.
+
+For columns with multiple streams, the sequences of positions in each
+stream are concatenated. That was an unfortunate decision on my part
+that we should fix at some point, because it makes code that uses the
+indexes error-prone.
+
+Because dictionaries are accessed randomly, there is not a position to
+record for the dictionary and the entire dictionary must be read even
+if only part of a stripe is being read.
+
+## Bloom Filter Index
+
+Bloom Filters are added to ORC indexes from Hive 1.2.0 onwards.
+Predicate pushdown can make use of bloom filters to better prune
+the row groups that do not satisfy the filter condition.
+The bloom filter indexes consist of a BLOOM_FILTER stream for each
+column specified through 'orc.bloom.filter.columns' table properties.
+A BLOOM_FILTER stream records a bloom filter entry for each row
+group (default to 10,000 rows) in a column. Only the row groups that
+satisfy min/max row index evaluation will be evaluated against the
+bloom filter index.
+
+Each BloomFilterEntry stores the number of hash functions ('k') used
+and the bitset backing the bloom filter. The original encoding (pre
+ORC-101) of bloom filters used the bitset field encoded as a repeating
+sequence of longs in the bitset field with a little endian encoding
+(0x1 is bit 0 and 0x2 is bit 1.) After ORC-101, the encoding is a
+sequence of bytes with a little endian encoding in the utf8bitset field.
+
+```message BloomFilter {
+ optional uint32 numHashFunctions = 1;
+ repeated fixed64 bitset = 2;
+ optional bytes utf8bitset = 3;
+}
+</code></pre></div></div>
+
+<p><code class="highlighter-rouge">message BloomFilterIndex {
+ repeated BloomFilter bloomFilter = 1;
+}
+</code></p>
+
+<p>Bloom filter internally uses two different hash functions to map a key
+to a position in the bit set. For tinyint, smallint, int, bigint, float
+and double types, Thomas Wang’s 64-bit integer hash function is used.
+Floats are converted to IEEE-754 32 bit representation
+(using Java’s Float.floatToIntBits(float)). Similary, Doubles are
+converted to IEEE-754 64 bit representation (using Java’s
+Double.doubleToLongBits(double)). All these primitive types
+are cast to long base type before being passed on to the hash function.
+For strings and binary types, Murmur3 64 bit hash algorithm is used.
+The 64 bit variant of Murmur3 considers only the most significant
+8 bytes of Murmur3 128-bit algorithm. The 64 bit hashcode generated
+from the above algorithms is used as a base to derive ‘k’ different
+hash functions. We use the idea mentioned in the paper “Less Hashing,
+Same Performance: Building a Better Bloom Filter” by Kirsch et. al. to
+quickly compute the k hashcodes.</p>
+
+<p>The algorithm for computing k hashcodes and setting the bit position
+in a bloom filter is as follows:</p>
+
+<ol>
+  <li>Get 64 bit base hash code from Murmur3 or Thomas Wang’s hash algorithm.</li>
+  <li>Split the above hashcode into two 32-bit hashcodes (say hash1 and hash2).</li>
+  <li>k’th hashcode is obtained by (where k &gt; 0):
+    <ul>
+      <li>combinedHash = hash1 + (k * hash2)</li>
+    </ul>
+  </li>
+  <li>If combinedHash is negative flip all the bits:
+    <ul>
+      <li>combinedHash = ~combinedHash</li>
+    </ul>
+  </li>
+  <li>Bit set position is obtained by performing modulo with m:
+    <ul>
+      <li>position = combinedHash % m</li>
+    </ul>
+  </li>
+  <li>Set the position in bit set. The LSB 6 bits identifies the long index
+within bitset and bit position within the long uses little endian order.
+    <ul>
+      <li>bitset[position »&gt; 6] |= (1L « position);</li>
+    </ul>
+  </li>
+</ol>
+
+<p>Bloom filter streams are interlaced with row group indexes. This placement
+makes it convenient to read the bloom filter stream and row index stream
+together in single read operation.</p>
+
+<p><img src="http://localhost:4000/img/BloomFilter.png" alt="bloom filter" /></p>
+
+      </article>
+    </div>
+
+    <div class="clear"></div>
+
+  </div>
+</section>
+
+
+  <footer role="contentinfo">
+  <p>The contents of this website are &copy;&nbsp;2018
+     <a href="https://www.apache.org/">Apache Software Foundation</a>
+     under the terms of the <a
+      href="https://www.apache.org/licenses/LICENSE-2.0.html">
+      Apache&nbsp;License&nbsp;v2</a>. Apache ORC and its logo are trademarks
+      of the Apache Software Foundation.</p>
+</footer>
+
+  <script>
+  var anchorForId = function (id) {
+    var anchor = document.createElement("a");
+    anchor.className = "header-link";
+    anchor.href      = "#" + id;
+    anchor.innerHTML = "<span class=\"sr-only\">Permalink</span><i class=\"fa fa-link\"></i>";
+    anchor.title = "Permalink";
+    return anchor;
+  };
+
+  var linkifyAnchors = function (level, containingElement) {
+    var headers = containingElement.getElementsByTagName("h" + level);
+    for (var h = 0; h < headers.length; h++) {
+      var header = headers[h];
+
+      if (typeof header.id !== "undefined" && header.id !== "") {
+        header.appendChild(anchorForId(header.id));
+      }
+    }
+  };
+
+  document.onreadystatechange = function () {
+    if (this.readyState === "complete") {
+      var contentBlock = document.getElementsByClassName("docs")[0] || document.getElementsByClassName("news")[0];
+      if (!contentBlock) {
+        return;
+      }
+      for (var level = 1; level <= 6; level++) {
+        linkifyAnchors(level, contentBlock);
+      }
+    }
+  };
+</script>
+
+
+</body>
+</html>


[10/11] orc git commit: Deploy docs with CVE.

Posted by om...@apache.org.
http://git-wip-us.apache.org/repos/asf/orc/blob/e58d3fc4/develop/make-release/index.html
----------------------------------------------------------------------
diff --git a/develop/make-release/index.html b/develop/make-release/index.html
new file mode 100644
index 0000000..3a2892f
--- /dev/null
+++ b/develop/make-release/index.html
@@ -0,0 +1,324 @@
+<!DOCTYPE HTML>
+<html lang="en-US">
+<head>
+  <meta charset="UTF-8">
+  <title>How To Release ORC</title>
+  <meta name="viewport" content="width=device-width,initial-scale=1">
+  <meta name="generator" content="Jekyll v3.7.3">
+  <link rel="stylesheet" href="//fonts.googleapis.com/css?family=Lato:300,300italic,400,400italic,700,700italic,900">
+  <link rel="stylesheet" href="/css/screen.css">
+  <link rel="icon" type="image/x-icon" href="/favicon.ico">
+  <!--[if lt IE 9]>
+  <script src="/js/html5shiv.min.js"></script>
+  <script src="/js/respond.min.js"></script>
+  <![endif]-->
+</head>
+
+
+<body class="wrap">
+  <header role="banner">
+  <nav class="mobile-nav show-on-mobiles">
+    <ul>
+  <li class="">
+    <a href="/">Home</a>
+  </li>
+  <li class="">
+    <a href="/docs/"><span class="show-on-mobiles">Docs</span>
+                     <span class="hide-on-mobiles">Documentation</span></a>
+  </li>
+  <li class="">
+    <a href="/talks/">Talks</a>
+  </li>
+  <li class="">
+    <a href="/news/">News</a>
+  </li>
+  <li class="">
+    <a href="/help/">Help</a>
+  </li>
+  <li class="current">
+    <a href="/develop/">Develop</a>
+  </li>
+</ul>
+
+  </nav>
+  <div class="grid">
+    <div class="unit one-third center-on-mobiles">
+      <h1>
+        <a href="/">
+          <span class="sr-only">Apache ORC</span>
+          <img src="/img/logo.png" width="249" height="101" alt="ORC Logo">
+        </a>
+      </h1>
+    </div>
+    <nav class="main-nav unit two-thirds hide-on-mobiles">
+      <ul>
+  <li class="">
+    <a href="/">Home</a>
+  </li>
+  <li class="">
+    <a href="/docs/"><span class="show-on-mobiles">Docs</span>
+                     <span class="hide-on-mobiles">Documentation</span></a>
+  </li>
+  <li class="">
+    <a href="/talks/">Talks</a>
+  </li>
+  <li class="">
+    <a href="/news/">News</a>
+  </li>
+  <li class="">
+    <a href="/help/">Help</a>
+  </li>
+  <li class="current">
+    <a href="/develop/">Develop</a>
+  </li>
+</ul>
+
+    </nav>
+  </div>
+</header>
+
+
+  <section class="standalone">
+  <div class="grid">
+
+    <div class="unit whole">
+      <article>
+        <h1>How To Release ORC</h1>
+        <h2 id="preparing-for-release">Preparing for release</h2>
+
+<p>Set version so that it isn’t a SNAPSHOT.</p>
+
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>% git checkout branch-X.Y
+% edit CMakeLists.txt
+% (mkdir build; cd build; cmake ..)
+</code></pre></div></div>
+
+<p>Commit the changes back to Apache along with a tag for the release candidate.</p>
+
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>% git commit -s -S -am 'Preparing for release X.Y.Z'
+% git remote add apache https://git-wip-us.apache.org/repos/asf/orc.git
+% git push apache branch-X.Y
+% git tag release-X.Y.Zrc0
+% git push apache release-X.Y.Zrc0
+</code></pre></div></div>
+
+<p>Generate the source tarball and checksums for the release.</p>
+
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>% wget https://github.com/apache/orc/archive/release-X.Y.Zrc0.tar.gz
+% tar xzf release-X.Y.Zrc0.tar.gz
+% mv orc-release-X.Y.Zrc0 orc-X.Y.Z
+% tar czf orc-X.Y.Zrc0.tar.gz orc-X.Y.Z
+% shasum -a 256 orc-X.Y.Zrc0.tar.gz &gt; orc-X.Y.Zrc0.tar.gz.sha256
+% gpg --detach-sig --armor orc-X.Y.Zrc0.tar.gz
+</code></pre></div></div>
+
+<p>Copy the artifacts into your personal Apache website.</p>
+
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>% sftp &lt;apacheid&gt;@home.apache.org
+sftp&gt; cd public_html
+sftp&gt; mkdir orc-X.Y.Zrc0
+sftp&gt; cd orc-X.Y.Zrc0
+sftp&gt; put orc-X.Y.Zrc0*
+sftp&gt; quit
+</code></pre></div></div>
+
+<p>Make sure your GPG key exists <a href="https://dist.apache.org/repos/dist/release/orc/KEYS">here</a> for others to verify signature in RC</p>
+
+<p>Click the version to release (X.Y.Z) <a href="https://issues.apache.org/jira/projects/ORC?selectedItem=com.atlassian.jira.jira-projects-plugin:release-page">here</a>
+to get the list of jiras that are fixed in X.Y.Z</p>
+
+<p>Send email with the vote:</p>
+
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>To: dev@orc.apache.org
+Subject: [VOTE] Should we release ORC X.Y.Zrc0?
+
+All,
+
+Should we release the following artifacts as ORC X.Y.Z?
+
+tar: http://home.apache.org/~omalley/orc-X.Y.Zrc0/
+tag: https://github.com/apache/orc/releases/tag/release-X.Y.Zrc0
+jiras: https://issues.apache.org/jira/browse/ORC/fixforversion/&lt;fixid&gt;
+
+Thanks!
+</code></pre></div></div>
+
+<h2 id="to-promote-a-release-candidate-rc-to-a-real-release">To promote a release candidate (RC) to a real release.</h2>
+
+<p>Update the final tag and remove the rc tag.</p>
+
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>% git tag rel/release-X.Y.Z -s
+% git push apache rel/release-X.Y.Z
+% git push apache :release-X.Y.Zrc0
+</code></pre></div></div>
+
+<p>Publish the artifacts to Maven central staging. Make sure to have this <a href="http://www.apache.org/dev/publishing-maven-artifacts.html#dev-env">setup</a> for Apache releases</p>
+
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>% cd java
+% mvn -Papache-release clean deploy
+</code></pre></div></div>
+
+<p>Publish from the staging area:</p>
+
+<ul>
+  <li>login to <a href="https://repository.apache.org/index.html#stagingRepositories">Maven staging</a></li>
+  <li>find your staging repository (search for org.apache.org)</li>
+  <li>close it</li>
+  <li>release it</li>
+</ul>
+
+<p>Publish the artifacts to Apache’s dist area.</p>
+
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>In a svn clone of https://dist.apache.org/repos/dist/release/orc
+% mkdir orc-X.Y.Z
+% cd orc-X.Y.Z
+copy release artifacts with a rename from orc-X.Y.Zrc0* to orc-X.Y.Z*
+% svn add .
+% svn commit ---username &lt;apacheid&gt; -m "ORC X.Y.Z Release"
+
+We keep the latest patch release for each of the last two branches, so remove
+extra releases (say I.J.K) from the Apache dist area.
+
+% cd ..
+% svn rm orc-I.J.K
+% svn commit --username &lt;apacheid&gt; -m "Removed old release ORC I.J.K"
+</code></pre></div></div>
+
+<p>Update the release branch with the version for the next release.</p>
+
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>edit CMakeLists.txt to change version to X.Y.(Z+1)-SNAPSHOT
+% cd build
+% cmake ..
+% git commit -a -s -S -am 'Preparing branch for post X.Y.Z development'
+% git push apache branch-X.Y
+</code></pre></div></div>
+
+<p>Update the site with the new release.</p>
+
+<ul>
+  <li>Check out the master branch (git co apache/master)</li>
+</ul>
+
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Change directory in to site.
+% pwd
+&lt;path-to-master-src&gt;
+% cd site
+% mkdir target
+% cd target
+Set up site/target to be a separate git workspace that tracks the asf-site branch.
+% git init
+% git remote add origin https://git-wip-us.apache.org/repos/asf/orc.git -t asf-site
+% git fetch origin
+% git checkout asf-site
+% cd ..
+% bundle install
+</code></pre></div></div>
+<ul>
+  <li>edit site/_data/releases.yml to add new release
+    <ul>
+      <li>update the state for the releases to match the changes in the Apache dist
+        <ul>
+          <li>latest = new release (only one of these!)</li>
+          <li>stable = other release still in dist</li>
+          <li>archived = removed from dist</li>
+        </ul>
+      </li>
+    </ul>
+  </li>
+  <li>create a new file _posts/YYYY-MM-DD-ORC-X.Y.Z.md for the news section</li>
+  <li>Run “bundle exec jekyll serve”</li>
+  <li>Check the website on http://0.0.0.0:4000/</li>
+  <li>If it looks good, use git add (from within site directory) to add the new files and commit to master with a message of “update site for X.Y.Z”.</li>
+</ul>
+
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>% pwd
+&lt;path-to-master-src&gt;/site
+% git commit -am "Update site for X.Y.Z"
+% git push origin master
+</code></pre></div></div>
+
+<ul>
+  <li>Change directory into site/target for publishing the site.</li>
+  <li>Add the new files that you just generated.
+    <ul>
+      <li>This assumes you’ve set up site/target to be a separate git workspace that tracks the asf-site branch.</li>
+    </ul>
+  </li>
+  <li>Commit to asf-site to publish the updated site.</li>
+</ul>
+
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>% cd target
+% pwd
+&lt;path-to-master-src&gt;/site/target
+% git commit -am "Publish site for X.Y.Z"
+% git push origin asf-site
+</code></pre></div></div>
+
+<p>Update ORC’s jira to reflect the released version.</p>
+
+<ul>
+  <li>Select the resolved issues and bulk transition them to closed with following query <a href="https://issues.apache.org/jira/issues/?filter=-1">here</a>.
+    <ul>
+      <li>query: project = ORC AND fixVersion = X.Y.Z and status = Resolved ORDER BY created desc</li>
+    </ul>
+  </li>
+  <li>Mark the version as released and set the date <a href="https://issues.apache.org/jira/projects/ORC?selectedItem=com.atlassian.jira.jira-projects-plugin:release-page">here</a>.</li>
+</ul>
+
+<p>It usually take up to 24 hours for the apache dist mirrors and maven central to update with the new release.</p>
+
+      </article>
+    </div>
+
+    <div class="clear"></div>
+
+  </div>
+</section>
+
+
+  <footer role="contentinfo">
+  <p>The contents of this website are &copy;&nbsp;2018
+     <a href="https://www.apache.org/">Apache Software Foundation</a>
+     under the terms of the <a
+      href="https://www.apache.org/licenses/LICENSE-2.0.html">
+      Apache&nbsp;License&nbsp;v2</a>. Apache ORC and its logo are trademarks
+      of the Apache Software Foundation.</p>
+</footer>
+
+  <script>
+  var anchorForId = function (id) {
+    var anchor = document.createElement("a");
+    anchor.className = "header-link";
+    anchor.href      = "#" + id;
+    anchor.innerHTML = "<span class=\"sr-only\">Permalink</span><i class=\"fa fa-link\"></i>";
+    anchor.title = "Permalink";
+    return anchor;
+  };
+
+  var linkifyAnchors = function (level, containingElement) {
+    var headers = containingElement.getElementsByTagName("h" + level);
+    for (var h = 0; h < headers.length; h++) {
+      var header = headers[h];
+
+      if (typeof header.id !== "undefined" && header.id !== "") {
+        header.appendChild(anchorForId(header.id));
+      }
+    }
+  };
+
+  document.onreadystatechange = function () {
+    if (this.readyState === "complete") {
+      var contentBlock = document.getElementsByClassName("docs")[0] || document.getElementsByClassName("news")[0];
+      if (!contentBlock) {
+        return;
+      }
+      for (var level = 1; level <= 6; level++) {
+        linkifyAnchors(level, contentBlock);
+      }
+    }
+  };
+</script>
+
+
+</body>
+</html>

http://git-wip-us.apache.org/repos/asf/orc/blob/e58d3fc4/docs/acid.html
----------------------------------------------------------------------
diff --git a/docs/acid.html b/docs/acid.html
index 71c980c..e500646 100644
--- a/docs/acid.html
+++ b/docs/acid.html
@@ -4,7 +4,7 @@
   <meta charset="UTF-8">
   <title>ACID support</title>
   <meta name="viewport" content="width=device-width,initial-scale=1">
-  <meta name="generator" content="Jekyll v2.4.0">
+  <meta name="generator" content="Jekyll v3.7.3">
   <link rel="stylesheet" href="//fonts.googleapis.com/css?family=Lato:300,300italic,400,400italic,700,700italic,900">
   <link rel="stylesheet" href="/css/screen.css">
   <link rel="icon" type="image/x-icon" href="/favicon.ico">
@@ -109,7 +109,7 @@
     
   
     
-      <option value="/docs/index.html">Background</option>
+      <option value="http://localhost:4000/docs/index.html">Background</option>
     
   
     
@@ -131,7 +131,7 @@
     
   
     
-      <option value="/docs/adopters.html">ORC Adopters</option>
+      <option value="http://localhost:4000/docs/adopters.html">ORC Adopters</option>
     
   
     
@@ -193,7 +193,7 @@
     
   
     
-      <option value="/docs/types.html">Types</option>
+      <option value="http://localhost:4000/docs/types.html">Types</option>
     
   
 
@@ -219,7 +219,7 @@
     
   
     
-      <option value="/docs/indexes.html">Indexes</option>
+      <option value="http://localhost:4000/docs/indexes.html">Indexes</option>
     
   
     
@@ -237,7 +237,7 @@
 
   
     
-      <option value="/docs/acid.html">ACID support</option>
+      <option value="http://localhost:4000/docs/acid.html">ACID support</option>
     
   
     
@@ -284,7 +284,7 @@
     
   
     
-      <option value="/docs/building.html">Building ORC</option>
+      <option value="http://localhost:4000/docs/building.html">Building ORC</option>
     
   
     
@@ -342,7 +342,7 @@
     
   
     
-      <option value="/docs/releases.html">Releases</option>
+      <option value="http://localhost:4000/docs/releases.html">Releases</option>
     
   
     
@@ -373,7 +373,7 @@
     
   
     
-      <option value="/docs/hive-ddl.html">Hive DDL</option>
+      <option value="http://localhost:4000/docs/hive-ddl.html">Hive DDL</option>
     
   
     
@@ -407,7 +407,7 @@
     
   
     
-      <option value="/docs/hive-config.html">Hive Configuration</option>
+      <option value="http://localhost:4000/docs/hive-config.html">Hive Configuration</option>
     
   
     
@@ -460,7 +460,7 @@
     
   
     
-      <option value="/docs/mapred.html">Using in MapRed</option>
+      <option value="http://localhost:4000/docs/mapred.html">Using in MapRed</option>
     
   
     
@@ -498,7 +498,7 @@
     
   
     
-      <option value="/docs/mapreduce.html">Using in MapReduce</option>
+      <option value="http://localhost:4000/docs/mapreduce.html">Using in MapReduce</option>
     
   
     
@@ -525,7 +525,7 @@
     
   
     
-      <option value="/docs/core-java.html">Using Core Java</option>
+      <option value="http://localhost:4000/docs/core-java.html">Using Core Java</option>
     
   
     
@@ -559,7 +559,7 @@
     
   
     
-      <option value="/docs/core-cpp.html">Using Core C++</option>
+      <option value="http://localhost:4000/docs/core-cpp.html">Using Core C++</option>
     
   
     
@@ -606,7 +606,7 @@
     
   
     
-      <option value="/docs/cpp-tools.html">C++ Tools</option>
+      <option value="http://localhost:4000/docs/cpp-tools.html">C++ Tools</option>
     
   
     
@@ -652,7 +652,7 @@
     
   
     
-      <option value="/docs/java-tools.html">Java Tools</option>
+      <option value="http://localhost:4000/docs/java-tools.html">Java Tools</option>
     
   
     
@@ -811,15 +811,14 @@ are the operation (insert, update, or delete), the triple that
 uniquely identifies the row (originalTransaction, bucket, rowId), and
 the current transaction.</p>
 
-<p><code>struct&lt;
-  operation: int,
+<pre><code class="language-struct&lt;">  operation: int,
   originalTransaction: bigInt,
   bucket: int,
   rowId: bigInt,
   currentTransaction: bigInt,
   row: struct&lt;...&gt;
 &gt;
-</code></p>
+</code></pre>
 
 <p>The serialization for the operation codes is:</p>
 
@@ -943,8 +942,6 @@ file that don’t need to be read in this task.</p>
   
 
   
-    
-  
 
   
     
@@ -964,29 +961,25 @@ file that don’t need to be read in this task.</p>
     
   
     
-      <li class=""><a href="/docs/index.html">Background</a></li>
+      <li class=""><a href="http://localhost:4000/docs/index.html">Background</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
   
     
-      <li class=""><a href="/docs/adopters.html">ORC Adopters</a></li>
+      <li class=""><a href="http://localhost:4000/docs/adopters.html">ORC Adopters</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -1018,15 +1011,13 @@ file that don’t need to be read in this task.</p>
     
   
     
-      <li class=""><a href="/docs/types.html">Types</a></li>
+      <li class=""><a href="http://localhost:4000/docs/types.html">Types</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -1048,19 +1039,17 @@ file that don’t need to be read in this task.</p>
     
   
     
-      <li class=""><a href="/docs/indexes.html">Indexes</a></li>
+      <li class=""><a href="http://localhost:4000/docs/indexes.html">Indexes</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
-      <li class="current"><a href="/docs/acid.html">ACID support</a></li>
+      <li class="current"><a href="http://localhost:4000/docs/acid.html">ACID support</a></li>
       
 
 
@@ -1075,8 +1064,6 @@ file that don’t need to be read in this task.</p>
   
 
   
-    
-  
 
   
     
@@ -1084,15 +1071,13 @@ file that don’t need to be read in this task.</p>
     
   
     
-      <li class=""><a href="/docs/building.html">Building ORC</a></li>
+      <li class=""><a href="http://localhost:4000/docs/building.html">Building ORC</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -1122,7 +1107,7 @@ file that don’t need to be read in this task.</p>
     
   
     
-      <li class=""><a href="/docs/releases.html">Releases</a></li>
+      <li class=""><a href="http://localhost:4000/docs/releases.html">Releases</a></li>
       
 
 
@@ -1137,8 +1122,6 @@ file that don’t need to be read in this task.</p>
   
 
   
-    
-  
 
   
     
@@ -1156,15 +1139,13 @@ file that don’t need to be read in this task.</p>
     
   
     
-      <li class=""><a href="/docs/hive-ddl.html">Hive DDL</a></li>
+      <li class=""><a href="http://localhost:4000/docs/hive-ddl.html">Hive DDL</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -1180,7 +1161,7 @@ file that don’t need to be read in this task.</p>
     
   
     
-      <li class=""><a href="/docs/hive-config.html">Hive Configuration</a></li>
+      <li class=""><a href="http://localhost:4000/docs/hive-config.html">Hive Configuration</a></li>
       
 
 
@@ -1195,8 +1176,6 @@ file that don’t need to be read in this task.</p>
   
 
   
-    
-  
 
   
     
@@ -1222,15 +1201,13 @@ file that don’t need to be read in this task.</p>
     
   
     
-      <li class=""><a href="/docs/mapred.html">Using in MapRed</a></li>
+      <li class=""><a href="http://localhost:4000/docs/mapred.html">Using in MapRed</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -1258,7 +1235,7 @@ file that don’t need to be read in this task.</p>
     
   
     
-      <li class=""><a href="/docs/mapreduce.html">Using in MapReduce</a></li>
+      <li class=""><a href="http://localhost:4000/docs/mapreduce.html">Using in MapReduce</a></li>
       
 
 
@@ -1273,8 +1250,6 @@ file that don’t need to be read in this task.</p>
   
 
   
-    
-  
 
   
     
@@ -1286,15 +1261,13 @@ file that don’t need to be read in this task.</p>
     
   
     
-      <li class=""><a href="/docs/core-java.html">Using Core Java</a></li>
+      <li class=""><a href="http://localhost:4000/docs/core-java.html">Using Core Java</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -1304,7 +1277,7 @@ file that don’t need to be read in this task.</p>
     
   
     
-      <li class=""><a href="/docs/core-cpp.html">Using Core C++</a></li>
+      <li class=""><a href="http://localhost:4000/docs/core-cpp.html">Using Core C++</a></li>
       
 
 
@@ -1319,8 +1292,6 @@ file that don’t need to be read in this task.</p>
   
 
   
-    
-  
 
   
     
@@ -1334,15 +1305,13 @@ file that don’t need to be read in this task.</p>
     
   
     
-      <li class=""><a href="/docs/cpp-tools.html">C++ Tools</a></li>
+      <li class=""><a href="http://localhost:4000/docs/cpp-tools.html">C++ Tools</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -1366,7 +1335,7 @@ file that don’t need to be read in this task.</p>
     
   
     
-      <li class=""><a href="/docs/java-tools.html">Java Tools</a></li>
+      <li class=""><a href="http://localhost:4000/docs/java-tools.html">Java Tools</a></li>
       
 
 

http://git-wip-us.apache.org/repos/asf/orc/blob/e58d3fc4/docs/adopters.html
----------------------------------------------------------------------
diff --git a/docs/adopters.html b/docs/adopters.html
index 7ee402b..c93dfed 100644
--- a/docs/adopters.html
+++ b/docs/adopters.html
@@ -4,7 +4,7 @@
   <meta charset="UTF-8">
   <title>ORC Adopters</title>
   <meta name="viewport" content="width=device-width,initial-scale=1">
-  <meta name="generator" content="Jekyll v2.4.0">
+  <meta name="generator" content="Jekyll v3.7.3">
   <link rel="stylesheet" href="//fonts.googleapis.com/css?family=Lato:300,300italic,400,400italic,700,700italic,900">
   <link rel="stylesheet" href="/css/screen.css">
   <link rel="icon" type="image/x-icon" href="/favicon.ico">
@@ -109,7 +109,7 @@
     
   
     
-      <option value="/docs/index.html">Background</option>
+      <option value="http://localhost:4000/docs/index.html">Background</option>
     
   
     
@@ -131,7 +131,7 @@
     
   
     
-      <option value="/docs/adopters.html">ORC Adopters</option>
+      <option value="http://localhost:4000/docs/adopters.html">ORC Adopters</option>
     
   
     
@@ -193,7 +193,7 @@
     
   
     
-      <option value="/docs/types.html">Types</option>
+      <option value="http://localhost:4000/docs/types.html">Types</option>
     
   
 
@@ -219,7 +219,7 @@
     
   
     
-      <option value="/docs/indexes.html">Indexes</option>
+      <option value="http://localhost:4000/docs/indexes.html">Indexes</option>
     
   
     
@@ -237,7 +237,7 @@
 
   
     
-      <option value="/docs/acid.html">ACID support</option>
+      <option value="http://localhost:4000/docs/acid.html">ACID support</option>
     
   
     
@@ -284,7 +284,7 @@
     
   
     
-      <option value="/docs/building.html">Building ORC</option>
+      <option value="http://localhost:4000/docs/building.html">Building ORC</option>
     
   
     
@@ -342,7 +342,7 @@
     
   
     
-      <option value="/docs/releases.html">Releases</option>
+      <option value="http://localhost:4000/docs/releases.html">Releases</option>
     
   
     
@@ -373,7 +373,7 @@
     
   
     
-      <option value="/docs/hive-ddl.html">Hive DDL</option>
+      <option value="http://localhost:4000/docs/hive-ddl.html">Hive DDL</option>
     
   
     
@@ -407,7 +407,7 @@
     
   
     
-      <option value="/docs/hive-config.html">Hive Configuration</option>
+      <option value="http://localhost:4000/docs/hive-config.html">Hive Configuration</option>
     
   
     
@@ -460,7 +460,7 @@
     
   
     
-      <option value="/docs/mapred.html">Using in MapRed</option>
+      <option value="http://localhost:4000/docs/mapred.html">Using in MapRed</option>
     
   
     
@@ -498,7 +498,7 @@
     
   
     
-      <option value="/docs/mapreduce.html">Using in MapReduce</option>
+      <option value="http://localhost:4000/docs/mapreduce.html">Using in MapReduce</option>
     
   
     
@@ -525,7 +525,7 @@
     
   
     
-      <option value="/docs/core-java.html">Using Core Java</option>
+      <option value="http://localhost:4000/docs/core-java.html">Using Core Java</option>
     
   
     
@@ -559,7 +559,7 @@
     
   
     
-      <option value="/docs/core-cpp.html">Using Core C++</option>
+      <option value="http://localhost:4000/docs/core-cpp.html">Using Core C++</option>
     
   
     
@@ -606,7 +606,7 @@
     
   
     
-      <option value="/docs/cpp-tools.html">C++ Tools</option>
+      <option value="http://localhost:4000/docs/cpp-tools.html">C++ Tools</option>
     
   
     
@@ -652,7 +652,7 @@
     
   
     
-      <option value="/docs/java-tools.html">Java Tools</option>
+      <option value="http://localhost:4000/docs/java-tools.html">Java Tools</option>
     
   
     
@@ -677,14 +677,14 @@
           <p>If your company or tool uses ORC, please let us know so that we can update
 this page.</p>
 
-<h3 id="apache-hadoophttpshadoopapacheorg"><a href="https://hadoop.apache.org/">Apache Hadoop</a></h3>
+<h3 id="apache-hadoop"><a href="https://hadoop.apache.org/">Apache Hadoop</a></h3>
 
 <p>ORC files have always supporting reading and writing from Hadoop’s MapReduce,
 but with the ORC 1.1.0 release it is now easier than ever without pulling in
 Hive’s exec jar and all of its dependencies. OrcStruct now also implements
 WritableComparable and can be serialized through the MapReduce shuffle.</p>
 
-<h3 id="apache-hivehttpshiveapacheorg"><a href="https://hive.apache.org/">Apache Hive</a></h3>
+<h3 id="apache-hive"><a href="https://hive.apache.org/">Apache Hive</a></h3>
 
 <p>Apache Hive was the original use case and home for ORC.  ORC’s strong
 type system, advanced compression, column projection, predicate push
@@ -692,25 +692,25 @@ down, and vectorization support make Hive <a href="https://hortonworks.com/blog/
 better</a>
 than any other format for your data.</p>
 
-<h3 id="apache-nifihttpsnifiapacheorg"><a href="https://nifi.apache.org/">Apache Nifi</a></h3>
+<h3 id="apache-nifi"><a href="https://nifi.apache.org/">Apache Nifi</a></h3>
 
 <p>Apache Nifi is <a href="https://issues.apache.org/jira/browse/NIFI-1663">adding
 support</a> for writing
 ORC files.</p>
 
-<h3 id="apache-pighttpspigapacheorg"><a href="https://pig.apache.org/">Apache Pig</a></h3>
+<h3 id="apache-pig"><a href="https://pig.apache.org/">Apache Pig</a></h3>
 
 <p>Apache Pig added support for reading and writing ORC files in <a href="https://hortonworks.com/blog/announcing-apache-pig-0-14-0/">Pig
 14.0</a>.</p>
 
-<h3 id="apache-sparkhttpssparkapacheorg"><a href="https://spark.apache.org/">Apache Spark</a></h3>
+<h3 id="apache-spark"><a href="https://spark.apache.org/">Apache Spark</a></h3>
 
 <p>Apache Spark has <a href="https://hortonworks.com/blog/bringing-orc-support-into-apache-spark/">added
 support</a>
 for reading and writing ORC files with support for column project and
 predicate push down.</p>
 
-<h3 id="eelhttpsgithubcom51zeroeel-sdk"><a href="https://github.com/51zero/eel-sdk">EEL</a></h3>
+<h3 id="eel"><a href="https://github.com/51zero/eel-sdk">EEL</a></h3>
 
 <p>EEL is a Scala BigData API that supports reading and writing data for
 various file formats and storage systems including to and from ORC. It
@@ -723,23 +723,23 @@ complex types such as maps, lists or nested structs. A typical use
 case would be to extract data from JDBC to ORC files housed in HDFS,
 or directly into Hive tables backed by an ORC file format.</p>
 
-<h3 id="facebookhttpsfacebookcom"><a href="https://facebook.com">Facebook</a></h3>
+<h3 id="facebook"><a href="https://facebook.com">Facebook</a></h3>
 
 <p>With more than 300 PB of data, Facebook was an <a href="https://code.facebook.com/posts/229861827208629/scaling-the-facebook-data-warehouse-to-300-pb/">early adopter of
 ORC</a> and quickly put it into production.</p>
 
-<h3 id="prestohttpsprestodbio"><a href="https://prestodb.io/">Presto</a></h3>
+<h3 id="presto"><a href="https://prestodb.io/">Presto</a></h3>
 
 <p>The Presto team has done a lot of work <a href="https://code.facebook.com/posts/370832626374903/even-faster-data-at-the-speed-of-presto-orc/">integrating
 ORC</a> into their SQL engine.</p>
 
-<h3 id="timberhttpstimberio"><a href="https://timber.io/">Timber</a></h3>
+<h3 id="timber"><a href="https://timber.io/">Timber</a></h3>
 
 <p>Timber adopted ORC for it’s S3 based logging platform that stores
 petabytes of log data. ORC has been key in ensuring a fast,
 cost-effective strategy for persisting and querying that data.</p>
 
-<h3 id="verticahttpwww8hpcomusensoftware-solutionsadvanced-sql-big-data-analytics"><a href="http://www8.hp.com/us/en/software-solutions/advanced-sql-big-data-analytics/">Vertica</a></h3>
+<h3 id="vertica"><a href="http://www8.hp.com/us/en/software-solutions/advanced-sql-big-data-analytics/">Vertica</a></h3>
 
 <p>HPE Vertica has contributed significantly to the ORC C++ library. ORC
 is a significant part of Vertica SQL-on-Hadoop (VSQLoH) which brings
@@ -790,8 +790,6 @@ Analytic Database to the Hadoop ecosystem.</p>
   
 
   
-    
-  
 
   
     
@@ -811,29 +809,25 @@ Analytic Database to the Hadoop ecosystem.</p>
     
   
     
-      <li class=""><a href="/docs/index.html">Background</a></li>
+      <li class=""><a href="http://localhost:4000/docs/index.html">Background</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
   
     
-      <li class="current"><a href="/docs/adopters.html">ORC Adopters</a></li>
+      <li class="current"><a href="http://localhost:4000/docs/adopters.html">ORC Adopters</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -865,15 +859,13 @@ Analytic Database to the Hadoop ecosystem.</p>
     
   
     
-      <li class=""><a href="/docs/types.html">Types</a></li>
+      <li class=""><a href="http://localhost:4000/docs/types.html">Types</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -895,19 +887,17 @@ Analytic Database to the Hadoop ecosystem.</p>
     
   
     
-      <li class=""><a href="/docs/indexes.html">Indexes</a></li>
+      <li class=""><a href="http://localhost:4000/docs/indexes.html">Indexes</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
-      <li class=""><a href="/docs/acid.html">ACID support</a></li>
+      <li class=""><a href="http://localhost:4000/docs/acid.html">ACID support</a></li>
       
 
 
@@ -922,8 +912,6 @@ Analytic Database to the Hadoop ecosystem.</p>
   
 
   
-    
-  
 
   
     
@@ -931,15 +919,13 @@ Analytic Database to the Hadoop ecosystem.</p>
     
   
     
-      <li class=""><a href="/docs/building.html">Building ORC</a></li>
+      <li class=""><a href="http://localhost:4000/docs/building.html">Building ORC</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -969,7 +955,7 @@ Analytic Database to the Hadoop ecosystem.</p>
     
   
     
-      <li class=""><a href="/docs/releases.html">Releases</a></li>
+      <li class=""><a href="http://localhost:4000/docs/releases.html">Releases</a></li>
       
 
 
@@ -984,8 +970,6 @@ Analytic Database to the Hadoop ecosystem.</p>
   
 
   
-    
-  
 
   
     
@@ -1003,15 +987,13 @@ Analytic Database to the Hadoop ecosystem.</p>
     
   
     
-      <li class=""><a href="/docs/hive-ddl.html">Hive DDL</a></li>
+      <li class=""><a href="http://localhost:4000/docs/hive-ddl.html">Hive DDL</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -1027,7 +1009,7 @@ Analytic Database to the Hadoop ecosystem.</p>
     
   
     
-      <li class=""><a href="/docs/hive-config.html">Hive Configuration</a></li>
+      <li class=""><a href="http://localhost:4000/docs/hive-config.html">Hive Configuration</a></li>
       
 
 
@@ -1042,8 +1024,6 @@ Analytic Database to the Hadoop ecosystem.</p>
   
 
   
-    
-  
 
   
     
@@ -1069,15 +1049,13 @@ Analytic Database to the Hadoop ecosystem.</p>
     
   
     
-      <li class=""><a href="/docs/mapred.html">Using in MapRed</a></li>
+      <li class=""><a href="http://localhost:4000/docs/mapred.html">Using in MapRed</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -1105,7 +1083,7 @@ Analytic Database to the Hadoop ecosystem.</p>
     
   
     
-      <li class=""><a href="/docs/mapreduce.html">Using in MapReduce</a></li>
+      <li class=""><a href="http://localhost:4000/docs/mapreduce.html">Using in MapReduce</a></li>
       
 
 
@@ -1120,8 +1098,6 @@ Analytic Database to the Hadoop ecosystem.</p>
   
 
   
-    
-  
 
   
     
@@ -1133,15 +1109,13 @@ Analytic Database to the Hadoop ecosystem.</p>
     
   
     
-      <li class=""><a href="/docs/core-java.html">Using Core Java</a></li>
+      <li class=""><a href="http://localhost:4000/docs/core-java.html">Using Core Java</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -1151,7 +1125,7 @@ Analytic Database to the Hadoop ecosystem.</p>
     
   
     
-      <li class=""><a href="/docs/core-cpp.html">Using Core C++</a></li>
+      <li class=""><a href="http://localhost:4000/docs/core-cpp.html">Using Core C++</a></li>
       
 
 
@@ -1166,8 +1140,6 @@ Analytic Database to the Hadoop ecosystem.</p>
   
 
   
-    
-  
 
   
     
@@ -1181,15 +1153,13 @@ Analytic Database to the Hadoop ecosystem.</p>
     
   
     
-      <li class=""><a href="/docs/cpp-tools.html">C++ Tools</a></li>
+      <li class=""><a href="http://localhost:4000/docs/cpp-tools.html">C++ Tools</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -1213,7 +1183,7 @@ Analytic Database to the Hadoop ecosystem.</p>
     
   
     
-      <li class=""><a href="/docs/java-tools.html">Java Tools</a></li>
+      <li class=""><a href="http://localhost:4000/docs/java-tools.html">Java Tools</a></li>
       
 
 

http://git-wip-us.apache.org/repos/asf/orc/blob/e58d3fc4/docs/building.html
----------------------------------------------------------------------
diff --git a/docs/building.html b/docs/building.html
index 378f541..befaa9b 100644
--- a/docs/building.html
+++ b/docs/building.html
@@ -4,7 +4,7 @@
   <meta charset="UTF-8">
   <title>Building ORC</title>
   <meta name="viewport" content="width=device-width,initial-scale=1">
-  <meta name="generator" content="Jekyll v2.4.0">
+  <meta name="generator" content="Jekyll v3.7.3">
   <link rel="stylesheet" href="//fonts.googleapis.com/css?family=Lato:300,300italic,400,400italic,700,700italic,900">
   <link rel="stylesheet" href="/css/screen.css">
   <link rel="icon" type="image/x-icon" href="/favicon.ico">
@@ -109,7 +109,7 @@
     
   
     
-      <option value="/docs/index.html">Background</option>
+      <option value="http://localhost:4000/docs/index.html">Background</option>
     
   
     
@@ -131,7 +131,7 @@
     
   
     
-      <option value="/docs/adopters.html">ORC Adopters</option>
+      <option value="http://localhost:4000/docs/adopters.html">ORC Adopters</option>
     
   
     
@@ -193,7 +193,7 @@
     
   
     
-      <option value="/docs/types.html">Types</option>
+      <option value="http://localhost:4000/docs/types.html">Types</option>
     
   
 
@@ -219,7 +219,7 @@
     
   
     
-      <option value="/docs/indexes.html">Indexes</option>
+      <option value="http://localhost:4000/docs/indexes.html">Indexes</option>
     
   
     
@@ -237,7 +237,7 @@
 
   
     
-      <option value="/docs/acid.html">ACID support</option>
+      <option value="http://localhost:4000/docs/acid.html">ACID support</option>
     
   
     
@@ -284,7 +284,7 @@
     
   
     
-      <option value="/docs/building.html">Building ORC</option>
+      <option value="http://localhost:4000/docs/building.html">Building ORC</option>
     
   
     
@@ -342,7 +342,7 @@
     
   
     
-      <option value="/docs/releases.html">Releases</option>
+      <option value="http://localhost:4000/docs/releases.html">Releases</option>
     
   
     
@@ -373,7 +373,7 @@
     
   
     
-      <option value="/docs/hive-ddl.html">Hive DDL</option>
+      <option value="http://localhost:4000/docs/hive-ddl.html">Hive DDL</option>
     
   
     
@@ -407,7 +407,7 @@
     
   
     
-      <option value="/docs/hive-config.html">Hive Configuration</option>
+      <option value="http://localhost:4000/docs/hive-config.html">Hive Configuration</option>
     
   
     
@@ -460,7 +460,7 @@
     
   
     
-      <option value="/docs/mapred.html">Using in MapRed</option>
+      <option value="http://localhost:4000/docs/mapred.html">Using in MapRed</option>
     
   
     
@@ -498,7 +498,7 @@
     
   
     
-      <option value="/docs/mapreduce.html">Using in MapReduce</option>
+      <option value="http://localhost:4000/docs/mapreduce.html">Using in MapReduce</option>
     
   
     
@@ -525,7 +525,7 @@
     
   
     
-      <option value="/docs/core-java.html">Using Core Java</option>
+      <option value="http://localhost:4000/docs/core-java.html">Using Core Java</option>
     
   
     
@@ -559,7 +559,7 @@
     
   
     
-      <option value="/docs/core-cpp.html">Using Core C++</option>
+      <option value="http://localhost:4000/docs/core-cpp.html">Using Core C++</option>
     
   
     
@@ -606,7 +606,7 @@
     
   
     
-      <option value="/docs/cpp-tools.html">C++ Tools</option>
+      <option value="http://localhost:4000/docs/cpp-tools.html">C++ Tools</option>
     
   
     
@@ -652,7 +652,7 @@
     
   
     
-      <option value="/docs/java-tools.html">Java Tools</option>
+      <option value="http://localhost:4000/docs/java-tools.html">Java Tools</option>
     
   
     
@@ -710,14 +710,14 @@ is in the docker subdirectory, for the list of packages required to build ORC:</
 
 <p>To build a normal release:</p>
 
-<pre><code class="language-shell">% mkdir build
-% cd build
+<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>% mkdir build
+% <span class="nb">cd </span>build
 % cmake ..
 % make package test-out
-</code></pre>
+</code></pre></div></div>
 
 <p>ORC’s C++ build supports three build types, which are controlled by adding
-<code>-DCMAKE_BUILD_TYPE=&lt;type&gt;</code> to the cmake command.</p>
+<code class="highlighter-rouge">-DCMAKE_BUILD_TYPE=&lt;type&gt;</code> to the cmake command.</p>
 
 <ul>
   <li><strong>RELWITHDEBINFO</strong> (default) - Optimized with debug information</li>
@@ -728,8 +728,8 @@ is in the docker subdirectory, for the list of packages required to build ORC:</
 <p>If your make command fails, it is useful to see the actual commands that make
 is invoking:</p>
 
-<pre><code class="language-shell">% make package test-out VERBOSE=1
-</code></pre>
+<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>% make package test-out <span class="nv">VERBOSE</span><span class="o">=</span>1
+</code></pre></div></div>
 
 <h2 id="building-just-java">Building just Java</h2>
 
@@ -742,29 +742,29 @@ is invoking:</p>
 
 <p>To build:</p>
 
-<pre><code class="language-shell">% cd java
+<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>% <span class="nb">cd </span>java
 % mvn package
-</code></pre>
+</code></pre></div></div>
 
 <h2 id="building-just-c">Building just C++</h2>
 
-<pre><code class="language-shell">% mkdir build
-% cd build
-% cmake .. -DBUILD_JAVA=OFF
+<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>% mkdir build
+% <span class="nb">cd </span>build
+% cmake .. <span class="nt">-DBUILD_JAVA</span><span class="o">=</span>OFF
 % make package test-out
-</code></pre>
+</code></pre></div></div>
 
 <h2 id="specify-third-party-libraries-for-c-build">Specify third-party libraries for C++ build</h2>
 
-<pre><code class="language-shell">% mkdir build
-% cd build
-% cmake .. -DSNAPPY_HOME=&lt;PATH&gt; \
-           -DZLIB_HOME=&lt;PATH&gt; \
-           -DLZ4_HOME=&lt;PATH&gt; \
-           -DGTEST_HOME=&lt;PATH&gt; \
-           -DPROTOBUF_HOME=&lt;PATH&gt;
+<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>% mkdir build
+% <span class="nb">cd </span>build
+% cmake .. <span class="nt">-DSNAPPY_HOME</span><span class="o">=</span>&lt;PATH&gt; <span class="se">\</span>
+           <span class="nt">-DZLIB_HOME</span><span class="o">=</span>&lt;PATH&gt; <span class="se">\</span>
+           <span class="nt">-DLZ4_HOME</span><span class="o">=</span>&lt;PATH&gt; <span class="se">\</span>
+           <span class="nt">-DGTEST_HOME</span><span class="o">=</span>&lt;PATH&gt; <span class="se">\</span>
+           <span class="nt">-DPROTOBUF_HOME</span><span class="o">=</span>&lt;PATH&gt;
 % make package test-out
-</code></pre>
+</code></pre></div></div>
 
           
 
@@ -822,8 +822,6 @@ is invoking:</p>
   
 
   
-    
-  
 
   
     
@@ -843,29 +841,25 @@ is invoking:</p>
     
   
     
-      <li class=""><a href="/docs/index.html">Background</a></li>
+      <li class=""><a href="http://localhost:4000/docs/index.html">Background</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
   
     
-      <li class=""><a href="/docs/adopters.html">ORC Adopters</a></li>
+      <li class=""><a href="http://localhost:4000/docs/adopters.html">ORC Adopters</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -897,15 +891,13 @@ is invoking:</p>
     
   
     
-      <li class=""><a href="/docs/types.html">Types</a></li>
+      <li class=""><a href="http://localhost:4000/docs/types.html">Types</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -927,19 +919,17 @@ is invoking:</p>
     
   
     
-      <li class=""><a href="/docs/indexes.html">Indexes</a></li>
+      <li class=""><a href="http://localhost:4000/docs/indexes.html">Indexes</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
-      <li class=""><a href="/docs/acid.html">ACID support</a></li>
+      <li class=""><a href="http://localhost:4000/docs/acid.html">ACID support</a></li>
       
 
 
@@ -954,8 +944,6 @@ is invoking:</p>
   
 
   
-    
-  
 
   
     
@@ -963,15 +951,13 @@ is invoking:</p>
     
   
     
-      <li class="current"><a href="/docs/building.html">Building ORC</a></li>
+      <li class="current"><a href="http://localhost:4000/docs/building.html">Building ORC</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -1001,7 +987,7 @@ is invoking:</p>
     
   
     
-      <li class=""><a href="/docs/releases.html">Releases</a></li>
+      <li class=""><a href="http://localhost:4000/docs/releases.html">Releases</a></li>
       
 
 
@@ -1016,8 +1002,6 @@ is invoking:</p>
   
 
   
-    
-  
 
   
     
@@ -1035,15 +1019,13 @@ is invoking:</p>
     
   
     
-      <li class=""><a href="/docs/hive-ddl.html">Hive DDL</a></li>
+      <li class=""><a href="http://localhost:4000/docs/hive-ddl.html">Hive DDL</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -1059,7 +1041,7 @@ is invoking:</p>
     
   
     
-      <li class=""><a href="/docs/hive-config.html">Hive Configuration</a></li>
+      <li class=""><a href="http://localhost:4000/docs/hive-config.html">Hive Configuration</a></li>
       
 
 
@@ -1074,8 +1056,6 @@ is invoking:</p>
   
 
   
-    
-  
 
   
     
@@ -1101,15 +1081,13 @@ is invoking:</p>
     
   
     
-      <li class=""><a href="/docs/mapred.html">Using in MapRed</a></li>
+      <li class=""><a href="http://localhost:4000/docs/mapred.html">Using in MapRed</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -1137,7 +1115,7 @@ is invoking:</p>
     
   
     
-      <li class=""><a href="/docs/mapreduce.html">Using in MapReduce</a></li>
+      <li class=""><a href="http://localhost:4000/docs/mapreduce.html">Using in MapReduce</a></li>
       
 
 
@@ -1152,8 +1130,6 @@ is invoking:</p>
   
 
   
-    
-  
 
   
     
@@ -1165,15 +1141,13 @@ is invoking:</p>
     
   
     
-      <li class=""><a href="/docs/core-java.html">Using Core Java</a></li>
+      <li class=""><a href="http://localhost:4000/docs/core-java.html">Using Core Java</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -1183,7 +1157,7 @@ is invoking:</p>
     
   
     
-      <li class=""><a href="/docs/core-cpp.html">Using Core C++</a></li>
+      <li class=""><a href="http://localhost:4000/docs/core-cpp.html">Using Core C++</a></li>
       
 
 
@@ -1198,8 +1172,6 @@ is invoking:</p>
   
 
   
-    
-  
 
   
     
@@ -1213,15 +1185,13 @@ is invoking:</p>
     
   
     
-      <li class=""><a href="/docs/cpp-tools.html">C++ Tools</a></li>
+      <li class=""><a href="http://localhost:4000/docs/cpp-tools.html">C++ Tools</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -1245,7 +1215,7 @@ is invoking:</p>
     
   
     
-      <li class=""><a href="/docs/java-tools.html">Java Tools</a></li>
+      <li class=""><a href="http://localhost:4000/docs/java-tools.html">Java Tools</a></li>
       
 
 

http://git-wip-us.apache.org/repos/asf/orc/blob/e58d3fc4/docs/core-cpp.html
----------------------------------------------------------------------
diff --git a/docs/core-cpp.html b/docs/core-cpp.html
index ec31d6f..2f00682 100644
--- a/docs/core-cpp.html
+++ b/docs/core-cpp.html
@@ -4,7 +4,7 @@
   <meta charset="UTF-8">
   <title>Using Core C++</title>
   <meta name="viewport" content="width=device-width,initial-scale=1">
-  <meta name="generator" content="Jekyll v2.4.0">
+  <meta name="generator" content="Jekyll v3.7.3">
   <link rel="stylesheet" href="//fonts.googleapis.com/css?family=Lato:300,300italic,400,400italic,700,700italic,900">
   <link rel="stylesheet" href="/css/screen.css">
   <link rel="icon" type="image/x-icon" href="/favicon.ico">
@@ -109,7 +109,7 @@
     
   
     
-      <option value="/docs/index.html">Background</option>
+      <option value="http://localhost:4000/docs/index.html">Background</option>
     
   
     
@@ -131,7 +131,7 @@
     
   
     
-      <option value="/docs/adopters.html">ORC Adopters</option>
+      <option value="http://localhost:4000/docs/adopters.html">ORC Adopters</option>
     
   
     
@@ -193,7 +193,7 @@
     
   
     
-      <option value="/docs/types.html">Types</option>
+      <option value="http://localhost:4000/docs/types.html">Types</option>
     
   
 
@@ -219,7 +219,7 @@
     
   
     
-      <option value="/docs/indexes.html">Indexes</option>
+      <option value="http://localhost:4000/docs/indexes.html">Indexes</option>
     
   
     
@@ -237,7 +237,7 @@
 
   
     
-      <option value="/docs/acid.html">ACID support</option>
+      <option value="http://localhost:4000/docs/acid.html">ACID support</option>
     
   
     
@@ -284,7 +284,7 @@
     
   
     
-      <option value="/docs/building.html">Building ORC</option>
+      <option value="http://localhost:4000/docs/building.html">Building ORC</option>
     
   
     
@@ -342,7 +342,7 @@
     
   
     
-      <option value="/docs/releases.html">Releases</option>
+      <option value="http://localhost:4000/docs/releases.html">Releases</option>
     
   
     
@@ -373,7 +373,7 @@
     
   
     
-      <option value="/docs/hive-ddl.html">Hive DDL</option>
+      <option value="http://localhost:4000/docs/hive-ddl.html">Hive DDL</option>
     
   
     
@@ -407,7 +407,7 @@
     
   
     
-      <option value="/docs/hive-config.html">Hive Configuration</option>
+      <option value="http://localhost:4000/docs/hive-config.html">Hive Configuration</option>
     
   
     
@@ -460,7 +460,7 @@
     
   
     
-      <option value="/docs/mapred.html">Using in MapRed</option>
+      <option value="http://localhost:4000/docs/mapred.html">Using in MapRed</option>
     
   
     
@@ -498,7 +498,7 @@
     
   
     
-      <option value="/docs/mapreduce.html">Using in MapReduce</option>
+      <option value="http://localhost:4000/docs/mapreduce.html">Using in MapReduce</option>
     
   
     
@@ -525,7 +525,7 @@
     
   
     
-      <option value="/docs/core-java.html">Using Core Java</option>
+      <option value="http://localhost:4000/docs/core-java.html">Using Core Java</option>
     
   
     
@@ -559,7 +559,7 @@
     
   
     
-      <option value="/docs/core-cpp.html">Using Core C++</option>
+      <option value="http://localhost:4000/docs/core-cpp.html">Using Core C++</option>
     
   
     
@@ -606,7 +606,7 @@
     
   
     
-      <option value="/docs/cpp-tools.html">C++ Tools</option>
+      <option value="http://localhost:4000/docs/cpp-tools.html">C++ Tools</option>
     
   
     
@@ -652,7 +652,7 @@
     
   
     
-      <option value="/docs/java-tools.html">Java Tools</option>
+      <option value="http://localhost:4000/docs/java-tools.html">Java Tools</option>
     
   
     
@@ -681,23 +681,23 @@ orc::ColumnVectorBatch vectorized classes.</p>
 
 <p>Data is passed to ORC as instances of orc::ColumnVectorBatch
 that contain the data a batch of rows. The focus is on speed and
-accessing the data fields directly. <code>numElements</code> is the number
+accessing the data fields directly. <code class="highlighter-rouge">numElements</code> is the number
 of rows. ColumnVectorBatch is the parent type of the different
 kinds of columns and has some fields that are shared across
-all of the column types. In particular, the <code>hasNulls</code> flag
+all of the column types. In particular, the <code class="highlighter-rouge">hasNulls</code> flag
 if there is any null in this column for this batch. For columns
-where <code>hasNulls == true</code> the <code>notNull</code> buffer is false if that
+where <code class="highlighter-rouge">hasNulls == true</code> the <code class="highlighter-rouge">notNull</code> buffer is false if that
 value is null.</p>
 
-<pre><code class="language-cpp">namespace orc {
-  struct ColumnVectorBatch {
-    uint64_t numElements;
-    DataBuffer&lt;char&gt; notNull;
-    bool hasNulls;
-    ...
-  }
-}
-</code></pre>
+<div class="language-cpp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">namespace</span> <span class="n">orc</span> <span class="p">{</span>
+  <span class="k">struct</span> <span class="n">ColumnVectorBatch</span> <span class="p">{</span>
+    <span class="kt">uint64_t</span> <span class="n">numElements</span><span class="p">;</span>
+    <span class="n">DataBuffer</span><span class="o">&lt;</span><span class="kt">char</span><span class="o">&gt;</span> <span class="n">notNull</span><span class="p">;</span>
+    <span class="kt">bool</span> <span class="n">hasNulls</span><span class="p">;</span>
+    <span class="p">...</span>
+  <span class="p">}</span>
+<span class="p">}</span>
+</code></pre></div></div>
 
 <p>The subtypes of ColumnVectorBatch are:</p>
 
@@ -788,11 +788,11 @@ value is null.</p>
 date, int, smallint, and tinyint). The data is represented as a
 buffer of int64_t where each value is sign-extended as necessary.</p>
 
-<pre><code class="language-cpp">  struct LongVectorBatch: public ColumnVectorBatch {
-    DataBuffer&lt;int64_t&gt; data;
-    ...
-  };
-</code></pre>
+<div class="language-cpp highlighter-rouge"><div class="highlight"><pre class="highlight"><code>  <span class="k">struct</span> <span class="n">LongVectorBatch</span><span class="o">:</span> <span class="k">public</span> <span class="n">ColumnVectorBatch</span> <span class="p">{</span>
+    <span class="n">DataBuffer</span><span class="o">&lt;</span><span class="kt">int64_t</span><span class="o">&gt;</span> <span class="n">data</span><span class="p">;</span>
+    <span class="p">...</span>
+  <span class="p">};</span>
+</code></pre></div></div>
 
 <p>TimestampVectorBatch handles timestamp values. The data is
 represented as two buffers of int64_t for seconds and nanoseconds
@@ -800,187 +800,187 @@ respectively. Note that we always assume data is in GMT timezone;
 therefore it is user’s responsibility to convert wall clock time
 from local timezone to GMT.</p>
 
-<pre><code class="language-cpp">  struct TimestampVectorBatch: public ColumnVectorBatch {
-    DataBuffer&lt;int64_t&gt; data;
-    DataBuffer&lt;int64_t&gt; nanoseconds;
-    ...
-  };
-</code></pre>
+<div class="language-cpp highlighter-rouge"><div class="highlight"><pre class="highlight"><code>  <span class="k">struct</span> <span class="n">TimestampVectorBatch</span><span class="o">:</span> <span class="k">public</span> <span class="n">ColumnVectorBatch</span> <span class="p">{</span>
+    <span class="n">DataBuffer</span><span class="o">&lt;</span><span class="kt">int64_t</span><span class="o">&gt;</span> <span class="n">data</span><span class="p">;</span>
+    <span class="n">DataBuffer</span><span class="o">&lt;</span><span class="kt">int64_t</span><span class="o">&gt;</span> <span class="n">nanoseconds</span><span class="p">;</span>
+    <span class="p">...</span>
+  <span class="p">};</span>
+</code></pre></div></div>
 
 <p>DoubleVectorBatch handles all of the floating point types
 (double, and float). The data is represented as a buffer of doubles.</p>
 
-<pre><code class="language-cpp">  struct DoubleVectorBatch: public ColumnVectorBatch {
-    DataBuffer&lt;double&gt; data;
-    ...
-  };
-</code></pre>
+<div class="language-cpp highlighter-rouge"><div class="highlight"><pre class="highlight"><code>  <span class="k">struct</span> <span class="n">DoubleVectorBatch</span><span class="o">:</span> <span class="k">public</span> <span class="n">ColumnVectorBatch</span> <span class="p">{</span>
+    <span class="n">DataBuffer</span><span class="o">&lt;</span><span class="kt">double</span><span class="o">&gt;</span> <span class="n">data</span><span class="p">;</span>
+    <span class="p">...</span>
+  <span class="p">};</span>
+</code></pre></div></div>
 
 <p>Decimal64VectorBatch handles decimal columns with precision no
 greater than 18. Decimal128VectorBatch handles the others. The data
 is represented as a buffer of int64_t and orc::Int128 respectively.</p>
 
-<pre><code class="language-cpp">  struct Decimal64VectorBatch: public ColumnVectorBatch {
-    DataBuffer&lt;int64_t&gt; values;
-    ...
-  };
+<div class="language-cpp highlighter-rouge"><div class="highlight"><pre class="highlight"><code>  <span class="k">struct</span> <span class="n">Decimal64VectorBatch</span><span class="o">:</span> <span class="k">public</span> <span class="n">ColumnVectorBatch</span> <span class="p">{</span>
+    <span class="n">DataBuffer</span><span class="o">&lt;</span><span class="kt">int64_t</span><span class="o">&gt;</span> <span class="n">values</span><span class="p">;</span>
+    <span class="p">...</span>
+  <span class="p">};</span>
 
-  struct Decimal128VectorBatch: public ColumnVectorBatch {
-    DataBuffer&lt;Int128&gt; values;
-    ...
-  };
-</code></pre>
+  <span class="k">struct</span> <span class="n">Decimal128VectorBatch</span><span class="o">:</span> <span class="k">public</span> <span class="n">ColumnVectorBatch</span> <span class="p">{</span>
+    <span class="n">DataBuffer</span><span class="o">&lt;</span><span class="n">Int128</span><span class="o">&gt;</span> <span class="n">values</span><span class="p">;</span>
+    <span class="p">...</span>
+  <span class="p">};</span>
+</code></pre></div></div>
 
 <p>StringVectorBatch handles all of the binary types (binary,
 char, string, and varchar). The data is represented as a char* buffer,
 and a length buffer.</p>
 
-<pre><code class="language-cpp">  struct StringVectorBatch: public ColumnVectorBatch {
-    DataBuffer&lt;char*&gt; data;
-    DataBuffer&lt;int64_t&gt; length;
-    ...
-  };
-</code></pre>
+<div class="language-cpp highlighter-rouge"><div class="highlight"><pre class="highlight"><code>  <span class="k">struct</span> <span class="n">StringVectorBatch</span><span class="o">:</span> <span class="k">public</span> <span class="n">ColumnVectorBatch</span> <span class="p">{</span>
+    <span class="n">DataBuffer</span><span class="o">&lt;</span><span class="kt">char</span><span class="o">*&gt;</span> <span class="n">data</span><span class="p">;</span>
+    <span class="n">DataBuffer</span><span class="o">&lt;</span><span class="kt">int64_t</span><span class="o">&gt;</span> <span class="n">length</span><span class="p">;</span>
+    <span class="p">...</span>
+  <span class="p">};</span>
+</code></pre></div></div>
 
 <p>StructVectorBatch handles the struct columns and represents
-the data as a buffer of <code>ColumnVectorBatch</code>.</p>
+the data as a buffer of <code class="highlighter-rouge">ColumnVectorBatch</code>.</p>
 
-<pre><code class="language-cpp">  struct StructVectorBatch: public ColumnVectorBatch {
-    std::vector&lt;ColumnVectorBatch*&gt; fields;
-    ...
-  };
-</code></pre>
+<div class="language-cpp highlighter-rouge"><div class="highlight"><pre class="highlight"><code>  <span class="k">struct</span> <span class="n">StructVectorBatch</span><span class="o">:</span> <span class="k">public</span> <span class="n">ColumnVectorBatch</span> <span class="p">{</span>
+    <span class="n">std</span><span class="o">::</span><span class="n">vector</span><span class="o">&lt;</span><span class="n">ColumnVectorBatch</span><span class="o">*&gt;</span> <span class="n">fields</span><span class="p">;</span>
+    <span class="p">...</span>
+  <span class="p">};</span>
+</code></pre></div></div>
 
-<p>UnionVectorBatch handles the union columns. It uses <code>tags</code>
-to indicate which subtype has the value and <code>offsets</code> indicates
+<p>UnionVectorBatch handles the union columns. It uses <code class="highlighter-rouge">tags</code>
+to indicate which subtype has the value and <code class="highlighter-rouge">offsets</code> indicates
 the offset in child batch of that subtype. A individual
-<code>ColumnVectorBatch</code> is used for each subtype.</p>
+<code class="highlighter-rouge">ColumnVectorBatch</code> is used for each subtype.</p>
 
-<pre><code class="language-cpp">  struct UnionVectorBatch: public ColumnVectorBatch {
-    DataBuffer&lt;unsigned char&gt; tags;
-    DataBuffer&lt;uint64_t&gt; offsets;
-    std::vector&lt;ColumnVectorBatch*&gt; children;
-    ...
-  };
-</code></pre>
+<div class="language-cpp highlighter-rouge"><div class="highlight"><pre class="highlight"><code>  <span class="k">struct</span> <span class="n">UnionVectorBatch</span><span class="o">:</span> <span class="k">public</span> <span class="n">ColumnVectorBatch</span> <span class="p">{</span>
+    <span class="n">DataBuffer</span><span class="o">&lt;</span><span class="kt">unsigned</span> <span class="kt">char</span><span class="o">&gt;</span> <span class="n">tags</span><span class="p">;</span>
+    <span class="n">DataBuffer</span><span class="o">&lt;</span><span class="kt">uint64_t</span><span class="o">&gt;</span> <span class="n">offsets</span><span class="p">;</span>
+    <span class="n">std</span><span class="o">::</span><span class="n">vector</span><span class="o">&lt;</span><span class="n">ColumnVectorBatch</span><span class="o">*&gt;</span> <span class="n">children</span><span class="p">;</span>
+    <span class="p">...</span>
+  <span class="p">};</span>
+</code></pre></div></div>
 
 <p>ListVectorBatch handles the array columns and represents
 the data as a buffer of integers for the offsets and a
-<code>ColumnVectorBatch</code> for the children values.</p>
+<code class="highlighter-rouge">ColumnVectorBatch</code> for the children values.</p>
 
-<pre><code class="language-cpp">  struct ListVectorBatch: public ColumnVectorBatch {
-    DataBuffer&lt;int64_t&gt; offsets;
-    ORC_UNIQUE_PTR&lt;ColumnVectorBatch&gt; elements;
-    ...
-  };
-</code></pre>
+<div class="language-cpp highlighter-rouge"><div class="highlight"><pre class="highlight"><code>  <span class="k">struct</span> <span class="n">ListVectorBatch</span><span class="o">:</span> <span class="k">public</span> <span class="n">ColumnVectorBatch</span> <span class="p">{</span>
+    <span class="n">DataBuffer</span><span class="o">&lt;</span><span class="kt">int64_t</span><span class="o">&gt;</span> <span class="n">offsets</span><span class="p">;</span>
+    <span class="n">ORC_UNIQUE_PTR</span><span class="o">&lt;</span><span class="n">ColumnVectorBatch</span><span class="o">&gt;</span> <span class="n">elements</span><span class="p">;</span>
+    <span class="p">...</span>
+  <span class="p">};</span>
+</code></pre></div></div>
 
 <p>MapVectorBatch handles the map columns and represents the data
-as two arrays of integers for the offsets and two <code>ColumnVectorBatch</code>s
+as two arrays of integers for the offsets and two <code class="highlighter-rouge">ColumnVectorBatch</code>s
 for the keys and values.</p>
 
-<pre><code class="language-cpp">  struct MapVectorBatch: public ColumnVectorBatch {
-    DataBuffer&lt;int64_t&gt; offsets;
-    ORC_UNIQUE_PTR&lt;ColumnVectorBatch&gt; keys;
-    ORC_UNIQUE_PTR&lt;ColumnVectorBatch&gt; elements;
-    ...
-  };
-</code></pre>
+<div class="language-cpp highlighter-rouge"><div class="highlight"><pre class="highlight"><code>  <span class="k">struct</span> <span class="n">MapVectorBatch</span><span class="o">:</span> <span class="k">public</span> <span class="n">ColumnVectorBatch</span> <span class="p">{</span>
+    <span class="n">DataBuffer</span><span class="o">&lt;</span><span class="kt">int64_t</span><span class="o">&gt;</span> <span class="n">offsets</span><span class="p">;</span>
+    <span class="n">ORC_UNIQUE_PTR</span><span class="o">&lt;</span><span class="n">ColumnVectorBatch</span><span class="o">&gt;</span> <span class="n">keys</span><span class="p">;</span>
+    <span class="n">ORC_UNIQUE_PTR</span><span class="o">&lt;</span><span class="n">ColumnVectorBatch</span><span class="o">&gt;</span> <span class="n">elements</span><span class="p">;</span>
+    <span class="p">...</span>
+  <span class="p">};</span>
+</code></pre></div></div>
 
 <h2 id="writing-orc-files">Writing ORC Files</h2>
 
-<p>To write an ORC file, you need to include <code>OrcFile.hh</code> and define
-the schema; then use <code>orc::OutputStream</code> and <code>orc::WriterOptions</code>
-to create a <code>orc::Writer</code> with the desired filename. This example
+<p>To write an ORC file, you need to include <code class="highlighter-rouge">OrcFile.hh</code> and define
+the schema; then use <code class="highlighter-rouge">orc::OutputStream</code> and <code class="highlighter-rouge">orc::WriterOptions</code>
+to create a <code class="highlighter-rouge">orc::Writer</code> with the desired filename. This example
 sets the required schema parameter, but there are many other
 options to control the ORC writer.</p>
 
-<pre><code class="language-cpp">ORC_UNIQUE_PTR&lt;OutputStream&gt; outStream =
-  writeLocalFile("my-file.orc");
-ORC_UNIQUE_PTR&lt;Type&gt; schema(
-  Type::buildTypeFromString("struct&lt;x:int,y:int&gt;"));
-WriterOptions options;
-ORC_UNIQUE_PTR&lt;Writer&gt; writer =
-  createWriter(*schema, outStream.get(), options);
-</code></pre>
+<div class="language-cpp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">ORC_UNIQUE_PTR</span><span class="o">&lt;</span><span class="n">OutputStream</span><span class="o">&gt;</span> <span class="n">outStream</span> <span class="o">=</span>
+  <span class="n">writeLocalFile</span><span class="p">(</span><span class="s">"my-file.orc"</span><span class="p">);</span>
+<span class="n">ORC_UNIQUE_PTR</span><span class="o">&lt;</span><span class="n">Type</span><span class="o">&gt;</span> <span class="n">schema</span><span class="p">(</span>
+  <span class="n">Type</span><span class="o">::</span><span class="n">buildTypeFromString</span><span class="p">(</span><span class="s">"struct&lt;x:int,y:int&gt;"</span><span class="p">));</span>
+<span class="n">WriterOptions</span> <span class="n">options</span><span class="p">;</span>
+<span class="n">ORC_UNIQUE_PTR</span><span class="o">&lt;</span><span class="n">Writer</span><span class="o">&gt;</span> <span class="n">writer</span> <span class="o">=</span>
+  <span class="n">createWriter</span><span class="p">(</span><span class="o">*</span><span class="n">schema</span><span class="p">,</span> <span class="n">outStream</span><span class="p">.</span><span class="n">get</span><span class="p">(),</span> <span class="n">options</span><span class="p">);</span>
+</code></pre></div></div>
 
 <p>Now you need to create a row batch, set the data, and write it to the file
-as the batch fills up. When the file is done, close the <code>Writer</code>.</p>
-
-<pre><code class="language-cpp">uint64_t batchSize = 1024, rowCount = 10000;
-ORC_UNIQUE_PTR&lt;ColumnVectorBatch&gt; batch =
-  writer-&gt;createRowBatch(batchSize);
-StructVectorBatch *root =
-  dynamic_cast&lt;StructVectorBatch *&gt;(batch.get());
-LongVectorBatch *x =
-  dynamic_cast&lt;LongVectorBatch *&gt;(root-&gt;fields[0]);
-LongVectorBatch *y =
-  dynamic_cast&lt;LongVectorBatch *&gt;(root-&gt;fields[1]);
-
-uint64_t rows = 0;
-for (uint64_t i = 0; i &lt; rowCount; ++i) {
-  x-&gt;data[rows] = i;
-  y-&gt;data[rows] = i * 3;
-  rows++;
-
-  if (rows == batchSize) {
-    root-&gt;numElements = rows;
-    x-&gt;numElements = rows;
-    y-&gt;numElements = rows;
-
-    writer-&gt;add(*batch);
-    rows = 0;
-  }
-}
-
-if (rows != 0) {
-  root-&gt;numElements = rows;
-  x-&gt;numElements = rows;
-  y-&gt;numElements = rows;
-
-  writer-&gt;add(*batch);
-  rows = 0;
-}
-
-writer-&gt;close();
-</code></pre>
+as the batch fills up. When the file is done, close the <code class="highlighter-rouge">Writer</code>.</p>
+
+<div class="language-cpp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kt">uint64_t</span> <span class="n">batchSize</span> <span class="o">=</span> <span class="mi">1024</span><span class="p">,</span> <span class="n">rowCount</span> <span class="o">=</span> <span class="mi">10000</span><span class="p">;</span>
+<span class="n">ORC_UNIQUE_PTR</span><span class="o">&lt;</span><span class="n">ColumnVectorBatch</span><span class="o">&gt;</span> <span class="n">batch</span> <span class="o">=</span>
+  <span class="n">writer</span><span class="o">-&gt;</span><span class="n">createRowBatch</span><span class="p">(</span><span class="n">batchSize</span><span class="p">);</span>
+<span class="n">StructVectorBatch</span> <span class="o">*</span><span class="n">root</span> <span class="o">=</span>
+  <span class="k">dynamic_cast</span><span class="o">&lt;</span><span class="n">StructVectorBatch</span> <span class="o">*&gt;</span><span class="p">(</span><span class="n">batch</span><span class="p">.</span><span class="n">get</span><span class="p">());</span>
+<span class="n">LongVectorBatch</span> <span class="o">*</span><span class="n">x</span> <span class="o">=</span>
+  <span class="k">dynamic_cast</span><span class="o">&lt;</span><span class="n">LongVectorBatch</span> <span class="o">*&gt;</span><span class="p">(</span><span class="n">root</span><span class="o">-&gt;</span><span class="n">fields</span><span class="p">[</span><span class="mi">0</span><span class="p">]);</span>
+<span class="n">LongVectorBatch</span> <span class="o">*</span><span class="n">y</span> <span class="o">=</span>
+  <span class="k">dynamic_cast</span><span class="o">&lt;</span><span class="n">LongVectorBatch</span> <span class="o">*&gt;</span><span class="p">(</span><span class="n">root</span><span class="o">-&gt;</span><span class="n">fields</span><span class="p">[</span><span class="mi">1</span><span class="p">]);</span>
+
+<span class="kt">uint64_t</span> <span class="n">rows</span> <span class="o">=</span> <span class="mi">0</span><span class="p">;</span>
+<span class="k">for</span> <span class="p">(</span><span class="kt">uint64_t</span> <span class="n">i</span> <span class="o">=</span> <span class="mi">0</span><span class="p">;</span> <span class="n">i</span> <span class="o">&lt;</span> <span class="n">rowCount</span><span class="p">;</span> <span class="o">++</span><span class="n">i</span><span class="p">)</span> <span class="p">{</span>
+  <span class="n">x</span><span class="o">-&gt;</span><span class="n">data</span><span class="p">[</span><span class="n">rows</span><span class="p">]</span> <span class="o">=</span> <span class="n">i</span><span class="p">;</span>
+  <span class="n">y</span><span class="o">-&gt;</span><span class="n">data</span><span class="p">[</span><span class="n">rows</span><span class="p">]</span> <span class="o">=</span> <span class="n">i</span> <span class="o">*</span> <span class="mi">3</span><span class="p">;</span>
+  <span class="n">rows</span><span class="o">++</span><span class="p">;</span>
+
+  <span class="k">if</span> <span class="p">(</span><span class="n">rows</span> <span class="o">==</span> <span class="n">batchSize</span><span class="p">)</span> <span class="p">{</span>
+    <span class="n">root</span><span class="o">-&gt;</span><span class="n">numElements</span> <span class="o">=</span> <span class="n">rows</span><span class="p">;</span>
+    <span class="n">x</span><span class="o">-&gt;</span><span class="n">numElements</span> <span class="o">=</span> <span class="n">rows</span><span class="p">;</span>
+    <span class="n">y</span><span class="o">-&gt;</span><span class="n">numElements</span> <span class="o">=</span> <span class="n">rows</span><span class="p">;</span>
+
+    <span class="n">writer</span><span class="o">-&gt;</span><span class="n">add</span><span class="p">(</span><span class="o">*</span><span class="n">batch</span><span class="p">);</span>
+    <span class="n">rows</span> <span class="o">=</span> <span class="mi">0</span><span class="p">;</span>
+  <span class="p">}</span>
+<span class="p">}</span>
+
+<span class="k">if</span> <span class="p">(</span><span class="n">rows</span> <span class="o">!=</span> <span class="mi">0</span><span class="p">)</span> <span class="p">{</span>
+  <span class="n">root</span><span class="o">-&gt;</span><span class="n">numElements</span> <span class="o">=</span> <span class="n">rows</span><span class="p">;</span>
+  <span class="n">x</span><span class="o">-&gt;</span><span class="n">numElements</span> <span class="o">=</span> <span class="n">rows</span><span class="p">;</span>
+  <span class="n">y</span><span class="o">-&gt;</span><span class="n">numElements</span> <span class="o">=</span> <span class="n">rows</span><span class="p">;</span>
+
+  <span class="n">writer</span><span class="o">-&gt;</span><span class="n">add</span><span class="p">(</span><span class="o">*</span><span class="n">batch</span><span class="p">);</span>
+  <span class="n">rows</span> <span class="o">=</span> <span class="mi">0</span><span class="p">;</span>
+<span class="p">}</span>
+
+<span class="n">writer</span><span class="o">-&gt;</span><span class="n">close</span><span class="p">();</span>
+</code></pre></div></div>
 
 <h2 id="reading-orc-files">Reading ORC Files</h2>
 
-<p>To read ORC files, include <code>OrcFile.hh</code> file to create a <code>orc::Reader</code>
+<p>To read ORC files, include <code class="highlighter-rouge">OrcFile.hh</code> file to create a <code class="highlighter-rouge">orc::Reader</code>
 that contains the metadata about the file. There are a few options to
-the <code>orc::Reader</code>, but far fewer than the writer and none of them are
+the <code class="highlighter-rouge">orc::Reader</code>, but far fewer than the writer and none of them are
 required. The reader has methods for getting the number of rows,
 schema, compression, etc. from the file.</p>
 
-<pre><code class="language-cpp">ORC_UNIQUE_PTR&lt;InputStream&gt; inStream =
-  readLocalFile("my-file.orc");
-ReaderOptions options;
-ORC_UNIQUE_PTR&lt;Reader&gt; reader =
-  createReader(inStream, options);
-</code></pre>
+<div class="language-cpp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">ORC_UNIQUE_PTR</span><span class="o">&lt;</span><span class="n">InputStream</span><span class="o">&gt;</span> <span class="n">inStream</span> <span class="o">=</span>
+  <span class="n">readLocalFile</span><span class="p">(</span><span class="s">"my-file.orc"</span><span class="p">);</span>
+<span class="n">ReaderOptions</span> <span class="n">options</span><span class="p">;</span>
+<span class="n">ORC_UNIQUE_PTR</span><span class="o">&lt;</span><span class="n">Reader</span><span class="o">&gt;</span> <span class="n">reader</span> <span class="o">=</span>
+  <span class="n">createReader</span><span class="p">(</span><span class="n">inStream</span><span class="p">,</span> <span class="n">options</span><span class="p">);</span>
+</code></pre></div></div>
 
-<p>To get the data, create a <code>orc::RowReader</code> object. By default,
+<p>To get the data, create a <code class="highlighter-rouge">orc::RowReader</code> object. By default,
 the RowReader reads all rows and all columns, but there are
 options to control the data that is read.</p>
 
-<pre><code class="language-cpp">RowReaderOptions rowReaderOptions;
-ORC_UNIQUE_PTR&lt;RowReader&gt; rowReader =
-  reader-&gt;createRowReader(rowReaderOptions);
-ORC_UNIQUE_PTR&lt;ColumnVectorBatch&gt; batch =
-  rowReader-&gt;createRowBatch(1024);
-</code></pre>
+<div class="language-cpp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">RowReaderOptions</span> <span class="n">rowReaderOptions</span><span class="p">;</span>
+<span class="n">ORC_UNIQUE_PTR</span><span class="o">&lt;</span><span class="n">RowReader</span><span class="o">&gt;</span> <span class="n">rowReader</span> <span class="o">=</span>
+  <span class="n">reader</span><span class="o">-&gt;</span><span class="n">createRowReader</span><span class="p">(</span><span class="n">rowReaderOptions</span><span class="p">);</span>
+<span class="n">ORC_UNIQUE_PTR</span><span class="o">&lt;</span><span class="n">ColumnVectorBatch</span><span class="o">&gt;</span> <span class="n">batch</span> <span class="o">=</span>
+  <span class="n">rowReader</span><span class="o">-&gt;</span><span class="n">createRowBatch</span><span class="p">(</span><span class="mi">1024</span><span class="p">);</span>
+</code></pre></div></div>
 
-<p>With a <code>orc::RowReader</code> the user can ask for the next batch until there
+<p>With a <code class="highlighter-rouge">orc::RowReader</code> the user can ask for the next batch until there
 are no more left. The reader will stop the batch at certain boundaries,
 so the returned batch may not be full, but it will always contain some rows.</p>
 
-<pre><code class="language-cpp">while (rowReader-&gt;next(*batch)) {
-  for (uint64_t r = 0; r &lt; batch-&gt;numElements; ++r) {
-    ... process row r from batch
-  }
-}
-</code></pre>
+<div class="language-cpp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">while</span> <span class="p">(</span><span class="n">rowReader</span><span class="o">-&gt;</span><span class="n">next</span><span class="p">(</span><span class="o">*</span><span class="n">batch</span><span class="p">))</span> <span class="p">{</span>
+  <span class="k">for</span> <span class="p">(</span><span class="kt">uint64_t</span> <span class="n">r</span> <span class="o">=</span> <span class="mi">0</span><span class="p">;</span> <span class="n">r</span> <span class="o">&lt;</span> <span class="n">batch</span><span class="o">-&gt;</span><span class="n">numElements</span><span class="p">;</span> <span class="o">++</span><span class="n">r</span><span class="p">)</span> <span class="p">{</span>
+    <span class="p">...</span> <span class="n">process</span> <span class="n">row</span> <span class="n">r</span> <span class="n">from</span> <span class="n">batch</span>
+  <span class="p">}</span>
+<span class="p">}</span>
+</code></pre></div></div>
 
           
 
@@ -1059,8 +1059,6 @@ so the returned batch may not be full, but it will always contain some rows.</p>
   
 
   
-    
-  
 
   
     
@@ -1080,29 +1078,25 @@ so the returned batch may not be full, but it will always contain some rows.</p>
     
   
     
-      <li class=""><a href="/docs/index.html">Background</a></li>
+      <li class=""><a href="http://localhost:4000/docs/index.html">Background</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
   
     
-      <li class=""><a href="/docs/adopters.html">ORC Adopters</a></li>
+      <li class=""><a href="http://localhost:4000/docs/adopters.html">ORC Adopters</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -1134,15 +1128,13 @@ so the returned batch may not be full, but it will always contain some rows.</p>
     
   
     
-      <li class=""><a href="/docs/types.html">Types</a></li>
+      <li class=""><a href="http://localhost:4000/docs/types.html">Types</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -1164,19 +1156,17 @@ so the returned batch may not be full, but it will always contain some rows.</p>
     
   
     
-      <li class=""><a href="/docs/indexes.html">Indexes</a></li>
+      <li class=""><a href="http://localhost:4000/docs/indexes.html">Indexes</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
-      <li class=""><a href="/docs/acid.html">ACID support</a></li>
+      <li class=""><a href="http://localhost:4000/docs/acid.html">ACID support</a></li>
       
 
 
@@ -1191,8 +1181,6 @@ so the returned batch may not be full, but it will always contain some rows.</p>
   
 
   
-    
-  
 
   
     
@@ -1200,15 +1188,13 @@ so the returned batch may not be full, but it will always contain some rows.</p>
     
   
     
-      <li class=""><a href="/docs/building.html">Building ORC</a></li>
+      <li class=""><a href="http://localhost:4000/docs/building.html">Building ORC</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -1238,7 +1224,7 @@ so the returned batch may not be full, but it will always contain some rows.</p>
     
   
     
-      <li class=""><a href="/docs/releases.html">Releases</a></li>
+      <li class=""><a href="http://localhost:4000/docs/releases.html">Releases</a></li>
       
 
 
@@ -1253,8 +1239,6 @@ so the returned batch may not be full, but it will always contain some rows.</p>
   
 
   
-    
-  
 
   
     
@@ -1272,15 +1256,13 @@ so the returned batch may not be full, but it will always contain some rows.</p>
     
   
     
-      <li class=""><a href="/docs/hive-ddl.html">Hive DDL</a></li>
+      <li class=""><a href="http://localhost:4000/docs/hive-ddl.html">Hive DDL</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -1296,7 +1278,7 @@ so the returned batch may not be full, but it will always contain some rows.</p>
     
   
     
-      <li class=""><a href="/docs/hive-config.html">Hive Configuration</a></li>
+      <li class=""><a href="http://localhost:4000/docs/hive-config.html">Hive Configuration</a></li>
       
 
 
@@ -1311,8 +1293,6 @@ so the returned batch may not be full, but it will always contain some rows.</p>
   
 
   
-    
-  
 
   
     
@@ -1338,15 +1318,13 @@ so the returned batch may not be full, but it will always contain some rows.</p>
     
   
     
-      <li class=""><a href="/docs/mapred.html">Using in MapRed</a></li>
+      <li class=""><a href="http://localhost:4000/docs/mapred.html">Using in MapRed</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -1374,7 +1352,7 @@ so the returned batch may not be full, but it will always contain some rows.</p>
     
   
     
-      <li class=""><a href="/docs/mapreduce.html">Using in MapReduce</a></li>
+      <li class=""><a href="http://localhost:4000/docs/mapreduce.html">Using in MapReduce</a></li>
       
 
 
@@ -1389,8 +1367,6 @@ so the returned batch may not be full, but it will always contain some rows.</p>
   
 
   
-    
-  
 
   
     
@@ -1402,15 +1378,13 @@ so the returned batch may not be full, but it will always contain some rows.</p>
     
   
     
-      <li class=""><a href="/docs/core-java.html">Using Core Java</a></li>
+      <li class=""><a href="http://localhost:4000/docs/core-java.html">Using Core Java</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -1420,7 +1394,7 @@ so the returned batch may not be full, but it will always contain some rows.</p>
     
   
     
-      <li class="current"><a href="/docs/core-cpp.html">Using Core C++</a></li>
+      <li class="current"><a href="http://localhost:4000/docs/core-cpp.html">Using Core C++</a></li>
       
 
 
@@ -1435,8 +1409,6 @@ so the returned batch may not be full, but it will always contain some rows.</p>
   
 
   
-    
-  
 
   
     
@@ -1450,15 +1422,13 @@ so the returned batch may not be full, but it will always contain some rows.</p>
     
   
     
-      <li class=""><a href="/docs/cpp-tools.html">C++ Tools</a></li>
+      <li class=""><a href="http://localhost:4000/docs/cpp-tools.html">C++ Tools</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -1482,7 +1452,7 @@ so the returned batch may not be full, but it will always contain some rows.</p>
     
   
     
-      <li class=""><a href="/docs/java-tools.html">Java Tools</a></li>
+      <li class=""><a href="http://localhost:4000/docs/java-tools.html">Java Tools</a></li>
       
 
 


[08/11] orc git commit: Deploy docs with CVE.

Posted by om...@apache.org.
http://git-wip-us.apache.org/repos/asf/orc/blob/e58d3fc4/docs/hive-config.html
----------------------------------------------------------------------
diff --git a/docs/hive-config.html b/docs/hive-config.html
index 159cdf7..5186b7e 100644
--- a/docs/hive-config.html
+++ b/docs/hive-config.html
@@ -4,7 +4,7 @@
   <meta charset="UTF-8">
   <title>Hive Configuration</title>
   <meta name="viewport" content="width=device-width,initial-scale=1">
-  <meta name="generator" content="Jekyll v2.4.0">
+  <meta name="generator" content="Jekyll v3.7.3">
   <link rel="stylesheet" href="//fonts.googleapis.com/css?family=Lato:300,300italic,400,400italic,700,700italic,900">
   <link rel="stylesheet" href="/css/screen.css">
   <link rel="icon" type="image/x-icon" href="/favicon.ico">
@@ -109,7 +109,7 @@
     
   
     
-      <option value="/docs/index.html">Background</option>
+      <option value="http://localhost:4000/docs/index.html">Background</option>
     
   
     
@@ -131,7 +131,7 @@
     
   
     
-      <option value="/docs/adopters.html">ORC Adopters</option>
+      <option value="http://localhost:4000/docs/adopters.html">ORC Adopters</option>
     
   
     
@@ -193,7 +193,7 @@
     
   
     
-      <option value="/docs/types.html">Types</option>
+      <option value="http://localhost:4000/docs/types.html">Types</option>
     
   
 
@@ -219,7 +219,7 @@
     
   
     
-      <option value="/docs/indexes.html">Indexes</option>
+      <option value="http://localhost:4000/docs/indexes.html">Indexes</option>
     
   
     
@@ -237,7 +237,7 @@
 
   
     
-      <option value="/docs/acid.html">ACID support</option>
+      <option value="http://localhost:4000/docs/acid.html">ACID support</option>
     
   
     
@@ -284,7 +284,7 @@
     
   
     
-      <option value="/docs/building.html">Building ORC</option>
+      <option value="http://localhost:4000/docs/building.html">Building ORC</option>
     
   
     
@@ -342,7 +342,7 @@
     
   
     
-      <option value="/docs/releases.html">Releases</option>
+      <option value="http://localhost:4000/docs/releases.html">Releases</option>
     
   
     
@@ -373,7 +373,7 @@
     
   
     
-      <option value="/docs/hive-ddl.html">Hive DDL</option>
+      <option value="http://localhost:4000/docs/hive-ddl.html">Hive DDL</option>
     
   
     
@@ -407,7 +407,7 @@
     
   
     
-      <option value="/docs/hive-config.html">Hive Configuration</option>
+      <option value="http://localhost:4000/docs/hive-config.html">Hive Configuration</option>
     
   
     
@@ -460,7 +460,7 @@
     
   
     
-      <option value="/docs/mapred.html">Using in MapRed</option>
+      <option value="http://localhost:4000/docs/mapred.html">Using in MapRed</option>
     
   
     
@@ -498,7 +498,7 @@
     
   
     
-      <option value="/docs/mapreduce.html">Using in MapReduce</option>
+      <option value="http://localhost:4000/docs/mapreduce.html">Using in MapReduce</option>
     
   
     
@@ -525,7 +525,7 @@
     
   
     
-      <option value="/docs/core-java.html">Using Core Java</option>
+      <option value="http://localhost:4000/docs/core-java.html">Using Core Java</option>
     
   
     
@@ -559,7 +559,7 @@
     
   
     
-      <option value="/docs/core-cpp.html">Using Core C++</option>
+      <option value="http://localhost:4000/docs/core-cpp.html">Using Core C++</option>
     
   
     
@@ -606,7 +606,7 @@
     
   
     
-      <option value="/docs/cpp-tools.html">C++ Tools</option>
+      <option value="http://localhost:4000/docs/cpp-tools.html">C++ Tools</option>
     
   
     
@@ -652,7 +652,7 @@
     
   
     
-      <option value="/docs/java-tools.html">Java Tools</option>
+      <option value="http://localhost:4000/docs/java-tools.html">Java Tools</option>
     
   
     
@@ -729,7 +729,7 @@ with the same options.</p>
 
 <p>For example, to create an ORC table without high level compression:</p>
 
-<p><code>CREATE TABLE istari (
+<p><code class="highlighter-rouge">CREATE TABLE istari (
   name STRING,
   color STRING
 ) STORED AS ORC TBLPROPERTIES ("orc.compress"="NONE");
@@ -815,8 +815,8 @@ with the same options.</p>
       available size within the block is more than 3.2Mb, a new
       smaller stripe will be inserted to fit within that space. This
       will make sure that no stripe written will cross block
-      boundaries and cause remote reads within a node local task.
-
+      boundaries and cause remote reads within a node local task.&lt;/t&gt;
+&lt;/tr&gt;
 <tr>
   <td>hive.exec.orc.default.compress</td>
   <td>ZLIB</td>
@@ -893,7 +893,7 @@ with the same options.</p>
       available? If the new stripe would straddle a block, Hadoop is &ge; 2.7,
       and this is enabled, it will end the block before the new stripe.</td>
 </tr>
-
+&lt;/table&gt;
 </td></tr></table>
 
           
@@ -961,8 +961,6 @@ with the same options.</p>
   
 
   
-    
-  
 
   
     
@@ -982,29 +980,25 @@ with the same options.</p>
     
   
     
-      <li class=""><a href="/docs/index.html">Background</a></li>
+      <li class=""><a href="http://localhost:4000/docs/index.html">Background</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
   
     
-      <li class=""><a href="/docs/adopters.html">ORC Adopters</a></li>
+      <li class=""><a href="http://localhost:4000/docs/adopters.html">ORC Adopters</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -1036,15 +1030,13 @@ with the same options.</p>
     
   
     
-      <li class=""><a href="/docs/types.html">Types</a></li>
+      <li class=""><a href="http://localhost:4000/docs/types.html">Types</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -1066,19 +1058,17 @@ with the same options.</p>
     
   
     
-      <li class=""><a href="/docs/indexes.html">Indexes</a></li>
+      <li class=""><a href="http://localhost:4000/docs/indexes.html">Indexes</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
-      <li class=""><a href="/docs/acid.html">ACID support</a></li>
+      <li class=""><a href="http://localhost:4000/docs/acid.html">ACID support</a></li>
       
 
 
@@ -1093,8 +1083,6 @@ with the same options.</p>
   
 
   
-    
-  
 
   
     
@@ -1102,15 +1090,13 @@ with the same options.</p>
     
   
     
-      <li class=""><a href="/docs/building.html">Building ORC</a></li>
+      <li class=""><a href="http://localhost:4000/docs/building.html">Building ORC</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -1140,7 +1126,7 @@ with the same options.</p>
     
   
     
-      <li class=""><a href="/docs/releases.html">Releases</a></li>
+      <li class=""><a href="http://localhost:4000/docs/releases.html">Releases</a></li>
       
 
 
@@ -1155,8 +1141,6 @@ with the same options.</p>
   
 
   
-    
-  
 
   
     
@@ -1174,15 +1158,13 @@ with the same options.</p>
     
   
     
-      <li class=""><a href="/docs/hive-ddl.html">Hive DDL</a></li>
+      <li class=""><a href="http://localhost:4000/docs/hive-ddl.html">Hive DDL</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -1198,7 +1180,7 @@ with the same options.</p>
     
   
     
-      <li class="current"><a href="/docs/hive-config.html">Hive Configuration</a></li>
+      <li class="current"><a href="http://localhost:4000/docs/hive-config.html">Hive Configuration</a></li>
       
 
 
@@ -1213,8 +1195,6 @@ with the same options.</p>
   
 
   
-    
-  
 
   
     
@@ -1240,15 +1220,13 @@ with the same options.</p>
     
   
     
-      <li class=""><a href="/docs/mapred.html">Using in MapRed</a></li>
+      <li class=""><a href="http://localhost:4000/docs/mapred.html">Using in MapRed</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -1276,7 +1254,7 @@ with the same options.</p>
     
   
     
-      <li class=""><a href="/docs/mapreduce.html">Using in MapReduce</a></li>
+      <li class=""><a href="http://localhost:4000/docs/mapreduce.html">Using in MapReduce</a></li>
       
 
 
@@ -1291,8 +1269,6 @@ with the same options.</p>
   
 
   
-    
-  
 
   
     
@@ -1304,15 +1280,13 @@ with the same options.</p>
     
   
     
-      <li class=""><a href="/docs/core-java.html">Using Core Java</a></li>
+      <li class=""><a href="http://localhost:4000/docs/core-java.html">Using Core Java</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -1322,7 +1296,7 @@ with the same options.</p>
     
   
     
-      <li class=""><a href="/docs/core-cpp.html">Using Core C++</a></li>
+      <li class=""><a href="http://localhost:4000/docs/core-cpp.html">Using Core C++</a></li>
       
 
 
@@ -1337,8 +1311,6 @@ with the same options.</p>
   
 
   
-    
-  
 
   
     
@@ -1352,15 +1324,13 @@ with the same options.</p>
     
   
     
-      <li class=""><a href="/docs/cpp-tools.html">C++ Tools</a></li>
+      <li class=""><a href="http://localhost:4000/docs/cpp-tools.html">C++ Tools</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -1384,7 +1354,7 @@ with the same options.</p>
     
   
     
-      <li class=""><a href="/docs/java-tools.html">Java Tools</a></li>
+      <li class=""><a href="http://localhost:4000/docs/java-tools.html">Java Tools</a></li>
       
 
 

http://git-wip-us.apache.org/repos/asf/orc/blob/e58d3fc4/docs/hive-ddl.html
----------------------------------------------------------------------
diff --git a/docs/hive-ddl.html b/docs/hive-ddl.html
index 8c360d3..d2e4f4f 100644
--- a/docs/hive-ddl.html
+++ b/docs/hive-ddl.html
@@ -4,7 +4,7 @@
   <meta charset="UTF-8">
   <title>Hive DDL</title>
   <meta name="viewport" content="width=device-width,initial-scale=1">
-  <meta name="generator" content="Jekyll v2.4.0">
+  <meta name="generator" content="Jekyll v3.7.3">
   <link rel="stylesheet" href="//fonts.googleapis.com/css?family=Lato:300,300italic,400,400italic,700,700italic,900">
   <link rel="stylesheet" href="/css/screen.css">
   <link rel="icon" type="image/x-icon" href="/favicon.ico">
@@ -109,7 +109,7 @@
     
   
     
-      <option value="/docs/index.html">Background</option>
+      <option value="http://localhost:4000/docs/index.html">Background</option>
     
   
     
@@ -131,7 +131,7 @@
     
   
     
-      <option value="/docs/adopters.html">ORC Adopters</option>
+      <option value="http://localhost:4000/docs/adopters.html">ORC Adopters</option>
     
   
     
@@ -193,7 +193,7 @@
     
   
     
-      <option value="/docs/types.html">Types</option>
+      <option value="http://localhost:4000/docs/types.html">Types</option>
     
   
 
@@ -219,7 +219,7 @@
     
   
     
-      <option value="/docs/indexes.html">Indexes</option>
+      <option value="http://localhost:4000/docs/indexes.html">Indexes</option>
     
   
     
@@ -237,7 +237,7 @@
 
   
     
-      <option value="/docs/acid.html">ACID support</option>
+      <option value="http://localhost:4000/docs/acid.html">ACID support</option>
     
   
     
@@ -284,7 +284,7 @@
     
   
     
-      <option value="/docs/building.html">Building ORC</option>
+      <option value="http://localhost:4000/docs/building.html">Building ORC</option>
     
   
     
@@ -342,7 +342,7 @@
     
   
     
-      <option value="/docs/releases.html">Releases</option>
+      <option value="http://localhost:4000/docs/releases.html">Releases</option>
     
   
     
@@ -373,7 +373,7 @@
     
   
     
-      <option value="/docs/hive-ddl.html">Hive DDL</option>
+      <option value="http://localhost:4000/docs/hive-ddl.html">Hive DDL</option>
     
   
     
@@ -407,7 +407,7 @@
     
   
     
-      <option value="/docs/hive-config.html">Hive Configuration</option>
+      <option value="http://localhost:4000/docs/hive-config.html">Hive Configuration</option>
     
   
     
@@ -460,7 +460,7 @@
     
   
     
-      <option value="/docs/mapred.html">Using in MapRed</option>
+      <option value="http://localhost:4000/docs/mapred.html">Using in MapRed</option>
     
   
     
@@ -498,7 +498,7 @@
     
   
     
-      <option value="/docs/mapreduce.html">Using in MapReduce</option>
+      <option value="http://localhost:4000/docs/mapreduce.html">Using in MapReduce</option>
     
   
     
@@ -525,7 +525,7 @@
     
   
     
-      <option value="/docs/core-java.html">Using Core Java</option>
+      <option value="http://localhost:4000/docs/core-java.html">Using Core Java</option>
     
   
     
@@ -559,7 +559,7 @@
     
   
     
-      <option value="/docs/core-cpp.html">Using Core C++</option>
+      <option value="http://localhost:4000/docs/core-cpp.html">Using Core C++</option>
     
   
     
@@ -606,7 +606,7 @@
     
   
     
-      <option value="/docs/cpp-tools.html">C++ Tools</option>
+      <option value="http://localhost:4000/docs/cpp-tools.html">C++ Tools</option>
     
   
     
@@ -652,7 +652,7 @@
     
   
     
-      <option value="/docs/java-tools.html">Java Tools</option>
+      <option value="http://localhost:4000/docs/java-tools.html">Java Tools</option>
     
   
     
@@ -677,33 +677,31 @@
           <p>ORC is well integrated into Hive, so storing your istari table as ORC
 is done by adding “STORED AS ORC”.</p>
 
-<p><code>CREATE TABLE istari (
+<p>```CREATE TABLE istari (
   name STRING,
   color STRING
-) STORED AS ORC;
-</code></p>
-
-<p>To modify a table so that new partitions of the istari table are
-stored as ORC files:</p>
+) STORED AS ORC;</p>
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
+To modify a table so that new partitions of the istari table are
+stored as ORC files:
 
-<p><code>ALTER TABLE istari SET FILEFORMAT ORC;
-</code></p>
+```ALTER TABLE istari SET FILEFORMAT ORC;
+</code></pre></div></div>
 
 <p>As of Hive 0.14, users can request an efficient merge of small ORC files
 together by issuing a CONCATENATE command on their table or partition. The
 files will be merged at the stripe level without reserialization.</p>
 
-<p><code>ALTER TABLE istari [PARTITION partition_spec] CONCATENATE;
-</code></p>
+<p>```ALTER TABLE istari [PARTITION partition_spec] CONCATENATE;</p>
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
+To get information about an ORC file, use the orcfiledump command.
 
-<p>To get information about an ORC file, use the orcfiledump command.</p>
-
-<p><code>% hive --orcfiledump &lt;path_to_file&gt;
-</code></p>
+```% hive --orcfiledump &lt;path_to_file&gt;
+</code></pre></div></div>
 
 <p>As of Hive 1.1, to display the data in the ORC file, use:</p>
 
-<p><code>% hive --orcfiledump -d &lt;path_to_file&gt;
+<p><code class="highlighter-rouge">% hive --orcfiledump -d &lt;path_to_file&gt;
 </code></p>
 
           
@@ -768,8 +766,6 @@ files will be merged at the stripe level without reserialization.</p>
   
 
   
-    
-  
 
   
     
@@ -789,29 +785,25 @@ files will be merged at the stripe level without reserialization.</p>
     
   
     
-      <li class=""><a href="/docs/index.html">Background</a></li>
+      <li class=""><a href="http://localhost:4000/docs/index.html">Background</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
   
     
-      <li class=""><a href="/docs/adopters.html">ORC Adopters</a></li>
+      <li class=""><a href="http://localhost:4000/docs/adopters.html">ORC Adopters</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -843,15 +835,13 @@ files will be merged at the stripe level without reserialization.</p>
     
   
     
-      <li class=""><a href="/docs/types.html">Types</a></li>
+      <li class=""><a href="http://localhost:4000/docs/types.html">Types</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -873,19 +863,17 @@ files will be merged at the stripe level without reserialization.</p>
     
   
     
-      <li class=""><a href="/docs/indexes.html">Indexes</a></li>
+      <li class=""><a href="http://localhost:4000/docs/indexes.html">Indexes</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
-      <li class=""><a href="/docs/acid.html">ACID support</a></li>
+      <li class=""><a href="http://localhost:4000/docs/acid.html">ACID support</a></li>
       
 
 
@@ -900,8 +888,6 @@ files will be merged at the stripe level without reserialization.</p>
   
 
   
-    
-  
 
   
     
@@ -909,15 +895,13 @@ files will be merged at the stripe level without reserialization.</p>
     
   
     
-      <li class=""><a href="/docs/building.html">Building ORC</a></li>
+      <li class=""><a href="http://localhost:4000/docs/building.html">Building ORC</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -947,7 +931,7 @@ files will be merged at the stripe level without reserialization.</p>
     
   
     
-      <li class=""><a href="/docs/releases.html">Releases</a></li>
+      <li class=""><a href="http://localhost:4000/docs/releases.html">Releases</a></li>
       
 
 
@@ -962,8 +946,6 @@ files will be merged at the stripe level without reserialization.</p>
   
 
   
-    
-  
 
   
     
@@ -981,15 +963,13 @@ files will be merged at the stripe level without reserialization.</p>
     
   
     
-      <li class="current"><a href="/docs/hive-ddl.html">Hive DDL</a></li>
+      <li class="current"><a href="http://localhost:4000/docs/hive-ddl.html">Hive DDL</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -1005,7 +985,7 @@ files will be merged at the stripe level without reserialization.</p>
     
   
     
-      <li class=""><a href="/docs/hive-config.html">Hive Configuration</a></li>
+      <li class=""><a href="http://localhost:4000/docs/hive-config.html">Hive Configuration</a></li>
       
 
 
@@ -1020,8 +1000,6 @@ files will be merged at the stripe level without reserialization.</p>
   
 
   
-    
-  
 
   
     
@@ -1047,15 +1025,13 @@ files will be merged at the stripe level without reserialization.</p>
     
   
     
-      <li class=""><a href="/docs/mapred.html">Using in MapRed</a></li>
+      <li class=""><a href="http://localhost:4000/docs/mapred.html">Using in MapRed</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -1083,7 +1059,7 @@ files will be merged at the stripe level without reserialization.</p>
     
   
     
-      <li class=""><a href="/docs/mapreduce.html">Using in MapReduce</a></li>
+      <li class=""><a href="http://localhost:4000/docs/mapreduce.html">Using in MapReduce</a></li>
       
 
 
@@ -1098,8 +1074,6 @@ files will be merged at the stripe level without reserialization.</p>
   
 
   
-    
-  
 
   
     
@@ -1111,15 +1085,13 @@ files will be merged at the stripe level without reserialization.</p>
     
   
     
-      <li class=""><a href="/docs/core-java.html">Using Core Java</a></li>
+      <li class=""><a href="http://localhost:4000/docs/core-java.html">Using Core Java</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -1129,7 +1101,7 @@ files will be merged at the stripe level without reserialization.</p>
     
   
     
-      <li class=""><a href="/docs/core-cpp.html">Using Core C++</a></li>
+      <li class=""><a href="http://localhost:4000/docs/core-cpp.html">Using Core C++</a></li>
       
 
 
@@ -1144,8 +1116,6 @@ files will be merged at the stripe level without reserialization.</p>
   
 
   
-    
-  
 
   
     
@@ -1159,15 +1129,13 @@ files will be merged at the stripe level without reserialization.</p>
     
   
     
-      <li class=""><a href="/docs/cpp-tools.html">C++ Tools</a></li>
+      <li class=""><a href="http://localhost:4000/docs/cpp-tools.html">C++ Tools</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -1191,7 +1159,7 @@ files will be merged at the stripe level without reserialization.</p>
     
   
     
-      <li class=""><a href="/docs/java-tools.html">Java Tools</a></li>
+      <li class=""><a href="http://localhost:4000/docs/java-tools.html">Java Tools</a></li>
       
 
 

http://git-wip-us.apache.org/repos/asf/orc/blob/e58d3fc4/docs/index.html
----------------------------------------------------------------------
diff --git a/docs/index.html b/docs/index.html
index bb12c87..73a37c3 100644
--- a/docs/index.html
+++ b/docs/index.html
@@ -4,7 +4,7 @@
   <meta charset="UTF-8">
   <title>Background</title>
   <meta name="viewport" content="width=device-width,initial-scale=1">
-  <meta name="generator" content="Jekyll v2.4.0">
+  <meta name="generator" content="Jekyll v3.7.3">
   <link rel="stylesheet" href="//fonts.googleapis.com/css?family=Lato:300,300italic,400,400italic,700,700italic,900">
   <link rel="stylesheet" href="/css/screen.css">
   <link rel="icon" type="image/x-icon" href="/favicon.ico">
@@ -109,7 +109,7 @@
     
   
     
-      <option value="/docs/index.html">Background</option>
+      <option value="http://localhost:4000/docs/index.html">Background</option>
     
   
     
@@ -131,7 +131,7 @@
     
   
     
-      <option value="/docs/adopters.html">ORC Adopters</option>
+      <option value="http://localhost:4000/docs/adopters.html">ORC Adopters</option>
     
   
     
@@ -193,7 +193,7 @@
     
   
     
-      <option value="/docs/types.html">Types</option>
+      <option value="http://localhost:4000/docs/types.html">Types</option>
     
   
 
@@ -219,7 +219,7 @@
     
   
     
-      <option value="/docs/indexes.html">Indexes</option>
+      <option value="http://localhost:4000/docs/indexes.html">Indexes</option>
     
   
     
@@ -237,7 +237,7 @@
 
   
     
-      <option value="/docs/acid.html">ACID support</option>
+      <option value="http://localhost:4000/docs/acid.html">ACID support</option>
     
   
     
@@ -284,7 +284,7 @@
     
   
     
-      <option value="/docs/building.html">Building ORC</option>
+      <option value="http://localhost:4000/docs/building.html">Building ORC</option>
     
   
     
@@ -342,7 +342,7 @@
     
   
     
-      <option value="/docs/releases.html">Releases</option>
+      <option value="http://localhost:4000/docs/releases.html">Releases</option>
     
   
     
@@ -373,7 +373,7 @@
     
   
     
-      <option value="/docs/hive-ddl.html">Hive DDL</option>
+      <option value="http://localhost:4000/docs/hive-ddl.html">Hive DDL</option>
     
   
     
@@ -407,7 +407,7 @@
     
   
     
-      <option value="/docs/hive-config.html">Hive Configuration</option>
+      <option value="http://localhost:4000/docs/hive-config.html">Hive Configuration</option>
     
   
     
@@ -460,7 +460,7 @@
     
   
     
-      <option value="/docs/mapred.html">Using in MapRed</option>
+      <option value="http://localhost:4000/docs/mapred.html">Using in MapRed</option>
     
   
     
@@ -498,7 +498,7 @@
     
   
     
-      <option value="/docs/mapreduce.html">Using in MapReduce</option>
+      <option value="http://localhost:4000/docs/mapreduce.html">Using in MapReduce</option>
     
   
     
@@ -525,7 +525,7 @@
     
   
     
-      <option value="/docs/core-java.html">Using Core Java</option>
+      <option value="http://localhost:4000/docs/core-java.html">Using Core Java</option>
     
   
     
@@ -559,7 +559,7 @@
     
   
     
-      <option value="/docs/core-cpp.html">Using Core C++</option>
+      <option value="http://localhost:4000/docs/core-cpp.html">Using Core C++</option>
     
   
     
@@ -606,7 +606,7 @@
     
   
     
-      <option value="/docs/cpp-tools.html">C++ Tools</option>
+      <option value="http://localhost:4000/docs/cpp-tools.html">C++ Tools</option>
     
   
     
@@ -652,7 +652,7 @@
     
   
     
-      <option value="/docs/java-tools.html">Java Tools</option>
+      <option value="http://localhost:4000/docs/java-tools.html">Java Tools</option>
     
   
     
@@ -748,8 +748,6 @@ format specification</a>.</p>
   
 
   
-    
-  
 
   
     
@@ -769,29 +767,25 @@ format specification</a>.</p>
     
   
     
-      <li class="current"><a href="/docs/index.html">Background</a></li>
+      <li class="current"><a href="http://localhost:4000/docs/index.html">Background</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
   
     
-      <li class=""><a href="/docs/adopters.html">ORC Adopters</a></li>
+      <li class=""><a href="http://localhost:4000/docs/adopters.html">ORC Adopters</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -823,15 +817,13 @@ format specification</a>.</p>
     
   
     
-      <li class=""><a href="/docs/types.html">Types</a></li>
+      <li class=""><a href="http://localhost:4000/docs/types.html">Types</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -853,19 +845,17 @@ format specification</a>.</p>
     
   
     
-      <li class=""><a href="/docs/indexes.html">Indexes</a></li>
+      <li class=""><a href="http://localhost:4000/docs/indexes.html">Indexes</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
-      <li class=""><a href="/docs/acid.html">ACID support</a></li>
+      <li class=""><a href="http://localhost:4000/docs/acid.html">ACID support</a></li>
       
 
 
@@ -880,8 +870,6 @@ format specification</a>.</p>
   
 
   
-    
-  
 
   
     
@@ -889,15 +877,13 @@ format specification</a>.</p>
     
   
     
-      <li class=""><a href="/docs/building.html">Building ORC</a></li>
+      <li class=""><a href="http://localhost:4000/docs/building.html">Building ORC</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -927,7 +913,7 @@ format specification</a>.</p>
     
   
     
-      <li class=""><a href="/docs/releases.html">Releases</a></li>
+      <li class=""><a href="http://localhost:4000/docs/releases.html">Releases</a></li>
       
 
 
@@ -942,8 +928,6 @@ format specification</a>.</p>
   
 
   
-    
-  
 
   
     
@@ -961,15 +945,13 @@ format specification</a>.</p>
     
   
     
-      <li class=""><a href="/docs/hive-ddl.html">Hive DDL</a></li>
+      <li class=""><a href="http://localhost:4000/docs/hive-ddl.html">Hive DDL</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -985,7 +967,7 @@ format specification</a>.</p>
     
   
     
-      <li class=""><a href="/docs/hive-config.html">Hive Configuration</a></li>
+      <li class=""><a href="http://localhost:4000/docs/hive-config.html">Hive Configuration</a></li>
       
 
 
@@ -1000,8 +982,6 @@ format specification</a>.</p>
   
 
   
-    
-  
 
   
     
@@ -1027,15 +1007,13 @@ format specification</a>.</p>
     
   
     
-      <li class=""><a href="/docs/mapred.html">Using in MapRed</a></li>
+      <li class=""><a href="http://localhost:4000/docs/mapred.html">Using in MapRed</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -1063,7 +1041,7 @@ format specification</a>.</p>
     
   
     
-      <li class=""><a href="/docs/mapreduce.html">Using in MapReduce</a></li>
+      <li class=""><a href="http://localhost:4000/docs/mapreduce.html">Using in MapReduce</a></li>
       
 
 
@@ -1078,8 +1056,6 @@ format specification</a>.</p>
   
 
   
-    
-  
 
   
     
@@ -1091,15 +1067,13 @@ format specification</a>.</p>
     
   
     
-      <li class=""><a href="/docs/core-java.html">Using Core Java</a></li>
+      <li class=""><a href="http://localhost:4000/docs/core-java.html">Using Core Java</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -1109,7 +1083,7 @@ format specification</a>.</p>
     
   
     
-      <li class=""><a href="/docs/core-cpp.html">Using Core C++</a></li>
+      <li class=""><a href="http://localhost:4000/docs/core-cpp.html">Using Core C++</a></li>
       
 
 
@@ -1124,8 +1098,6 @@ format specification</a>.</p>
   
 
   
-    
-  
 
   
     
@@ -1139,15 +1111,13 @@ format specification</a>.</p>
     
   
     
-      <li class=""><a href="/docs/cpp-tools.html">C++ Tools</a></li>
+      <li class=""><a href="http://localhost:4000/docs/cpp-tools.html">C++ Tools</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -1171,7 +1141,7 @@ format specification</a>.</p>
     
   
     
-      <li class=""><a href="/docs/java-tools.html">Java Tools</a></li>
+      <li class=""><a href="http://localhost:4000/docs/java-tools.html">Java Tools</a></li>
       
 
 

http://git-wip-us.apache.org/repos/asf/orc/blob/e58d3fc4/docs/indexes.html
----------------------------------------------------------------------
diff --git a/docs/indexes.html b/docs/indexes.html
index 0a81f43..9cf8abc 100644
--- a/docs/indexes.html
+++ b/docs/indexes.html
@@ -4,7 +4,7 @@
   <meta charset="UTF-8">
   <title>Indexes</title>
   <meta name="viewport" content="width=device-width,initial-scale=1">
-  <meta name="generator" content="Jekyll v2.4.0">
+  <meta name="generator" content="Jekyll v3.7.3">
   <link rel="stylesheet" href="//fonts.googleapis.com/css?family=Lato:300,300italic,400,400italic,700,700italic,900">
   <link rel="stylesheet" href="/css/screen.css">
   <link rel="icon" type="image/x-icon" href="/favicon.ico">
@@ -109,7 +109,7 @@
     
   
     
-      <option value="/docs/index.html">Background</option>
+      <option value="http://localhost:4000/docs/index.html">Background</option>
     
   
     
@@ -131,7 +131,7 @@
     
   
     
-      <option value="/docs/adopters.html">ORC Adopters</option>
+      <option value="http://localhost:4000/docs/adopters.html">ORC Adopters</option>
     
   
     
@@ -193,7 +193,7 @@
     
   
     
-      <option value="/docs/types.html">Types</option>
+      <option value="http://localhost:4000/docs/types.html">Types</option>
     
   
 
@@ -219,7 +219,7 @@
     
   
     
-      <option value="/docs/indexes.html">Indexes</option>
+      <option value="http://localhost:4000/docs/indexes.html">Indexes</option>
     
   
     
@@ -237,7 +237,7 @@
 
   
     
-      <option value="/docs/acid.html">ACID support</option>
+      <option value="http://localhost:4000/docs/acid.html">ACID support</option>
     
   
     
@@ -284,7 +284,7 @@
     
   
     
-      <option value="/docs/building.html">Building ORC</option>
+      <option value="http://localhost:4000/docs/building.html">Building ORC</option>
     
   
     
@@ -342,7 +342,7 @@
     
   
     
-      <option value="/docs/releases.html">Releases</option>
+      <option value="http://localhost:4000/docs/releases.html">Releases</option>
     
   
     
@@ -373,7 +373,7 @@
     
   
     
-      <option value="/docs/hive-ddl.html">Hive DDL</option>
+      <option value="http://localhost:4000/docs/hive-ddl.html">Hive DDL</option>
     
   
     
@@ -407,7 +407,7 @@
     
   
     
-      <option value="/docs/hive-config.html">Hive Configuration</option>
+      <option value="http://localhost:4000/docs/hive-config.html">Hive Configuration</option>
     
   
     
@@ -460,7 +460,7 @@
     
   
     
-      <option value="/docs/mapred.html">Using in MapRed</option>
+      <option value="http://localhost:4000/docs/mapred.html">Using in MapRed</option>
     
   
     
@@ -498,7 +498,7 @@
     
   
     
-      <option value="/docs/mapreduce.html">Using in MapReduce</option>
+      <option value="http://localhost:4000/docs/mapreduce.html">Using in MapReduce</option>
     
   
     
@@ -525,7 +525,7 @@
     
   
     
-      <option value="/docs/core-java.html">Using Core Java</option>
+      <option value="http://localhost:4000/docs/core-java.html">Using Core Java</option>
     
   
     
@@ -559,7 +559,7 @@
     
   
     
-      <option value="/docs/core-cpp.html">Using Core C++</option>
+      <option value="http://localhost:4000/docs/core-cpp.html">Using Core C++</option>
     
   
     
@@ -606,7 +606,7 @@
     
   
     
-      <option value="/docs/cpp-tools.html">C++ Tools</option>
+      <option value="http://localhost:4000/docs/cpp-tools.html">C++ Tools</option>
     
   
     
@@ -652,7 +652,7 @@
     
   
     
-      <option value="/docs/java-tools.html">Java Tools</option>
+      <option value="http://localhost:4000/docs/java-tools.html">Java Tools</option>
     
   
     
@@ -753,8 +753,6 @@ would be read.</p>
   
 
   
-    
-  
 
   
     
@@ -774,29 +772,25 @@ would be read.</p>
     
   
     
-      <li class=""><a href="/docs/index.html">Background</a></li>
+      <li class=""><a href="http://localhost:4000/docs/index.html">Background</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
   
     
-      <li class=""><a href="/docs/adopters.html">ORC Adopters</a></li>
+      <li class=""><a href="http://localhost:4000/docs/adopters.html">ORC Adopters</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -828,15 +822,13 @@ would be read.</p>
     
   
     
-      <li class=""><a href="/docs/types.html">Types</a></li>
+      <li class=""><a href="http://localhost:4000/docs/types.html">Types</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -858,19 +850,17 @@ would be read.</p>
     
   
     
-      <li class="current"><a href="/docs/indexes.html">Indexes</a></li>
+      <li class="current"><a href="http://localhost:4000/docs/indexes.html">Indexes</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
-      <li class=""><a href="/docs/acid.html">ACID support</a></li>
+      <li class=""><a href="http://localhost:4000/docs/acid.html">ACID support</a></li>
       
 
 
@@ -885,8 +875,6 @@ would be read.</p>
   
 
   
-    
-  
 
   
     
@@ -894,15 +882,13 @@ would be read.</p>
     
   
     
-      <li class=""><a href="/docs/building.html">Building ORC</a></li>
+      <li class=""><a href="http://localhost:4000/docs/building.html">Building ORC</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -932,7 +918,7 @@ would be read.</p>
     
   
     
-      <li class=""><a href="/docs/releases.html">Releases</a></li>
+      <li class=""><a href="http://localhost:4000/docs/releases.html">Releases</a></li>
       
 
 
@@ -947,8 +933,6 @@ would be read.</p>
   
 
   
-    
-  
 
   
     
@@ -966,15 +950,13 @@ would be read.</p>
     
   
     
-      <li class=""><a href="/docs/hive-ddl.html">Hive DDL</a></li>
+      <li class=""><a href="http://localhost:4000/docs/hive-ddl.html">Hive DDL</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -990,7 +972,7 @@ would be read.</p>
     
   
     
-      <li class=""><a href="/docs/hive-config.html">Hive Configuration</a></li>
+      <li class=""><a href="http://localhost:4000/docs/hive-config.html">Hive Configuration</a></li>
       
 
 
@@ -1005,8 +987,6 @@ would be read.</p>
   
 
   
-    
-  
 
   
     
@@ -1032,15 +1012,13 @@ would be read.</p>
     
   
     
-      <li class=""><a href="/docs/mapred.html">Using in MapRed</a></li>
+      <li class=""><a href="http://localhost:4000/docs/mapred.html">Using in MapRed</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -1068,7 +1046,7 @@ would be read.</p>
     
   
     
-      <li class=""><a href="/docs/mapreduce.html">Using in MapReduce</a></li>
+      <li class=""><a href="http://localhost:4000/docs/mapreduce.html">Using in MapReduce</a></li>
       
 
 
@@ -1083,8 +1061,6 @@ would be read.</p>
   
 
   
-    
-  
 
   
     
@@ -1096,15 +1072,13 @@ would be read.</p>
     
   
     
-      <li class=""><a href="/docs/core-java.html">Using Core Java</a></li>
+      <li class=""><a href="http://localhost:4000/docs/core-java.html">Using Core Java</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -1114,7 +1088,7 @@ would be read.</p>
     
   
     
-      <li class=""><a href="/docs/core-cpp.html">Using Core C++</a></li>
+      <li class=""><a href="http://localhost:4000/docs/core-cpp.html">Using Core C++</a></li>
       
 
 
@@ -1129,8 +1103,6 @@ would be read.</p>
   
 
   
-    
-  
 
   
     
@@ -1144,15 +1116,13 @@ would be read.</p>
     
   
     
-      <li class=""><a href="/docs/cpp-tools.html">C++ Tools</a></li>
+      <li class=""><a href="http://localhost:4000/docs/cpp-tools.html">C++ Tools</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -1176,7 +1146,7 @@ would be read.</p>
     
   
     
-      <li class=""><a href="/docs/java-tools.html">Java Tools</a></li>
+      <li class=""><a href="http://localhost:4000/docs/java-tools.html">Java Tools</a></li>
       
 
 

http://git-wip-us.apache.org/repos/asf/orc/blob/e58d3fc4/docs/java-tools.html
----------------------------------------------------------------------
diff --git a/docs/java-tools.html b/docs/java-tools.html
index 25efb43..ad6f0b4 100644
--- a/docs/java-tools.html
+++ b/docs/java-tools.html
@@ -4,7 +4,7 @@
   <meta charset="UTF-8">
   <title>Java Tools</title>
   <meta name="viewport" content="width=device-width,initial-scale=1">
-  <meta name="generator" content="Jekyll v2.4.0">
+  <meta name="generator" content="Jekyll v3.7.3">
   <link rel="stylesheet" href="//fonts.googleapis.com/css?family=Lato:300,300italic,400,400italic,700,700italic,900">
   <link rel="stylesheet" href="/css/screen.css">
   <link rel="icon" type="image/x-icon" href="/favicon.ico">
@@ -109,7 +109,7 @@
     
   
     
-      <option value="/docs/index.html">Background</option>
+      <option value="http://localhost:4000/docs/index.html">Background</option>
     
   
     
@@ -131,7 +131,7 @@
     
   
     
-      <option value="/docs/adopters.html">ORC Adopters</option>
+      <option value="http://localhost:4000/docs/adopters.html">ORC Adopters</option>
     
   
     
@@ -193,7 +193,7 @@
     
   
     
-      <option value="/docs/types.html">Types</option>
+      <option value="http://localhost:4000/docs/types.html">Types</option>
     
   
 
@@ -219,7 +219,7 @@
     
   
     
-      <option value="/docs/indexes.html">Indexes</option>
+      <option value="http://localhost:4000/docs/indexes.html">Indexes</option>
     
   
     
@@ -237,7 +237,7 @@
 
   
     
-      <option value="/docs/acid.html">ACID support</option>
+      <option value="http://localhost:4000/docs/acid.html">ACID support</option>
     
   
     
@@ -284,7 +284,7 @@
     
   
     
-      <option value="/docs/building.html">Building ORC</option>
+      <option value="http://localhost:4000/docs/building.html">Building ORC</option>
     
   
     
@@ -342,7 +342,7 @@
     
   
     
-      <option value="/docs/releases.html">Releases</option>
+      <option value="http://localhost:4000/docs/releases.html">Releases</option>
     
   
     
@@ -373,7 +373,7 @@
     
   
     
-      <option value="/docs/hive-ddl.html">Hive DDL</option>
+      <option value="http://localhost:4000/docs/hive-ddl.html">Hive DDL</option>
     
   
     
@@ -407,7 +407,7 @@
     
   
     
-      <option value="/docs/hive-config.html">Hive Configuration</option>
+      <option value="http://localhost:4000/docs/hive-config.html">Hive Configuration</option>
     
   
     
@@ -460,7 +460,7 @@
     
   
     
-      <option value="/docs/mapred.html">Using in MapRed</option>
+      <option value="http://localhost:4000/docs/mapred.html">Using in MapRed</option>
     
   
     
@@ -498,7 +498,7 @@
     
   
     
-      <option value="/docs/mapreduce.html">Using in MapReduce</option>
+      <option value="http://localhost:4000/docs/mapreduce.html">Using in MapReduce</option>
     
   
     
@@ -525,7 +525,7 @@
     
   
     
-      <option value="/docs/core-java.html">Using Core Java</option>
+      <option value="http://localhost:4000/docs/core-java.html">Using Core Java</option>
     
   
     
@@ -559,7 +559,7 @@
     
   
     
-      <option value="/docs/core-cpp.html">Using Core C++</option>
+      <option value="http://localhost:4000/docs/core-cpp.html">Using Core C++</option>
     
   
     
@@ -606,7 +606,7 @@
     
   
     
-      <option value="/docs/cpp-tools.html">C++ Tools</option>
+      <option value="http://localhost:4000/docs/cpp-tools.html">C++ Tools</option>
     
   
     
@@ -652,7 +652,7 @@
     
   
     
-      <option value="/docs/java-tools.html">Java Tools</option>
+      <option value="http://localhost:4000/docs/java-tools.html">Java Tools</option>
     
   
     
@@ -691,8 +691,8 @@ supports both the local file system and HDFS.</p>
 
 <p>The command line looks like:</p>
 
-<pre><code class="language-shell">% java -jar orc-tools-X.Y.Z-uber.jar &lt;sub-command&gt; &lt;args&gt;
-</code></pre>
+<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>% java <span class="nt">-jar</span> orc-tools-X.Y.Z-uber.jar &lt;sub-command&gt; &lt;args&gt;
+</code></pre></div></div>
 
 <h2 id="java-meta">Java Meta</h2>
 
@@ -718,9 +718,9 @@ equivalent to the Hive ORC File Dump command.</p>
 
 <p>An example of the output is given below:</p>
 
-<pre><code class="language-shell">% java -jar orc-tools-X.Y.Z-uber.jar meta examples/TestOrcFile.test1.orc
-Processing data file examples/TestOrcFile.test1.orc [length: 1711]
-Structure for examples/TestOrcFile.test1.orc
+<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>% java <span class="nt">-jar</span> orc-tools-X.Y.Z-uber.jar meta examples/TestOrcFile.test1.orc
+Processing data file examples/TestOrcFile.test1.orc <span class="o">[</span>length: 1711]
+Structure <span class="k">for </span>examples/TestOrcFile.test1.orc
 File Version: 0.12 with HIVE_8732
 Rows: 2
 Compression: ZLIB
@@ -733,56 +733,56 @@ string&gt;&gt;&gt;
 
 Stripe Statistics:
   Stripe 1:
-    Column 0: count: 2 hasNull: false
-    Column 1: count: 2 hasNull: false true: 1
-    Column 2: count: 2 hasNull: false min: 1 max: 100 sum: 101
-    Column 3: count: 2 hasNull: false min: 1024 max: 2048 sum: 3072
-    Column 4: count: 2 hasNull: false min: 65536 max: 65536 sum: 131072
-    Column 5: count: 2 hasNull: false min: 9223372036854775807 max: 9223372036854775807
-    Column 6: count: 2 hasNull: false min: 1.0 max: 2.0 sum: 3.0
-    Column 7: count: 2 hasNull: false min: -15.0 max: -5.0 sum: -20.0
-    Column 8: count: 2 hasNull: false sum: 5
-    Column 9: count: 2 hasNull: false min: bye max: hi sum: 5
-    Column 10: count: 2 hasNull: false
-    Column 11: count: 2 hasNull: false
-    Column 12: count: 4 hasNull: false
-    Column 13: count: 4 hasNull: false min: 1 max: 2 sum: 6
-    Column 14: count: 4 hasNull: false min: bye max: sigh sum: 14
-    Column 15: count: 2 hasNull: false
-    Column 16: count: 5 hasNull: false
-    Column 17: count: 5 hasNull: false min: -100000 max: 100000000 sum: 99901241
-    Column 18: count: 5 hasNull: false min: bad max: in sum: 15
-    Column 19: count: 2 hasNull: false
-    Column 20: count: 2 hasNull: false min: chani max: mauddib sum: 12
-    Column 21: count: 2 hasNull: false
-    Column 22: count: 2 hasNull: false min: 1 max: 5 sum: 6
-    Column 23: count: 2 hasNull: false min: chani max: mauddib sum: 12
+    Column 0: count: 2 hasNull: <span class="nb">false
+    </span>Column 1: count: 2 hasNull: <span class="nb">false true</span>: 1
+    Column 2: count: 2 hasNull: <span class="nb">false </span>min: 1 max: 100 sum: 101
+    Column 3: count: 2 hasNull: <span class="nb">false </span>min: 1024 max: 2048 sum: 3072
+    Column 4: count: 2 hasNull: <span class="nb">false </span>min: 65536 max: 65536 sum: 131072
+    Column 5: count: 2 hasNull: <span class="nb">false </span>min: 9223372036854775807 max: 9223372036854775807
+    Column 6: count: 2 hasNull: <span class="nb">false </span>min: 1.0 max: 2.0 sum: 3.0
+    Column 7: count: 2 hasNull: <span class="nb">false </span>min: <span class="nt">-15</span>.0 max: <span class="nt">-5</span>.0 sum: <span class="nt">-20</span>.0
+    Column 8: count: 2 hasNull: <span class="nb">false </span>sum: 5
+    Column 9: count: 2 hasNull: <span class="nb">false </span>min: bye max: hi sum: 5
+    Column 10: count: 2 hasNull: <span class="nb">false
+    </span>Column 11: count: 2 hasNull: <span class="nb">false
+    </span>Column 12: count: 4 hasNull: <span class="nb">false
+    </span>Column 13: count: 4 hasNull: <span class="nb">false </span>min: 1 max: 2 sum: 6
+    Column 14: count: 4 hasNull: <span class="nb">false </span>min: bye max: sigh sum: 14
+    Column 15: count: 2 hasNull: <span class="nb">false
+    </span>Column 16: count: 5 hasNull: <span class="nb">false
+    </span>Column 17: count: 5 hasNull: <span class="nb">false </span>min: <span class="nt">-100000</span> max: 100000000 sum: 99901241
+    Column 18: count: 5 hasNull: <span class="nb">false </span>min: bad max: <span class="k">in </span>sum: 15
+    Column 19: count: 2 hasNull: <span class="nb">false
+    </span>Column 20: count: 2 hasNull: <span class="nb">false </span>min: chani max: mauddib sum: 12
+    Column 21: count: 2 hasNull: <span class="nb">false
+    </span>Column 22: count: 2 hasNull: <span class="nb">false </span>min: 1 max: 5 sum: 6
+    Column 23: count: 2 hasNull: <span class="nb">false </span>min: chani max: mauddib sum: 12
 
 File Statistics:
-  Column 0: count: 2 hasNull: false
-  Column 1: count: 2 hasNull: false true: 1
-  Column 2: count: 2 hasNull: false min: 1 max: 100 sum: 101
-  Column 3: count: 2 hasNull: false min: 1024 max: 2048 sum: 3072
-  Column 4: count: 2 hasNull: false min: 65536 max: 65536 sum: 131072
-  Column 5: count: 2 hasNull: false min: 9223372036854775807 max: 9223372036854775807
-  Column 6: count: 2 hasNull: false min: 1.0 max: 2.0 sum: 3.0
-  Column 7: count: 2 hasNull: false min: -15.0 max: -5.0 sum: -20.0
-  Column 8: count: 2 hasNull: false sum: 5
-  Column 9: count: 2 hasNull: false min: bye max: hi sum: 5
-  Column 10: count: 2 hasNull: false
-  Column 11: count: 2 hasNull: false
-  Column 12: count: 4 hasNull: false
-  Column 13: count: 4 hasNull: false min: 1 max: 2 sum: 6
-  Column 14: count: 4 hasNull: false min: bye max: sigh sum: 14
-  Column 15: count: 2 hasNull: false
-  Column 16: count: 5 hasNull: false
-  Column 17: count: 5 hasNull: false min: -100000 max: 100000000 sum: 99901241
-  Column 18: count: 5 hasNull: false min: bad max: in sum: 15
-  Column 19: count: 2 hasNull: false
-  Column 20: count: 2 hasNull: false min: chani max: mauddib sum: 12
-  Column 21: count: 2 hasNull: false
-  Column 22: count: 2 hasNull: false min: 1 max: 5 sum: 6
-  Column 23: count: 2 hasNull: false min: chani max: mauddib sum: 12
+  Column 0: count: 2 hasNull: <span class="nb">false
+  </span>Column 1: count: 2 hasNull: <span class="nb">false true</span>: 1
+  Column 2: count: 2 hasNull: <span class="nb">false </span>min: 1 max: 100 sum: 101
+  Column 3: count: 2 hasNull: <span class="nb">false </span>min: 1024 max: 2048 sum: 3072
+  Column 4: count: 2 hasNull: <span class="nb">false </span>min: 65536 max: 65536 sum: 131072
+  Column 5: count: 2 hasNull: <span class="nb">false </span>min: 9223372036854775807 max: 9223372036854775807
+  Column 6: count: 2 hasNull: <span class="nb">false </span>min: 1.0 max: 2.0 sum: 3.0
+  Column 7: count: 2 hasNull: <span class="nb">false </span>min: <span class="nt">-15</span>.0 max: <span class="nt">-5</span>.0 sum: <span class="nt">-20</span>.0
+  Column 8: count: 2 hasNull: <span class="nb">false </span>sum: 5
+  Column 9: count: 2 hasNull: <span class="nb">false </span>min: bye max: hi sum: 5
+  Column 10: count: 2 hasNull: <span class="nb">false
+  </span>Column 11: count: 2 hasNull: <span class="nb">false
+  </span>Column 12: count: 4 hasNull: <span class="nb">false
+  </span>Column 13: count: 4 hasNull: <span class="nb">false </span>min: 1 max: 2 sum: 6
+  Column 14: count: 4 hasNull: <span class="nb">false </span>min: bye max: sigh sum: 14
+  Column 15: count: 2 hasNull: <span class="nb">false
+  </span>Column 16: count: 5 hasNull: <span class="nb">false
+  </span>Column 17: count: 5 hasNull: <span class="nb">false </span>min: <span class="nt">-100000</span> max: 100000000 sum: 99901241
+  Column 18: count: 5 hasNull: <span class="nb">false </span>min: bad max: <span class="k">in </span>sum: 15
+  Column 19: count: 2 hasNull: <span class="nb">false
+  </span>Column 20: count: 2 hasNull: <span class="nb">false </span>min: chani max: mauddib sum: 12
+  Column 21: count: 2 hasNull: <span class="nb">false
+  </span>Column 22: count: 2 hasNull: <span class="nb">false </span>min: 1 max: 5 sum: 6
+  Column 23: count: 2 hasNull: <span class="nb">false </span>min: chani max: mauddib sum: 12
 
 Stripes:
   Stripe: offset: 3 data: 243 rows: 2 tail: 199 index: 570
@@ -865,7 +865,7 @@ File length: 1711 bytes
 Padding length: 0 bytes
 Padding ratio: 0%
 ______________________________________________________________________
-</code></pre>
+</code></pre></div></div>
 
 <h2 id="java-data">Java Data</h2>
 
@@ -993,8 +993,6 @@ to include all of the observed values.</p>
   
 
   
-    
-  
 
   
     
@@ -1014,29 +1012,25 @@ to include all of the observed values.</p>
     
   
     
-      <li class=""><a href="/docs/index.html">Background</a></li>
+      <li class=""><a href="http://localhost:4000/docs/index.html">Background</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
   
     
-      <li class=""><a href="/docs/adopters.html">ORC Adopters</a></li>
+      <li class=""><a href="http://localhost:4000/docs/adopters.html">ORC Adopters</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -1068,15 +1062,13 @@ to include all of the observed values.</p>
     
   
     
-      <li class=""><a href="/docs/types.html">Types</a></li>
+      <li class=""><a href="http://localhost:4000/docs/types.html">Types</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -1098,19 +1090,17 @@ to include all of the observed values.</p>
     
   
     
-      <li class=""><a href="/docs/indexes.html">Indexes</a></li>
+      <li class=""><a href="http://localhost:4000/docs/indexes.html">Indexes</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
-      <li class=""><a href="/docs/acid.html">ACID support</a></li>
+      <li class=""><a href="http://localhost:4000/docs/acid.html">ACID support</a></li>
       
 
 
@@ -1125,8 +1115,6 @@ to include all of the observed values.</p>
   
 
   
-    
-  
 
   
     
@@ -1134,15 +1122,13 @@ to include all of the observed values.</p>
     
   
     
-      <li class=""><a href="/docs/building.html">Building ORC</a></li>
+      <li class=""><a href="http://localhost:4000/docs/building.html">Building ORC</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -1172,7 +1158,7 @@ to include all of the observed values.</p>
     
   
     
-      <li class=""><a href="/docs/releases.html">Releases</a></li>
+      <li class=""><a href="http://localhost:4000/docs/releases.html">Releases</a></li>
       
 
 
@@ -1187,8 +1173,6 @@ to include all of the observed values.</p>
   
 
   
-    
-  
 
   
     
@@ -1206,15 +1190,13 @@ to include all of the observed values.</p>
     
   
     
-      <li class=""><a href="/docs/hive-ddl.html">Hive DDL</a></li>
+      <li class=""><a href="http://localhost:4000/docs/hive-ddl.html">Hive DDL</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -1230,7 +1212,7 @@ to include all of the observed values.</p>
     
   
     
-      <li class=""><a href="/docs/hive-config.html">Hive Configuration</a></li>
+      <li class=""><a href="http://localhost:4000/docs/hive-config.html">Hive Configuration</a></li>
       
 
 
@@ -1245,8 +1227,6 @@ to include all of the observed values.</p>
   
 
   
-    
-  
 
   
     
@@ -1272,15 +1252,13 @@ to include all of the observed values.</p>
     
   
     
-      <li class=""><a href="/docs/mapred.html">Using in MapRed</a></li>
+      <li class=""><a href="http://localhost:4000/docs/mapred.html">Using in MapRed</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -1308,7 +1286,7 @@ to include all of the observed values.</p>
     
   
     
-      <li class=""><a href="/docs/mapreduce.html">Using in MapReduce</a></li>
+      <li class=""><a href="http://localhost:4000/docs/mapreduce.html">Using in MapReduce</a></li>
       
 
 
@@ -1323,8 +1301,6 @@ to include all of the observed values.</p>
   
 
   
-    
-  
 
   
     
@@ -1336,15 +1312,13 @@ to include all of the observed values.</p>
     
   
     
-      <li class=""><a href="/docs/core-java.html">Using Core Java</a></li>
+      <li class=""><a href="http://localhost:4000/docs/core-java.html">Using Core Java</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -1354,7 +1328,7 @@ to include all of the observed values.</p>
     
   
     
-      <li class=""><a href="/docs/core-cpp.html">Using Core C++</a></li>
+      <li class=""><a href="http://localhost:4000/docs/core-cpp.html">Using Core C++</a></li>
       
 
 
@@ -1369,8 +1343,6 @@ to include all of the observed values.</p>
   
 
   
-    
-  
 
   
     
@@ -1384,15 +1356,13 @@ to include all of the observed values.</p>
     
   
     
-      <li class=""><a href="/docs/cpp-tools.html">C++ Tools</a></li>
+      <li class=""><a href="http://localhost:4000/docs/cpp-tools.html">C++ Tools</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -1416,7 +1386,7 @@ to include all of the observed values.</p>
     
   
     
-      <li class="current"><a href="/docs/java-tools.html">Java Tools</a></li>
+      <li class="current"><a href="http://localhost:4000/docs/java-tools.html">Java Tools</a></li>
       
 
 

http://git-wip-us.apache.org/repos/asf/orc/blob/e58d3fc4/docs/mapred.html
----------------------------------------------------------------------
diff --git a/docs/mapred.html b/docs/mapred.html
index ab932db..d23b006 100644
--- a/docs/mapred.html
+++ b/docs/mapred.html
@@ -4,7 +4,7 @@
   <meta charset="UTF-8">
   <title>Using in MapRed</title>
   <meta name="viewport" content="width=device-width,initial-scale=1">
-  <meta name="generator" content="Jekyll v2.4.0">
+  <meta name="generator" content="Jekyll v3.7.3">
   <link rel="stylesheet" href="//fonts.googleapis.com/css?family=Lato:300,300italic,400,400italic,700,700italic,900">
   <link rel="stylesheet" href="/css/screen.css">
   <link rel="icon" type="image/x-icon" href="/favicon.ico">
@@ -109,7 +109,7 @@
     
   
     
-      <option value="/docs/index.html">Background</option>
+      <option value="http://localhost:4000/docs/index.html">Background</option>
     
   
     
@@ -131,7 +131,7 @@
     
   
     
-      <option value="/docs/adopters.html">ORC Adopters</option>
+      <option value="http://localhost:4000/docs/adopters.html">ORC Adopters</option>
     
   
     
@@ -193,7 +193,7 @@
     
   
     
-      <option value="/docs/types.html">Types</option>
+      <option value="http://localhost:4000/docs/types.html">Types</option>
     
   
 
@@ -219,7 +219,7 @@
     
   
     
-      <option value="/docs/indexes.html">Indexes</option>
+      <option value="http://localhost:4000/docs/indexes.html">Indexes</option>
     
   
     
@@ -237,7 +237,7 @@
 
   
     
-      <option value="/docs/acid.html">ACID support</option>
+      <option value="http://localhost:4000/docs/acid.html">ACID support</option>
     
   
     
@@ -284,7 +284,7 @@
     
   
     
-      <option value="/docs/building.html">Building ORC</option>
+      <option value="http://localhost:4000/docs/building.html">Building ORC</option>
     
   
     
@@ -342,7 +342,7 @@
     
   
     
-      <option value="/docs/releases.html">Releases</option>
+      <option value="http://localhost:4000/docs/releases.html">Releases</option>
     
   
     
@@ -373,7 +373,7 @@
     
   
     
-      <option value="/docs/hive-ddl.html">Hive DDL</option>
+      <option value="http://localhost:4000/docs/hive-ddl.html">Hive DDL</option>
     
   
     
@@ -407,7 +407,7 @@
     
   
     
-      <option value="/docs/hive-config.html">Hive Configuration</option>
+      <option value="http://localhost:4000/docs/hive-config.html">Hive Configuration</option>
     
   
     
@@ -460,7 +460,7 @@
     
   
     
-      <option value="/docs/mapred.html">Using in MapRed</option>
+      <option value="http://localhost:4000/docs/mapred.html">Using in MapRed</option>
     
   
     
@@ -498,7 +498,7 @@
     
   
     
-      <option value="/docs/mapreduce.html">Using in MapReduce</option>
+      <option value="http://localhost:4000/docs/mapreduce.html">Using in MapReduce</option>
     
   
     
@@ -525,7 +525,7 @@
     
   
     
-      <option value="/docs/core-java.html">Using Core Java</option>
+      <option value="http://localhost:4000/docs/core-java.html">Using Core Java</option>
     
   
     
@@ -559,7 +559,7 @@
     
   
     
-      <option value="/docs/core-cpp.html">Using Core C++</option>
+      <option value="http://localhost:4000/docs/core-cpp.html">Using Core C++</option>
     
   
     
@@ -606,7 +606,7 @@
     
   
     
-      <option value="/docs/cpp-tools.html">C++ Tools</option>
+      <option value="http://localhost:4000/docs/cpp-tools.html">C++ Tools</option>
     
   
     
@@ -652,7 +652,7 @@
     
   
     
-      <option value="/docs/java-tools.html">Java Tools</option>
+      <option value="http://localhost:4000/docs/java-tools.html">Java Tools</option>
     
   
     
@@ -676,31 +676,31 @@
           <h1>Using in MapRed</h1>
           <p>This page describes how to read and write ORC files from Hadoop’s
 older org.apache.hadoop.mapred MapReduce APIs. If you want to use the
-new org.apache.hadoop.mapreduce API, please look at the <a href="/docs/mapreduce.html">next
+new org.apache.hadoop.mapreduce API, please look at the <a href="http://localhost:4000/docs/mapreduce.html">next
 page</a>.</p>
 
 <h2 id="reading-orc-files">Reading ORC files</h2>
 
-<p>Add ORC and your desired version of Hadoop to your <code>pom.xml</code>:</p>
-
-<pre><code class="language-xml">&lt;dependencies&gt;
-  &lt;dependency&gt;
-    &lt;groupId&gt;org.apache.orc&lt;/groupId&gt;
-    &lt;artifactId&gt;orc-mapreduce&lt;/artifactId&gt;
-    &lt;version&gt;1.1.0&lt;/version&gt;
-  &lt;/dependency&gt;
-  &lt;dependency&gt;
-    &lt;groupId&gt;org.apache.hadoop&lt;/groupId&gt;
-    &lt;artifactId&gt;hadoop-mapreduce-client-core&lt;/artifactId&gt;
-    &lt;version&gt;2.7.0&lt;/version&gt;
-  &lt;/dependency&gt;
-&lt;/dependencies&gt;
-</code></pre>
+<p>Add ORC and your desired version of Hadoop to your <code class="highlighter-rouge">pom.xml</code>:</p>
+
+<div class="language-xml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nt">&lt;dependencies&gt;</span>
+  <span class="nt">&lt;dependency&gt;</span>
+    <span class="nt">&lt;groupId&gt;</span>org.apache.orc<span class="nt">&lt;/groupId&gt;</span>
+    <span class="nt">&lt;artifactId&gt;</span>orc-mapreduce<span class="nt">&lt;/artifactId&gt;</span>
+    <span class="nt">&lt;version&gt;</span>1.1.0<span class="nt">&lt;/version&gt;</span>
+  <span class="nt">&lt;/dependency&gt;</span>
+  <span class="nt">&lt;dependency&gt;</span>
+    <span class="nt">&lt;groupId&gt;</span>org.apache.hadoop<span class="nt">&lt;/groupId&gt;</span>
+    <span class="nt">&lt;artifactId&gt;</span>hadoop-mapreduce-client-core<span class="nt">&lt;/artifactId&gt;</span>
+    <span class="nt">&lt;version&gt;</span>2.7.0<span class="nt">&lt;/version&gt;</span>
+  <span class="nt">&lt;/dependency&gt;</span>
+<span class="nt">&lt;/dependencies&gt;</span>
+</code></pre></div></div>
 
 <p>Set the minimal properties in your JobConf:</p>
 
 <ul>
-  <li><strong>mapreduce.job.inputformat.class</strong> = <a href="/api/orc-mapreduce/index.html?org/apache/orc/mapred/OrcInputFormat.html">org.apache.orc.mapred.OrcInputFormat</a></li>
+  <li><strong>mapreduce.job.inputformat.class</strong> = <a href="http://localhost:4000/api/orc-mapreduce/index.html?org/apache/orc/mapred/OrcInputFormat.html">org.apache.orc.mapred.OrcInputFormat</a></li>
   <li><strong>mapreduce.input.fileinputformat.inputdir</strong> = your input directory</li>
 </ul>
 
@@ -723,7 +723,7 @@ the key and a value based on the table below expanded recursively.</p>
   <tbody>
     <tr>
       <td>array</td>
-      <td><a href="/api/orc-mapreduce/index.html?org/apache/orc/mapred/OrcStruct.html">org.apache.orc.mapred.OrcList</a></td>
+      <td><a href="http://localhost:4000/api/orc-mapreduce/index.html?org/apache/orc/mapred/OrcStruct.html">org.apache.orc.mapred.OrcList</a></td>
     </tr>
     <tr>
       <td>binary</td>
@@ -743,11 +743,11 @@ the key and a value based on the table below expanded recursively.</p>
     </tr>
     <tr>
       <td>date</td>
-      <td><a href="/api/hive-storage-api/index.html?org/apache/hadoop/hive/serde2/io/DateWritable.html">org.apache.hadoop.hive.serde2.io.DateWritable</a></td>
+      <td><a href="http://localhost:4000/api/hive-storage-api/index.html?org/apache/hadoop/hive/serde2/io/DateWritable.html">org.apache.hadoop.hive.serde2.io.DateWritable</a></td>
     </tr>
     <tr>
       <td>decimal</td>
-      <td><a href="/api/hive-storage-api/index.html?org/apache/hadoop/hive/serde2/io/HiveDecimalWritable.html">org.apache.hadoop.hive.serde2.io.HiveDecimalWritable</a></td>
+      <td><a href="http://localhost:4000/api/hive-storage-api/index.html?org/apache/hadoop/hive/serde2/io/HiveDecimalWritable.html">org.apache.hadoop.hive.serde2.io.HiveDecimalWritable</a></td>
     </tr>
     <tr>
       <td>double</td>
@@ -763,7 +763,7 @@ the key and a value based on the table below expanded recursively.</p>
     </tr>
     <tr>
       <td>map</td>
-      <td><a href="/api/orc-mapreduce/index.html?org/apache/orc/mapred/OrcMap.html">org.apache.orc.mapred.OrcMap</a></td>
+      <td><a href="http://localhost:4000/api/orc-mapreduce/index.html?org/apache/orc/mapred/OrcMap.html">org.apache.orc.mapred.OrcMap</a></td>
     </tr>
     <tr>
       <td>smallint</td>
@@ -775,11 +775,11 @@ the key and a value based on the table below expanded recursively.</p>
     </tr>
     <tr>
       <td>struct</td>
-      <td><a href="/api/orc-mapreduce/index.html?org/apache/orc/mapred/OrcStruct.html">org.apache.orc.mapred.OrcStruct</a></td>
+      <td><a href="http://localhost:4000/api/orc-mapreduce/index.html?org/apache/orc/mapred/OrcStruct.html">org.apache.orc.mapred.OrcStruct</a></td>
     </tr>
     <tr>
       <td>timestamp</td>
-      <td><a href="/api/orc-mapreduce/index.html?org/apache/orc/mapred/OrcTimestamp.html">org.apache.orc.mapred.OrcTimestamp</a></td>
+      <td><a href="http://localhost:4000/api/orc-mapreduce/index.html?org/apache/orc/mapred/OrcTimestamp.html">org.apache.orc.mapred.OrcTimestamp</a></td>
     </tr>
     <tr>
       <td>tinyint</td>
@@ -787,7 +787,7 @@ the key and a value based on the table below expanded recursively.</p>
     </tr>
     <tr>
       <td>uniontype</td>
-      <td><a href="/api/orc-mapreduce/index.html?org/apache/orc/mapred/OrcUnion.html">org.apache.orc.mapred.OrcUnion</a></td>
+      <td><a href="http://localhost:4000/api/orc-mapreduce/index.html?org/apache/orc/mapred/OrcUnion.html">org.apache.orc.mapred.OrcUnion</a></td>
     </tr>
     <tr>
       <td>varchar</td>
@@ -797,33 +797,33 @@ the key and a value based on the table below expanded recursively.</p>
 </table>
 
 <p>Let’s assume that your input directory contains ORC files with the
-schema <code>struct&lt;s:string,i:int&gt;</code> and you want to use the string field
+schema <code class="highlighter-rouge">struct&lt;s:string,i:int&gt;</code> and you want to use the string field
 as the key to the MapReduce shuffle and the integer as the value. The
 mapper code would look like:</p>
 
-<pre><code class="language-java">public class MyMapper
-    implements Mapper&lt;NullWritable,OrcStruct,Text,IntWritable&gt; {
+<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">public</span> <span class="kd">class</span> <span class="nc">MyMapper</span>
+    <span class="kd">implements</span> <span class="n">Mapper</span><span class="o">&lt;</span><span class="n">NullWritable</span><span class="o">,</span><span class="n">OrcStruct</span><span class="o">,</span><span class="n">Text</span><span class="o">,</span><span class="n">IntWritable</span><span class="o">&gt;</span> <span class="o">{</span>
 
-  // Input should be: struct&lt;s:string,i:int&gt;
-  public void map(NullWritable key, OrcStruct value,
-                  OutputCollector&lt;Text,IntWritable&gt; output,
-                  Reporter reporter) throws IOException {
-    output.collect((Text) value.getFieldValue(0),
-                   (IntWritable) value.getFieldValue(1));
-  }
+  <span class="c1">// Input should be: struct&lt;s:string,i:int&gt;</span>
+  <span class="kd">public</span> <span class="kt">void</span> <span class="nf">map</span><span class="o">(</span><span class="n">NullWritable</span> <span class="n">key</span><span class="o">,</span> <span class="n">OrcStruct</span> <span class="n">value</span><span class="o">,</span>
+                  <span class="n">OutputCollector</span><span class="o">&lt;</span><span class="n">Text</span><span class="o">,</span><span class="n">IntWritable</span><span class="o">&gt;</span> <span class="n">output</span><span class="o">,</span>
+                  <span class="n">Reporter</span> <span class="n">reporter</span><span class="o">)</span> <span class="kd">throws</span> <span class="n">IOException</span> <span class="o">{</span>
+    <span class="n">output</span><span class="o">.</span><span class="na">collect</span><span class="o">((</span><span class="n">Text</span><span class="o">)</span> <span class="n">value</span><span class="o">.</span><span class="na">getFieldValue</span><span class="o">(</span><span class="mi">0</span><span class="o">),</span>
+                   <span class="o">(</span><span class="n">IntWritable</span><span class="o">)</span> <span class="n">value</span><span class="o">.</span><span class="na">getFieldValue</span><span class="o">(</span><span class="mi">1</span><span class="o">));</span>
+  <span class="o">}</span>
 
-  public void configure(JobConf conf) { }
+  <span class="kd">public</span> <span class="kt">void</span> <span class="nf">configure</span><span class="o">(</span><span class="n">JobConf</span> <span class="n">conf</span><span class="o">)</span> <span class="o">{</span> <span class="o">}</span>
 
-  public void close() { }
-}
-</code></pre>
+  <span class="kd">public</span> <span class="kt">void</span> <span class="nf">close</span><span class="o">()</span> <span class="o">{</span> <span class="o">}</span>
+<span class="o">}</span>
+</code></pre></div></div>
 
 <h2 id="writing-orc-files">Writing ORC files</h2>
 
 <p>To write ORC files from your MapReduce job, you’ll need to set</p>
 
 <ul>
-  <li><strong>mapreduce.job.outputformat.class</strong> = <a href="/api/orc-mapreduce/index.html?org/apache/orc/mapred/OrcOutputFormat.html">org.apache.orc.mapred.OrcOutputFormat</a></li>
+  <li><strong>mapreduce.job.outputformat.class</strong> = <a href="http://localhost:4000/api/orc-mapreduce/index.html?org/apache/orc/mapred/OrcOutputFormat.html">org.apache.orc.mapred.OrcOutputFormat</a></li>
   <li><strong>mapreduce.output.fileoutputformat.outputdir</strong> = your output directory</li>
   <li><strong>orc.mapred.output.schema</strong> = the schema to write to the ORC file</li>
 </ul>
@@ -833,38 +833,38 @@ file and typically uses the OrcStruct.createValue(TypeDescription)
 function. For our example, let’s assume that the shuffle types are
 (Text, IntWritable) from the previous section and the reduce should
 gather the integer for each key together and write them as a list. The
-output schema would be <code>struct&lt;key:string,ints:array&lt;int&gt;&gt;</code>. As always
+output schema would be <code class="highlighter-rouge">struct&lt;key:string,ints:array&lt;int&gt;&gt;</code>. As always
 with MapReduce, if your method stores the values, you need to copy their
 value before getting the next.</p>
 
-<pre><code class="language-java">public static class MyReducer
-  implements Reducer&lt;Text,IntWritable,NullWritable,OrcStruct&gt; {
-
-  private TypeDescription schema =
-    TypeDescription.fromString("struct&lt;key:string,ints:array&lt;int&gt;&gt;");
-  // createValue creates the correct value type for the schema
-  private OrcStruct pair = (OrcStruct) OrcStruct.createValue(schema);
-  // get a handle to the list of ints
-  private OrcList&lt;IntWritable&gt; values =
-    (OrcList&lt;IntWritable&gt;) pair.getFieldValue(1);
-  private final NullWritable nada = NullWritable.get();
-
-  public void reduce(Text key, Iterator&lt;IntWritable&gt; iterator,
-                     OutputCollector&lt;NullWritable, OrcStruct&gt; output,
-                     Reporter reporter) throws IOException {
-    pair.setFieldValue(0, key);
-    values.clear();
-    while (iterator.hasNext()) {
-      values.add(new IntWritable(iterator.next().get()));
-    }
-    output.collect(nada, pair);
-  }
-
-  public void configure(JobConf conf) { }
-
-  public void close() { }
-}
-</code></pre>
+<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">public</span> <span class="kd">static</span> <span class="kd">class</span> <span class="nc">MyReducer</span>
+  <span class="kd">implements</span> <span class="n">Reducer</span><span class="o">&lt;</span><span class="n">Text</span><span class="o">,</span><span class="n">IntWritable</span><span class="o">,</span><span class="n">NullWritable</span><span class="o">,</span><span class="n">OrcStruct</span><span class="o">&gt;</span> <span class="o">{</span>
+
+  <span class="kd">private</span> <span class="n">TypeDescription</span> <span class="n">schema</span> <span class="o">=</span>
+    <span class="n">TypeDescription</span><span class="o">.</span><span class="na">fromString</span><span class="o">(</span><span class="s">"struct&lt;key:string,ints:array&lt;int&gt;&gt;"</span><span class="o">);</span>
+  <span class="c1">// createValue creates the correct value type for the schema</span>
+  <span class="kd">private</span> <span class="n">OrcStruct</span> <span class="n">pair</span> <span class="o">=</span> <span class="o">(</span><span class="n">OrcStruct</span><span class="o">)</span> <span class="n">OrcStruct</span><span class="o">.</span><span class="na">createValue</span><span class="o">(</span><span class="n">schema</span><span class="o">);</span>
+  <span class="c1">// get a handle to the list of ints</span>
+  <span class="kd">private</span> <span class="n">OrcList</span><span class="o">&lt;</span><span class="n">IntWritable</span><span class="o">&gt;</span> <span class="n">values</span> <span class="o">=</span>
+    <span class="o">(</span><span class="n">OrcList</span><span class="o">&lt;</span><span class="n">IntWritable</span><span class="o">&gt;)</span> <span class="n">pair</span><span class="o">.</span><span class="na">getFieldValue</span><span class="o">(</span><span class="mi">1</span><span class="o">);</span>
+  <span class="kd">private</span> <span class="kd">final</span> <span class="n">NullWritable</span> <span class="n">nada</span> <span class="o">=</span> <span class="n">NullWritable</span><span class="o">.</span><span class="na">get</span><span class="o">();</span>
+
+  <span class="kd">public</span> <span class="kt">void</span> <span class="nf">reduce</span><span class="o">(</span><span class="n">Text</span> <span class="n">key</span><span class="o">,</span> <span class="n">Iterator</span><span class="o">&lt;</span><span class="n">IntWritable</span><span class="o">&gt;</span> <span class="n">iterator</span><span class="o">,</span>
+                     <span class="n">OutputCollector</span><span class="o">&lt;</span><span class="n">NullWritable</span><span class="o">,</span> <span class="n">OrcStruct</span><span class="o">&gt;</span> <span class="n">output</span><span class="o">,</span>
+                     <span class="n">Reporter</span> <span class="n">reporter</span><span class="o">)</span> <span class="kd">throws</span> <span class="n">IOException</span> <span class="o">{</span>
+    <span class="n">pair</span><span class="o">.</span><span class="na">setFieldValue</span><span class="o">(</span><span class="mi">0</span><span class="o">,</span> <span class="n">key</span><span class="o">);</span>
+    <span class="n">values</span><span class="o">.</span><span class="na">clear</span><span class="o">();</span>
+    <span class="k">while</span> <span class="o">(</span><span class="n">iterator</span><span class="o">.</span><span class="na">hasNext</span><span class="o">())</span> <span class="o">{</span>
+      <span class="n">values</span><span class="o">.</span><span class="na">add</span><span class="o">(</span><span class="k">new</span> <span class="n">IntWritable</span><span class="o">(</span><span class="n">iterator</span><span class="o">.</span><span class="na">next</span><span class="o">().</span><span class="na">get</span><span class="o">()));</span>
+    <span class="o">}</span>
+    <span class="n">output</span><span class="o">.</span><span class="na">collect</span><span class="o">(</span><span class="n">nada</span><span class="o">,</span> <span class="n">pair</span><span class="o">);</span>
+  <span class="o">}</span>
+
+  <span class="kd">public</span> <span class="kt">void</span> <span class="nf">configure</span><span class="o">(</span><span class="n">JobConf</span> <span class="n">conf</span><span class="o">)</span> <span class="o">{</span> <span class="o">}</span>
+
+  <span class="kd">public</span> <span class="kt">void</span> <span class="nf">close</span><span class="o">()</span> <span class="o">{</span> <span class="o">}</span>
+<span class="o">}</span>
+</code></pre></div></div>
 
 <h2 id="sending-orcstruct-orclist-orcmap-or-orcunion-through-the-shuffle">Sending OrcStruct, OrcList, OrcMap, or OrcUnion through the Shuffle</h2>
 
@@ -873,9 +873,9 @@ MapReduce shuffle. The complex ORC types, since they are generic
 types, need to have their full type information provided to create the
 object. To enable MapReduce to properly instantiate the OrcStruct and
 other ORC types, we need to wrap it in either an
-<a href="/api/orc-mapreduce/index.html?org/apache/orc/mapred/OrcKey.html">OrcKey</a>
+<a href="http://localhost:4000/api/orc-mapreduce/index.html?org/apache/orc/mapred/OrcKey.html">OrcKey</a>
 for the shuffle key or
-<a href="/api/orc-mapreduce/index.html?org/apache/orc/mapred/OrcValue.html">OrcValue</a>
+<a href="http://localhost:4000/api/orc-mapreduce/index.html?org/apache/orc/mapred/OrcValue.html">OrcValue</a>
 for the shuffle value.</p>
 
 <p>To send two OrcStructs through the shuffle, define the following properties
@@ -892,38 +892,38 @@ in the JobConf:</p>
 respectively. These objects should be created once and reused as the mapper
 runs.</p>
 
-<pre><code class="language-java">public static class MyMapperWithShuffle
-  implements Mapper&lt;NullWritable,OrcStruct,OrcKey,OrcValue&gt; {
+<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">public</span> <span class="kd">static</span> <span class="kd">class</span> <span class="nc">MyMapperWithShuffle</span>
+  <span class="kd">implements</span> <span class="n">Mapper</span><span class="o">&lt;</span><span class="n">NullWritable</span><span class="o">,</span><span class="n">OrcStruct</span><span class="o">,</span><span class="n">OrcKey</span><span class="o">,</span><span class="n">OrcValue</span><span class="o">&gt;</span> <span class="o">{</span>
 
-  // create wrapper objects
-  private OrcKey keyWrapper = new OrcKey();
-  private OrcValue valueWrapper = new OrcValue();
+  <span class="c1">// create wrapper objects</span>
+  <span class="kd">private</span> <span class="n">OrcKey</span> <span class="n">keyWrapper</span> <span class="o">=</span> <span class="k">new</span> <span class="n">OrcKey</span><span class="o">();</span>
+  <span class="kd">private</span> <span class="n">OrcValue</span> <span class="n">valueWrapper</span> <span class="o">=</span> <span class="k">new</span> <span class="n">OrcValue</span><span class="o">();</span>
 
-  // create a new structure to pass as the value in the shuffle
-  private OrcStruct outStruct = (OrcStruct) OrcStruct.createValue
-    (TypeDescription.fromString("struct&lt;i:int,j:int&gt;"));
+  <span class="c1">// create a new structure to pass as the value in the shuffle</span>
+  <span class="kd">private</span> <span class="n">OrcStruct</span> <span class="n">outStruct</span> <span class="o">=</span> <span class="o">(</span><span class="n">OrcStruct</span><span class="o">)</span> <span class="n">OrcStruct</span><span class="o">.</span><span class="na">createValue</span>
+    <span class="o">(</span><span class="n">TypeDescription</span><span class="o">.</span><span class="na">fromString</span><span class="o">(</span><span class="s">"struct&lt;i:int,j:int&gt;"</span><span class="o">));</span>
 
-  // get the two fields of the outStruct
-  private IntWritable i = (IntWritable) outStruct.getFieldValue("i");
-  private IntWritable j = (IntWritable) outStruct.getFieldValue("j");
+  <span class="c1">// get the two fields of the outStruct</span>
+  <span class="kd">private</span> <span class="n">IntWritable</span> <span class="n">i</span> <span class="o">=</span> <span class="o">(</span><span class="n">IntWritable</span><span class="o">)</span> <span class="n">outStruct</span><span class="o">.</span><span class="na">getFieldValue</span><span class="o">(</span><span class="s">"i"</span><span class="o">);</span>
+  <span class="kd">private</span> <span class="n">IntWritable</span> <span class="n">j</span> <span class="o">=</span> <span class="o">(</span><span class="n">IntWritable</span><span class="o">)</span> <span class="n">outStruct</span><span class="o">.</span><span class="na">getFieldValue</span><span class="o">(</span><span class="s">"j"</span><span class="o">);</span>
 
-  // Assume the input has type: struct&lt;s:string,i:int&gt;
-  public void map(NullWritable key, OrcStruct value,
-                  OutputCollector&lt;OrcKey,OrcValue&gt; output,
-                  Reporter reporter) throws IOException {
-    keyWrapper.key = value;
-    valueWrapper.value = outStruct;
-    int val = ((IntWritable) value.getFieldValue("i")).get();
-    i.set(val * 2);
-    j.set(val * val);
-    output.collect(keyWrapper, valueWrapper);
-  }
+  <span class="c1">// Assume the input has type: struct&lt;s:string,i:int&gt;</span>
+  <span class="kd">public</span> <span class="kt">void</span> <span class="nf">map</span><span class="o">(</span><span class="n">NullWritable</span> <span class="n">key</span><span class="o">,</span> <span class="n">OrcStruct</span> <span class="n">value</span><span class="o">,</span>
+                  <span class="n">OutputCollector</span><span class="o">&lt;</span><span class="n">OrcKey</span><span class="o">,</span><span class="n">OrcValue</span><span class="o">&gt;</span> <span class="n">output</span><span class="o">,</span>
+                  <span class="n">Reporter</span> <span class="n">reporter</span><span class="o">)</span> <span class="kd">throws</span> <span class="n">IOException</span> <span class="o">{</span>
+    <span class="n">keyWrapper</span><span class="o">.</span><span class="na">key</span> <span class="o">=</span> <span class="n">value</span><span class="o">;</span>
+    <span class="n">valueWrapper</span><span class="o">.</span><span class="na">value</span> <span class="o">=</span> <span class="n">outStruct</span><span class="o">;</span>
+    <span class="kt">int</span> <span class="n">val</span> <span class="o">=</span> <span class="o">((</span><span class="n">IntWritable</span><span class="o">)</span> <span class="n">value</span><span class="o">.</span><span class="na">getFieldValue</span><span class="o">(</span><span class="s">"i"</span><span class="o">)).</span><span class="na">get</span><span class="o">();</span>
+    <span class="n">i</span><span class="o">.</span><span class="na">set</span><span class="o">(</span><span class="n">val</span> <span class="o">*</span> <span class="mi">2</span><span class="o">);</span>
+    <span class="n">j</span><span class="o">.</span><span class="na">set</span><span class="o">(</span><span class="n">val</span> <span class="o">*</span> <span class="n">val</span><span class="o">);</span>
+    <span class="n">output</span><span class="o">.</span><span class="na">collect</span><span class="o">(</span><span class="n">keyWrapper</span><span class="o">,</span> <span class="n">valueWrapper</span><span class="o">);</span>
+  <span class="o">}</span>
 
-  public void configure(JobConf conf) { }
+  <span class="kd">public</span> <span class="kt">void</span> <span class="nf">configure</span><span class="o">(</span><span class="n">JobConf</span> <span class="n">conf</span><span class="o">)</span> <span class="o">{</span> <span class="o">}</span>
 
-  public void close() { }
-}
-</code></pre>
+  <span class="kd">public</span> <span class="kt">void</span> <span class="nf">close</span><span class="o">()</span> <span class="o">{</span> <span class="o">}</span>
+<span class="o">}</span>
+</code></pre></div></div>
 
 <p>The reducer code accesses the underlying OrcStructs by using the
 OrcKey.key and OrcValue.value fields.</p>
@@ -996,8 +996,6 @@ OrcKey.key and OrcValue.value fields.</p>
   
 
   
-    
-  
 
   
     
@@ -1017,29 +1015,25 @@ OrcKey.key and OrcValue.value fields.</p>
     
   
     
-      <li class=""><a href="/docs/index.html">Background</a></li>
+      <li class=""><a href="http://localhost:4000/docs/index.html">Background</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
   
     
-      <li class=""><a href="/docs/adopters.html">ORC Adopters</a></li>
+      <li class=""><a href="http://localhost:4000/docs/adopters.html">ORC Adopters</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -1071,15 +1065,13 @@ OrcKey.key and OrcValue.value fields.</p>
     
   
     
-      <li class=""><a href="/docs/types.html">Types</a></li>
+      <li class=""><a href="http://localhost:4000/docs/types.html">Types</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -1101,19 +1093,17 @@ OrcKey.key and OrcValue.value fields.</p>
     
   
     
-      <li class=""><a href="/docs/indexes.html">Indexes</a></li>
+      <li class=""><a href="http://localhost:4000/docs/indexes.html">Indexes</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
-      <li class=""><a href="/docs/acid.html">ACID support</a></li>
+      <li class=""><a href="http://localhost:4000/docs/acid.html">ACID support</a></li>
       
 
 
@@ -1128,8 +1118,6 @@ OrcKey.key and OrcValue.value fields.</p>
   
 
   
-    
-  
 
   
     
@@ -1137,15 +1125,13 @@ OrcKey.key and OrcValue.value fields.</p>
     
   
     
-      <li class=""><a href="/docs/building.html">Building ORC</a></li>
+      <li class=""><a href="http://localhost:4000/docs/building.html">Building ORC</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -1175,7 +1161,7 @@ OrcKey.key and OrcValue.value fields.</p>
     
   
     
-      <li class=""><a href="/docs/releases.html">Releases</a></li>
+      <li class=""><a href="http://localhost:4000/docs/releases.html">Releases</a></li>
       
 
 
@@ -1190,8 +1176,6 @@ OrcKey.key and OrcValue.value fields.</p>
   
 
   
-    
-  
 
   
     
@@ -1209,15 +1193,13 @@ OrcKey.key and OrcValue.value fields.</p>
     
   
     
-      <li class=""><a href="/docs/hive-ddl.html">Hive DDL</a></li>
+      <li class=""><a href="http://localhost:4000/docs/hive-ddl.html">Hive DDL</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -1233,7 +1215,7 @@ OrcKey.key and OrcValue.value fields.</p>
     
   
     
-      <li class=""><a href="/docs/hive-config.html">Hive Configuration</a></li>
+      <li class=""><a href="http://localhost:4000/docs/hive-config.html">Hive Configuration</a></li>
       
 
 
@@ -1248,8 +1230,6 @@ OrcKey.key and OrcValue.value fields.</p>
   
 
   
-    
-  
 
   
     
@@ -1275,15 +1255,13 @@ OrcKey.key and OrcValue.value fields.</p>
     
   
     
-      <li class="current"><a href="/docs/mapred.html">Using in MapRed</a></li>
+      <li class="current"><a href="http://localhost:4000/docs/mapred.html">Using in MapRed</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -1311,7 +1289,7 @@ OrcKey.key and OrcValue.value fields.</p>
     
   
     
-      <li class=""><a href="/docs/mapreduce.html">Using in MapReduce</a></li>
+      <li class=""><a href="http://localhost:4000/docs/mapreduce.html">Using in MapReduce</a></li>
       
 
 
@@ -1326,8 +1304,6 @@ OrcKey.key and OrcValue.value fields.</p>
   
 
   
-    
-  
 
   
     
@@ -1339,15 +1315,13 @@ OrcKey.key and OrcValue.value fields.</p>
     
   
     
-      <li class=""><a href="/docs/core-java.html">Using Core Java</a></li>
+      <li class=""><a href="http://localhost:4000/docs/core-java.html">Using Core Java</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -1357,7 +1331,7 @@ OrcKey.key and OrcValue.value fields.</p>
     
   
     
-      <li class=""><a href="/docs/core-cpp.html">Using Core C++</a></li>
+      <li class=""><a href="http://localhost:4000/docs/core-cpp.html">Using Core C++</a></li>
       
 
 
@@ -1372,8 +1346,6 @@ OrcKey.key and OrcValue.value fields.</p>
   
 
   
-    
-  
 
   
     
@@ -1387,15 +1359,13 @@ OrcKey.key and OrcValue.value fields.</p>
     
   
     
-      <li class=""><a href="/docs/cpp-tools.html">C++ Tools</a></li>
+      <li class=""><a href="http://localhost:4000/docs/cpp-tools.html">C++ Tools</a></li>
       
 
 
   
 
   
-    
-  
 
   
     
@@ -1419,7 +1389,7 @@ OrcKey.key and OrcValue.value fields.</p>
     
   
     
-      <li class=""><a href="/docs/java-tools.html">Java Tools</a></li>
+      <li class=""><a href="http://localhost:4000/docs/java-tools.html">Java Tools</a></li>