You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@orc.apache.org by om...@apache.org on 2018/05/02 15:35:26 UTC
orc git commit: Push site updates to deploy.
Repository: orc
Updated Branches:
refs/heads/asf-site da021334f -> be01d14ac
Push site updates to deploy.
Signed-off-by: Owen O'Malley <om...@apache.org>
Project: http://git-wip-us.apache.org/repos/asf/orc/repo
Commit: http://git-wip-us.apache.org/repos/asf/orc/commit/be01d14a
Tree: http://git-wip-us.apache.org/repos/asf/orc/tree/be01d14a
Diff: http://git-wip-us.apache.org/repos/asf/orc/diff/be01d14a
Branch: refs/heads/asf-site
Commit: be01d14ac25f6c241ff131afb3893a618fefa6b1
Parents: da02133
Author: Owen O'Malley <om...@apache.org>
Authored: Wed May 2 08:34:33 2018 -0700
Committer: Owen O'Malley <om...@apache.org>
Committed: Wed May 2 08:34:33 2018 -0700
----------------------------------------------------------------------
docs/hive-config.html | 7 +++++++
security.html | 4 ++--
specification/ORCv2.html | 21 ++++++---------------
3 files changed, 15 insertions(+), 17 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/orc/blob/be01d14a/docs/hive-config.html
----------------------------------------------------------------------
diff --git a/docs/hive-config.html b/docs/hive-config.html
index bc2f68c..159cdf7 100644
--- a/docs/hive-config.html
+++ b/docs/hive-config.html
@@ -886,6 +886,13 @@ with the same options.</p>
the compression level of higher level compression codec. Value can be
SPEED or COMPRESSION.</td>
</tr>
+<tr>
+ <td>orc.write.variable.length.blocks</td>
+ <td>false</td>
+ <td>Should the ORC writer use HDFS variable length blocks, if they are
+ available? If the new stripe would straddle a block, Hadoop is ≥ 2.7,
+ and this is enabled, it will end the block before the new stripe.</td>
+</tr>
</td></tr></table>
http://git-wip-us.apache.org/repos/asf/orc/blob/be01d14a/security.html
----------------------------------------------------------------------
diff --git a/security.html b/security.html
index 56e06ed..0db181b 100644
--- a/security.html
+++ b/security.html
@@ -102,12 +102,12 @@ address. All mail sent to this address that does not relate to an
undisclosed security problem in Apache ORC will be ignored.</p>
<p>The ORC security mailing list address is:
-<a href="mailto:private@orc.apache.org">private@orc.apache.org</a>.
+<a href="mailto:security@orc.apache.org">security@orc.apache.org</a>.
This is a private mailing list and only members of the ORC project
are subscribed.</p>
<p>Please note that we do not use a team GnuPG key. If you wish to
-encrypt your e-mail to private@orc.apache.org then please use the GnuPG
+encrypt your e-mail to security@orc.apache.org then please use the GnuPG
keys from <a href="https://people.apache.org/keys/group/orc.asc">ORC GPG keys</a> for
the members of the
<a href="https://people.apache.org/phonebook.html?ctte=orc">ORC PMC</a>.</p>
http://git-wip-us.apache.org/repos/asf/orc/blob/be01d14a/specification/ORCv2.html
----------------------------------------------------------------------
diff --git a/specification/ORCv2.html b/specification/ORCv2.html
index b78fc0a..db8aecf 100644
--- a/specification/ORCv2.html
+++ b/specification/ORCv2.html
@@ -92,7 +92,7 @@ developers on the project.</p>
<p>The list of things that we plan to change:</p>
<ul>
- <li>Create a decimal representation with fixed scale using rle.</li>
+ <li>Move decimal encoding to RLEv3 and remove variable length encoding.</li>
<li>Create a better float/double encoding that splits mantissa and
exponent.</li>
<li>Create a dictionary encoding for float, double, and decimal.</li>
@@ -1283,13 +1283,10 @@ value.</p>
<h2 id="decimal-columns">Decimal Columns</h2>
-<p>Decimal was introduced in Hive 0.11 with infinite precision (the total
-number of digits). In Hive 0.13, the definition was change to limit
-the precision to a maximum of 38 digits, which conveniently uses 127
-bits plus a sign bit. The current encoding of decimal columns stores
-the integer representation of the value as an unbounded length zigzag
-encoded base 128 varint. The scale is stored in the SECONDARY stream
-as an signed integer.</p>
+<p>Since Hive 0.13, all decimals have had fixed precision and scale.
+The goal is to use RLEv3 for the value and use the fixed scale from
+the type. As an interim solution, we are using RLE v2 for short decimals
+(precision <= 18) and the old encoding for long decimals.</p>
<table>
<thead>
@@ -1311,13 +1308,7 @@ as an signed integer.</p>
<td style="text-align: left"> </td>
<td style="text-align: left">DATA</td>
<td style="text-align: left">No</td>
- <td style="text-align: left">Unbounded base 128 varints</td>
- </tr>
- <tr>
- <td style="text-align: left"> </td>
- <td style="text-align: left">SECONDARY</td>
- <td style="text-align: left">No</td>
- <td style="text-align: left">Unsigned Integer RLE v1</td>
+ <td style="text-align: left">Signed Integer RLE v2</td>
</tr>
<tr>
<td style="text-align: left">DIRECT_V2</td>