You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tajo.apache.org by ji...@apache.org on 2016/05/19 02:00:27 UTC
svn commit: r1744489 [42/43] - in /tajo/site/docs: 0.11.3/ 0.11.3/_sources/
0.11.3/_sources/backup_and_restore/ 0.11.3/_sources/configuration/
0.11.3/_sources/functions/ 0.11.3/_sources/index/
0.11.3/_sources/partitioning/ 0.11.3/_sources/sql_language/...
Modified: tajo/site/docs/current/table_management/json.html
URL: http://svn.apache.org/viewvc/tajo/site/docs/current/table_management/json.html?rev=1744489&r1=1744488&r2=1744489&view=diff
==============================================================================
--- tajo/site/docs/current/table_management/json.html (original)
+++ tajo/site/docs/current/table_management/json.html Thu May 19 02:00:25 2016
@@ -7,7 +7,7 @@
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
- <title>JSON — Apache Tajo 0.11.0 documentation</title>
+ <title>JSON — Apache Tajo 0.11.3 documentation</title>
@@ -28,7 +28,7 @@
- <link rel="top" title="Apache Tajo 0.11.0 documentation" href="../index.html"/>
+ <link rel="top" title="Apache Tajo 0.11.3 documentation" href="../index.html"/>
<link rel="up" title="Data Formats" href="data_formats.html"/>
<link rel="next" title="RCFile" href="rcfile.html"/>
<link rel="prev" title="TEXT" href="text.html"/>
@@ -217,9 +217,9 @@
Tajo supports JSON as its data format. In this section, you will get an overview of how to create JSON tables and query on them.</p>
<div class="section" id="how-to-create-a-json-table">
<h2>How to Create a JSON Table ?<a class="headerlink" href="#how-to-create-a-json-table" title="Permalink to this headline">¶</a></h2>
-<p>You can create a JSON table using the <code class="docutils literal"><span class="pre">CREATE</span> <span class="pre">TABLE</span></code> statement. (For more information, please refer to <a class="reference internal" href="../sql_language/ddl.html"><em>Data Definition Language</em></a>.)
+<p>You can create a JSON table using the <tt class="docutils literal"><span class="pre">CREATE</span> <span class="pre">TABLE</span></tt> statement. (For more information, please refer to <a class="reference internal" href="../sql_language/ddl.html"><em>Data Definition Language</em></a>.)
For example, please consider an example data as follows:</p>
-<div class="highlight-bash"><div class="highlight"><pre>$ hdfs dfs -cat /table1/table.json
+<div class="highlight-bash"><div class="highlight"><pre><span class="nv">$ </span>hdfs dfs -cat /table1/table.json
<span class="o">{</span> <span class="s2">"title"</span> : <span class="s2">"Hand of the King"</span>, <span class="s2">"name"</span> : <span class="o">{</span> <span class="s2">"first_name"</span>: <span class="s2">"Eddard"</span>, <span class="s2">"last_name"</span>: <span class="s2">"Stark"</span><span class="o">}}</span>
<span class="o">{</span> <span class="s2">"title"</span> : <span class="s2">"Assassin"</span>, <span class="s2">"name"</span> : <span class="o">{</span> <span class="s2">"first_name"</span>: <span class="s2">"Arya"</span>, <span class="s2">"last_name"</span>: <span class="s2">"Stark"</span><span class="o">}}</span>
<span class="o">{</span> <span class="s2">"title"</span> : <span class="s2">"Dancing Master"</span>, <span class="s2">"name"</span> : <span class="o">{</span> <span class="s2">"first_name"</span>: <span class="s2">"Syrio"</span>, <span class="s2">"last_name"</span>: <span class="s2">"Forel"</span><span class="o">}}</span>
@@ -249,29 +249,29 @@ Second is a simpler alternative to allev
</div>
<div class="admonition warning">
<p class="first admonition-title">Warning</p>
-<p class="last">If you create a table with the second way, every column is assumed as the <code class="docutils literal"><span class="pre">TEXT</span></code> type.
+<p class="last">If you create a table with the second way, every column is assumed as the <tt class="docutils literal"><span class="pre">TEXT</span></tt> type.
So, you need to perform type casting if you want to handle them as other types.</p>
</div>
</div>
<div class="section" id="physical-properties">
<h2>Physical Properties<a class="headerlink" href="#physical-properties" title="Permalink to this headline">¶</a></h2>
<p>Some table storage formats provide parameters for enabling or disabling features and adjusting physical parameters.
-The <code class="docutils literal"><span class="pre">WITH</span></code> clause in the CREATE TABLE statement allows users to set those parameters.</p>
+The <tt class="docutils literal"><span class="pre">WITH</span></tt> clause in the CREATE TABLE statement allows users to set those parameters.</p>
<p>The JSON format provides the following physical properties.</p>
<ul class="simple">
-<li><code class="docutils literal"><span class="pre">text.delimiter</span></code>: delimiter character. <code class="docutils literal"><span class="pre">|</span></code> or <code class="docutils literal"><span class="pre">\u0001</span></code> is usually used, and the default field delimiter is <code class="docutils literal"><span class="pre">|</span></code>.</li>
-<li><code class="docutils literal"><span class="pre">text.null</span></code>: <code class="docutils literal"><span class="pre">NULL</span></code> character. The default <code class="docutils literal"><span class="pre">NULL</span></code> character is an empty string <code class="docutils literal"><span class="pre">''</span></code>. Hive’s default <code class="docutils literal"><span class="pre">NULL</span></code> character is <code class="docutils literal"><span class="pre">'\\N'</span></code>.</li>
-<li><code class="docutils literal"><span class="pre">compression.codec</span></code>: Compression codec. You can enable compression feature and set specified compression algorithm. The compression algorithm used to compress files. The compression codec name should be the fully qualified class name inherited from <a class="reference external" href="https://hadoop.apache.org/docs/current/api/org/apache/hadoop/io/compress/CompressionCodec.html">org.apache.hadoop.io.compress.CompressionCodec</a>. By default, compression is disabled.</li>
-<li><code class="docutils literal"><span class="pre">timezone</span></code>: the time zone that the table uses for writting. When table rows are read or written, <code class="docutils literal"><span class="pre">`timestamp`</span></code> and <code class="docutils literal"><span class="pre">`time`</span></code> column values are adjusted by this timezone if it is set. Time zone can be an abbreviation form like ‘PST’ or ‘DST’. Also, it accepts an offset-based form like ‘UTC+9’ or a location-based form like ‘Asia/Seoul’.</li>
-<li><code class="docutils literal"><span class="pre">text.error-tolerance.max-num</span></code>: the maximum number of permissible parsing errors. This value should be an integer value. By default, <code class="docutils literal"><span class="pre">text.error-tolerance.max-num</span></code> is <code class="docutils literal"><span class="pre">0</span></code>. According to the value, parsing errors will be handled in different ways.<ul>
-<li>If <code class="docutils literal"><span class="pre">text.error-tolerance.max-num</span> <span class="pre"><</span> <span class="pre">0</span></code>, all parsing errors are ignored.</li>
-<li>If <code class="docutils literal"><span class="pre">text.error-tolerance.max-num</span> <span class="pre">==</span> <span class="pre">0</span></code>, any parsing error is not allowed. If any error occurs, the query will be failed. (default)</li>
-<li>If <code class="docutils literal"><span class="pre">text.error-tolerance.max-num</span> <span class="pre">></span> <span class="pre">0</span></code>, the given number of parsing errors in each task will be pemissible.</li>
+<li><tt class="docutils literal"><span class="pre">text.delimiter</span></tt>: delimiter character. <tt class="docutils literal"><span class="pre">|</span></tt> or <tt class="docutils literal"><span class="pre">\u0001</span></tt> is usually used, and the default field delimiter is <tt class="docutils literal"><span class="pre">|</span></tt>.</li>
+<li><tt class="docutils literal"><span class="pre">text.null</span></tt>: <tt class="docutils literal"><span class="pre">NULL</span></tt> character. The default <tt class="docutils literal"><span class="pre">NULL</span></tt> character is an empty string <tt class="docutils literal"><span class="pre">''</span></tt>. Hive’s default <tt class="docutils literal"><span class="pre">NULL</span></tt> character is <tt class="docutils literal"><span class="pre">'\\N'</span></tt>.</li>
+<li><tt class="docutils literal"><span class="pre">compression.codec</span></tt>: Compression codec. You can enable compression feature and set specified compression algorithm. The compression algorithm used to compress files. The compression codec name should be the fully qualified class name inherited from <a class="reference external" href="https://hadoop.apache.org/docs/current/api/org/apache/hadoop/io/compress/CompressionCodec.html">org.apache.hadoop.io.compress.CompressionCodec</a>. By default, compression is disabled.</li>
+<li><tt class="docutils literal"><span class="pre">timezone</span></tt>: the time zone that the table uses for writting. When table rows are read or written, <tt class="docutils literal"><span class="pre">`timestamp`</span></tt> and <tt class="docutils literal"><span class="pre">`time`</span></tt> column values are adjusted by this timezone if it is set. Time zone can be an abbreviation form like ‘PST’ or ‘DST’. Also, it accepts an offset-based form like ‘UTC+9’ or a location-based form like ‘Asia/Seoul’.</li>
+<li><tt class="docutils literal"><span class="pre">text.error-tolerance.max-num</span></tt>: the maximum number of permissible parsing errors. This value should be an integer value. By default, <tt class="docutils literal"><span class="pre">text.error-tolerance.max-num</span></tt> is <tt class="docutils literal"><span class="pre">0</span></tt>. According to the value, parsing errors will be handled in different ways.<ul>
+<li>If <tt class="docutils literal"><span class="pre">text.error-tolerance.max-num</span> <span class="pre"><</span> <span class="pre">0</span></tt>, all parsing errors are ignored.</li>
+<li>If <tt class="docutils literal"><span class="pre">text.error-tolerance.max-num</span> <span class="pre">==</span> <span class="pre">0</span></tt>, any parsing error is not allowed. If any error occurs, the query will be failed. (default)</li>
+<li>If <tt class="docutils literal"><span class="pre">text.error-tolerance.max-num</span> <span class="pre">></span> <span class="pre">0</span></tt>, the given number of parsing errors in each task will be pemissible.</li>
</ul>
</li>
-<li><code class="docutils literal"><span class="pre">text.skip.headerlines</span></code>: Number of header lines to be skipped. Some text files often have a header which has a kind of metadata(e.g.: column names), thus this option can be useful.</li>
+<li><tt class="docutils literal"><span class="pre">text.skip.headerlines</span></tt>: Number of header lines to be skipped. Some text files often have a header which has a kind of metadata(e.g.: column names), thus this option can be useful.</li>
</ul>
-<p>The following example is to set a custom field delimiter, <code class="docutils literal"><span class="pre">NULL</span></code> character, and compression codec:</p>
+<p>The following example is to set a custom field delimiter, <tt class="docutils literal"><span class="pre">NULL</span></tt> character, and compression codec:</p>
<div class="highlight-sql"><div class="highlight"><pre><span class="k">CREATE</span> <span class="k">TABLE</span> <span class="n">table1</span> <span class="p">(</span>
<span class="n">id</span> <span class="nb">int</span><span class="p">,</span>
<span class="n">name</span> <span class="nb">text</span><span class="p">,</span>
@@ -284,16 +284,16 @@ The <code class="docutils literal"><span
</div>
<div class="admonition warning">
<p class="first admonition-title">Warning</p>
-<p class="last">Be careful when using <code class="docutils literal"><span class="pre">\n</span></code> as the field delimiter because <em>TEXT</em> format tables use <code class="docutils literal"><span class="pre">\n</span></code> as the line delimiter.
+<p class="last">Be careful when using <tt class="docutils literal"><span class="pre">\n</span></tt> as the field delimiter because <em>TEXT</em> format tables use <tt class="docutils literal"><span class="pre">\n</span></tt> as the line delimiter.
At the moment, Tajo does not provide a way to specify the line delimiter.</p>
</div>
</div>
<div class="section" id="null-value-handling-issues">
<h2>Null Value Handling Issues<a class="headerlink" href="#null-value-handling-issues" title="Permalink to this headline">¶</a></h2>
-<p>In default, <code class="docutils literal"><span class="pre">NULL</span></code> character in <em>TEXT</em> format is an empty string <code class="docutils literal"><span class="pre">''</span></code>.
-In other words, an empty field is basically recognized as a <code class="docutils literal"><span class="pre">NULL</span></code> value in Tajo.
-If a field domain is <code class="docutils literal"><span class="pre">TEXT</span></code>, an empty field is recognized as a string value <code class="docutils literal"><span class="pre">''</span></code> instead of <code class="docutils literal"><span class="pre">NULL</span></code> value.
-Besides, You can also use your own <code class="docutils literal"><span class="pre">NULL</span></code> character by specifying a physical property <code class="docutils literal"><span class="pre">text.null</span></code>.</p>
+<p>In default, <tt class="docutils literal"><span class="pre">NULL</span></tt> character in <em>TEXT</em> format is an empty string <tt class="docutils literal"><span class="pre">''</span></tt>.
+In other words, an empty field is basically recognized as a <tt class="docutils literal"><span class="pre">NULL</span></tt> value in Tajo.
+If a field domain is <tt class="docutils literal"><span class="pre">TEXT</span></tt>, an empty field is recognized as a string value <tt class="docutils literal"><span class="pre">''</span></tt> instead of <tt class="docutils literal"><span class="pre">NULL</span></tt> value.
+Besides, You can also use your own <tt class="docutils literal"><span class="pre">NULL</span></tt> character by specifying a physical property <tt class="docutils literal"><span class="pre">text.null</span></tt>.</p>
</div>
</div>
@@ -315,7 +315,7 @@ Besides, You can also use your own <code
<div role="contentinfo">
<p>
- © Copyright 2015, Apache Tajo Team.
+ © Copyright 2016, Apache Tajo Team.
</p>
</div>
@@ -335,7 +335,7 @@ Besides, You can also use your own <code
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT:'../',
- VERSION:'0.11.0',
+ VERSION:'0.11.3',
COLLAPSE_INDEX:false,
FILE_SUFFIX:'.html',
HAS_SOURCE: true
Modified: tajo/site/docs/current/table_management/orc.html
URL: http://svn.apache.org/viewvc/tajo/site/docs/current/table_management/orc.html?rev=1744489&r1=1744488&r2=1744489&view=diff
==============================================================================
--- tajo/site/docs/current/table_management/orc.html (original)
+++ tajo/site/docs/current/table_management/orc.html Thu May 19 02:00:25 2016
@@ -7,7 +7,7 @@
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
- <title>ORC — Apache Tajo 0.11.0 documentation</title>
+ <title>ORC — Apache Tajo 0.11.3 documentation</title>
@@ -28,7 +28,7 @@
- <link rel="top" title="Apache Tajo 0.11.0 documentation" href="../index.html"/>
+ <link rel="top" title="Apache Tajo 0.11.3 documentation" href="../index.html"/>
<link rel="up" title="Data Formats" href="data_formats.html"/>
<link rel="next" title="SequenceFile" href="sequencefile.html"/>
<link rel="prev" title="Parquet" href="parquet.html"/>
@@ -218,8 +218,8 @@ writing, and processing data.
For more details, please refer to <a class="reference external" href="https://cwiki.apache.org/confluence/display/Hive/LanguageManual+ORC">ORC Files</a> at Hive wiki.</p>
<div class="section" id="how-to-create-an-orc-table">
<h2>How to Create an ORC Table?<a class="headerlink" href="#how-to-create-an-orc-table" title="Permalink to this headline">¶</a></h2>
-<p>If you are not familiar with <code class="docutils literal"><span class="pre">CREATE</span> <span class="pre">TABLE</span></code> statement, please refer to Data Definition Language <a class="reference internal" href="../sql_language/ddl.html"><em>Data Definition Language</em></a>.</p>
-<p>In order to specify a certain file format for your table, you need to use the <code class="docutils literal"><span class="pre">USING</span></code> clause in your <code class="docutils literal"><span class="pre">CREATE</span> <span class="pre">TABLE</span></code>
+<p>If you are not familiar with <tt class="docutils literal"><span class="pre">CREATE</span> <span class="pre">TABLE</span></tt> statement, please refer to Data Definition Language <a class="reference internal" href="../sql_language/ddl.html"><em>Data Definition Language</em></a>.</p>
+<p>In order to specify a certain file format for your table, you need to use the <tt class="docutils literal"><span class="pre">USING</span></tt> clause in your <tt class="docutils literal"><span class="pre">CREATE</span> <span class="pre">TABLE</span></tt>
statement. Below is an example statement for creating a table using orc files.</p>
<div class="highlight-sql"><div class="highlight"><pre><span class="k">CREATE</span> <span class="k">TABLE</span> <span class="n">table1</span> <span class="p">(</span>
<span class="n">id</span> <span class="nb">int</span><span class="p">,</span>
@@ -233,14 +233,14 @@ statement. Below is an example statement
<div class="section" id="physical-properties">
<h2>Physical Properties<a class="headerlink" href="#physical-properties" title="Permalink to this headline">¶</a></h2>
<p>Some table storage formats provide parameters for enabling or disabling features and adjusting physical parameters.
-The <code class="docutils literal"><span class="pre">WITH</span></code> clause in the CREATE TABLE statement allows users to set those parameters.</p>
+The <tt class="docutils literal"><span class="pre">WITH</span></tt> clause in the CREATE TABLE statement allows users to set those parameters.</p>
<p>Now, ORC file provides the following physical properties.</p>
<ul class="simple">
-<li><code class="docutils literal"><span class="pre">orc.max.merge.distance</span></code>: When ORC file is read, if stripes are too closer and the distance is lower than this value, they are merged and read at once. Default is 1MB.</li>
-<li><code class="docutils literal"><span class="pre">orc.stripe.size</span></code>: It decides size of each stripe. Default is 64MB.</li>
-<li><code class="docutils literal"><span class="pre">orc.compression.kind</span></code>: It means the compression algorithm used to compress and write data. It should be one of <code class="docutils literal"><span class="pre">none</span></code>, <code class="docutils literal"><span class="pre">snappy</span></code>, <code class="docutils literal"><span class="pre">zlib</span></code>. Default is <code class="docutils literal"><span class="pre">none</span></code>.</li>
-<li><code class="docutils literal"><span class="pre">orc.buffer.size</span></code>: It decides size of writing buffer. Default is 256KB.</li>
-<li><code class="docutils literal"><span class="pre">orc.rowindex.stride</span></code>: Define the default ORC index stride in number of rows. (Stride is the number of rows an index entry represents.) Default is 10000.</li>
+<li><tt class="docutils literal"><span class="pre">orc.max.merge.distance</span></tt>: When ORC file is read, if stripes are too closer and the distance is lower than this value, they are merged and read at once. Default is 1MB.</li>
+<li><tt class="docutils literal"><span class="pre">orc.stripe.size</span></tt>: It decides size of each stripe. Default is 64MB.</li>
+<li><tt class="docutils literal"><span class="pre">orc.compression.kind</span></tt>: It means the compression algorithm used to compress and write data. It should be one of <tt class="docutils literal"><span class="pre">none</span></tt>, <tt class="docutils literal"><span class="pre">snappy</span></tt>, <tt class="docutils literal"><span class="pre">zlib</span></tt>. Default is <tt class="docutils literal"><span class="pre">none</span></tt>.</li>
+<li><tt class="docutils literal"><span class="pre">orc.buffer.size</span></tt>: It decides size of writing buffer. Default is 256KB.</li>
+<li><tt class="docutils literal"><span class="pre">orc.rowindex.stride</span></tt>: Define the default ORC index stride in number of rows. (Stride is the number of rows an index entry represents.) Default is 10000.</li>
</ul>
</div>
<div class="section" id="compatibility-issues-with-apache-hive">
@@ -268,7 +268,7 @@ We are currently working on adding suppo
<div role="contentinfo">
<p>
- © Copyright 2015, Apache Tajo Team.
+ © Copyright 2016, Apache Tajo Team.
</p>
</div>
@@ -288,7 +288,7 @@ We are currently working on adding suppo
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT:'../',
- VERSION:'0.11.0',
+ VERSION:'0.11.3',
COLLAPSE_INDEX:false,
FILE_SUFFIX:'.html',
HAS_SOURCE: true
Modified: tajo/site/docs/current/table_management/parquet.html
URL: http://svn.apache.org/viewvc/tajo/site/docs/current/table_management/parquet.html?rev=1744489&r1=1744488&r2=1744489&view=diff
==============================================================================
--- tajo/site/docs/current/table_management/parquet.html (original)
+++ tajo/site/docs/current/table_management/parquet.html Thu May 19 02:00:25 2016
@@ -7,7 +7,7 @@
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
- <title>Parquet — Apache Tajo 0.11.0 documentation</title>
+ <title>Parquet — Apache Tajo 0.11.3 documentation</title>
@@ -28,7 +28,7 @@
- <link rel="top" title="Apache Tajo 0.11.0 documentation" href="../index.html"/>
+ <link rel="top" title="Apache Tajo 0.11.3 documentation" href="../index.html"/>
<link rel="up" title="Data Formats" href="data_formats.html"/>
<link rel="next" title="ORC" href="orc.html"/>
<link rel="prev" title="RCFile" href="rcfile.html"/>
@@ -219,8 +219,8 @@ regardless of the choice of data process
For more details, please refer to <a class="reference external" href="http://parquet.io/">Parquet File Format</a>.</p>
<div class="section" id="how-to-create-a-parquet-table">
<h2>How to Create a Parquet Table?<a class="headerlink" href="#how-to-create-a-parquet-table" title="Permalink to this headline">¶</a></h2>
-<p>If you are not familiar with <code class="docutils literal"><span class="pre">CREATE</span> <span class="pre">TABLE</span></code> statement, please refer to Data Definition Language <a class="reference internal" href="../sql_language/ddl.html"><em>Data Definition Language</em></a>.</p>
-<p>In order to specify a certain file format for your table, you need to use the <code class="docutils literal"><span class="pre">USING</span></code> clause in your <code class="docutils literal"><span class="pre">CREATE</span> <span class="pre">TABLE</span></code>
+<p>If you are not familiar with <tt class="docutils literal"><span class="pre">CREATE</span> <span class="pre">TABLE</span></tt> statement, please refer to Data Definition Language <a class="reference internal" href="../sql_language/ddl.html"><em>Data Definition Language</em></a>.</p>
+<p>In order to specify a certain file format for your table, you need to use the <tt class="docutils literal"><span class="pre">USING</span></tt> clause in your <tt class="docutils literal"><span class="pre">CREATE</span> <span class="pre">TABLE</span></tt>
statement. Below is an example statement for creating a table using parquet files.</p>
<div class="highlight-sql"><div class="highlight"><pre><span class="k">CREATE</span> <span class="k">TABLE</span> <span class="n">table1</span> <span class="p">(</span>
<span class="n">id</span> <span class="nb">int</span><span class="p">,</span>
@@ -234,13 +234,13 @@ statement. Below is an example statement
<div class="section" id="physical-properties">
<h2>Physical Properties<a class="headerlink" href="#physical-properties" title="Permalink to this headline">¶</a></h2>
<p>Some table storage formats provide parameters for enabling or disabling features and adjusting physical parameters.
-The <code class="docutils literal"><span class="pre">WITH</span></code> clause in the CREATE TABLE statement allows users to set those parameters.</p>
+The <tt class="docutils literal"><span class="pre">WITH</span></tt> clause in the CREATE TABLE statement allows users to set those parameters.</p>
<p>Now, Parquet file provides the following physical properties.</p>
<ul class="simple">
-<li><code class="docutils literal"><span class="pre">parquet.block.size</span></code>: The block size is the size of a row group being buffered in memory. This limits the memory usage when writing. Larger values will improve the I/O when reading but consume more memory when writing. Default size is 134217728 bytes (= 128 * 1024 * 1024).</li>
-<li><code class="docutils literal"><span class="pre">parquet.page.size</span></code>: The page size is for compression. When reading, each page can be decompressed independently. A block is composed of pages. The page is the smallest unit that must be read fully to access a single record. If this value is too small, the compression will deteriorate. Default size is 1048576 bytes (= 1 * 1024 * 1024).</li>
-<li><code class="docutils literal"><span class="pre">parquet.compression</span></code>: The compression algorithm used to compress pages. It should be one of <code class="docutils literal"><span class="pre">uncompressed</span></code>, <code class="docutils literal"><span class="pre">snappy</span></code>, <code class="docutils literal"><span class="pre">gzip</span></code>, <code class="docutils literal"><span class="pre">lzo</span></code>. Default is <code class="docutils literal"><span class="pre">uncompressed</span></code>.</li>
-<li><code class="docutils literal"><span class="pre">parquet.enable.dictionary</span></code>: The boolean value is to enable/disable dictionary encoding. It should be one of either <code class="docutils literal"><span class="pre">true</span></code> or <code class="docutils literal"><span class="pre">false</span></code>. Default is <code class="docutils literal"><span class="pre">true</span></code>.</li>
+<li><tt class="docutils literal"><span class="pre">parquet.block.size</span></tt>: The block size is the size of a row group being buffered in memory. This limits the memory usage when writing. Larger values will improve the I/O when reading but consume more memory when writing. Default size is 134217728 bytes (= 128 * 1024 * 1024).</li>
+<li><tt class="docutils literal"><span class="pre">parquet.page.size</span></tt>: The page size is for compression. When reading, each page can be decompressed independently. A block is composed of pages. The page is the smallest unit that must be read fully to access a single record. If this value is too small, the compression will deteriorate. Default size is 1048576 bytes (= 1 * 1024 * 1024).</li>
+<li><tt class="docutils literal"><span class="pre">parquet.compression</span></tt>: The compression algorithm used to compress pages. It should be one of <tt class="docutils literal"><span class="pre">uncompressed</span></tt>, <tt class="docutils literal"><span class="pre">snappy</span></tt>, <tt class="docutils literal"><span class="pre">gzip</span></tt>, <tt class="docutils literal"><span class="pre">lzo</span></tt>. Default is <tt class="docutils literal"><span class="pre">uncompressed</span></tt>.</li>
+<li><tt class="docutils literal"><span class="pre">parquet.enable.dictionary</span></tt>: The boolean value is to enable/disable dictionary encoding. It should be one of either <tt class="docutils literal"><span class="pre">true</span></tt> or <tt class="docutils literal"><span class="pre">false</span></tt>. Default is <tt class="docutils literal"><span class="pre">true</span></tt>.</li>
</ul>
</div>
<div class="section" id="compatibility-issues-with-apache-hive">
@@ -269,7 +269,7 @@ However, we are currently working on add
<div role="contentinfo">
<p>
- © Copyright 2015, Apache Tajo Team.
+ © Copyright 2016, Apache Tajo Team.
</p>
</div>
@@ -289,7 +289,7 @@ However, we are currently working on add
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT:'../',
- VERSION:'0.11.0',
+ VERSION:'0.11.3',
COLLAPSE_INDEX:false,
FILE_SUFFIX:'.html',
HAS_SOURCE: true
Modified: tajo/site/docs/current/table_management/rcfile.html
URL: http://svn.apache.org/viewvc/tajo/site/docs/current/table_management/rcfile.html?rev=1744489&r1=1744488&r2=1744489&view=diff
==============================================================================
--- tajo/site/docs/current/table_management/rcfile.html (original)
+++ tajo/site/docs/current/table_management/rcfile.html Thu May 19 02:00:25 2016
@@ -7,7 +7,7 @@
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
- <title>RCFile — Apache Tajo 0.11.0 documentation</title>
+ <title>RCFile — Apache Tajo 0.11.3 documentation</title>
@@ -28,7 +28,7 @@
- <link rel="top" title="Apache Tajo 0.11.0 documentation" href="../index.html"/>
+ <link rel="top" title="Apache Tajo 0.11.3 documentation" href="../index.html"/>
<link rel="up" title="Data Formats" href="data_formats.html"/>
<link rel="next" title="Parquet" href="parquet.html"/>
<link rel="prev" title="JSON" href="json.html"/>
@@ -217,8 +217,8 @@
which shares many similarities with SequenceFile.</p>
<div class="section" id="how-to-create-a-rcfile-table">
<h2>How to Create a RCFile Table?<a class="headerlink" href="#how-to-create-a-rcfile-table" title="Permalink to this headline">¶</a></h2>
-<p>If you are not familiar with the <code class="docutils literal"><span class="pre">CREATE</span> <span class="pre">TABLE</span></code> statement, please refer to the Data Definition Language <a class="reference internal" href="../sql_language/ddl.html"><em>Data Definition Language</em></a>.</p>
-<p>In order to specify a certain file format for your table, you need to use the <code class="docutils literal"><span class="pre">USING</span></code> clause in your <code class="docutils literal"><span class="pre">CREATE</span> <span class="pre">TABLE</span></code>
+<p>If you are not familiar with the <tt class="docutils literal"><span class="pre">CREATE</span> <span class="pre">TABLE</span></tt> statement, please refer to the Data Definition Language <a class="reference internal" href="../sql_language/ddl.html"><em>Data Definition Language</em></a>.</p>
+<p>In order to specify a certain file format for your table, you need to use the <tt class="docutils literal"><span class="pre">USING</span></tt> clause in your <tt class="docutils literal"><span class="pre">CREATE</span> <span class="pre">TABLE</span></tt>
statement. Below is an example statement for creating a table using RCFile.</p>
<div class="highlight-sql"><div class="highlight"><pre><span class="k">CREATE</span> <span class="k">TABLE</span> <span class="n">table1</span> <span class="p">(</span>
<span class="n">id</span> <span class="nb">int</span><span class="p">,</span>
@@ -232,12 +232,12 @@ statement. Below is an example statement
<div class="section" id="physical-properties">
<h2>Physical Properties<a class="headerlink" href="#physical-properties" title="Permalink to this headline">¶</a></h2>
<p>Some table storage formats provide parameters for enabling or disabling features and adjusting physical parameters.
-The <code class="docutils literal"><span class="pre">WITH</span></code> clause in the CREATE TABLE statement allows users to set those parameters.</p>
+The <tt class="docutils literal"><span class="pre">WITH</span></tt> clause in the CREATE TABLE statement allows users to set those parameters.</p>
<p>Now, the RCFile storage type provides the following physical properties.</p>
<ul class="simple">
-<li><code class="docutils literal"><span class="pre">rcfile.serde</span></code> : custom (De)serializer class. <code class="docutils literal"><span class="pre">org.apache.tajo.storage.BinarySerializerDeserializer</span></code> is the default (de)serializer class.</li>
-<li><code class="docutils literal"><span class="pre">rcfile.null</span></code> : NULL character. It is only used when a table uses <code class="docutils literal"><span class="pre">org.apache.tajo.storage.TextSerializerDeserializer</span></code>. The default NULL character is an empty string <code class="docutils literal"><span class="pre">''</span></code>. Hive’s default NULL character is <code class="docutils literal"><span class="pre">'\\N'</span></code>.</li>
-<li><code class="docutils literal"><span class="pre">compression.codec</span></code> : Compression codec. You can enable compression feature and set specified compression algorithm. The compression algorithm used to compress files. The compression codec name should be the fully qualified class name inherited from <a class="reference external" href="https://hadoop.apache.org/docs/current/api/org/apache/hadoop/io/compress/CompressionCodec.html">org.apache.hadoop.io.compress.CompressionCodec</a>. By default, compression is disabled.</li>
+<li><tt class="docutils literal"><span class="pre">rcfile.serde</span></tt> : custom (De)serializer class. <tt class="docutils literal"><span class="pre">org.apache.tajo.storage.BinarySerializerDeserializer</span></tt> is the default (de)serializer class.</li>
+<li><tt class="docutils literal"><span class="pre">rcfile.null</span></tt> : NULL character. It is only used when a table uses <tt class="docutils literal"><span class="pre">org.apache.tajo.storage.TextSerializerDeserializer</span></tt>. The default NULL character is an empty string <tt class="docutils literal"><span class="pre">''</span></tt>. Hive’s default NULL character is <tt class="docutils literal"><span class="pre">'\\N'</span></tt>.</li>
+<li><tt class="docutils literal"><span class="pre">compression.codec</span></tt> : Compression codec. You can enable compression feature and set specified compression algorithm. The compression algorithm used to compress files. The compression codec name should be the fully qualified class name inherited from <a class="reference external" href="https://hadoop.apache.org/docs/current/api/org/apache/hadoop/io/compress/CompressionCodec.html">org.apache.hadoop.io.compress.CompressionCodec</a>. By default, compression is disabled.</li>
</ul>
<p>The following is an example for creating a table using RCFile that uses compression.</p>
<div class="highlight-sql"><div class="highlight"><pre><span class="k">CREATE</span> <span class="k">TABLE</span> <span class="n">table1</span> <span class="p">(</span>
@@ -253,14 +253,14 @@ The <code class="docutils literal"><span
<h2>RCFile (De)serializers<a class="headerlink" href="#rcfile-de-serializers" title="Permalink to this headline">¶</a></h2>
<p>Tajo provides two built-in (De)serializer for RCFile:</p>
<ul class="simple">
-<li><code class="docutils literal"><span class="pre">org.apache.tajo.storage.TextSerializerDeserializer</span></code>: stores column values in a plain-text form.</li>
-<li><code class="docutils literal"><span class="pre">org.apache.tajo.storage.BinarySerializerDeserializer</span></code>: stores column values in a binary file format.</li>
+<li><tt class="docutils literal"><span class="pre">org.apache.tajo.storage.TextSerializerDeserializer</span></tt>: stores column values in a plain-text form.</li>
+<li><tt class="docutils literal"><span class="pre">org.apache.tajo.storage.BinarySerializerDeserializer</span></tt>: stores column values in a binary file format.</li>
</ul>
<p>The RCFile format can store some metadata in the RCFile header. Tajo writes the (de)serializer class name into
the metadata header of each RCFile when the RCFile is created in Tajo.</p>
<div class="admonition note">
<p class="first admonition-title">Note</p>
-<p class="last"><code class="docutils literal"><span class="pre">org.apache.tajo.storage.BinarySerializerDeserializer</span></code> is the default (de) serializer for RCFile.</p>
+<p class="last"><tt class="docutils literal"><span class="pre">org.apache.tajo.storage.BinarySerializerDeserializer</span></tt> is the default (de) serializer for RCFile.</p>
</div>
</div>
<div class="section" id="compatibility-issues-with-apache-hive">
@@ -271,16 +271,16 @@ In other words, Tajo can process RCFiles
by setting a physical property.</p>
<p>In Hive, there are two SerDe, and they correspond to the following (de)serializer in Tajo.</p>
<ul class="simple">
-<li><code class="docutils literal"><span class="pre">org.apache.hadoop.hive.serde2.columnar.ColumnarSerDe</span></code>: corresponds to <code class="docutils literal"><span class="pre">TextSerializerDeserializer</span></code> in Tajo.</li>
-<li><code class="docutils literal"><span class="pre">org.apache.hadoop.hive.serde2.columnar.LazyBinaryColumnarSerDe</span></code>: corresponds to <code class="docutils literal"><span class="pre">BinarySerializerDeserializer</span></code> in Tajo.</li>
+<li><tt class="docutils literal"><span class="pre">org.apache.hadoop.hive.serde2.columnar.ColumnarSerDe</span></tt>: corresponds to <tt class="docutils literal"><span class="pre">TextSerializerDeserializer</span></tt> in Tajo.</li>
+<li><tt class="docutils literal"><span class="pre">org.apache.hadoop.hive.serde2.columnar.LazyBinaryColumnarSerDe</span></tt>: corresponds to <tt class="docutils literal"><span class="pre">BinarySerializerDeserializer</span></tt> in Tajo.</li>
</ul>
<p>The compatibility issue mostly occurs when a user creates an external table pointing to data of an existing table.
The following section explains two cases: 1) the case where Tajo reads RCFile written by Hive, and
2) the case where Hive reads RCFile written by Tajo.</p>
<div class="section" id="when-tajo-reads-rcfile-generated-in-hive">
<h3>When Tajo reads RCFile generated in Hive<a class="headerlink" href="#when-tajo-reads-rcfile-generated-in-hive" title="Permalink to this headline">¶</a></h3>
-<p>To create an external RCFile table generated with <code class="docutils literal"><span class="pre">ColumnarSerDe</span></code> in Hive,
-you should set the physical property <code class="docutils literal"><span class="pre">rcfile.serde</span></code> in Tajo as follows:</p>
+<p>To create an external RCFile table generated with <tt class="docutils literal"><span class="pre">ColumnarSerDe</span></tt> in Hive,
+you should set the physical property <tt class="docutils literal"><span class="pre">rcfile.serde</span></tt> in Tajo as follows:</p>
<div class="highlight-sql"><div class="highlight"><pre><span class="k">CREATE</span> <span class="k">EXTERNAL</span> <span class="k">TABLE</span> <span class="n">table1</span> <span class="p">(</span>
<span class="n">id</span> <span class="nb">int</span><span class="p">,</span>
<span class="n">name</span> <span class="nb">text</span><span class="p">,</span>
@@ -290,8 +290,8 @@ you should set the physical property <co
<span class="k">LOCATION</span> <span class="s1">'....'</span><span class="p">;</span>
</pre></div>
</div>
-<p>To create an external RCFile table generated with <code class="docutils literal"><span class="pre">LazyBinaryColumnarSerDe</span></code> in Hive,
-you should set the physical property <code class="docutils literal"><span class="pre">rcfile.serde</span></code> in Tajo as follows:</p>
+<p>To create an external RCFile table generated with <tt class="docutils literal"><span class="pre">LazyBinaryColumnarSerDe</span></tt> in Hive,
+you should set the physical property <tt class="docutils literal"><span class="pre">rcfile.serde</span></tt> in Tajo as follows:</p>
<div class="highlight-sql"><div class="highlight"><pre><span class="k">CREATE</span> <span class="k">EXTERNAL</span> <span class="k">TABLE</span> <span class="n">table1</span> <span class="p">(</span>
<span class="n">id</span> <span class="nb">int</span><span class="p">,</span>
<span class="n">name</span> <span class="nb">text</span><span class="p">,</span>
@@ -303,14 +303,14 @@ you should set the physical property <co
</div>
<div class="admonition note">
<p class="first admonition-title">Note</p>
-<p class="last">As we mentioned above, <code class="docutils literal"><span class="pre">BinarySerializerDeserializer</span></code> is the default (de) serializer for RCFile.
-So, you can omit the <code class="docutils literal"><span class="pre">rcfile.serde</span></code> only for <code class="docutils literal"><span class="pre">org.apache.tajo.storage.BinarySerializerDeserializer</span></code>.</p>
+<p class="last">As we mentioned above, <tt class="docutils literal"><span class="pre">BinarySerializerDeserializer</span></tt> is the default (de) serializer for RCFile.
+So, you can omit the <tt class="docutils literal"><span class="pre">rcfile.serde</span></tt> only for <tt class="docutils literal"><span class="pre">org.apache.tajo.storage.BinarySerializerDeserializer</span></tt>.</p>
</div>
</div>
<div class="section" id="when-hive-reads-rcfile-generated-in-tajo">
<h3>When Hive reads RCFile generated in Tajo<a class="headerlink" href="#when-hive-reads-rcfile-generated-in-tajo" title="Permalink to this headline">¶</a></h3>
-<p>To create an external RCFile table written by Tajo with <code class="docutils literal"><span class="pre">TextSerializerDeserializer</span></code>,
-you should set the <code class="docutils literal"><span class="pre">SERDE</span></code> as follows:</p>
+<p>To create an external RCFile table written by Tajo with <tt class="docutils literal"><span class="pre">TextSerializerDeserializer</span></tt>,
+you should set the <tt class="docutils literal"><span class="pre">SERDE</span></tt> as follows:</p>
<div class="highlight-sql"><div class="highlight"><pre><span class="k">CREATE</span> <span class="k">TABLE</span> <span class="n">table1</span> <span class="p">(</span>
<span class="n">id</span> <span class="nb">int</span><span class="p">,</span>
<span class="n">name</span> <span class="n">string</span><span class="p">,</span>
@@ -320,8 +320,8 @@ you should set the <code class="docutils
<span class="k">LOCATION</span> <span class="s1">'<hdfs_location>'</span><span class="p">;</span>
</pre></div>
</div>
-<p>To create an external RCFile table written by Tajo with <code class="docutils literal"><span class="pre">BinarySerializerDeserializer</span></code>,
-you should set the <code class="docutils literal"><span class="pre">SERDE</span></code> as follows:</p>
+<p>To create an external RCFile table written by Tajo with <tt class="docutils literal"><span class="pre">BinarySerializerDeserializer</span></tt>,
+you should set the <tt class="docutils literal"><span class="pre">SERDE</span></tt> as follows:</p>
<div class="highlight-sql"><div class="highlight"><pre><span class="k">CREATE</span> <span class="k">TABLE</span> <span class="n">table1</span> <span class="p">(</span>
<span class="n">id</span> <span class="nb">int</span><span class="p">,</span>
<span class="n">name</span> <span class="n">string</span><span class="p">,</span>
@@ -353,7 +353,7 @@ you should set the <code class="docutils
<div role="contentinfo">
<p>
- © Copyright 2015, Apache Tajo Team.
+ © Copyright 2016, Apache Tajo Team.
</p>
</div>
@@ -373,7 +373,7 @@ you should set the <code class="docutils
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT:'../',
- VERSION:'0.11.0',
+ VERSION:'0.11.3',
COLLAPSE_INDEX:false,
FILE_SUFFIX:'.html',
HAS_SOURCE: true
Modified: tajo/site/docs/current/table_management/sequencefile.html
URL: http://svn.apache.org/viewvc/tajo/site/docs/current/table_management/sequencefile.html?rev=1744489&r1=1744488&r2=1744489&view=diff
==============================================================================
--- tajo/site/docs/current/table_management/sequencefile.html (original)
+++ tajo/site/docs/current/table_management/sequencefile.html Thu May 19 02:00:25 2016
@@ -7,7 +7,7 @@
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
- <title>SequenceFile — Apache Tajo 0.11.0 documentation</title>
+ <title>SequenceFile — Apache Tajo 0.11.3 documentation</title>
@@ -28,7 +28,7 @@
- <link rel="top" title="Apache Tajo 0.11.0 documentation" href="../index.html"/>
+ <link rel="top" title="Apache Tajo 0.11.3 documentation" href="../index.html"/>
<link rel="up" title="Data Formats" href="data_formats.html"/>
<link rel="next" title="Compression" href="compression.html"/>
<link rel="prev" title="ORC" href="orc.html"/>
@@ -217,7 +217,7 @@
<h2>Introduce<a class="headerlink" href="#introduce" title="Permalink to this headline">¶</a></h2>
<p>SequenceFiles are flat files consisting of binary key/value pairs.
SequenceFile is basic file format which provided by Hadoop, and Hive also provides it to create a table.</p>
-<p>The <code class="docutils literal"><span class="pre">USING</span> <span class="pre">sequencefile</span></code> keywords let you create a SequecneFile. Here is an example statement to create a table using <code class="docutils literal"><span class="pre">SequecneFile</span></code>:</p>
+<p>The <tt class="docutils literal"><span class="pre">USING</span> <span class="pre">sequencefile</span></tt> keywords let you create a SequecneFile. Here is an example statement to create a table using <tt class="docutils literal"><span class="pre">SequecneFile</span></tt>:</p>
<div class="highlight-sql"><div class="highlight"><pre><span class="k">CREATE</span> <span class="k">TABLE</span> <span class="n">table1</span> <span class="p">(</span><span class="n">id</span> <span class="nb">int</span><span class="p">,</span> <span class="n">name</span> <span class="nb">text</span><span class="p">,</span> <span class="n">score</span> <span class="nb">float</span><span class="p">,</span> <span class="k">type</span> <span class="nb">text</span><span class="p">)</span>
<span class="k">USING</span> <span class="n">sequencefile</span><span class="p">;</span>
</pre></div>
@@ -237,7 +237,7 @@ SequenceFile is basic file format which
<li>BinarySerializerDeserializer: This class can read and write data in binary file format.</li>
</ul>
</div></blockquote>
-<p>The default is the SerDe for plain text file in Tajo. The above example statement created the table using TextSerializerDeserializer.If you want to use BinarySerializerDeserializer, you can specify it by <code class="docutils literal"><span class="pre">sequencefile.serde</span></code> keywords:</p>
+<p>The default is the SerDe for plain text file in Tajo. The above example statement created the table using TextSerializerDeserializer.If you want to use BinarySerializerDeserializer, you can specify it by <tt class="docutils literal"><span class="pre">sequencefile.serde</span></tt> keywords:</p>
<div class="highlight-sql"><div class="highlight"><pre><span class="k">CREATE</span> <span class="k">TABLE</span> <span class="n">table1</span> <span class="p">(</span><span class="n">id</span> <span class="nb">int</span><span class="p">,</span> <span class="n">name</span> <span class="nb">text</span><span class="p">,</span> <span class="n">score</span> <span class="nb">float</span><span class="p">,</span> <span class="k">type</span> <span class="nb">text</span><span class="p">)</span>
<span class="k">USING</span> <span class="n">sequencefile</span> <span class="k">with</span> <span class="p">(</span><span class="s1">'sequencefile.serde'</span><span class="o">=</span><span class="s1">'org.apache.tajo.storage.BinarySerializerDeserializer'</span><span class="p">)</span>
</pre></div>
@@ -260,7 +260,7 @@ SequenceFile is basic file format which
<li>BlockCompressWriter : Block-compressed files, both keys & values are collected in ‘blocks’ separately and compressed. The size of the ‘block’ is configurable.</li>
</ul>
</div></blockquote>
-<p>The default is Uncompressed Writer in Tajo. If you want to use RecordCompressWriter, you can specify it by <code class="docutils literal"><span class="pre">compression.type</span></code> keywords and <code class="docutils literal"><span class="pre">compression.codec</span></code> keywords:</p>
+<p>The default is Uncompressed Writer in Tajo. If you want to use RecordCompressWriter, you can specify it by <tt class="docutils literal"><span class="pre">compression.type</span></tt> keywords and <tt class="docutils literal"><span class="pre">compression.codec</span></tt> keywords:</p>
<div class="highlight-sql"><div class="highlight"><pre><span class="k">CREATE</span> <span class="k">TABLE</span> <span class="n">table1</span> <span class="p">(</span><span class="n">id</span> <span class="nb">int</span><span class="p">,</span> <span class="n">name</span> <span class="nb">text</span><span class="p">,</span> <span class="n">score</span> <span class="nb">float</span><span class="p">,</span> <span class="k">type</span> <span class="nb">text</span><span class="p">)</span>
<span class="k">USING</span> <span class="n">sequencefile</span> <span class="k">with</span> <span class="p">(</span><span class="s1">'compression.type'</span><span class="o">=</span><span class="s1">'RECORD'</span><span class="p">,</span><span class="s1">'compression.codec'</span><span class="o">=</span><span class="s1">'org.apache.hadoop.io.compress.SnappyCodec'</span><span class="p">)</span>
</pre></div>
@@ -272,7 +272,7 @@ SequenceFile is basic file format which
<span class="n">hive</span><span class="o">></span> <span class="k">CREATE</span> <span class="k">TABLE</span> <span class="n">table1</span> <span class="p">(</span><span class="n">id</span> <span class="nb">int</span><span class="p">,</span> <span class="n">name</span> <span class="n">string</span><span class="p">,</span> <span class="n">score</span> <span class="nb">float</span><span class="p">,</span> <span class="k">type</span> <span class="n">string</span><span class="p">)</span> <span class="n">STORED</span> <span class="k">AS</span> <span class="n">sequencefile</span><span class="p">;;</span>
</pre></div>
</div>
-<p>And if you want to use BlockCompressWriter, you can specify it by <code class="docutils literal"><span class="pre">compression.type</span></code> keywords and <code class="docutils literal"><span class="pre">compression.codec</span></code> keywords:</p>
+<p>And if you want to use BlockCompressWriter, you can specify it by <tt class="docutils literal"><span class="pre">compression.type</span></tt> keywords and <tt class="docutils literal"><span class="pre">compression.codec</span></tt> keywords:</p>
<div class="highlight-sql"><div class="highlight"><pre><span class="k">CREATE</span> <span class="k">TABLE</span> <span class="n">table1</span> <span class="p">(</span><span class="n">id</span> <span class="nb">int</span><span class="p">,</span> <span class="n">name</span> <span class="nb">text</span><span class="p">,</span> <span class="n">score</span> <span class="nb">float</span><span class="p">,</span> <span class="k">type</span> <span class="nb">text</span><span class="p">)</span>
<span class="k">USING</span> <span class="n">sequencefile</span> <span class="k">with</span> <span class="p">(</span><span class="s1">'compression.type'</span><span class="o">=</span><span class="s1">'BLOCK'</span><span class="p">,</span><span class="s1">'compression.codec'</span><span class="o">=</span><span class="s1">'org.apache.hadoop.io.compress.SnappyCodec'</span><span class="p">)</span>
</pre></div>
@@ -321,7 +321,7 @@ Here is an example statement for this ca
<div role="contentinfo">
<p>
- © Copyright 2015, Apache Tajo Team.
+ © Copyright 2016, Apache Tajo Team.
</p>
</div>
@@ -341,7 +341,7 @@ Here is an example statement for this ca
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT:'../',
- VERSION:'0.11.0',
+ VERSION:'0.11.3',
COLLAPSE_INDEX:false,
FILE_SUFFIX:'.html',
HAS_SOURCE: true
Modified: tajo/site/docs/current/table_management/table_overview.html
URL: http://svn.apache.org/viewvc/tajo/site/docs/current/table_management/table_overview.html?rev=1744489&r1=1744488&r2=1744489&view=diff
==============================================================================
--- tajo/site/docs/current/table_management/table_overview.html (original)
+++ tajo/site/docs/current/table_management/table_overview.html Thu May 19 02:00:25 2016
@@ -7,7 +7,7 @@
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
- <title>Overview of Tajo Tables — Apache Tajo 0.11.0 documentation</title>
+ <title>Overview of Tajo Tables — Apache Tajo 0.11.3 documentation</title>
@@ -28,7 +28,7 @@
- <link rel="top" title="Apache Tajo 0.11.0 documentation" href="../index.html"/>
+ <link rel="top" title="Apache Tajo 0.11.3 documentation" href="../index.html"/>
<link rel="up" title="Table Management" href="../table_management.html"/>
<link rel="next" title="Tablespaces" href="tablespaces.html"/>
<link rel="prev" title="Table Management" href="../table_management.html"/>
@@ -221,7 +221,7 @@
</div>
<div class="section" id="managed-table">
<h2>Managed Table<a class="headerlink" href="#managed-table" title="Permalink to this headline">¶</a></h2>
-<p><code class="docutils literal"><span class="pre">CREATE</span> <span class="pre">TABLE</span></code> statement lets you create a table located in the warehouse directory specified by the configuration property <code class="docutils literal"><span class="pre">tajo.warehouse.directory</span></code> or <code class="docutils literal"><span class="pre">${tajo.root}/warehouse</span></code> by default. For example:</p>
+<p><tt class="docutils literal"><span class="pre">CREATE</span> <span class="pre">TABLE</span></tt> statement lets you create a table located in the warehouse directory specified by the configuration property <tt class="docutils literal"><span class="pre">tajo.warehouse.directory</span></tt> or <tt class="docutils literal"><span class="pre">${tajo.root}/warehouse</span></tt> by default. For example:</p>
<div class="highlight-sql"><div class="highlight"><pre><span class="k">CREATE</span> <span class="k">TABLE</span> <span class="n">employee</span> <span class="p">(</span>
<span class="n">id</span> <span class="nb">int</span><span class="p">,</span>
<span class="n">name</span> <span class="nb">text</span><span class="p">,</span>
@@ -232,7 +232,7 @@
</div>
<div class="section" id="external-table">
<h2>External Table<a class="headerlink" href="#external-table" title="Permalink to this headline">¶</a></h2>
-<p><code class="docutils literal"><span class="pre">CREATE</span> <span class="pre">EXTERNAL</span> <span class="pre">TABLE</span></code> statement lets you create a table located in a specify location so that Tajo does not use a default data warehouse location for the table. External tables are in common used if you already have data generated. LOCATION clause must be required for an external table.</p>
+<p><tt class="docutils literal"><span class="pre">CREATE</span> <span class="pre">EXTERNAL</span> <span class="pre">TABLE</span></tt> statement lets you create a table located in a specify location so that Tajo does not use a default data warehouse location for the table. External tables are in common used if you already have data generated. LOCATION clause must be required for an external table.</p>
<div class="highlight-sql"><div class="highlight"><pre><span class="k">CREATE</span> <span class="k">EXTERNAL</span> <span class="k">TABLE</span> <span class="n">employee</span> <span class="p">(</span>
<span class="n">id</span> <span class="nb">int</span><span class="p">,</span>
<span class="n">name</span> <span class="nb">text</span><span class="p">,</span>
@@ -243,16 +243,16 @@
<p>The location can be a directory located in HDFS, Amazon S3, HBase, or local file system (if a Tajo cluster runs in a single machine). URI examples are as follows:</p>
<blockquote>
<div><ul class="simple">
-<li>HDFS - <code class="docutils literal"><span class="pre">hdfs://hostname:8020/table1</span></code></li>
-<li>Amazon S3 - <code class="docutils literal"><span class="pre">s3://bucket-name/table1</span></code></li>
-<li>local file system - <code class="docutils literal"><span class="pre">file:///dir/table1</span></code></li>
-<li>Openstack Swift - <code class="docutils literal"><span class="pre">swift://bucket-name/table1</span></code></li>
+<li>HDFS - <tt class="docutils literal"><span class="pre">hdfs://hostname:8020/table1</span></tt></li>
+<li>Amazon S3 - <tt class="docutils literal"><span class="pre">s3://bucket-name/table1</span></tt></li>
+<li>local file system - <tt class="docutils literal"><span class="pre">file:///dir/table1</span></tt></li>
+<li>Openstack Swift - <tt class="docutils literal"><span class="pre">swift://bucket-name/table1</span></tt></li>
</ul>
</div></blockquote>
<div class="section" id="table-properties">
<h3>Table Properties<a class="headerlink" href="#table-properties" title="Permalink to this headline">¶</a></h3>
<p>All table formats provide parameters for enabling or disabling features and adjusting physical parameters.
-The <code class="docutils literal"><span class="pre">WITH</span></code> clause in the CREATE TABLE statement allows users to set those properties.</p>
+The <tt class="docutils literal"><span class="pre">WITH</span></tt> clause in the CREATE TABLE statement allows users to set those properties.</p>
<p>The following example is to set a custom field delimiter, NULL character, and compression codec:</p>
<div class="highlight-sql"><div class="highlight"><pre><span class="k">CREATE</span> <span class="k">TABLE</span> <span class="n">table1</span> <span class="p">(</span>
<span class="n">id</span> <span class="nb">int</span><span class="p">,</span>
@@ -275,7 +275,7 @@ The <code class="docutils literal"><span
</div>
<div class="section" id="time-zone">
<h4>Time zone<a class="headerlink" href="#time-zone" title="Permalink to this headline">¶</a></h4>
-<p>In Tajo, a table property <code class="docutils literal"><span class="pre">timezone</span></code> allows users to specify a time zone that the table uses for reading or writing.</p>
+<p>In Tajo, a table property <tt class="docutils literal"><span class="pre">timezone</span></tt> allows users to specify a time zone that the table uses for reading or writing.</p>
<p>You can specify a table time zone as follows:</p>
<div class="highlight-sql"><div class="highlight"><pre><span class="k">CREATE</span> <span class="k">EXTERNAL</span> <span class="k">TABLE</span> <span class="n">table1</span> <span class="p">(</span>
<span class="n">t_timestamp</span> <span class="k">TIMESTAMP</span><span class="p">,</span>
@@ -307,7 +307,7 @@ The <code class="docutils literal"><span
<div role="contentinfo">
<p>
- © Copyright 2015, Apache Tajo Team.
+ © Copyright 2016, Apache Tajo Team.
</p>
</div>
@@ -327,7 +327,7 @@ The <code class="docutils literal"><span
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT:'../',
- VERSION:'0.11.0',
+ VERSION:'0.11.3',
COLLAPSE_INDEX:false,
FILE_SUFFIX:'.html',
HAS_SOURCE: true
Modified: tajo/site/docs/current/table_management/tablespaces.html
URL: http://svn.apache.org/viewvc/tajo/site/docs/current/table_management/tablespaces.html?rev=1744489&r1=1744488&r2=1744489&view=diff
==============================================================================
--- tajo/site/docs/current/table_management/tablespaces.html (original)
+++ tajo/site/docs/current/table_management/tablespaces.html Thu May 19 02:00:25 2016
@@ -7,7 +7,7 @@
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
- <title>Tablespaces — Apache Tajo 0.11.0 documentation</title>
+ <title>Tablespaces — Apache Tajo 0.11.3 documentation</title>
@@ -28,7 +28,7 @@
- <link rel="top" title="Apache Tajo 0.11.0 documentation" href="../index.html"/>
+ <link rel="top" title="Apache Tajo 0.11.3 documentation" href="../index.html"/>
<link rel="up" title="Table Management" href="../table_management.html"/>
<link rel="next" title="Data Formats" href="data_formats.html"/>
<link rel="prev" title="Overview of Tajo Tables" href="table_overview.html"/>
@@ -229,7 +229,7 @@ If the tablespace is not specified, the
</div>
<div class="section" id="configuration">
<h2>Configuration<a class="headerlink" href="#configuration" title="Permalink to this headline">¶</a></h2>
-<p>By default, Tajo use <code class="docutils literal"><span class="pre">${tajo.rootdir}/warehouse</span></code> in <a class="reference internal" href="../configuration/tajo-site-xml.html"><em>conf/tajo-site.xml</em></a> as a default tablespace. It also allows users to register additional tablespaces using <code class="docutils literal"><span class="pre">storage-site.json</span></code> file like below.</p>
+<p>By default, Tajo use <tt class="docutils literal"><span class="pre">${tajo.rootdir}/warehouse</span></tt> in <a class="reference internal" href="../configuration/tajo-site-xml.html"><em>conf/tajo-site.xml</em></a> as a default tablespace. It also allows users to register additional tablespaces using <tt class="docutils literal"><span class="pre">storage-site.json</span></tt> file like below.</p>
<div class="section" id="conf-storage-site-json-file">
<h3>conf/storage-site.json file<a class="headerlink" href="#conf-storage-site-json-file" title="Permalink to this headline">¶</a></h3>
<p>The configuration file has the following struct:</p>
@@ -283,7 +283,7 @@ If the tablespace is not specified, the
<div role="contentinfo">
<p>
- © Copyright 2015, Apache Tajo Team.
+ © Copyright 2016, Apache Tajo Team.
</p>
</div>
@@ -303,7 +303,7 @@ If the tablespace is not specified, the
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT:'../',
- VERSION:'0.11.0',
+ VERSION:'0.11.3',
COLLAPSE_INDEX:false,
FILE_SUFFIX:'.html',
HAS_SOURCE: true
Modified: tajo/site/docs/current/table_management/text.html
URL: http://svn.apache.org/viewvc/tajo/site/docs/current/table_management/text.html?rev=1744489&r1=1744488&r2=1744489&view=diff
==============================================================================
--- tajo/site/docs/current/table_management/text.html (original)
+++ tajo/site/docs/current/table_management/text.html Thu May 19 02:00:25 2016
@@ -7,7 +7,7 @@
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
- <title>TEXT — Apache Tajo 0.11.0 documentation</title>
+ <title>TEXT — Apache Tajo 0.11.3 documentation</title>
@@ -28,7 +28,7 @@
- <link rel="top" title="Apache Tajo 0.11.0 documentation" href="../index.html"/>
+ <link rel="top" title="Apache Tajo 0.11.3 documentation" href="../index.html"/>
<link rel="up" title="Data Formats" href="data_formats.html"/>
<link rel="next" title="JSON" href="json.html"/>
<link rel="prev" title="Data Formats" href="data_formats.html"/>
@@ -214,14 +214,14 @@
<div class="section" id="text">
<h1>TEXT<a class="headerlink" href="#text" title="Permalink to this headline">¶</a></h1>
<p>A character-separated values plain text file represents a tabular data set consisting of rows and columns.
-Each row is a plain text line. A line is usually broken by a character line feed <code class="docutils literal"><span class="pre">\n</span></code> or carriage-return <code class="docutils literal"><span class="pre">\r</span></code>.
-The line feed <code class="docutils literal"><span class="pre">\n</span></code> is the default delimiter in Tajo. Each record consists of multiple fields, separated by
-some other character or string, most commonly a literal vertical bar <code class="docutils literal"><span class="pre">|</span></code>, comma <code class="docutils literal"><span class="pre">,</span></code> or tab <code class="docutils literal"><span class="pre">\t</span></code>.
+Each row is a plain text line. A line is usually broken by a character line feed <tt class="docutils literal"><span class="pre">\n</span></tt> or carriage-return <tt class="docutils literal"><span class="pre">\r</span></tt>.
+The line feed <tt class="docutils literal"><span class="pre">\n</span></tt> is the default delimiter in Tajo. Each record consists of multiple fields, separated by
+some other character or string, most commonly a literal vertical bar <tt class="docutils literal"><span class="pre">|</span></tt>, comma <tt class="docutils literal"><span class="pre">,</span></tt> or tab <tt class="docutils literal"><span class="pre">\t</span></tt>.
The vertical bar is used as the default field delimiter in Tajo.</p>
<div class="section" id="how-to-create-a-text-table">
<h2>How to Create a TEXT Table ?<a class="headerlink" href="#how-to-create-a-text-table" title="Permalink to this headline">¶</a></h2>
-<p>If you are not familiar with the <code class="docutils literal"><span class="pre">CREATE</span> <span class="pre">TABLE</span></code> statement, please refer to the Data Definition Language <a class="reference internal" href="../sql_language/ddl.html"><em>Data Definition Language</em></a>.</p>
-<p>In order to specify a certain file format for your table, you need to use the <code class="docutils literal"><span class="pre">USING</span></code> clause in your <code class="docutils literal"><span class="pre">CREATE</span> <span class="pre">TABLE</span></code>
+<p>If you are not familiar with the <tt class="docutils literal"><span class="pre">CREATE</span> <span class="pre">TABLE</span></tt> statement, please refer to the Data Definition Language <a class="reference internal" href="../sql_language/ddl.html"><em>Data Definition Language</em></a>.</p>
+<p>In order to specify a certain file format for your table, you need to use the <tt class="docutils literal"><span class="pre">USING</span></tt> clause in your <tt class="docutils literal"><span class="pre">CREATE</span> <span class="pre">TABLE</span></tt>
statement. The below is an example statement for creating a table using <em>TEXT</em> format.</p>
<div class="highlight-sql"><div class="highlight"><pre><span class="k">CREATE</span> <span class="k">TABLE</span>
<span class="n">table1</span> <span class="p">(</span>
@@ -236,23 +236,23 @@ statement. The below is an example state
<div class="section" id="physical-properties">
<h2>Physical Properties<a class="headerlink" href="#physical-properties" title="Permalink to this headline">¶</a></h2>
<p>Some table storage formats provide parameters for enabling or disabling features and adjusting physical parameters.
-The <code class="docutils literal"><span class="pre">WITH</span></code> clause in the CREATE TABLE statement allows users to set those parameters.</p>
+The <tt class="docutils literal"><span class="pre">WITH</span></tt> clause in the CREATE TABLE statement allows users to set those parameters.</p>
<p><em>TEXT</em> format provides the following physical properties.</p>
<ul class="simple">
-<li><code class="docutils literal"><span class="pre">text.delimiter</span></code>: delimiter character. <code class="docutils literal"><span class="pre">|</span></code> or <code class="docutils literal"><span class="pre">\u0001</span></code> is usually used, and the default field delimiter is <code class="docutils literal"><span class="pre">|</span></code>.</li>
-<li><code class="docutils literal"><span class="pre">text.null</span></code>: <code class="docutils literal"><span class="pre">NULL</span></code> character. The default <code class="docutils literal"><span class="pre">NULL</span></code> character is an empty string <code class="docutils literal"><span class="pre">''</span></code>. Hive’s default <code class="docutils literal"><span class="pre">NULL</span></code> character is <code class="docutils literal"><span class="pre">'\\N'</span></code>.</li>
-<li><code class="docutils literal"><span class="pre">compression.codec</span></code>: Compression codec. You can enable compression feature and set specified compression algorithm. The compression algorithm used to compress files. The compression codec name should be the fully qualified class name inherited from <a class="reference external" href="https://hadoop.apache.org/docs/current/api/org/apache/hadoop/io/compress/CompressionCodec.html">org.apache.hadoop.io.compress.CompressionCodec</a>. By default, compression is disabled.</li>
-<li><code class="docutils literal"><span class="pre">text.serde</span></code>: custom (De)serializer class. <code class="docutils literal"><span class="pre">org.apache.tajo.storage.text.CSVLineSerDe</span></code> is the default (De)serializer class.</li>
-<li><code class="docutils literal"><span class="pre">timezone</span></code>: the time zone that the table uses for writting. When table rows are read or written, <code class="docutils literal"><span class="pre">`timestamp`</span></code> and <code class="docutils literal"><span class="pre">`time`</span></code> column values are adjusted by this timezone if it is set. Time zone can be an abbreviation form like ‘PST’ or ‘DST’. Also, it accepts an offset-based form like ‘UTC+9’ or a location-based form like ‘Asia/Seoul’.</li>
-<li><code class="docutils literal"><span class="pre">text.error-tolerance.max-num</span></code>: the maximum number of permissible parsing errors. This value should be an integer value. By default, <code class="docutils literal"><span class="pre">text.error-tolerance.max-num</span></code> is <code class="docutils literal"><span class="pre">0</span></code>. According to the value, parsing errors will be handled in different ways.<ul>
-<li>If <code class="docutils literal"><span class="pre">text.error-tolerance.max-num</span> <span class="pre"><</span> <span class="pre">0</span></code>, all parsing errors are ignored.</li>
-<li>If <code class="docutils literal"><span class="pre">text.error-tolerance.max-num</span> <span class="pre">==</span> <span class="pre">0</span></code>, any parsing error is not allowed. If any error occurs, the query will be failed. (default)</li>
-<li>If <code class="docutils literal"><span class="pre">text.error-tolerance.max-num</span> <span class="pre">></span> <span class="pre">0</span></code>, the given number of parsing errors in each task will be pemissible.</li>
+<li><tt class="docutils literal"><span class="pre">text.delimiter</span></tt>: delimiter character. <tt class="docutils literal"><span class="pre">|</span></tt> or <tt class="docutils literal"><span class="pre">\u0001</span></tt> is usually used, and the default field delimiter is <tt class="docutils literal"><span class="pre">|</span></tt>.</li>
+<li><tt class="docutils literal"><span class="pre">text.null</span></tt>: <tt class="docutils literal"><span class="pre">NULL</span></tt> character. The default <tt class="docutils literal"><span class="pre">NULL</span></tt> character is an empty string <tt class="docutils literal"><span class="pre">''</span></tt>. Hive’s default <tt class="docutils literal"><span class="pre">NULL</span></tt> character is <tt class="docutils literal"><span class="pre">'\\N'</span></tt>.</li>
+<li><tt class="docutils literal"><span class="pre">compression.codec</span></tt>: Compression codec. You can enable compression feature and set specified compression algorithm. The compression algorithm used to compress files. The compression codec name should be the fully qualified class name inherited from <a class="reference external" href="https://hadoop.apache.org/docs/current/api/org/apache/hadoop/io/compress/CompressionCodec.html">org.apache.hadoop.io.compress.CompressionCodec</a>. By default, compression is disabled.</li>
+<li><tt class="docutils literal"><span class="pre">text.serde</span></tt>: custom (De)serializer class. <tt class="docutils literal"><span class="pre">org.apache.tajo.storage.text.CSVLineSerDe</span></tt> is the default (De)serializer class.</li>
+<li><tt class="docutils literal"><span class="pre">timezone</span></tt>: the time zone that the table uses for writting. When table rows are read or written, <tt class="docutils literal"><span class="pre">`timestamp`</span></tt> and <tt class="docutils literal"><span class="pre">`time`</span></tt> column values are adjusted by this timezone if it is set. Time zone can be an abbreviation form like ‘PST’ or ‘DST’. Also, it accepts an offset-based form like ‘UTC+9’ or a location-based form like ‘Asia/Seoul’.</li>
+<li><tt class="docutils literal"><span class="pre">text.error-tolerance.max-num</span></tt>: the maximum number of permissible parsing errors. This value should be an integer value. By default, <tt class="docutils literal"><span class="pre">text.error-tolerance.max-num</span></tt> is <tt class="docutils literal"><span class="pre">0</span></tt>. According to the value, parsing errors will be handled in different ways.<ul>
+<li>If <tt class="docutils literal"><span class="pre">text.error-tolerance.max-num</span> <span class="pre"><</span> <span class="pre">0</span></tt>, all parsing errors are ignored.</li>
+<li>If <tt class="docutils literal"><span class="pre">text.error-tolerance.max-num</span> <span class="pre">==</span> <span class="pre">0</span></tt>, any parsing error is not allowed. If any error occurs, the query will be failed. (default)</li>
+<li>If <tt class="docutils literal"><span class="pre">text.error-tolerance.max-num</span> <span class="pre">></span> <span class="pre">0</span></tt>, the given number of parsing errors in each task will be pemissible.</li>
</ul>
</li>
-<li><code class="docutils literal"><span class="pre">text.skip.headerlines</span></code>: Number of header lines to be skipped. Some text files often have a header which has a kind of metadata(e.g.: column names), thus this option can be useful.</li>
+<li><tt class="docutils literal"><span class="pre">text.skip.headerlines</span></tt>: Number of header lines to be skipped. Some text files often have a header which has a kind of metadata(e.g.: column names), thus this option can be useful.</li>
</ul>
-<p>The following example is to set a custom field delimiter, <code class="docutils literal"><span class="pre">NULL</span></code> character, and compression codec:</p>
+<p>The following example is to set a custom field delimiter, <tt class="docutils literal"><span class="pre">NULL</span></tt> character, and compression codec:</p>
<div class="highlight-sql"><div class="highlight"><pre><span class="k">CREATE</span> <span class="k">TABLE</span> <span class="n">table1</span> <span class="p">(</span>
<span class="n">id</span> <span class="nb">int</span><span class="p">,</span>
<span class="n">name</span> <span class="nb">text</span><span class="p">,</span>
@@ -265,7 +265,7 @@ The <code class="docutils literal"><span
</div>
<div class="admonition warning">
<p class="first admonition-title">Warning</p>
-<p class="last">Be careful when using <code class="docutils literal"><span class="pre">\n</span></code> as the field delimiter because <em>TEXT</em> format tables use <code class="docutils literal"><span class="pre">\n</span></code> as the line delimiter.
+<p class="last">Be careful when using <tt class="docutils literal"><span class="pre">\n</span></tt> as the field delimiter because <em>TEXT</em> format tables use <tt class="docutils literal"><span class="pre">\n</span></tt> as the line delimiter.
At the moment, Tajo does not provide a way to specify the line delimiter.</p>
</div>
</div>
@@ -274,7 +274,7 @@ At the moment, Tajo does not provide a w
<p>The <em>TEXT</em> format not only provides reading and writing interfaces for text data but also allows users to process custom
plain text file formats with user-defined (De)serializer classes.
With custom (de)serializers, Tajo can process any text files no matter which the internal structure is.</p>
-<p>In order to specify a custom (De)serializer, set a physical property <code class="docutils literal"><span class="pre">text.serde</span></code>.
+<p>In order to specify a custom (De)serializer, set a physical property <tt class="docutils literal"><span class="pre">text.serde</span></tt>.
The property value should be a fully qualified class name.</p>
<p>For example:</p>
<div class="highlight-sql"><div class="highlight"><pre><span class="k">CREATE</span> <span class="k">TABLE</span> <span class="n">table1</span> <span class="p">(</span>
@@ -288,24 +288,24 @@ The property value should be a fully qua
</div>
<div class="section" id="null-value-handling-issues">
<h2>Null Value Handling Issues<a class="headerlink" href="#null-value-handling-issues" title="Permalink to this headline">¶</a></h2>
-<p>In default, <code class="docutils literal"><span class="pre">NULL</span></code> character in <em>TEXT</em> format is an empty string <code class="docutils literal"><span class="pre">''</span></code>.
-In other words, an empty field is basically recognized as a <code class="docutils literal"><span class="pre">NULL</span></code> value in Tajo.
-If a field domain is <code class="docutils literal"><span class="pre">TEXT</span></code>, an empty field is recognized as a string value <code class="docutils literal"><span class="pre">''</span></code> instead of <code class="docutils literal"><span class="pre">NULL</span></code> value.
-Besides, You can also use your own <code class="docutils literal"><span class="pre">NULL</span></code> character by specifying a physical property <code class="docutils literal"><span class="pre">text.null</span></code>.</p>
+<p>In default, <tt class="docutils literal"><span class="pre">NULL</span></tt> character in <em>TEXT</em> format is an empty string <tt class="docutils literal"><span class="pre">''</span></tt>.
+In other words, an empty field is basically recognized as a <tt class="docutils literal"><span class="pre">NULL</span></tt> value in Tajo.
+If a field domain is <tt class="docutils literal"><span class="pre">TEXT</span></tt>, an empty field is recognized as a string value <tt class="docutils literal"><span class="pre">''</span></tt> instead of <tt class="docutils literal"><span class="pre">NULL</span></tt> value.
+Besides, You can also use your own <tt class="docutils literal"><span class="pre">NULL</span></tt> character by specifying a physical property <tt class="docutils literal"><span class="pre">text.null</span></tt>.</p>
</div>
<div class="section" id="compatibility-issues-with-apache-hive">
<h2>Compatibility Issues with Apache Hive™<a class="headerlink" href="#compatibility-issues-with-apache-hive" title="Permalink to this headline">¶</a></h2>
<p><em>TEXT</em> tables generated in Tajo can be processed directly by Apache Hive™ without further processing.
In this section, we explain some compatibility issue for users who use both Hive and Tajo.</p>
<p>If you set a custom field delimiter, the <em>TEXT</em> tables cannot be directly used in Hive.
-In order to specify the custom field delimiter in Hive, you need to use <code class="docutils literal"><span class="pre">ROW</span> <span class="pre">FORMAT</span> <span class="pre">DELIMITED</span> <span class="pre">FIELDS</span> <span class="pre">TERMINATED</span> <span class="pre">BY</span></code>
-clause in a Hive’s <code class="docutils literal"><span class="pre">CREATE</span> <span class="pre">TABLE</span></code> statement as follows:</p>
+In order to specify the custom field delimiter in Hive, you need to use <tt class="docutils literal"><span class="pre">ROW</span> <span class="pre">FORMAT</span> <span class="pre">DELIMITED</span> <span class="pre">FIELDS</span> <span class="pre">TERMINATED</span> <span class="pre">BY</span></tt>
+clause in a Hive’s <tt class="docutils literal"><span class="pre">CREATE</span> <span class="pre">TABLE</span></tt> statement as follows:</p>
<div class="highlight-sql"><div class="highlight"><pre><span class="k">CREATE</span> <span class="k">TABLE</span> <span class="n">table1</span> <span class="p">(</span><span class="n">id</span> <span class="nb">int</span><span class="p">,</span> <span class="n">name</span> <span class="n">string</span><span class="p">,</span> <span class="n">score</span> <span class="nb">float</span><span class="p">,</span> <span class="k">type</span> <span class="n">string</span><span class="p">)</span>
<span class="k">ROW</span> <span class="n">FORMAT</span> <span class="n">DELIMITED</span> <span class="n">FIELDS</span> <span class="n">TERMINATED</span> <span class="k">BY</span> <span class="s1">'|'</span>
<span class="n">STORED</span> <span class="k">AS</span> <span class="nb">TEXT</span>
</pre></div>
</div>
-<p>To the best of our knowledge, there is not way to specify a custom <code class="docutils literal"><span class="pre">NULL</span></code> character in Hive.</p>
+<p>To the best of our knowledge, there is not way to specify a custom <tt class="docutils literal"><span class="pre">NULL</span></tt> character in Hive.</p>
</div>
</div>
@@ -327,7 +327,7 @@ clause in a Hive’s <code class="do
<div role="contentinfo">
<p>
- © Copyright 2015, Apache Tajo Team.
+ © Copyright 2016, Apache Tajo Team.
</p>
</div>
@@ -347,7 +347,7 @@ clause in a Hive’s <code class="do
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT:'../',
- VERSION:'0.11.0',
+ VERSION:'0.11.3',
COLLAPSE_INDEX:false,
FILE_SUFFIX:'.html',
HAS_SOURCE: true
Modified: tajo/site/docs/current/table_partitioning.html
URL: http://svn.apache.org/viewvc/tajo/site/docs/current/table_partitioning.html?rev=1744489&r1=1744488&r2=1744489&view=diff
==============================================================================
--- tajo/site/docs/current/table_partitioning.html (original)
+++ tajo/site/docs/current/table_partitioning.html Thu May 19 02:00:25 2016
@@ -7,7 +7,7 @@
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
- <title>Table Partitioning — Apache Tajo 0.11.0 documentation</title>
+ <title>Table Partitioning — Apache Tajo 0.11.3 documentation</title>
@@ -28,7 +28,7 @@
- <link rel="top" title="Apache Tajo 0.11.0 documentation" href="index.html"/>
+ <link rel="top" title="Apache Tajo 0.11.3 documentation" href="index.html"/>
<link rel="next" title="Introduction to Partitioning" href="partitioning/intro_to_partitioning.html"/>
<link rel="prev" title="Compression" href="table_management/compression.html"/>
@@ -236,7 +236,7 @@
<div role="contentinfo">
<p>
- © Copyright 2015, Apache Tajo Team.
+ © Copyright 2016, Apache Tajo Team.
</p>
</div>
@@ -256,7 +256,7 @@
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT:'./',
- VERSION:'0.11.0',
+ VERSION:'0.11.3',
COLLAPSE_INDEX:false,
FILE_SUFFIX:'.html',
HAS_SOURCE: true
Modified: tajo/site/docs/current/tajo_client_api.html
URL: http://svn.apache.org/viewvc/tajo/site/docs/current/tajo_client_api.html?rev=1744489&r1=1744488&r2=1744489&view=diff
==============================================================================
--- tajo/site/docs/current/tajo_client_api.html (original)
+++ tajo/site/docs/current/tajo_client_api.html Thu May 19 02:00:25 2016
@@ -7,7 +7,7 @@
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
- <title>Tajo Client API — Apache Tajo 0.11.0 documentation</title>
+ <title>Tajo Client API — Apache Tajo 0.11.3 documentation</title>
@@ -28,7 +28,7 @@
- <link rel="top" title="Apache Tajo 0.11.0 documentation" href="index.html"/>
+ <link rel="top" title="Apache Tajo 0.11.3 documentation" href="index.html"/>
<link rel="next" title="FAQ" href="faq.html"/>
<link rel="prev" title="Tajo JDBC Driver" href="jdbc_driver.html"/>
@@ -231,7 +231,7 @@
<div role="contentinfo">
<p>
- © Copyright 2015, Apache Tajo Team.
+ © Copyright 2016, Apache Tajo Team.
</p>
</div>
@@ -251,7 +251,7 @@
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT:'./',
- VERSION:'0.11.0',
+ VERSION:'0.11.3',
COLLAPSE_INDEX:false,
FILE_SUFFIX:'.html',
HAS_SOURCE: true