You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tajo.apache.org by ji...@apache.org on 2016/02/04 01:29:12 UTC
svn commit: r1728394 [36/42] - in /tajo/site/docs: 0.11.1/ 0.11.1/_sources/
0.11.1/_sources/backup_and_restore/ 0.11.1/_sources/configuration/
0.11.1/_sources/functions/ 0.11.1/_sources/index/
0.11.1/_sources/partitioning/ 0.11.1/_sources/sql_language/...
Modified: tajo/site/docs/current/_sources/index/how_to_use.txt
URL: http://svn.apache.org/viewvc/tajo/site/docs/current/_sources/index/how_to_use.txt?rev=1728394&r1=1728393&r2=1728394&view=diff
==============================================================================
--- tajo/site/docs/current/_sources/index/how_to_use.txt (original)
+++ tajo/site/docs/current/_sources/index/how_to_use.txt Thu Feb 4 00:29:05 2016
@@ -1,18 +1,19 @@
-*************************************
+*****************
How to use index?
-*************************************
+*****************
--------------------------------------
+---------------
1. Create index
--------------------------------------
+---------------
-The first step for utilizing index is index creation. You can create index using SQL (:doc:`/sql_language/ddl`) or Tajo API (:doc:`/tajo_client_api`). For example, you can create a BST index on the lineitem table by submitting the following SQL to Tajo.
+The first step for utilizing index is to create an index. You can create an index using SQL (:doc:`/sql_language/ddl`) or Tajo API (:doc:`/tajo_client_api`).
+For example, the following SQL statement will create a BST index on the lineitem table.
.. code-block:: sql
default> create index l_orderkey_idx on lineitem (l_orderkey);
-If the index is created successfully, you can see the information about that index as follows: ::
+If the index is created successfully, you can see the index information as follows: ::
default> \d lineitem
@@ -48,17 +49,18 @@ If the index is created successfully, yo
For more information about index creation, please refer to the above links.
--------------------------------------
+-----------------------------
2. Enable/disable index scans
--------------------------------------
+-----------------------------
-When an index is successfully created, you must enable the index scan feature as follows:
+Reading data using index is disabled by default.
+So, exploiting the created index, you need a further step, enabling 'index scan' as following:
.. code-block:: sql
default> \set INDEX_ENABLED true
-If you don't want to use the index scan feature anymore, you can simply disable it as follows:
+If you don't want to use index scan anymore, you can simply disable it as follows:
.. code-block:: sql
@@ -66,4 +68,12 @@ If you don't want to use the index scan
.. note::
- Once the index scan feature is enabled, Tajo currently always performs the index scan regardless of its efficiency. You should set this option when the expected number of retrieved tuples is sufficiently small.
+ Once index scan is enabled, Tajo will perform 'index scan' if possible. In some cases, it may cause performance
+ degradation. If you always want to get better performance, you should either enable or disable 'index scan'
+ according to selectivity. Usually, the performance gain of index will increase when the selectivity is low.
+
+---------------------------
+3. Index backup and restore
+---------------------------
+
+Tajo currently provides only the catalog backup and restore for index. Please refer to :doc:`/backup_and_restore/catalog` for more information about catalog backup and restore.
\ No newline at end of file
Modified: tajo/site/docs/current/_sources/index/types.txt
URL: http://svn.apache.org/viewvc/tajo/site/docs/current/_sources/index/types.txt?rev=1728394&r1=1728393&r2=1728394&view=diff
==============================================================================
--- tajo/site/docs/current/_sources/index/types.txt (original)
+++ tajo/site/docs/current/_sources/index/types.txt Thu Feb 4 00:29:05 2016
@@ -2,6 +2,6 @@
Index Types
*************************************
-Currently, Tajo supports only one type of index, ``TWO_LEVEL_BIN_TREE``, shortly ``BST``. The BST index is a kind of binary search tree which is extended to be permanently stored on disk. It consists of two levels of nodes; a leaf node indexes the keys with the positions of data in an HDFS block and a root node indexes the keys with the leaf node indices.
+Currently, Tajo supports only one type of index, ``TWO_LEVEL_BIN_TREE``, shortly ``BST``. The BST index is a kind of binary search tree which is extended to be permanently stored on disk. It consists of two levels of nodes; a leaf node indexes the keys with the offsets to data stored on HDFS, and a root node indexes the keys with the offsets to the leaf nodes.
-When an index scan is started, the query engine first reads the root node and finds the search key. If it finds a leaf node corresponding to the search key, it subsequently finds the search key in that leaf node. Finally, it directly reads a tuple corresponding to the search key from HDFS.
\ No newline at end of file
+When an index scan is started, the query engine first reads the root node and finds the search key. If it successfully finds a leaf node corresponding to the search key, it subsequently finds the search key in that leaf node. Finally, it directly reads a tuple corresponding to the search key from HDFS.
\ No newline at end of file
Modified: tajo/site/docs/current/_sources/index_overview.txt
URL: http://svn.apache.org/viewvc/tajo/site/docs/current/_sources/index_overview.txt?rev=1728394&r1=1728393&r2=1728394&view=diff
==============================================================================
--- tajo/site/docs/current/_sources/index_overview.txt (original)
+++ tajo/site/docs/current/_sources/index_overview.txt Thu Feb 4 00:29:05 2016
@@ -4,12 +4,6 @@ Index (Experimental Feature)
An index is a data structure that is used for efficient query processing. Using an index, the Tajo query engine can directly retrieve search values.
-This is still an experimental feature. In order to use indexes, you must check out the source code of the ``index_support`` branch::
-
- git clone -b index_support https://git-wip-us.apache.org/repos/asf/tajo.git tajo-index
-
-For the source code build, please refer to :doc:`getting_started`.
-
The following sections describe the supported index types, the query execution with an index, and the future works.
.. toctree::
Modified: tajo/site/docs/current/_sources/partitioning/column_partitioning.txt
URL: http://svn.apache.org/viewvc/tajo/site/docs/current/_sources/partitioning/column_partitioning.txt?rev=1728394&r1=1728393&r2=1728394&view=diff
==============================================================================
--- tajo/site/docs/current/_sources/partitioning/column_partitioning.txt (original)
+++ tajo/site/docs/current/_sources/partitioning/column_partitioning.txt Thu Feb 4 00:29:05 2016
@@ -11,29 +11,61 @@ How to Create a Column Partitioned Table
You can create a partitioned table by using the ``PARTITION BY`` clause. For a column partitioned table, you should use
the ``PARTITION BY COLUMN`` clause with partition keys.
-For example, assume there is a table ``orders`` composed of the following schema. ::
+For example, assume a table with the following schema.
- id INT,
- item_name TEXT,
- price FLOAT
+.. code-block:: sql
+
+ id INT,
+ name TEXT,
+ gender char(1),
+ grade TEXT,
+ country TEXT,
+ city TEXT,
+ phone TEXT
+ );
-Also, assume that you want to use ``order_date TEXT`` and ``ship_date TEXT`` as the partition keys.
-Then, you should create a table as follows:
+If you want to make country as partitioned column, your Tajo definition would be this:
.. code-block:: sql
- CREATE TABLE orders (
- id INT,
- item_name TEXT,
- price
- ) PARTITION BY COLUMN (order_date TEXT, ship_date TEXT);
+ CREATE TABLE student (
+ id INT,
+ name TEXT,
+ gender char(1),
+ grade TEXT,
+ city TEXT,
+ phone TEXT
+ ) PARTITION BY COLUMN (country TEXT);
+
+Let us assume you want to use more partition columns and parquet file format. Here's an example statement to create a table:
+
+.. code-block:: sql
+
+ CREATE TABLE student (
+ id INT,
+ name TEXT,
+ gender char(1),
+ grade TEXT,
+ phone TEXT
+ ) USING PARQUET
+ PARTITION BY COLUMN (country TEXT, city TEXT);
+
+The statement above creates the student table with id, name, grade, etc. The table is also partitioned and data is stored in parquet files.
+
+You might have noticed that while the partitioning key columns are a part of the table DDL, theyâre only listed in the ``PARTITION BY`` clause. In Tajo, as data is written to disk, each partition of data will be automatically split out into different folders, e.g. country=USA/city=NEWYORK. During a read operation, Tajo will use the folder structure to quickly locate the right partitions and also return the partitioning columns as columns in the result set.
+
==================================================
-Partition Pruning on Column Partitioned Tables
+Querying Partitioned Tables
==================================================
-The following predicates in the ``WHERE`` clause can be used to prune unqualified column partitions without processing
-during query planning phase.
+If a table created using the ``PARTITION BY`` clause, a query can do partition pruning and scan only a fraction of the table relevant to the partitions specified by the query. Tajo currently does partition pruning if the partition predicates are specified in the WHERE clause. For example, if table student is partitioned on column country and column city, the following query retrieves rows in ``country=KOREA\city=SEOUL`` directory.
+
+.. code-block:: sql
+
+ SELECT * FROM student WHERE country = 'KOREA' AND city = 'SEOUL';
+
+The following predicates in the ``WHERE`` clause can be used to prune column partitions during query planning phase.
* ``=``
* ``<>``
@@ -44,9 +76,160 @@ during query planning phase.
* LIKE predicates with a leading wild-card character
* IN list predicates
+
+==================================================
+Add data to Partition Table
+==================================================
+
+Tajo provides a very useful feature of dynamic partitioning. You don't need to use any syntax with both ``INSERT INTO ... SELECT`` and ``Create Table As Select(CTAS)`` statments for dynamic partitioning. Tajo will automatically filter the data, create directories, move filtered data to appropriate directory and create partition over it.
+
+For example, assume there are both ``student_source`` and ``student`` tables composed of the following schema.
+
+.. code-block:: sql
+
+ CREATE TABLE student_source (
+ id INT,
+ name TEXT,
+ gender char(1),
+ grade TEXT,
+ country TEXT,
+ city TEXT,
+ phone TEXT
+ );
+
+ CREATE TABLE student (
+ id INT,
+ name TEXT,
+ gender char(1),
+ grade TEXT,
+ phone TEXT
+ ) PARTITION BY COLUMN (country TEXT, city TEXT);
+
+
+How to INSERT dynamically to partition table
+--------------------------------------------------------
+
+If you want to load an entire country or an entire city in one fell swoop:
+
+.. code-block:: sql
+
+ INSERT OVERWRITE INTO student
+ SELECT id, name, gender, grade, phone, country, city
+ FROM student_source;
+
+
+How to CTAS dynamically to partition table
+--------------------------------------------------------
+
+when a partition table is created:
+
+.. code-block:: sql
+
+ DROP TABLE if exists student;
+
+ CREATE TABLE student (
+ id INT,
+ name TEXT,
+ gender char(1),
+ grade TEXT,
+ phone TEXT
+ ) PARTITION BY COLUMN (country TEXT, city TEXT)
+ AS SELECT id, name, gender, grade, phone, country, city
+ FROM student_source;
+
+
+.. note::
+
+ When loading data into a partition, itâs necessary to include the partition columns as the last columns in the query. The column names in the source query donât need to match the partition column names.
+
+
==================================================
Compatibility Issues with Apache Hiveâ¢
==================================================
If partitioned tables of Hive are created as external tables in Tajo, Tajo can process the Hive partitioned tables directly.
-There haven't known compatibility issues yet.
\ No newline at end of file
+
+
+How to create partition table
+--------------------------------------------------------
+
+If you create a partition table as follows in Tajo:
+
+.. code-block:: sql
+
+ default> CREATE TABLE student (
+ id INT,
+ name TEXT,
+ gender char(1),
+ grade TEXT,
+ phone TEXT
+ ) PARTITION BY COLUMN (country TEXT, city TEXT);
+
+
+And then you can get table information in Hive:
+
+.. code-block:: sql
+
+ hive> desc student;
+ OK
+ id int
+ name string
+ gender char(1)
+ grade string
+ phone string
+ country string
+ city string
+
+ # Partition Information
+ # col_name data_type comment
+
+ country string
+ city string
+
+
+Or as you create the table in Hive:
+
+.. code-block:: sql
+
+ hive > CREATE TABLE student (
+ id int,
+ name string,
+ gender char(1),
+ grade string,
+ phone string
+ ) PARTITIONED BY (country string, city string)
+ ROW FORMAT DELIMITED
+ FIELDS TERMINATED BY '|' ;
+
+You will see table information in Tajo:
+
+.. code-block:: sql
+
+ default> \d student;
+ table name: default.student
+ table uri: hdfs://your_hdfs_namespace/user/hive/warehouse/student
+ store type: TEXT
+ number of rows: 0
+ volume: 0 B
+ Options:
+ 'text.null'='\\N'
+ 'transient_lastDdlTime'='1438756422'
+ 'text.delimiter'='|'
+
+ schema:
+ id INT4
+ name TEXT
+ gender CHAR(1)
+ grade TEXT
+ phone TEXT
+
+ Partitions:
+ type:COLUMN
+ columns::default.student.country (TEXT), default.student.city (TEXT)
+
+
+How to add data to partition table
+--------------------------------------------------------
+
+In Tajo, you can add data dynamically to partition table of Hive with both ``INSERT INTO ... SELECT`` and ``Create Table As Select (CTAS)`` statments. Tajo will automatically filter the data to HiveMetastore, create directories and move filtered data to appropriate directory on the distributed file system.
+
Modified: tajo/site/docs/current/_sources/sql_language.txt
URL: http://svn.apache.org/viewvc/tajo/site/docs/current/_sources/sql_language.txt?rev=1728394&r1=1728393&r2=1728394&view=diff
==============================================================================
--- tajo/site/docs/current/_sources/sql_language.txt (original)
+++ tajo/site/docs/current/_sources/sql_language.txt Thu Feb 4 00:29:05 2016
@@ -12,4 +12,5 @@ SQL Language
sql_language/queries
sql_language/joins
sql_language/sql_expression
- sql_language/predicates
\ No newline at end of file
+ sql_language/predicates
+ sql_language/explain
\ No newline at end of file
Added: tajo/site/docs/current/_sources/sql_language/explain.txt
URL: http://svn.apache.org/viewvc/tajo/site/docs/current/_sources/sql_language/explain.txt?rev=1728394&view=auto
==============================================================================
--- tajo/site/docs/current/_sources/sql_language/explain.txt (added)
+++ tajo/site/docs/current/_sources/sql_language/explain.txt Thu Feb 4 00:29:05 2016
@@ -0,0 +1,93 @@
+************************
+EXPLAIN
+************************
+
+*Synopsis*
+
+.. code-block:: sql
+
+ EXPLAIN [GLOBAL] statement
+
+
+*Description*
+
+Show the logical or global execution plan of a statement.
+
+
+*Examples*
+
+Logical plan:
+
+.. code-block:: sql
+
+ default> EXPLAIN SELECT l_orderkey, count(*) FROM lineitem GROUP BY l_orderkey;
+ explain
+ -------------------------------
+ GROUP_BY(1)(l_orderkey)
+ => exprs: (count())
+ => target list: default.lineitem.l_orderkey (INT8), ?count (INT8)
+ => out schema:{(2) default.lineitem.l_orderkey (INT8), ?count (INT8)}
+ => in schema:{(1) default.lineitem.l_orderkey (INT8)}
+ SCAN(0) on default.lineitem
+ => target list: default.lineitem.l_orderkey (INT8)
+ => out schema: {(1) default.lineitem.l_orderkey (INT8)}
+ => in schema: {(16) default.lineitem.l_orderkey (INT8), default.lineitem.l_partkey (INT8), default.lineitem.l_suppkey (INT8), default.lineitem.l_linenumber (INT8), default.lineitem.l_quantity (FLOAT8), default.lineitem.l_extendedprice (FLOAT8), default.lineitem.l_discount (FLOAT8), default.lineitem.l_tax (FLOAT8), default.lineitem.l_returnflag (TEXT), default.lineitem.l_linestatus (TEXT), default.lineitem.l_shipdate (DATE), default.lineitem.l_commitdate (DATE), default.lineitem.l_receiptdate (DATE), default.lineitem.l_shipinstruct (TEXT), default.lineitem.l_shipmode (TEXT), default.lineitem.l_comment (TEXT)}
+
+
+Global plan:
+
+.. code-block:: sql
+
+ default> EXPLAIN GLOBAL SELECT l_orderkey, count(*) FROM lineitem GROUP BY l_orderkey;
+ explain
+ -------------------------------
+ -------------------------------------------------------------------------------
+ Execution Block Graph (TERMINAL - eb_0000000000000_0000_000003)
+ -------------------------------------------------------------------------------
+ |-eb_0000000000000_0000_000003
+ |-eb_0000000000000_0000_000002
+ |-eb_0000000000000_0000_000001
+ -------------------------------------------------------------------------------
+ Order of Execution
+ -------------------------------------------------------------------------------
+ 1: eb_0000000000000_0000_000001
+ 2: eb_0000000000000_0000_000002
+ 3: eb_0000000000000_0000_000003
+ -------------------------------------------------------------------------------
+
+ =======================================================
+ Block Id: eb_0000000000000_0000_000001 [LEAF]
+ =======================================================
+
+ [Outgoing]
+ [q_0000000000000_0000] 1 => 2 (type=HASH_SHUFFLE, key=default.lineitem.l_orderkey (INT8), num=32)
+
+ GROUP_BY(5)(l_orderkey)
+ => exprs: (count())
+ => target list: default.lineitem.l_orderkey (INT8), ?count_1 (INT8)
+ => out schema:{(2) default.lineitem.l_orderkey (INT8), ?count_1 (INT8)}
+ => in schema:{(1) default.lineitem.l_orderkey (INT8)}
+ SCAN(0) on default.lineitem
+ => target list: default.lineitem.l_orderkey (INT8)
+ => out schema: {(1) default.lineitem.l_orderkey (INT8)}
+ => in schema: {(16) default.lineitem.l_orderkey (INT8), default.lineitem.l_partkey (INT8), default.lineitem.l_suppkey (INT8), default.lineitem.l_linenumber (INT8), default.lineitem.l_quantity (FLOAT8), default.lineitem.l_extendedprice (FLOAT8), default.lineitem.l_discount (FLOAT8), default.lineitem.l_tax (FLOAT8), default.lineitem.l_returnflag (TEXT), default.lineitem.l_linestatus (TEXT), default.lineitem.l_shipdate (DATE), default.lineitem.l_commitdate (DATE), default.lineitem.l_receiptdate (DATE), default.lineitem.l_shipinstruct (TEXT), default.lineitem.l_shipmode (TEXT), default.lineitem.l_comment (TEXT)}
+
+ =======================================================
+ Block Id: eb_0000000000000_0000_000002 [ROOT]
+ =======================================================
+
+ [Incoming]
+ [q_0000000000000_0000] 1 => 2 (type=HASH_SHUFFLE, key=default.lineitem.l_orderkey (INT8), num=32)
+
+ GROUP_BY(1)(l_orderkey)
+ => exprs: (count(?count_1 (INT8)))
+ => target list: default.lineitem.l_orderkey (INT8), ?count (INT8)
+ => out schema:{(2) default.lineitem.l_orderkey (INT8), ?count (INT8)}
+ => in schema:{(2) default.lineitem.l_orderkey (INT8), ?count_1 (INT8)}
+ SCAN(6) on eb_0000000000000_0000_000001
+ => out schema: {(2) default.lineitem.l_orderkey (INT8), ?count_1 (INT8)}
+ => in schema: {(2) default.lineitem.l_orderkey (INT8), ?count_1 (INT8)}
+
+ =======================================================
+ Block Id: eb_0000000000000_0000_000003 [TERMINAL]
+ =======================================================
\ No newline at end of file
Modified: tajo/site/docs/current/_sources/sql_language/joins.txt
URL: http://svn.apache.org/viewvc/tajo/site/docs/current/_sources/sql_language/joins.txt?rev=1728394&r1=1728393&r2=1728394&view=diff
==============================================================================
--- tajo/site/docs/current/_sources/sql_language/joins.txt (original)
+++ tajo/site/docs/current/_sources/sql_language/joins.txt Thu Feb 4 00:29:05 2016
@@ -155,6 +155,6 @@ Thus, they can perform join without expe
Tajo provides a session variable for broadcast join configuration. (For more detailed information of session variables, please refer to :doc:`/tsql/variables`.)
-* ``DIST_QUERY_BROADCAST_JOIN_THRESHOLD`` is a threshold for broadcast join. Only the relations who are larger than this value can be broadcasted.
+* ``BROADCAST_NON_CROSS_JOIN_THRESHOLD`` and ``BROADCAST_CROSS_JOIN_THRESHOLD`` are thresholds for broadcast join. Only the relations who are larger than this threshold can be broadcasted.
-You can also apply this configuration system widely by setting ``tajo.dist-query.join.broadcast.threshold-bytes`` in ``${TAJO_HOME}/conf/tajo-site.xml``.
\ No newline at end of file
+You can also apply this configuration system widely by setting ``tajo.dist-query.broadcast.non-cross-join.threshold-kb`` or ``tajo.dist-query.broadcast.cross-join.threshold-kb`` in ``${TAJO_HOME}/conf/tajo-site.xml``.
\ No newline at end of file
Modified: tajo/site/docs/current/_sources/sql_language/predicates.txt
URL: http://svn.apache.org/viewvc/tajo/site/docs/current/_sources/sql_language/predicates.txt?rev=1728394&r1=1728393&r2=1728394&view=diff
==============================================================================
--- tajo/site/docs/current/_sources/sql_language/predicates.txt (original)
+++ tajo/site/docs/current/_sources/sql_language/predicates.txt Thu Feb 4 00:29:05 2016
@@ -1,19 +1,19 @@
-*****************
+***********
Predicates
-*****************
+***********
-=====================
+=============
IN Predicate
-=====================
+=============
-IN predicate provides row and array comparison.
+IN predicate provides a comparison of row, array, and result of a subquery.
*Synopsis*
.. code-block:: sql
- column_reference IN (val1, val2, ..., valN)
- column_reference NOT IN (val1, val2, ..., valN)
+ column_reference (NOT) IN (val1, val2, ..., valN)
+ column_reference (NOT) IN (SELECT ... FROM ...) AS alias_name
Examples are as follows:
@@ -26,7 +26,7 @@ Examples are as follows:
-- this statement filters lists down all the records where col1 value is neither 1, 2 nor 3:
SELECT col1, col2 FROM table1 WHERE col1 NOT IN (1, 2, 3);
-You can use 'IN clause' on text data domain as follows:
+You can use `IN clause` on text data domain as follows:
.. code-block:: sql
@@ -34,6 +34,25 @@ You can use 'IN clause' on text data dom
SELECT col1, col2 FROM table1 WHERE col2 NOT IN ('tajo', 'hadoop');
+Finally, you can use subqueries in the `IN clause`.
+
+.. code-block:: sql
+
+ SELECT col1, col2
+ FROM table1
+ WHERE col3 IN (
+ SELECT avg(col2) as avg_col2
+ FROM table2
+ GROUP BY col1
+ HAVING avg_col2 > 100);
+
+ SELECT col1, col2
+ FROM table1
+ WHERE col3 NOT IN (
+ SELECT avg(col2) as avg_col2
+ FROM table2
+ GROUP BY col1
+ HAVING avg_col2 > 100);
==================================
String Pattern Matching Predicates
Modified: tajo/site/docs/current/_sources/sql_language/queries.txt
URL: http://svn.apache.org/viewvc/tajo/site/docs/current/_sources/sql_language/queries.txt?rev=1728394&r1=1728393&r2=1728394&view=diff
==============================================================================
--- tajo/site/docs/current/_sources/sql_language/queries.txt (original)
+++ tajo/site/docs/current/_sources/sql_language/queries.txt Thu Feb 4 00:29:05 2016
@@ -1,10 +1,10 @@
-**************************
+*******
Queries
-**************************
+*******
-=====================
+========
Overview
-=====================
+========
*Synopsis*
@@ -15,13 +15,13 @@ Overview
[WHERE <condition>]
[GROUP BY <expression> [, ...]]
[HAVING <condition>]
- [ORDER BY <expression> [ASC|DESC] [NULL FIRST|NULL LAST] [, ...]]
+ [ORDER BY <expression> [ASC|DESC] [NULLS (FIRST|LAST)] [, ...]]
-=====================
+===========
From Clause
-=====================
+===========
*Synopsis*
@@ -120,18 +120,20 @@ both join tables. These common columns a
**Subqueries**
-Subqueries allow users to specify a derived table. It requires enclosing a SQL statement in parentheses and an alias name.
-For example:
+A subquery is a query that is nested inside another query. It can be embedded in the FROM and WHERE clauses.
+
+Example:
.. code-block:: sql
- FROM (SELECT * FROM table1) AS alias_name
+ FROM (SELECT col1, sum(col2) FROM table1 WHERE col3 > 0 group by col1 order by col1) AS alias_name
+ WHERE col1 IN (SELECT col1 FROM table1 WHERE col2 > 0 AND col2 < 100) AS alias_name
For more detailed information, please refer to :doc:`joins`.
-=====================
+============
Where Clause
-=====================
+============
The syntax of the WHERE Clause is
@@ -181,15 +183,15 @@ Aggregation functions can be used with `
SELECT l_partkey, COUNT(distinct l_quantity), SUM(distinct l_extendedprice) AS total FROM lineitem GROUP BY l_partkey;
-==========================
+=========================
Orderby and Limit Clauses
-==========================
+=========================
*Synopsis*
.. code-block:: sql
- FROM ... ORDER BY <sort_expr> [(ASC|DESC)] [NULL (FIRST|LAST) [,...]
+ FROM ... ORDER BY <sort_expr> [(ASC|DESC)] [NULLS (FIRST|LAST) [,...]
``sort_expr`` can be a column reference, aliased column reference, or a complex expression.
``ASC`` indicates an ascending order of ``sort_expr`` values. ``DESC`` indicates a descending order of ``sort_expr`` values.
@@ -199,9 +201,9 @@ Orderby and Limit Clauses
before or after non-null values in the sort ordering. By default, null values are dealt as if larger than any non-null value;
that is, ``NULLS FIRST`` is the default for ``DESC`` order, and ``NULLS LAST`` otherwise.
-==========================
+================
Window Functions
-==========================
+================
A window function performs a calculation across multiple table rows that belong to some window frame.
Modified: tajo/site/docs/current/_sources/table_management.txt
URL: http://svn.apache.org/viewvc/tajo/site/docs/current/_sources/table_management.txt?rev=1728394&r1=1728393&r2=1728394&view=diff
==============================================================================
--- tajo/site/docs/current/_sources/table_management.txt (original)
+++ tajo/site/docs/current/_sources/table_management.txt Thu Feb 4 00:29:05 2016
@@ -1,6 +1,6 @@
-******************
+****************
Table Management
-******************
+****************
In Tajo, a table is a logical view of one data sources. Logically, one table consists of a logical schema, partitions, URL, and various properties. Physically, A table can be a directory in HDFS, a single file, one HBase table, or a RDBMS table. In order to make good use of Tajo, users need to understand features and physical characteristics of their physical layout. This section explains all about table management.
@@ -9,5 +9,5 @@ In Tajo, a table is a logical view of on
table_management/table_overview
table_management/tablespaces
- table_management/file_formats
- table_management/compression
\ No newline at end of file
+ table_management/data_formats
+ table_management/compression
Modified: tajo/site/docs/current/_sources/table_management/compression.txt
URL: http://svn.apache.org/viewvc/tajo/site/docs/current/_sources/table_management/compression.txt?rev=1728394&r1=1728393&r2=1728394&view=diff
==============================================================================
--- tajo/site/docs/current/_sources/table_management/compression.txt (original)
+++ tajo/site/docs/current/_sources/table_management/compression.txt Thu Feb 4 00:29:05 2016
@@ -1,5 +1,22 @@
-*********************************
+***********
Compression
-*********************************
+***********
-.. todo::
\ No newline at end of file
+Using compression can make data size compact, thereby enabling efficient use of network bandwidth and storage. Most of Tajo data formats support data compression feature.
+Currently, compression configuration affects only for stored data format and it is enabled when a table is created with the proper table property(See `Create Table <../sql_language/ddl.html#create-table>`_).
+
+===========================================
+Compression Properties for each Data Format
+===========================================
+
+ .. csv-table:: Compression Properties
+
+ **Data Format**,**Property Name**,**Avaliable Values**
+ :doc:`text</table_management/text>`/:doc:`json</table_management/json>`/:doc:`rcfile</table_management/rcfile>`/:doc:`sequencefile</table_management/sequencefile>` [#f1]_,compression.codec,Fully Qualified Classname in Hadoop [#f2]_
+ :doc:`parquet</table_management/parquet>`,parquet.compression,uncompressed/snappy/gzip/lzo
+ :doc:`orc</table_management/orc>`,orc.compression.kind,none/snappy/zlib
+
+.. rubric:: Footnotes
+
+.. [#f1] For sequence file, you should specify 'compression.type' in addition to 'compression.codec'. Refer to :doc:`/table_management/sequencefile`.
+.. [#f2] All classes are available if they implement `org.apache.hadoop.io.compress.CompressionCodec <https://hadoop.apache.org/docs/current/api/org/apache/hadoop/io/compress/CompressionCodec.html>`_.
Added: tajo/site/docs/current/_sources/table_management/data_formats.txt
URL: http://svn.apache.org/viewvc/tajo/site/docs/current/_sources/table_management/data_formats.txt?rev=1728394&view=auto
==============================================================================
--- tajo/site/docs/current/_sources/table_management/data_formats.txt (added)
+++ tajo/site/docs/current/_sources/table_management/data_formats.txt Thu Feb 4 00:29:05 2016
@@ -0,0 +1,15 @@
+************
+Data Formats
+************
+
+Currently, Tajo provides following data formats:
+
+.. toctree::
+ :maxdepth: 1
+
+ text
+ json
+ rcfile
+ parquet
+ orc
+ sequencefile
\ No newline at end of file
Modified: tajo/site/docs/current/_sources/table_management/table_overview.txt
URL: http://svn.apache.org/viewvc/tajo/site/docs/current/_sources/table_management/table_overview.txt?rev=1728394&r1=1728393&r2=1728394&view=diff
==============================================================================
--- tajo/site/docs/current/_sources/table_management/table_overview.txt (original)
+++ tajo/site/docs/current/_sources/table_management/table_overview.txt Thu Feb 4 00:29:05 2016
@@ -1,10 +1,12 @@
-*************************************
+***********************
Overview of Tajo Tables
-*************************************
+***********************
+========
Overview
========
+===========
Tablespaces
===========
@@ -12,8 +14,9 @@ Tablespaces is a physical location where
Please refer to :doc:`/table_management/tablespaces` if you want to know more information about tablespaces.
+=============
Managed Table
-================
+=============
``CREATE TABLE`` statement lets you create a table located in the warehouse directory specified by the configuration property ``tajo.warehouse.directory`` or ``${tajo.root}/warehouse`` by default. For example:
@@ -26,8 +29,9 @@ Managed Table
);
+==============
External Table
-================
+==============
``CREATE EXTERNAL TABLE`` statement lets you create a table located in a specify location so that Tajo does not use a default data warehouse location for the table. External tables are in common used if you already have data generated. LOCATION clause must be required for an external table.
@@ -65,7 +69,7 @@ The following example is to set a custom
'text.null'='\\N',
'compression.codec'='org.apache.hadoop.io.compress.SnappyCodec');
-Each physical table layout has its own specialized properties. They will be addressed in :doc:`/table_management/file_formats`.
+Each physical table layout has its own specialized properties. They will be addressed in :doc:`/table_management/data_formats`.
Common Table Properties
@@ -75,7 +79,8 @@ There are some common table properties w
Compression
-----------
-.. todo::
+
+See :doc:`compression`.
Time zone
---------
Modified: tajo/site/docs/current/_sources/table_management/tablespaces.txt
URL: http://svn.apache.org/viewvc/tajo/site/docs/current/_sources/table_management/tablespaces.txt?rev=1728394&r1=1728393&r2=1728394&view=diff
==============================================================================
--- tajo/site/docs/current/_sources/table_management/tablespaces.txt (original)
+++ tajo/site/docs/current/_sources/table_management/tablespaces.txt Thu Feb 4 00:29:05 2016
@@ -1,15 +1,36 @@
-*************************************
+***********
Tablespaces
-*************************************
+***********
-Tablespaces in Tajo allow users to define locations in the storage system where the files or data objects representing database objects can be stored. Once defined, a tablespace can be referred to by name when creating a database or a table. Especially, it is very useful when a Tajo cluster instance should use heterogeneous storage systems such as HDFS, MySQL, and Oracle.
+Tablespaces in Tajo allow users to define locations in the storage system where the files or data objects representing database objects can be stored.
+Once defined, a tablespace can be referred to by name when creating a database or a table.
+Especially, it is very useful when a Tajo cluster instance should use heterogeneous storage systems such as HDFS, MySQL, and Oracle.
+
+============================================
+External Table, Managed Table and Tablespace
+============================================
+
+Tajo has two types of table. One is external table. It needs **location** property when the table is created. Using this property, you can create an external table indicating existing external data source.
+For example, if there is already your data as Text/JSON files or HBase table, you can register it as tajo external table.
+Other one is managed table, which means internal table, that is created in a speficied tablespace.
+
+Tablespace is a pre-defined physical location where data stored on. It is supported for only managed tables.
+When you create a managed table, you can use the **tablespace** keyword to specify the location of data will be stored.
+If the tablespace is not specified, the default tablespace of the table's database is used.
+.. note::
+
+ For creating a table, see :doc:`/sql_language/ddl`.
+
+=============
Configuration
=============
-By default, Tajo use in ``${tajo.rootdir}/warehouse`` in ``conf/tajo-site.xml`` as a default tablespace. It also allows users to register additional tablespaces.
+By default, Tajo use ``${tajo.rootdir}/warehouse`` in :doc:`conf/tajo-site.xml</configuration/tajo-site-xml>` as a default tablespace. It also allows users to register additional tablespaces using ``storage-site.json`` file like below.
-``conf/storage-site.json`` file.
+---------------------------
+conf/storage-site.json file
+---------------------------
The configuration file has the following struct:
@@ -17,8 +38,8 @@ The configuration file has the following
{
"spaces": {
- "${table_space_name}": {
- "uri": "hbase://quorum1:port,quorum2:port/"
+ "${tablespace_name}": {
+ "uri": "hbase:zk://quorum1:port,quorum2:port/"
}
}
}
@@ -29,8 +50,8 @@ The following is an example for two tabl
{
"spaces": {
- "hbase-cluster1": {
- "uri": "hbase://quorum1:port,quorum2:port/"
+ "hbase_cluster1": {
+ "uri": "hbase:zk://quorum1:port,quorum2:port/"
},
"ssd": {
@@ -39,6 +60,8 @@ The following is an example for two tabl
}
}
+For more details, see :doc:`conf/storage-site.json</configuration/storage-site-json>`.
+
.. note::
Modified: tajo/site/docs/current/_sources/time_zone.txt
URL: http://svn.apache.org/viewvc/tajo/site/docs/current/_sources/time_zone.txt?rev=1728394&r1=1728393&r2=1728394&view=diff
==============================================================================
--- tajo/site/docs/current/_sources/time_zone.txt (original)
+++ tajo/site/docs/current/_sources/time_zone.txt Thu Feb 4 00:29:05 2016
@@ -19,7 +19,7 @@ You can set the system time zone in *con
.. code-block:: xml
<name>tajo.timezone</name>
- <property>GMT+9</property>
+ <value>GMT+9</value>
==================
Modified: tajo/site/docs/current/_sources/tsql/variables.txt
URL: http://svn.apache.org/viewvc/tajo/site/docs/current/_sources/tsql/variables.txt?rev=1728394&r1=1728393&r2=1728394&view=diff
==============================================================================
--- tajo/site/docs/current/_sources/tsql/variables.txt (original)
+++ tajo/site/docs/current/_sources/tsql/variables.txt Thu Feb 4 00:29:05 2016
@@ -2,8 +2,10 @@
Session Variables
*********************************
+Once a tajo client connects to the Tajo master, it assigns a unique session. This session is kept until the client is disconnected or it is expired.
-Each client connection to TajoMaster creates a unique session, and the client and TajoMaster uses the session until disconnect. A session provides session variables which are used for various configs per session.
+For the sake of more convenient user configuration, Tajo provides `session variables`.
+With session variables, different configurations are allowed for each session.
``tsql`` provides the meta command ``\set`` to manipulate session variables. Just ``\set`` command shows all session variables. ::
@@ -28,35 +30,477 @@ Each client connection to TajoMaster cre
Also, ``\unset key`` will unset the session variable named *key*.
-Now, tajo provides the following session variables.
+Currently, tajo provides the following session variables.
+
+.. describe:: BROADCAST_NON_CROSS_JOIN_THRESHOLD
+
+A threshold for non-cross joins. When a non-cross join query is executed with the broadcast join, the whole size of broadcasted tables won't exceed this threshold.
+
+ * Configuration name: :ref:`tajo.dist-query.broadcast.non-cross-join.threshold-kb`
+ * Property value: Integer
+ * Unit: KB
+ * Default value: 5120
+ * Example
+
+.. code-block:: sh
+
+ \set BROADCAST_NON_CROSS_JOIN_THRESHOLD 5120
+
+.. describe:: BROADCAST_CROSS_JOIN_THRESHOLD
+
+A threshold for cross joins. When a cross join query is executed, the whole size of broadcasted tables won't exceed this threshold.
+
+ * Configuration name: :ref:`tajo.dist-query.broadcast.cross-join.threshold-kb`
+ * Property value: Integer
+ * Unit: KB
+ * Default value: 1024
+ * Example
+
+.. code-block:: sh
+
+ \set BROADCAST_CROSS_JOIN_THRESHOLD 1024
+
+.. warning::
+ In Tajo, the broadcast join is only the way to perform cross joins. Since the cross join is a very expensive operation, this value need to be tuned carefully.
+
+.. describe:: JOIN_TASK_INPUT_SIZE
+
+The repartition join is executed in two stages. When a join query is executed with the repartition join, this value indicates the amount of input data processed by each task at the second stage.
+As a result, it determines the degree of the parallel processing of the join query.
+
+ * Configuration name: :ref:`tajo.dist-query.join.task-volume-mb`
+ * Property value: Integer
+ * Unit: MB
+ * Default value: 64
+ * Example
+
+.. code-block:: sh
+
+ \set JOIN_TASK_INPUT_SIZE 64
+
+.. describe:: JOIN_PER_SHUFFLE_SIZE
+
+The repartition join is executed in two stages. When a join query is executed with the repartition join,
+this value indicates the output size of each task at the first stage, which determines the number of partitions to be shuffled between two stages.
+
+ * Configuration name: :ref:`tajo.dist-query.join.partition-volume-mb`
+ * Property value: Integer
+ * Unit: MB
+ * Default value: 128
+ * Example
+
+.. code-block:: sh
+
+ \set JOIN_PER_SHUFFLE_SIZE 128
+
+.. describe:: HASH_JOIN_SIZE_LIMIT
+
+This value provides the criterion to decide the algorithm to perform a join in a task.
+If the input data is smaller than this value, join is performed with the in-memory hash join.
+Otherwise, the sort-merge join is used.
+
+ * Configuration name: :ref:`tajo.executor.join.common.in-memory-hash-threshold-mb`
+ * Property value: Integer
+ * Unit: MB
+ * Default value: 64
+ * Example
+
+.. code-block:: sh
+
+ \set HASH_JOIN_SIZE_LIMIT 64
+
+.. warning::
+ This value is the size of the input stored on file systems. So, when the input data is loaded into JVM heap,
+ its actual size is usually much larger than the configured value, which means that too large threshold can cause unexpected OutOfMemory errors.
+ This value should be tuned carefully.
+
+.. describe:: INNER_HASH_JOIN_SIZE_LIMIT
+
+This value provides the criterion to decide the algorithm to perform an inner join in a task.
+If the input data is smaller than this value, the inner join is performed with the in-memory hash join.
+Otherwise, the sort-merge join is used.
+
+ * Configuration name: :ref:`tajo.executor.join.inner.in-memory-hash-threshold-mb`
+ * Property value: Integer
+ * Unit: MB
+ * Default value: 64
+ * Example
+
+.. code-block:: sh
+
+ \set INNER_HASH_JOIN_SIZE_LIMIT 64
+
+.. warning::
+ This value is the size of the input stored on file systems. So, when the input data is loaded into JVM heap,
+ its actual size is usually much larger than the configured value, which means that too large threshold can cause unexpected OutOfMemory errors.
+ This value should be tuned carefully.
+
+.. describe:: OUTER_HASH_JOIN_SIZE_LIMIT
+
+This value provides the criterion to decide the algorithm to perform an outer join in a task.
+If the input data is smaller than this value, the outer join is performed with the in-memory hash join.
+Otherwise, the sort-merge join is used.
+
+ * Configuration name: :ref:`tajo.executor.join.outer.in-memory-hash-threshold-mb`
+ * Property value: Integer
+ * Unit: MB
+ * Default value: 64
+ * Example
+
+.. code-block:: sh
+
+ \set OUTER_HASH_JOIN_SIZE_LIMIT 64
+
+.. warning::
+ This value is the size of the input stored on file systems. So, when the input data is loaded into JVM heap,
+ its actual size is usually much larger than the configured value, which means that too large threshold can cause unexpected OutOfMemory errors.
+ This value should be tuned carefully.
+
+.. describe:: JOIN_HASH_TABLE_SIZE
+
+The initial size of hash table for in-memory hash join.
+
+ * Configuration name: :ref:`tajo.executor.join.hash-table.size`
+ * Property value: Integer
+ * Default value: 100000
+ * Example
+
+.. code-block:: sh
+
+ \set JOIN_HASH_TABLE_SIZE 100000
+
+.. describe:: SORT_TASK_INPUT_SIZE
+
+The sort operation is executed in two stages. When a sort query is executed, this value indicates the amount of input data processed by each task at the second stage.
+As a result, it determines the degree of the parallel processing of the sort query.
+
+ * Configuration name: :ref:`tajo.dist-query.sort.task-volume-mb`
+ * Property value: Integer
+ * Unit: MB
+ * Default value: 64
+ * Example
+
+.. code-block:: sh
+
+ \set SORT_TASK_INPUT_SIZE 64
+
+.. describe:: EXTSORT_BUFFER_SIZE
+
+A threshold to choose the sort algorithm. If the input data is larger than this threshold, the external sort algorithm is used.
+
+ * Configuration name: :ref:`tajo.executor.external-sort.buffer-mb`
+ * Property value: Integer
+ * Unit: MB
+ * Default value: 200
+ * Example
+
+.. code-block:: sh
+
+ \set EXTSORT_BUFFER_SIZE 200
+
+.. describe:: SORT_LIST_SIZE
+
+The initial size of list for in-memory sort.
+
+ * Configuration name: :ref:`tajo.executor.sort.list.size`
+ * Property value: Integer
+ * Default value: 100000
+ * Example
+
+.. code-block:: sh
+
+ \set SORT_LIST_SIZE 100000
+
+.. describe:: GROUPBY_MULTI_LEVEL_ENABLED
+
+A flag to enable the multi-level algorithm for distinct aggregation. If this value is set, 3-phase aggregation algorithm is used.
+Otherwise, 2-phase aggregation algorithm is used.
+
+ * Configuration name: :ref:`tajo.dist-query.groupby.multi-level-aggr`
+ * Property value: Boolean
+ * Default value: true
+ * Example
+
+.. code-block:: sh
+
+ \set GROUPBY_MULTI_LEVEL_ENABLED true
+
+.. describe:: GROUPBY_PER_SHUFFLE_SIZE
+
+The aggregation is executed in two stages. When an aggregation query is executed,
+this value indicates the output size of each task at the first stage, which determines the number of partitions to be shuffled between two stages.
+
+ * Configuration name: :ref:`tajo.dist-query.groupby.partition-volume-mb`
+ * Property value: Integer
+ * Unit: MB
+ * Default value: 256
+ * Example
+
+.. code-block:: sh
+
+ \set GROUPBY_PER_SHUFFLE_SIZE 256
+
+.. describe:: GROUPBY_TASK_INPUT_SIZE
+
+The aggregation operation is executed in two stages. When an aggregation query is executed, this value indicates the amount of input data processed by each task at the second stage.
+As a result, it determines the degree of the parallel processing of the aggregation query.
+
+ * Configuration name: :ref:`tajo.dist-query.groupby.task-volume-mb`
+ * Property value: Integer
+ * Unit: MB
+ * Default value: 64
+ * Example
+
+.. code-block:: sh
+
+ \set GROUPBY_TASK_INPUT_SIZE 64
+
+.. describe:: HASH_GROUPBY_SIZE_LIMIT
+
+This value provides the criterion to decide the algorithm to perform an aggregation in a task.
+If the input data is smaller than this value, the aggregation is performed with the in-memory hash aggregation.
+Otherwise, the sort-based aggregation is used.
+
+ * Configuration name: :ref:`tajo.executor.groupby.in-memory-hash-threshold-mb`
+ * Property value: Integer
+ * Unit: MB
+ * Default value: 64
+ * Example
+
+.. code-block:: sh
+
+ \set HASH_GROUPBY_SIZE_LIMIT 64
+
+.. warning::
+ This value is the size of the input stored on file systems. So, when the input data is loaded into JVM heap,
+ its actual size is usually much larger than the configured value, which means that too large threshold can cause unexpected OutOfMemory errors.
+ This value should be tuned carefully.
+
+.. describe:: AGG_HASH_TABLE_SIZE
+
+The initial size of hash table for in-memory aggregation.
+
+ * Configuration name: :ref:`tajo.executor.aggregate.hash-table.size`
+ * Property value: Integer
+ * Default value: 10000
+ * Example
+
+.. code-block:: sh
+
+ \set AGG_HASH_TABLE_SIZE 10000
+
+.. describe:: TIMEZONE
+
+Refer to :doc:`/time_zone`.
+
+ * Configuration name: :ref:`tajo.timezone`
+ * Property value: Time zone id
+ * Default value: Default time zone of JVM
+ * Example
+
+.. code-block:: sh
+
+ \set TIMEZONE GMT+9
+
+.. describe:: DATE_ORDER
+
+Date order specification.
+
+ * Configuration name: :ref:`tajo.datetime.date-order`
+ * Property value: One of YMD, DMY, MDY.
+ * Default value: YMD
+ * Example
+
+.. code-block:: sh
+
+ \set DATE_ORDER YMD
+
+.. describe:: PARTITION_NO_RESULT_OVERWRITE_ENABLED
+
+If this value is true, a partitioned table is overwritten even if a subquery leads to no result. Otherwise, the table data will be kept if there is no result.
+
+ * Configuration name: :ref:`tajo.partition.overwrite.even-if-no-result`
+ * Property value: Boolean
+ * Default value: false
+ * Example
+
+.. code-block:: sh
+
+ \set PARTITION_NO_RESULT_OVERWRITE_ENABLED false
+
+.. describe:: TABLE_PARTITION_PER_SHUFFLE_SIZE
+
+In Tajo, storing a partition table is executed in two stages.
+This value indicates the output size of a task of the former stage, which determines the number of partitions to be shuffled between two stages.
+
+ * Configuration name: :ref:`tajo.dist-query.table-partition.task-volume-mb`
+ * Property value: Integer
+ * Unit: MB
+ * Default value: 256
+ * Example
+
+.. code-block:: sh
+
+ \set TABLE_PARTITION_PER_SHUFFLE_SIZE 256
+
+.. describe:: ARITHABORT
+
+A flag to indicate how to handle the errors caused by invalid arithmetic operations. If true, a running query will be terminated with an overflow or a divide-by-zero.
+
+ * Configuration name: :ref:`tajo.behavior.arithmetic-abort`
+ * Property value: Boolean
+ * Default value: false
+ * Example
+
+.. code-block:: sh
+
+ \set ARITHABORT false
+
+.. describe:: MAX_OUTPUT_FILE_SIZE
+
+Maximum per-output file size. 0 means infinite.
+
+ * Property value: Integer
+ * Unit: MB
+ * Default value: 0
+ * Example
+
+.. code-block:: sh
+
+ \set MAX_OUTPUT_FILE_SIZE 0
+
+.. describe:: SESSION_EXPIRY_TIME
+
+Session expiry time.
+
+ * Property value: Integer
+ * Unit: seconds
+ * Default value: 3600
+ * Example
+
+.. code-block:: sh
+
+ \set SESSION_EXPIRY_TIME 3600
+
+.. describe:: CLI_COLUMNS
+
+Sets the width for the wrapped format.
+
+ * Property value: Integer
+ * Default value: 120
+ * Example
+
+.. code-block:: sh
+
+ \set CLI_COLUMNS 120
+
+.. describe:: CLI_NULL_CHAR
+
+Sets the string to be printed in place of a null value.
+
+ * Property value: String
+ * Default value: ''
+ * Example
+
+.. code-block:: sh
+
+ \set CLI_NULL_CHAR ''
+
+.. describe:: CLI_PAGE_ROWS
+
+Sets the number of rows for paging.
+
+ * Property value: Integer
+ * Default value: 100
+ * Example
+
+.. code-block:: sh
+
+ \set CLI_PAGE_ROWS 100
+
+.. describe:: CLI_PAGING_ENABLED
+
+Enable paging of result display.
+
+ * Property value: Boolean
+ * Default value: true
+ * Example
+
+.. code-block:: sh
+
+ \set CLI_PAGING_ENABLED true
+
+.. describe:: CLI_DISPLAY_ERROR_TRACE
+
+Enable display of error trace.
+
+ * Property value: Boolean
+ * Default value: true
+ * Example
+
+.. code-block:: sh
+
+ \set CLI_DISPLAY_ERROR_TRACE true
+
+.. describe:: CLI_FORMATTER_CLASS
+
+Sets the output format class to display results.
+
+ * Property value: Class name
+ * Default value: org.apache.tajo.cli.tsql.DefaultTajoCliOutputFormatter
+ * Example
+
+.. code-block:: sh
+
+ \set CLI_FORMATTER_CLASS org.apache.tajo.cli.tsql.DefaultTajoCliOutputFormatter
+
+.. describe:: ON_ERROR_STOP
+
+tsql will exit if an error occurs.
+
+ * Property value: Boolean
+ * Default value: false
+ * Example
+
+.. code-block:: sh
+
+ \set ON_ERROR_STOP false
+
+.. describe:: NULL_CHAR
+
+Null char of text file output. This value is used when the table property `text.null` is not specified.
+
+ * Property value: String
+ * Default value: '\\N'
+ * Example
+
+.. code-block:: sh
+
+ \set NULL_CHAR '\\N'
+
+.. describe:: DEBUG_ENABLED
+
+A flag to enable debug mode.
+
+ * Property value: Boolean
+ * Default value: false
+ * Example
+
+.. code-block:: sh
+
+ \set DEBUG_ENABLED false
+
+.. describe:: FETCH_ROWNUM
+
+The number of rows to be fetched from Master each time.
+
+ * Property value: Integer
+ * Default value: 200
+ * Example
+
+.. code-block:: sh
+
+ \set FETCH_ROWNUM 200
-* ``DIST_QUERY_BROADCAST_JOIN_THRESHOLD``
-* ``DIST_QUERY_JOIN_TASK_VOLUME``
-* ``DIST_QUERY_SORT_TASK_VOLUME``
-* ``DIST_QUERY_GROUPBY_TASK_VOLUME``
-* ``DIST_QUERY_JOIN_PARTITION_VOLUME``
-* ``DIST_QUERY_GROUPBY_PARTITION_VOLUME``
-* ``DIST_QUERY_TABLE_PARTITION_VOLUME``
-* ``EXECUTOR_EXTERNAL_SORT_BUFFER_SIZE``
-* ``EXECUTOR_HASH_JOIN_SIZE_THRESHOLD``
-* ``EXECUTOR_INNER_HASH_JOIN_SIZE_THRESHOLD``
-* ``EXECUTOR_OUTER_HASH_JOIN_SIZE_THRESHOLD``
-* ``EXECUTOR_GROUPBY_INMEMORY_HASH_THRESHOLD``
-* ``MAX_OUTPUT_FILE_SIZE``
-* ``CODEGEN``
-* ``CLIENT_SESSION_EXPIRY_TIME``
-* ``CLI_MAX_COLUMN``
-* ``CLI_NULL_CHAR``
-* ``CLI_PRINT_PAUSE_NUM_RECORDS``
-* ``CLI_PRINT_PAUSE``
-* ``CLI_PRINT_ERROR_TRACE``
-* ``CLI_OUTPUT_FORMATTER_CLASS``
-* ``CLI_ERROR_STOP``
-* ``TIMEZONE``
-* ``DATE_ORDER``
-* ``TEXT_NULL``
-* ``DEBUG_ENABLED``
-* ``BEHAVIOR_ARITHMETIC_ABORT``
-* ``RESULT_SET_FETCH_ROWNUM``
Modified: tajo/site/docs/current/_static/basic.css
URL: http://svn.apache.org/viewvc/tajo/site/docs/current/_static/basic.css?rev=1728394&r1=1728393&r2=1728394&view=diff
==============================================================================
--- tajo/site/docs/current/_static/basic.css (original)
+++ tajo/site/docs/current/_static/basic.css Thu Feb 4 00:29:05 2016
@@ -4,7 +4,7 @@
*
* Sphinx stylesheet -- basic theme.
*
- * :copyright: Copyright 2007-2015 by the Sphinx team, see AUTHORS.
+ * :copyright: Copyright 2007-2016 by the Sphinx team, see AUTHORS.
* :license: BSD, see LICENSE for details.
*
*/
Modified: tajo/site/docs/current/_static/doctools.js
URL: http://svn.apache.org/viewvc/tajo/site/docs/current/_static/doctools.js?rev=1728394&r1=1728393&r2=1728394&view=diff
==============================================================================
--- tajo/site/docs/current/_static/doctools.js (original)
+++ tajo/site/docs/current/_static/doctools.js Thu Feb 4 00:29:05 2016
@@ -4,7 +4,7 @@
*
* Sphinx JavaScript utilities for all documentation.
*
- * :copyright: Copyright 2007-2015 by the Sphinx team, see AUTHORS.
+ * :copyright: Copyright 2007-2016 by the Sphinx team, see AUTHORS.
* :license: BSD, see LICENSE for details.
*
*/
Modified: tajo/site/docs/current/_static/pygments.css
URL: http://svn.apache.org/viewvc/tajo/site/docs/current/_static/pygments.css?rev=1728394&r1=1728393&r2=1728394&view=diff
==============================================================================
--- tajo/site/docs/current/_static/pygments.css (original)
+++ tajo/site/docs/current/_static/pygments.css Thu Feb 4 00:29:05 2016
@@ -9,8 +9,10 @@
.highlight .o { color: #ce5c00; font-weight: bold } /* Operator */
.highlight .x { color: #000000 } /* Other */
.highlight .p { color: #000000; font-weight: bold } /* Punctuation */
+.highlight .ch { color: #8f5902; font-style: italic } /* Comment.Hashbang */
.highlight .cm { color: #8f5902; font-style: italic } /* Comment.Multiline */
.highlight .cp { color: #8f5902; font-style: italic } /* Comment.Preproc */
+.highlight .cpf { color: #8f5902; font-style: italic } /* Comment.PreprocFile */
.highlight .c1 { color: #8f5902; font-style: italic } /* Comment.Single */
.highlight .cs { color: #8f5902; font-style: italic } /* Comment.Special */
.highlight .gd { color: #a40000 } /* Generic.Deleted */
Modified: tajo/site/docs/current/_static/searchtools.js
URL: http://svn.apache.org/viewvc/tajo/site/docs/current/_static/searchtools.js?rev=1728394&r1=1728393&r2=1728394&view=diff
==============================================================================
--- tajo/site/docs/current/_static/searchtools.js (original)
+++ tajo/site/docs/current/_static/searchtools.js Thu Feb 4 00:29:05 2016
@@ -4,12 +4,13 @@
*
* Sphinx JavaScript utilties for the full-text search.
*
- * :copyright: Copyright 2007-2015 by the Sphinx team, see AUTHORS.
+ * :copyright: Copyright 2007-2016 by the Sphinx team, see AUTHORS.
* :license: BSD, see LICENSE for details.
*
*/
+/* Non-minified version JS is _stemmer.js if file is provided */
/**
* Porter Stemmer
*/
@@ -373,8 +374,7 @@ var Search = {
}
// lookup as search terms in fulltext
- results = results.concat(this.performTermsSearch(searchterms, excluded, terms, Scorer.term))
- .concat(this.performTermsSearch(searchterms, excluded, titleterms, Scorer.title));
+ results = results.concat(this.performTermsSearch(searchterms, excluded, terms, titleterms));
// let the scorer override scores with a custom scoring function
if (Scorer.score) {
@@ -538,23 +538,47 @@ var Search = {
/**
* search for full-text terms in the index
*/
- performTermsSearch : function(searchterms, excluded, terms, score) {
+ performTermsSearch : function(searchterms, excluded, terms, titleterms) {
var filenames = this._index.filenames;
var titles = this._index.titles;
- var i, j, file, files;
+ var i, j, file;
var fileMap = {};
+ var scoreMap = {};
var results = [];
// perform the search on the required terms
for (i = 0; i < searchterms.length; i++) {
var word = searchterms[i];
+ var files = [];
+ var _o = [
+ {files: terms[word], score: Scorer.term},
+ {files: titleterms[word], score: Scorer.title}
+ ];
+
// no match but word was a required one
- if ((files = terms[word]) === undefined)
+ if ($u.every(_o, function(o){return o.files === undefined;})) {
break;
- if (files.length === undefined) {
- files = [files];
}
+ // found search word in contents
+ $u.each(_o, function(o) {
+ var _files = o.files;
+ if (_files === undefined)
+ return
+
+ if (_files.length === undefined)
+ _files = [_files];
+ files = files.concat(_files);
+
+ // set score for the word in each file to Scorer.term
+ for (j = 0; j < _files.length; j++) {
+ file = _files[j];
+ if (!(file in scoreMap))
+ scoreMap[file] = {}
+ scoreMap[file][word] = o.score;
+ }
+ });
+
// create the mapping
for (j = 0; j < files.length; j++) {
file = files[j];
@@ -576,7 +600,9 @@ var Search = {
// ensure that none of the excluded terms is in the search result
for (i = 0; i < excluded.length; i++) {
if (terms[excluded[i]] == file ||
- $u.contains(terms[excluded[i]] || [], file)) {
+ titleterms[excluded[i]] == file ||
+ $u.contains(terms[excluded[i]] || [], file) ||
+ $u.contains(titleterms[excluded[i]] || [], file)) {
valid = false;
break;
}
@@ -584,6 +610,9 @@ var Search = {
// if we have still a valid result we can add it to the result list
if (valid) {
+ // select one (max) score for the file.
+ // for better ranking, we should calculate ranking by using words statistics like basic tf-idf...
+ var score = $u.max($u.map(fileMap[file], function(w){return scoreMap[file][w]}));
results.push([filenames[file], titles[file], '', null, score]);
}
}
Modified: tajo/site/docs/current/_static/websupport.js
URL: http://svn.apache.org/viewvc/tajo/site/docs/current/_static/websupport.js?rev=1728394&r1=1728393&r2=1728394&view=diff
==============================================================================
--- tajo/site/docs/current/_static/websupport.js (original)
+++ tajo/site/docs/current/_static/websupport.js Thu Feb 4 00:29:05 2016
@@ -4,7 +4,7 @@
*
* sphinx.websupport utilties for all documentation.
*
- * :copyright: Copyright 2007-2015 by the Sphinx team, see AUTHORS.
+ * :copyright: Copyright 2007-2016 by the Sphinx team, see AUTHORS.
* :license: BSD, see LICENSE for details.
*
*/
Modified: tajo/site/docs/current/backup_and_restore.html
URL: http://svn.apache.org/viewvc/tajo/site/docs/current/backup_and_restore.html?rev=1728394&r1=1728393&r2=1728394&view=diff
==============================================================================
--- tajo/site/docs/current/backup_and_restore.html (original)
+++ tajo/site/docs/current/backup_and_restore.html Thu Feb 4 00:29:05 2016
@@ -77,6 +77,7 @@
<li class="toctree-l2"><a class="reference internal" href="configuration/service_config_defaults.html">Cluster Service Configuration Defaults</a></li>
<li class="toctree-l2"><a class="reference internal" href="configuration/tajo-site-xml.html">The tajo-site.xml File</a></li>
<li class="toctree-l2"><a class="reference internal" href="configuration/catalog-site-xml.html">The catalog-site.xml File</a></li>
+<li class="toctree-l2"><a class="reference internal" href="configuration/storage-site-json.html">The storage-site.json File</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="tsql.html">Tajo Shell (TSQL)</a><ul>
@@ -99,6 +100,7 @@
<li class="toctree-l2"><a class="reference internal" href="sql_language/joins.html">Joins</a></li>
<li class="toctree-l2"><a class="reference internal" href="sql_language/sql_expression.html">SQL Expressions</a></li>
<li class="toctree-l2"><a class="reference internal" href="sql_language/predicates.html">Predicates</a></li>
+<li class="toctree-l2"><a class="reference internal" href="sql_language/explain.html">EXPLAIN</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="time_zone.html">Time Zone</a><ul>
@@ -119,7 +121,7 @@
<li class="toctree-l1"><a class="reference internal" href="table_management.html">Table Management</a><ul>
<li class="toctree-l2"><a class="reference internal" href="table_management/table_overview.html">Overview of Tajo Tables</a></li>
<li class="toctree-l2"><a class="reference internal" href="table_management/tablespaces.html">Tablespaces</a></li>
-<li class="toctree-l2"><a class="reference internal" href="table_management/file_formats.html">File Formats</a></li>
+<li class="toctree-l2"><a class="reference internal" href="table_management/data_formats.html">Data Formats</a></li>
<li class="toctree-l2"><a class="reference internal" href="table_management/compression.html">Compression</a></li>
</ul>
</li>
@@ -231,7 +233,7 @@
<div role="contentinfo">
<p>
- © Copyright 2014, Apache Tajo Team.
+ © Copyright 2015, Apache Tajo Team.
</p>
</div>
Modified: tajo/site/docs/current/backup_and_restore/catalog.html
URL: http://svn.apache.org/viewvc/tajo/site/docs/current/backup_and_restore/catalog.html?rev=1728394&r1=1728393&r2=1728394&view=diff
==============================================================================
--- tajo/site/docs/current/backup_and_restore/catalog.html (original)
+++ tajo/site/docs/current/backup_and_restore/catalog.html Thu Feb 4 00:29:05 2016
@@ -78,6 +78,7 @@
<li class="toctree-l2"><a class="reference internal" href="../configuration/service_config_defaults.html">Cluster Service Configuration Defaults</a></li>
<li class="toctree-l2"><a class="reference internal" href="../configuration/tajo-site-xml.html">The tajo-site.xml File</a></li>
<li class="toctree-l2"><a class="reference internal" href="../configuration/catalog-site-xml.html">The catalog-site.xml File</a></li>
+<li class="toctree-l2"><a class="reference internal" href="../configuration/storage-site-json.html">The storage-site.json File</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="../tsql.html">Tajo Shell (TSQL)</a><ul>
@@ -100,6 +101,7 @@
<li class="toctree-l2"><a class="reference internal" href="../sql_language/joins.html">Joins</a></li>
<li class="toctree-l2"><a class="reference internal" href="../sql_language/sql_expression.html">SQL Expressions</a></li>
<li class="toctree-l2"><a class="reference internal" href="../sql_language/predicates.html">Predicates</a></li>
+<li class="toctree-l2"><a class="reference internal" href="../sql_language/explain.html">EXPLAIN</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="../time_zone.html">Time Zone</a><ul>
@@ -120,7 +122,7 @@
<li class="toctree-l1"><a class="reference internal" href="../table_management.html">Table Management</a><ul>
<li class="toctree-l2"><a class="reference internal" href="../table_management/table_overview.html">Overview of Tajo Tables</a></li>
<li class="toctree-l2"><a class="reference internal" href="../table_management/tablespaces.html">Tablespaces</a></li>
-<li class="toctree-l2"><a class="reference internal" href="../table_management/file_formats.html">File Formats</a></li>
+<li class="toctree-l2"><a class="reference internal" href="../table_management/data_formats.html">Data Formats</a></li>
<li class="toctree-l2"><a class="reference internal" href="../table_management/compression.html">Compression</a></li>
</ul>
</li>
@@ -270,7 +272,7 @@ CREATE EXTERNAL TABLE customer (c_custke
<div role="contentinfo">
<p>
- © Copyright 2014, Apache Tajo Team.
+ © Copyright 2015, Apache Tajo Team.
</p>
</div>
Modified: tajo/site/docs/current/configuration.html
URL: http://svn.apache.org/viewvc/tajo/site/docs/current/configuration.html?rev=1728394&r1=1728393&r2=1728394&view=diff
==============================================================================
--- tajo/site/docs/current/configuration.html (original)
+++ tajo/site/docs/current/configuration.html Thu Feb 4 00:29:05 2016
@@ -77,6 +77,7 @@
<li class="toctree-l2"><a class="reference internal" href="configuration/service_config_defaults.html">Cluster Service Configuration Defaults</a></li>
<li class="toctree-l2"><a class="reference internal" href="configuration/tajo-site-xml.html">The tajo-site.xml File</a></li>
<li class="toctree-l2"><a class="reference internal" href="configuration/catalog-site-xml.html">The catalog-site.xml File</a></li>
+<li class="toctree-l2"><a class="reference internal" href="configuration/storage-site-json.html">The storage-site.json File</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="tsql.html">Tajo Shell (TSQL)</a><ul>
@@ -99,6 +100,7 @@
<li class="toctree-l2"><a class="reference internal" href="sql_language/joins.html">Joins</a></li>
<li class="toctree-l2"><a class="reference internal" href="sql_language/sql_expression.html">SQL Expressions</a></li>
<li class="toctree-l2"><a class="reference internal" href="sql_language/predicates.html">Predicates</a></li>
+<li class="toctree-l2"><a class="reference internal" href="sql_language/explain.html">EXPLAIN</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="time_zone.html">Time Zone</a><ul>
@@ -119,7 +121,7 @@
<li class="toctree-l1"><a class="reference internal" href="table_management.html">Table Management</a><ul>
<li class="toctree-l2"><a class="reference internal" href="table_management/table_overview.html">Overview of Tajo Tables</a></li>
<li class="toctree-l2"><a class="reference internal" href="table_management/tablespaces.html">Tablespaces</a></li>
-<li class="toctree-l2"><a class="reference internal" href="table_management/file_formats.html">File Formats</a></li>
+<li class="toctree-l2"><a class="reference internal" href="table_management/data_formats.html">Data Formats</a></li>
<li class="toctree-l2"><a class="reference internal" href="table_management/compression.html">Compression</a></li>
</ul>
</li>
@@ -217,6 +219,7 @@
<li class="toctree-l1"><a class="reference internal" href="configuration/service_config_defaults.html">Cluster Service Configuration Defaults</a></li>
<li class="toctree-l1"><a class="reference internal" href="configuration/tajo-site-xml.html">The tajo-site.xml File</a></li>
<li class="toctree-l1"><a class="reference internal" href="configuration/catalog-site-xml.html">The catalog-site.xml File</a></li>
+<li class="toctree-l1"><a class="reference internal" href="configuration/storage-site-json.html">The storage-site.json File</a></li>
</ul>
</div>
</div>
@@ -239,7 +242,7 @@
<div role="contentinfo">
<p>
- © Copyright 2014, Apache Tajo Team.
+ © Copyright 2015, Apache Tajo Team.
</p>
</div>
Modified: tajo/site/docs/current/configuration/catalog-site-xml.html
URL: http://svn.apache.org/viewvc/tajo/site/docs/current/configuration/catalog-site-xml.html?rev=1728394&r1=1728393&r2=1728394&view=diff
==============================================================================
--- tajo/site/docs/current/configuration/catalog-site-xml.html (original)
+++ tajo/site/docs/current/configuration/catalog-site-xml.html Thu Feb 4 00:29:05 2016
@@ -30,7 +30,7 @@
<link rel="top" title="Apache Tajo 0.11.0 documentation" href="../index.html"/>
<link rel="up" title="Configuration" href="../configuration.html"/>
- <link rel="next" title="Tajo Shell (TSQL)" href="../tsql.html"/>
+ <link rel="next" title="The storage-site.json File" href="storage-site-json.html"/>
<link rel="prev" title="The tajo-site.xml File" href="tajo-site-xml.html"/>
@@ -78,6 +78,7 @@
<li class="toctree-l2"><a class="reference internal" href="service_config_defaults.html">Cluster Service Configuration Defaults</a></li>
<li class="toctree-l2"><a class="reference internal" href="tajo-site-xml.html">The tajo-site.xml File</a></li>
<li class="toctree-l2 current"><a class="current reference internal" href="">The catalog-site.xml File</a></li>
+<li class="toctree-l2"><a class="reference internal" href="storage-site-json.html">The storage-site.json File</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="../tsql.html">Tajo Shell (TSQL)</a><ul>
@@ -100,6 +101,7 @@
<li class="toctree-l2"><a class="reference internal" href="../sql_language/joins.html">Joins</a></li>
<li class="toctree-l2"><a class="reference internal" href="../sql_language/sql_expression.html">SQL Expressions</a></li>
<li class="toctree-l2"><a class="reference internal" href="../sql_language/predicates.html">Predicates</a></li>
+<li class="toctree-l2"><a class="reference internal" href="../sql_language/explain.html">EXPLAIN</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="../time_zone.html">Time Zone</a><ul>
@@ -120,7 +122,7 @@
<li class="toctree-l1"><a class="reference internal" href="../table_management.html">Table Management</a><ul>
<li class="toctree-l2"><a class="reference internal" href="../table_management/table_overview.html">Overview of Tajo Tables</a></li>
<li class="toctree-l2"><a class="reference internal" href="../table_management/tablespaces.html">Tablespaces</a></li>
-<li class="toctree-l2"><a class="reference internal" href="../table_management/file_formats.html">File Formats</a></li>
+<li class="toctree-l2"><a class="reference internal" href="../table_management/data_formats.html">Data Formats</a></li>
<li class="toctree-l2"><a class="reference internal" href="../table_management/compression.html">Compression</a></li>
</ul>
</li>
@@ -217,7 +219,7 @@
<div class="rst-footer-buttons" role="navigation" aria-label="footer navigation">
- <a href="../tsql.html" class="btn btn-neutral float-right" title="Tajo Shell (TSQL)"/>Next <span class="fa fa-arrow-circle-right"></span></a>
+ <a href="storage-site-json.html" class="btn btn-neutral float-right" title="The storage-site.json File"/>Next <span class="fa fa-arrow-circle-right"></span></a>
<a href="tajo-site-xml.html" class="btn btn-neutral" title="The tajo-site.xml File"><span class="fa fa-arrow-circle-left"></span> Previous</a>
@@ -229,7 +231,7 @@
<div role="contentinfo">
<p>
- © Copyright 2014, Apache Tajo Team.
+ © Copyright 2015, Apache Tajo Team.
</p>
</div>
Modified: tajo/site/docs/current/configuration/catalog_configuration.html
URL: http://svn.apache.org/viewvc/tajo/site/docs/current/configuration/catalog_configuration.html?rev=1728394&r1=1728393&r2=1728394&view=diff
==============================================================================
--- tajo/site/docs/current/configuration/catalog_configuration.html (original)
+++ tajo/site/docs/current/configuration/catalog_configuration.html Thu Feb 4 00:29:05 2016
@@ -78,6 +78,7 @@
<li class="toctree-l2"><a class="reference internal" href="service_config_defaults.html">Cluster Service Configuration Defaults</a></li>
<li class="toctree-l2"><a class="reference internal" href="tajo-site-xml.html">The tajo-site.xml File</a></li>
<li class="toctree-l2"><a class="reference internal" href="catalog-site-xml.html">The catalog-site.xml File</a></li>
+<li class="toctree-l2"><a class="reference internal" href="storage-site-json.html">The storage-site.json File</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="../tsql.html">Tajo Shell (TSQL)</a><ul>
@@ -100,6 +101,7 @@
<li class="toctree-l2"><a class="reference internal" href="../sql_language/joins.html">Joins</a></li>
<li class="toctree-l2"><a class="reference internal" href="../sql_language/sql_expression.html">SQL Expressions</a></li>
<li class="toctree-l2"><a class="reference internal" href="../sql_language/predicates.html">Predicates</a></li>
+<li class="toctree-l2"><a class="reference internal" href="../sql_language/explain.html">EXPLAIN</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="../time_zone.html">Time Zone</a><ul>
@@ -120,7 +122,7 @@
<li class="toctree-l1"><a class="reference internal" href="../table_management.html">Table Management</a><ul>
<li class="toctree-l2"><a class="reference internal" href="../table_management/table_overview.html">Overview of Tajo Tables</a></li>
<li class="toctree-l2"><a class="reference internal" href="../table_management/tablespaces.html">Tablespaces</a></li>
-<li class="toctree-l2"><a class="reference internal" href="../table_management/file_formats.html">File Formats</a></li>
+<li class="toctree-l2"><a class="reference internal" href="../table_management/data_formats.html">Data Formats</a></li>
<li class="toctree-l2"><a class="reference internal" href="../table_management/compression.html">Compression</a></li>
</ul>
</li>
@@ -288,16 +290,16 @@ Query OK, <span class="m">0</span> rows
</div>
<p>Second, you must install the proper JDBC driver on the TajoMaster node. And then, you need to set the <code class="docutils literal"><span class="pre">TAJO_CLASSPATH</span></code> variable in <code class="docutils literal"><span class="pre">conf/tajo-env.sh</span></code> as follows:</p>
<div class="highlight-sh"><div class="highlight"><pre><span class="o">(</span>MySQL<span class="o">)</span>
-<span class="nv">$ </span><span class="nb">export </span><span class="nv">TAJO_CLASSPATH</span><span class="o">=</span>/usr/local/mysql/lib/mysql-connector-java-x.x.x.jar
+$ <span class="nb">export</span> <span class="nv">TAJO_CLASSPATH</span><span class="o">=</span>/usr/local/mysql/lib/mysql-connector-java-x.x.x.jar
<span class="o">(</span>MariaDB<span class="o">)</span>
-<span class="nv">$ </span><span class="nb">export </span><span class="nv">TAJO_CLASSPATH</span><span class="o">=</span>/usr/local/mariadb/lib/mariadb-java-client-x.x.x.jar
+$ <span class="nb">export</span> <span class="nv">TAJO_CLASSPATH</span><span class="o">=</span>/usr/local/mariadb/lib/mariadb-java-client-x.x.x.jar
<span class="o">(</span>PostgreSQL<span class="o">)</span>
-<span class="nv">$ </span><span class="nb">export </span><span class="nv">TAJO_CLASSPATH</span><span class="o">=</span>/usr/share/java/postgresql-jdbc4.jar
+$ <span class="nb">export</span> <span class="nv">TAJO_CLASSPATH</span><span class="o">=</span>/usr/share/java/postgresql-jdbc4.jar
<span class="o">(</span>Oracle<span class="o">)</span>
-<span class="nv">$ </span><span class="nb">export </span><span class="nv">TAJO_CLASSPATH</span><span class="o">=</span>/path/to/oracle/driver/ojdbc7.jar
+$ <span class="nb">export</span> <span class="nv">TAJO_CLASSPATH</span><span class="o">=</span>/path/to/oracle/driver/ojdbc7.jar
</pre></div>
</div>
<p>Alternatively, you can copy the jdbc driver into <code class="docutils literal"><span class="pre">$TAJO_HOME/lib</span></code>.</p>
@@ -357,17 +359,17 @@ Query OK, <span class="m">0</span> rows
<h2>HiveCatalogStore Configuration<a class="headerlink" href="#hivecatalogstore-configuration" title="Permalink to this headline">¶</a></h2>
<p>Tajo support HiveCatalogStore to integrate with hive. If you want to use HiveCatalogStore, you just do as follows.</p>
<p>First, you must compile source code and get a binary archive as follows:</p>
-<div class="highlight-sh"><div class="highlight"><pre><span class="nv">$ </span>git clone https://git-wip-us.apache.org/repos/asf/tajo.git tajo
-<span class="nv">$ </span>mvn clean install -DskipTests -Pdist -Dtar
-<span class="nv">$ </span>ls tajo-dist/target/tajo-x.y.z-SNAPSHOT.tar.gz
+<div class="highlight-sh"><div class="highlight"><pre>$ git clone https://git-wip-us.apache.org/repos/asf/tajo.git tajo
+$ mvn clean install -DskipTests -Pdist -Dtar
+$ ls tajo-dist/target/tajo-x.y.z-SNAPSHOT.tar.gz
</pre></div>
</div>
<p>Second, you must set your hive home directory to HIVE_HOME variable in <code class="docutils literal"><span class="pre">conf/tajo-env.sh</span></code> with it as follows:</p>
-<div class="highlight-sh"><div class="highlight"><pre><span class="nb">export </span><span class="nv">HIVE_HOME</span><span class="o">=</span>/path/to/your/hive/directory
+<div class="highlight-sh"><div class="highlight"><pre><span class="nb">export</span> <span class="nv">HIVE_HOME</span><span class="o">=</span>/path/to/your/hive/directory
</pre></div>
</div>
<p>Third, if you need to use jdbc to connect HiveMetaStore, you have to prepare mysql jdbc driver on host which can be ran TajoMaster. If you prepare it, you should set jdbc driver file path to <code class="docutils literal"><span class="pre">HIVE_JDBC_DRIVER_DIR</span></code> variable in conf/tajo-env.sh with it as follows:</p>
-<div class="highlight-sh"><div class="highlight"><pre><span class="nb">export </span><span class="nv">HIVE_JDBC_DRIVER_DIR</span><span class="o">=</span>/path/to/your/mysql_jdbc_driver/mysql-connector-java-x.x.x-bin.jar
+<div class="highlight-sh"><div class="highlight"><pre><span class="nb">export</span> <span class="nv">HIVE_JDBC_DRIVER_DIR</span><span class="o">=</span>/path/to/your/mysql_jdbc_driver/mysql-connector-java-x.x.x-bin.jar
</pre></div>
</div>
<p>Lastly, you should add the following config to <code class="docutils literal"><span class="pre">conf/catalog-site.xml</span></code> :</p>
@@ -398,7 +400,7 @@ Query OK, <span class="m">0</span> rows
<div role="contentinfo">
<p>
- © Copyright 2014, Apache Tajo Team.
+ © Copyright 2015, Apache Tajo Team.
</p>
</div>
Modified: tajo/site/docs/current/configuration/cluster_setup.html
URL: http://svn.apache.org/viewvc/tajo/site/docs/current/configuration/cluster_setup.html?rev=1728394&r1=1728393&r2=1728394&view=diff
==============================================================================
--- tajo/site/docs/current/configuration/cluster_setup.html (original)
+++ tajo/site/docs/current/configuration/cluster_setup.html Thu Feb 4 00:29:05 2016
@@ -78,6 +78,7 @@
<li class="toctree-l2"><a class="reference internal" href="service_config_defaults.html">Cluster Service Configuration Defaults</a></li>
<li class="toctree-l2"><a class="reference internal" href="tajo-site-xml.html">The tajo-site.xml File</a></li>
<li class="toctree-l2"><a class="reference internal" href="catalog-site-xml.html">The catalog-site.xml File</a></li>
+<li class="toctree-l2"><a class="reference internal" href="storage-site-json.html">The storage-site.json File</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="../tsql.html">Tajo Shell (TSQL)</a><ul>
@@ -100,6 +101,7 @@
<li class="toctree-l2"><a class="reference internal" href="../sql_language/joins.html">Joins</a></li>
<li class="toctree-l2"><a class="reference internal" href="../sql_language/sql_expression.html">SQL Expressions</a></li>
<li class="toctree-l2"><a class="reference internal" href="../sql_language/predicates.html">Predicates</a></li>
+<li class="toctree-l2"><a class="reference internal" href="../sql_language/explain.html">EXPLAIN</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="../time_zone.html">Time Zone</a><ul>
@@ -120,7 +122,7 @@
<li class="toctree-l1"><a class="reference internal" href="../table_management.html">Table Management</a><ul>
<li class="toctree-l2"><a class="reference internal" href="../table_management/table_overview.html">Overview of Tajo Tables</a></li>
<li class="toctree-l2"><a class="reference internal" href="../table_management/tablespaces.html">Tablespaces</a></li>
-<li class="toctree-l2"><a class="reference internal" href="../table_management/file_formats.html">File Formats</a></li>
+<li class="toctree-l2"><a class="reference internal" href="../table_management/data_formats.html">Data Formats</a></li>
<li class="toctree-l2"><a class="reference internal" href="../table_management/compression.html">Compression</a></li>
</ul>
</li>
@@ -304,7 +306,7 @@ $ $HADOOP_HOME/bin/hadoop fs -chmod g+w
<div role="contentinfo">
<p>
- © Copyright 2014, Apache Tajo Team.
+ © Copyright 2015, Apache Tajo Team.
</p>
</div>
Modified: tajo/site/docs/current/configuration/ha_configuration.html
URL: http://svn.apache.org/viewvc/tajo/site/docs/current/configuration/ha_configuration.html?rev=1728394&r1=1728393&r2=1728394&view=diff
==============================================================================
--- tajo/site/docs/current/configuration/ha_configuration.html (original)
+++ tajo/site/docs/current/configuration/ha_configuration.html Thu Feb 4 00:29:05 2016
@@ -78,6 +78,7 @@
<li class="toctree-l2"><a class="reference internal" href="service_config_defaults.html">Cluster Service Configuration Defaults</a></li>
<li class="toctree-l2"><a class="reference internal" href="tajo-site-xml.html">The tajo-site.xml File</a></li>
<li class="toctree-l2"><a class="reference internal" href="catalog-site-xml.html">The catalog-site.xml File</a></li>
+<li class="toctree-l2"><a class="reference internal" href="storage-site-json.html">The storage-site.json File</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="../tsql.html">Tajo Shell (TSQL)</a><ul>
@@ -100,6 +101,7 @@
<li class="toctree-l2"><a class="reference internal" href="../sql_language/joins.html">Joins</a></li>
<li class="toctree-l2"><a class="reference internal" href="../sql_language/sql_expression.html">SQL Expressions</a></li>
<li class="toctree-l2"><a class="reference internal" href="../sql_language/predicates.html">Predicates</a></li>
+<li class="toctree-l2"><a class="reference internal" href="../sql_language/explain.html">EXPLAIN</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="../time_zone.html">Time Zone</a><ul>
@@ -120,7 +122,7 @@
<li class="toctree-l1"><a class="reference internal" href="../table_management.html">Table Management</a><ul>
<li class="toctree-l2"><a class="reference internal" href="../table_management/table_overview.html">Overview of Tajo Tables</a></li>
<li class="toctree-l2"><a class="reference internal" href="../table_management/tablespaces.html">Tablespaces</a></li>
-<li class="toctree-l2"><a class="reference internal" href="../table_management/file_formats.html">File Formats</a></li>
+<li class="toctree-l2"><a class="reference internal" href="../table_management/data_formats.html">Data Formats</a></li>
<li class="toctree-l2"><a class="reference internal" href="../table_management/compression.html">Compression</a></li>
</ul>
</li>
@@ -352,7 +354,7 @@ host2.domain.com
<div role="contentinfo">
<p>
- © Copyright 2014, Apache Tajo Team.
+ © Copyright 2015, Apache Tajo Team.
</p>
</div>
Modified: tajo/site/docs/current/configuration/preliminary.html
URL: http://svn.apache.org/viewvc/tajo/site/docs/current/configuration/preliminary.html?rev=1728394&r1=1728393&r2=1728394&view=diff
==============================================================================
--- tajo/site/docs/current/configuration/preliminary.html (original)
+++ tajo/site/docs/current/configuration/preliminary.html Thu Feb 4 00:29:05 2016
@@ -78,6 +78,7 @@
<li class="toctree-l2"><a class="reference internal" href="service_config_defaults.html">Cluster Service Configuration Defaults</a></li>
<li class="toctree-l2"><a class="reference internal" href="tajo-site-xml.html">The tajo-site.xml File</a></li>
<li class="toctree-l2"><a class="reference internal" href="catalog-site-xml.html">The catalog-site.xml File</a></li>
+<li class="toctree-l2"><a class="reference internal" href="storage-site-json.html">The storage-site.json File</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="../tsql.html">Tajo Shell (TSQL)</a><ul>
@@ -100,6 +101,7 @@
<li class="toctree-l2"><a class="reference internal" href="../sql_language/joins.html">Joins</a></li>
<li class="toctree-l2"><a class="reference internal" href="../sql_language/sql_expression.html">SQL Expressions</a></li>
<li class="toctree-l2"><a class="reference internal" href="../sql_language/predicates.html">Predicates</a></li>
+<li class="toctree-l2"><a class="reference internal" href="../sql_language/explain.html">EXPLAIN</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="../time_zone.html">Time Zone</a><ul>
@@ -120,7 +122,7 @@
<li class="toctree-l1"><a class="reference internal" href="../table_management.html">Table Management</a><ul>
<li class="toctree-l2"><a class="reference internal" href="../table_management/table_overview.html">Overview of Tajo Tables</a></li>
<li class="toctree-l2"><a class="reference internal" href="../table_management/tablespaces.html">Tablespaces</a></li>
-<li class="toctree-l2"><a class="reference internal" href="../table_management/file_formats.html">File Formats</a></li>
+<li class="toctree-l2"><a class="reference internal" href="../table_management/data_formats.html">Data Formats</a></li>
<li class="toctree-l2"><a class="reference internal" href="../table_management/compression.html">Compression</a></li>
</ul>
</li>
@@ -257,7 +259,7 @@
<div role="contentinfo">
<p>
- © Copyright 2014, Apache Tajo Team.
+ © Copyright 2015, Apache Tajo Team.
</p>
</div>
Modified: tajo/site/docs/current/configuration/service_config_defaults.html
URL: http://svn.apache.org/viewvc/tajo/site/docs/current/configuration/service_config_defaults.html?rev=1728394&r1=1728393&r2=1728394&view=diff
==============================================================================
--- tajo/site/docs/current/configuration/service_config_defaults.html (original)
+++ tajo/site/docs/current/configuration/service_config_defaults.html Thu Feb 4 00:29:05 2016
@@ -78,6 +78,7 @@
<li class="toctree-l2 current"><a class="current reference internal" href="">Cluster Service Configuration Defaults</a></li>
<li class="toctree-l2"><a class="reference internal" href="tajo-site-xml.html">The tajo-site.xml File</a></li>
<li class="toctree-l2"><a class="reference internal" href="catalog-site-xml.html">The catalog-site.xml File</a></li>
+<li class="toctree-l2"><a class="reference internal" href="storage-site-json.html">The storage-site.json File</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="../tsql.html">Tajo Shell (TSQL)</a><ul>
@@ -100,6 +101,7 @@
<li class="toctree-l2"><a class="reference internal" href="../sql_language/joins.html">Joins</a></li>
<li class="toctree-l2"><a class="reference internal" href="../sql_language/sql_expression.html">SQL Expressions</a></li>
<li class="toctree-l2"><a class="reference internal" href="../sql_language/predicates.html">Predicates</a></li>
+<li class="toctree-l2"><a class="reference internal" href="../sql_language/explain.html">EXPLAIN</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="../time_zone.html">Time Zone</a><ul>
@@ -120,7 +122,7 @@
<li class="toctree-l1"><a class="reference internal" href="../table_management.html">Table Management</a><ul>
<li class="toctree-l2"><a class="reference internal" href="../table_management/table_overview.html">Overview of Tajo Tables</a></li>
<li class="toctree-l2"><a class="reference internal" href="../table_management/tablespaces.html">Tablespaces</a></li>
-<li class="toctree-l2"><a class="reference internal" href="../table_management/file_formats.html">File Formats</a></li>
+<li class="toctree-l2"><a class="reference internal" href="../table_management/data_formats.html">Data Formats</a></li>
<li class="toctree-l2"><a class="reference internal" href="../table_management/compression.html">Compression</a></li>
</ul>
</li>
@@ -309,7 +311,7 @@
<div role="contentinfo">
<p>
- © Copyright 2014, Apache Tajo Team.
+ © Copyright 2015, Apache Tajo Team.
</p>
</div>