You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tajo.apache.org by hy...@apache.org on 2014/03/11 18:09:23 UTC
git commit: TAJO-669: Add cluster setup documentation. (hyunsik)
Repository: incubator-tajo
Updated Branches:
refs/heads/master d4e64bb39 -> 44b28e6a1
TAJO-669: Add cluster setup documentation. (hyunsik)
Project: http://git-wip-us.apache.org/repos/asf/incubator-tajo/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-tajo/commit/44b28e6a
Tree: http://git-wip-us.apache.org/repos/asf/incubator-tajo/tree/44b28e6a
Diff: http://git-wip-us.apache.org/repos/asf/incubator-tajo/diff/44b28e6a
Branch: refs/heads/master
Commit: 44b28e6a154904a6365c788769b2fbff6914fc1e
Parents: d4e64bb
Author: Hyunsik Choi <hy...@apache.org>
Authored: Wed Mar 12 02:08:48 2014 +0900
Committer: Hyunsik Choi <hy...@apache.org>
Committed: Wed Mar 12 02:08:48 2014 +0900
----------------------------------------------------------------------
CHANGES.txt | 2 +
tajo-docs/src/main/sphinx/cli.rst | 2 +-
.../main/sphinx/configuration/cluster_setup.rst | 64 ++++++++++++++++++--
.../main/sphinx/getting_started/building.rst | 2 +-
.../main/sphinx/getting_started/first_query.rst | 26 +++++---
.../main/sphinx/getting_started/local_setup.rst | 17 +++---
6 files changed, 92 insertions(+), 21 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-tajo/blob/44b28e6a/CHANGES.txt
----------------------------------------------------------------------
diff --git a/CHANGES.txt b/CHANGES.txt
index afae236..13fa2df 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -526,6 +526,8 @@ Release 0.8.0 - unreleased
TASKS
+ TAJO-669: Add cluster setup documentation. (hyunsik)
+
TAJO-681: Embed sphinx rtd theme into tajo-docs. (hyunsik)
TAJO-657: Missing table stat in RCFile. (jinho)
http://git-wip-us.apache.org/repos/asf/incubator-tajo/blob/44b28e6a/tajo-docs/src/main/sphinx/cli.rst
----------------------------------------------------------------------
diff --git a/tajo-docs/src/main/sphinx/cli.rst b/tajo-docs/src/main/sphinx/cli.rst
index f2fe60c..6c91d2f 100644
--- a/tajo-docs/src/main/sphinx/cli.rst
+++ b/tajo-docs/src/main/sphinx/cli.rst
@@ -1,5 +1,5 @@
*****************************
-Command Line Interface (TSQL)
+Tajo Shell (TSQL)
*****************************
==========
http://git-wip-us.apache.org/repos/asf/incubator-tajo/blob/44b28e6a/tajo-docs/src/main/sphinx/configuration/cluster_setup.rst
----------------------------------------------------------------------
diff --git a/tajo-docs/src/main/sphinx/configuration/cluster_setup.rst b/tajo-docs/src/main/sphinx/configuration/cluster_setup.rst
index d566ecb..e9715f3 100644
--- a/tajo-docs/src/main/sphinx/configuration/cluster_setup.rst
+++ b/tajo-docs/src/main/sphinx/configuration/cluster_setup.rst
@@ -1,10 +1,66 @@
*******************************************
-Tajo run modes: Standalone and Distributed
+Custer Setup
*******************************************
+Fully Distributed Mode
+==========================================
+A fully distributed mode enables a Tajo instance to run on `Hadoop Distributed File System (HDFS) <http://wiki.apache.org/hadoop/HDFS>`_. In this mode, a number of Tajo workers run across a number of the physical nodes where HDFS data nodes run.
-==========================================
-Standalone Mode
-==========================================
+In this section, we explain how to setup the cluster mode.
+
+
+Settings
+--------------------------------------------------------
+
+Please add the following configs to tajo-site.xml file:
+
+.. code-block:: xml
+
+ <property>
+ <name>tajo.rootdir</name>
+ <value>hdfs://hostname:port/tajo</value>
+ </property>
+
+ <property>
+ <name>tajo.master.umbilical-rpc.address</name>
+ <value>hostname:26001</value>
+ </property>
+
+ <property>
+ <name>tajo.master.client-rpc.addres</name>
+ <value>hostname:26002</value>
+ </property>
+
+ <property>
+ <name>tajo.catalog.client-rpc.address</name>
+ <value>hostname:26005</value>
+ </property>
+
+
+Make base directories and set permissions
+--------------------------------------------------------
+
+If you want to know Tajo’s configuration in more detail, see Configuration page.
+Before launching the tajo, you should create the tajo root dir and set the permission as follows: ::
+
+ $ $HADOOP_HOME/bin/hadoop fs -mkdir /tajo
+ $ $HADOOP_HOME/bin/hadoop fs -chmod g+w /tajo
+
+
+Launch a Tajo cluster
+--------------------------------------------------------
+
+Then, execute ``start-tajo.sh`` ::
+
+ $ $TAJO_HOME/bin/start-tajo.sh
+
+.. note::
+
+ In default, each worker is set to very little resource capacity. In order to increase parallel degree, please read
+ :doc:`/configuration/worker_configuration`.
+
+.. note::
+
+ In default, TajoMaster listens on 127.0.0.1 for clients. To allow remote clients to access TajoMaster, please set tajo.master.client-rpc.address config to tajo-site.xml. In order to know how to change the listen port, please refer :doc:`/configuration/configuration_defaults`.
http://git-wip-us.apache.org/repos/asf/incubator-tajo/blob/44b28e6a/tajo-docs/src/main/sphinx/getting_started/building.rst
----------------------------------------------------------------------
diff --git a/tajo-docs/src/main/sphinx/getting_started/building.rst b/tajo-docs/src/main/sphinx/getting_started/building.rst
index b5a7381..fbe003b 100644
--- a/tajo-docs/src/main/sphinx/getting_started/building.rst
+++ b/tajo-docs/src/main/sphinx/getting_started/building.rst
@@ -1,5 +1,5 @@
*****************
-Build Source Code
+Build source code
*****************
You prepare the prerequisites and the source code, you can build the source code now.
http://git-wip-us.apache.org/repos/asf/incubator-tajo/blob/44b28e6a/tajo-docs/src/main/sphinx/getting_started/first_query.rst
----------------------------------------------------------------------
diff --git a/tajo-docs/src/main/sphinx/getting_started/first_query.rst b/tajo-docs/src/main/sphinx/getting_started/first_query.rst
index fd43619..aa5f4d2 100644
--- a/tajo-docs/src/main/sphinx/getting_started/first_query.rst
+++ b/tajo-docs/src/main/sphinx/getting_started/first_query.rst
@@ -2,7 +2,9 @@
First query execution
************************
-First of all, we need to prepare some data for query execution. For example, you can make a simple text-based table as follows: ::
+First of all, we need to prepare some table for query execution. For example, you can make a simple text-based table as follows:
+
+.. code-block:: bash
$ mkdir /home/x/table1
$ cd /home/x/table1
@@ -14,7 +16,14 @@ First of all, we need to prepare some data for query execution. For example, you
5|mno|5.6|e
<CTRL + D>
-This schema of this table is (int, text, float, text). ::
+
+Apache Tajo™ provides a SQL shell which allows users to interactively submit SQL queries. In order to use this shell, please execute ``bin/tsql`` ::
+
+ $ $TAJO_HOME/bin/tsql
+ tajo>
+
+In order to load the table we created above, we should think of a schema of the table.
+Here, we assume the schema as (int, text, float, text). ::
$ $TAJO_HOME/bin/tsql
tajo> create external table table1 (
@@ -24,7 +33,7 @@ This schema of this table is (int, text, float, text). ::
type text)
using csv with ('csvfile.delimiter'='|') location 'file:/home/x/table1';
-In order to load an external table, you need to use ‘create external table’ statement.
+To load an external table, you need to use ‘create external table’ statement.
In the location clause, you should use the absolute directory path with an appropriate scheme.
If the table resides in HDFS, you should use ‘hdfs’ instead of ‘file’.
@@ -33,7 +42,7 @@ If you want to know DDL statements in more detail, please see Query Language. ::
tajo> \d
table1
-‘d’ command shows the list of tables. ::
+ ``\d`` command shows the list of tables. ::
tajo> \d table1
@@ -48,7 +57,7 @@ If you want to know DDL statements in more detail, please see Query Language. ::
score FLOAT
type TEXT
-‘d [table name]’ command shows the description of a given table.
+``\d [table name]`` command shows the description of a given table.
Also, you can execute SQL queries as follows: ::
@@ -62,7 +71,8 @@ Also, you can execute SQL queries as follows: ::
4, jkl, 4.5, d
5, mno, 5.6, e
- tajo>
-
-
+ tajo> exit
+ bye
+Feel free to enjoy Tajo with SQL standards.
+If you want to know more explanation for SQL supported by Tajo, please refer :doc:`/sql_language`.
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/incubator-tajo/blob/44b28e6a/tajo-docs/src/main/sphinx/getting_started/local_setup.rst
----------------------------------------------------------------------
diff --git a/tajo-docs/src/main/sphinx/getting_started/local_setup.rst b/tajo-docs/src/main/sphinx/getting_started/local_setup.rst
index 328c612..e012f1b 100644
--- a/tajo-docs/src/main/sphinx/getting_started/local_setup.rst
+++ b/tajo-docs/src/main/sphinx/getting_started/local_setup.rst
@@ -2,7 +2,11 @@
Setting up a local Tajo cluster
**********************************
-First of all, you need to add the environment variables to conf/tajo-env.sh. ::
+Apache Tajo™ provides two run modes: local mode and fully distributed mode. Here, we explain only the local mode where a Tajo instance runs on a local file system. A local mode Tajo instance can start up with very simple configurations.
+
+First of all, you need to add the environment variables to conf/tajo-env.sh.
+
+.. code-block:: bash
# Hadoop home. Required
export HADOOP_HOME= ...
@@ -10,13 +14,12 @@ First of all, you need to add the environment variables to conf/tajo-env.sh. ::
# The java implementation to use. Required.
export JAVA_HOME= ...
-To launch the tajo master, execute start-tajo.sh. ::
+To launch the tajo master, execute start-tajo.sh.
- $ $TAJO_HOME/bin/start-tajo.sh
+.. code-block:: bash
-After then, you can use tsql, which is the command line shell of Tajo. ::
+ $ $TAJO_HOME/bin/start-tajo.sh
- $ $TAJO_HOME/bin/tsql
- tajo>
+.. note::
-If you want to how to use tsql, read Tajo Interactive Shell document.
\ No newline at end of file
+ If you want to how to setup a fully distributed mode of Tajo, please see :doc:`/configuration/cluster_setup`.