You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tajo.apache.org by ji...@apache.org on 2015/03/09 06:32:05 UTC
svn commit: r1665129 - in /tajo/site/docs/devel:
_sources/hbase_integration.txt _sources/swift_integration.txt
hbase_integration.html swift_integration.html
Author: jihoonson
Date: Mon Mar 9 05:32:04 2015
New Revision: 1665129
URL: http://svn.apache.org/r1665129
Log:
Missing source files
Added:
tajo/site/docs/devel/_sources/hbase_integration.txt
tajo/site/docs/devel/_sources/swift_integration.txt
tajo/site/docs/devel/hbase_integration.html
tajo/site/docs/devel/swift_integration.html
Added: tajo/site/docs/devel/_sources/hbase_integration.txt
URL: http://svn.apache.org/viewvc/tajo/site/docs/devel/_sources/hbase_integration.txt?rev=1665129&view=auto
==============================================================================
--- tajo/site/docs/devel/_sources/hbase_integration.txt (added)
+++ tajo/site/docs/devel/_sources/hbase_integration.txt Mon Mar 9 05:32:04 2015
@@ -0,0 +1,183 @@
+*************************************
+HBase Integration
+*************************************
+
+Apache Tajo⢠storage supports integration with Apache HBaseâ¢.
+This integration allows Tajo to access all tables used in Apache HBase.
+
+In order to use this feature, you need to build add some configs into ``conf/tajo-env.sh`` and then add some properties into a table create statement.
+
+This section describes how to setup HBase integration.
+
+First, you need to set your HBase home directory to the environment variable ``HBASE_HOME`` in conf/tajo-env.sh as follows: ::
+
+ export HBASE_HOME=/path/to/your/hbase/directory
+
+If you set the directory, Tajo will add HBase library file to classpath.
+
+
+
+========================
+CREATE TABLE
+========================
+
+*Synopsis*
+
+.. code-block:: sql
+
+ CREATE [EXTERNAL] TABLE [IF NOT EXISTS] <table_name> [(<column_name> <data_type>, ... )]
+ USING hbase
+ WITH ('table'='<hbase_table_name>'
+ , 'columns'=':key,<column_family_name>:<qualifier_name>, ...'
+ , 'hbase.zookeeper.quorum'='<zookeeper_address>'
+ , 'hbase.zookeeper.property.clientPort'='<zookeeper_client_port>'
+ )
+
+Options
+
+* ``table`` : Set hbase origin table name. If you want to create an external table, the table must exists on HBase. The other way, if you want to create a managed table, the table must doesn't exist on HBase.
+* ``columns`` : :key means HBase row key. The number of columns entry need to equals to the number of Tajo table column
+* ``hbase.zookeeper.quorum`` : Set zookeeper quorum address. You can use different zookeeper cluster on the same Tajo database. If you don't set the zookeeper address, Tajo will refer the property of hbase-site.xml file.
+* ``hbase.zookeeper.property.clientPort`` : Set zookeeper client port. If you don't set the port, Tajo will refer the property of hbase-site.xml file.
+
+``IF NOT EXISTS`` allows ``CREATE [EXTERNAL] TABLE`` statement to avoid an error which occurs when the table does not exist.
+
+
+
+========================
+ DROP TABLE
+========================
+
+*Synopsis*
+
+.. code-block:: sql
+
+ DROP TABLE [IF EXISTS] <table_name> [PURGE]
+
+``IF EXISTS`` allows ``DROP TABLE`` statement to avoid an error which occurs when the table does not exist. ``DROP TABLE`` statement removes a table from Tajo catalog, but it does not remove the contents on HBase cluster. If ``PURGE`` option is given, ``DROP TABLE`` statement will eliminate the entry in the catalog as well as the contents on HBase cluster.
+
+
+========================
+INSERT (OVERWRITE) INTO
+========================
+
+INSERT OVERWRITE statement overwrites a table data of an existing table. Tajo's INSERT OVERWRITE statement follows ``INSERT INTO SELECT`` statement of SQL. The examples are as follows:
+
+.. code-block:: sql
+
+ -- when a target table schema and output schema are equivalent to each other
+ INSERT OVERWRITE INTO t1 SELECT l_orderkey, l_partkey, l_quantity FROM lineitem;
+ -- or
+ INSERT OVERWRITE INTO t1 SELECT * FROM lineitem;
+
+ -- when the output schema are smaller than the target table schema
+ INSERT OVERWRITE INTO t1 SELECT l_orderkey FROM lineitem;
+
+ -- when you want to specify certain target columns
+ INSERT OVERWRITE INTO t1 (col1, col3) SELECT l_orderkey, l_quantity FROM lineitem;
+
+
+.. note::
+
+ If you don't set row key option, You are never able to use your table data. Because Tajo need to have some key columns for sorting before creating result data.
+
+
+
+========================
+Usage
+========================
+
+In order to create a new HBase table which is to be managed by Tajo, use the USING clause on CREATE TABLE:
+
+.. code-block:: sql
+
+ CREATE EXTERNAL TABLE blog (rowkey text, author text, register_date text, title text)
+ USING hbase WITH (
+ 'table'='blog'
+ , 'columns'=':key,info:author,info:date,content:title');
+
+After executing the command above, you should be able to see the new table in the HBase shell:
+
+.. code-block:: sql
+
+ $ hbase shell
+ create 'blog', {NAME=>'info'}, {NAME=>'content'}
+ put 'blog', 'hyunsik-02', 'content:title', 'Getting started with Tajo on your desktop'
+ put 'blog', 'hyunsik-02', 'info:author', 'Hyunsik Choi'
+ put 'blog', 'hyunsik-02', 'info:date', '2014-12-03'
+ put 'blog', 'blrunner-01', 'content:title', 'Apache Tajo: A Big Data Warehouse System on Hadoop'
+ put 'blog', 'blrunner-01', 'info:author', 'Jaehwa Jung'
+ put 'blog', 'blrunner-01', 'info:date', '2014-10-31'
+ put 'blog', 'jhkim-01', 'content:title', 'APACHE TAJO⢠v0.9 HAS ARRIVED!'
+ put 'blog', 'jhkim-01', 'info:author', 'Jinho Kim'
+ put 'blog', 'jhkim-01', 'info:date', '2014-10-22'
+
+And then create the table and query the table meta data with ``\d`` option:
+
+.. code-block:: sql
+
+ default> \d blog;
+
+ table name: default.blog
+ table path:
+ store type: HBASE
+ number of rows: unknown
+ volume: 0 B
+ Options:
+ 'columns'=':key,info:author,info:date,content:title'
+ 'table'='blog'
+
+ schema:
+ rowkey TEXT
+ author TEXT
+ register_date TEXT
+ title TEXT
+
+
+And then query the table as follows:
+
+.. code-block:: sql
+
+ default> SELECT * FROM blog;
+ rowkey, author, register_date, title
+ -------------------------------
+ blrunner-01, Jaehwa Jung, 2014-10-31, Apache Tajo: A Big Data Warehouse System on Hadoop
+ hyunsik-02, Hyunsik Choi, 2014-12-03, Getting started with Tajo on your desktop
+ jhkim-01, Jinho Kim, 2014-10-22, APACHE TAJO⢠v0.9 HAS ARRIVED!
+
+ default> SELECT * FROM blog WHERE rowkey = 'blrunner-01';
+ Progress: 100%, response time: 2.043 sec
+ rowkey, author, register_date, title
+ -------------------------------
+ blrunner-01, Jaehwa Jung, 2014-10-31, Apache Tajo: A Big Data Warehouse System on Hadoop
+
+
+Here's how to insert data the HBase table:
+
+.. code-block:: sql
+
+ CREATE TABLE blog_backup(rowkey text, author text, register_date text, title text)
+ USING hbase WITH (
+ 'table'='blog_backup'
+ , 'columns'=':key,info:author,info:date,content:title');
+ INSERT OVERWRITE INTO blog_backup SELECT * FROM blog;
+
+
+Use HBase shell to verify that the data actually got loaded:
+
+.. code-block:: sql
+
+ hbase(main):004:0> scan 'blog_backup'
+ ROW COLUMN+CELL
+ blrunner-01 column=content:title, timestamp=1421227531054, value=Apache Tajo: A Big Data Warehouse System on Hadoop
+ blrunner-01 column=info:author, timestamp=1421227531054, value=Jaehwa Jung
+ blrunner-01 column=info:date, timestamp=1421227531054, value=2014-10-31
+ hyunsik-02 column=content:title, timestamp=1421227531054, value=Getting started with Tajo on your desktop
+ hyunsik-02 column=info:author, timestamp=1421227531054, value=Hyunsik Choi
+ hyunsik-02 column=info:date, timestamp=1421227531054, value=2014-12-03
+ jhkim-01 column=content:title, timestamp=1421227531054, value=APACHE TAJO\xE2\x84\xA2 v0.9 HAS ARRIVED!
+ jhkim-01 column=info:author, timestamp=1421227531054, value=Jinho Kim
+ jhkim-01 column=info:date, timestamp=1421227531054, value=2014-10-22
+ 3 row(s) in 0.0470 seconds
+
+
Added: tajo/site/docs/devel/_sources/swift_integration.txt
URL: http://svn.apache.org/viewvc/tajo/site/docs/devel/_sources/swift_integration.txt?rev=1665129&view=auto
==============================================================================
--- tajo/site/docs/devel/_sources/swift_integration.txt (added)
+++ tajo/site/docs/devel/_sources/swift_integration.txt Mon Mar 9 05:32:04 2015
@@ -0,0 +1,110 @@
+*************************************
+OpenStack Swift Integration
+*************************************
+
+Tajo supports OpenStack Swift as one of the underlying storage types.
+In Tajo, Swift objects are represented and recognized by the same URI format as in Hadoop.
+
+You don't need to run Hadoop to run Tajo on Swift, but need to configure it.
+You will also need to configure Swift and Tajo.
+
+For details, please see the following sections.
+
+======================
+Swift configuration
+======================
+
+This step is not mandatory, but is strongly recommended to configure the Swift's proxy-server with ``list_endpoints`` for better performance.
+More information is available `here <http://docs.openstack.org/developer/swift/middleware.html#module-swift.common.middleware.list_endpoints>`_.
+
+======================
+Hadoop configurations
+======================
+
+You need to configure Hadoop to specify how to access Swift objects.
+Here is an example of ``${HADOOP_HOME}/etc/hadoop/core-site.xml``.
+
+-----------------------
+Common configurations
+-----------------------
+
+.. code-block:: xml
+
+ <property>
+ <name>fs.swift.impl</name>
+ <value>org.apache.hadoop.fs.swift.snative.SwiftNativeFileSystem</value>
+ <description>File system implementation for Swift</description>
+ </property>
+ <property>
+ <name>fs.swift.blocksize</name>
+ <value>131072</value>
+ <description>Split size in KB</description>
+ </property>
+
+----------------------------
+Configurations per provider
+----------------------------
+
+.. code-block:: xml
+
+ <property>
+ <name>fs.swift.service.${PROVIDER}.auth.url</name>
+ <value>http://127.0.0.1/v2.0/tokens</value>
+ <description>Keystone authenticaiton URL</description>
+ </property>
+ <property>
+ <name>fs.swift.service.${PROVIDER}.auth.endpoint.prefix</name>
+ <value>/endpoints/AUTH_</value>
+ <description>Keystone endpoints prefix</description>
+ </property>
+ <property>
+ <name>fs.swift.service.${PROVIDER}.http.port</name>
+ <value>8080</value>
+ <description>HTTP port</description>
+ </property>
+ <property>
+ <name>fs.swift.service.${PROVIDER}.region</name>
+ <value>regionOne</value>
+ <description>Region name</description>
+ </property>
+ <property>
+ <name>fs.swift.service.${PROVIDER}.tenant</name>
+ <value>demo</value>
+ <description>Tenant name</description>
+ </property>
+ <property>
+ <name>fs.swift.service.${PROVIDER}.username</name>
+ <value>tajo</value>
+ </property>
+ <property>
+ <name>fs.swift.service.${PROVIDER}.password</name>
+ <value>tajo_password</value>
+ </property>
+ <property>
+ <name>fs.swift.service.${PROVIDER}.location-aware</name>
+ <value>true</value>
+ <description>Flag to enable the location-aware computing</description>
+ </property>
+
+======================
+Tajo configuration
+======================
+
+Finally, you need to configure the classpath of Tajo by adding the following line to ``${TAJO_HOME}/conf/tajo-evn.sh``.
+
+.. code-block:: sh
+
+ export TAJO_CLASSPATH=$HADOOP_HOME/share/hadoop/tools/lib/hadoop-openstack-x.x.x.jar
+
+======================
+Querying on Swift
+======================
+
+Given a provider name *tajo* and a Swift container name *demo*, you can create a Tajo table with data on Swift as follows.
+
+.. code-block:: sql
+
+ default> create external table swift_table (id int32, name text, score float, type text) using text with ('text.delimiter'='|') location 'swift://demo.tajo/test.tbl';
+
+Once a table is created, you can execute any SQL queries on that table as other tables stored on HDFS.
+For query execution details, please refer to :doc:`sql_language`.
\ No newline at end of file
Added: tajo/site/docs/devel/hbase_integration.html
URL: http://svn.apache.org/viewvc/tajo/site/docs/devel/hbase_integration.html?rev=1665129&view=auto
==============================================================================
--- tajo/site/docs/devel/hbase_integration.html (added)
+++ tajo/site/docs/devel/hbase_integration.html Mon Mar 9 05:32:04 2015
@@ -0,0 +1,406 @@
+
+
+<!DOCTYPE html>
+<!--[if IE 8]><html class="no-js lt-ie9" lang="en" > <![endif]-->
+<!--[if gt IE 8]><!--> <html class="no-js" lang="en" > <!--<![endif]-->
+<head>
+ <meta charset="utf-8">
+ <meta name="viewport" content="width=device-width, initial-scale=1.0">
+
+ <title>HBase Integration — Apache Tajo 0.11.0 documentation</title>
+
+
+
+
+
+
+ <link href='https://fonts.googleapis.com/css?family=Lato:400,700|Roboto+Slab:400,700|Inconsolata:400,700' rel='stylesheet' type='text/css'>
+
+
+
+
+
+
+
+
+
+ <link rel="stylesheet" href="_static/css/theme.css" type="text/css" />
+
+
+
+ <link rel="top" title="Apache Tajo 0.11.0 documentation" href="index.html"/>
+ <link rel="next" title="OpenStack Swift Integration" href="swift_integration.html"/>
+ <link rel="prev" title="HCatalog Integration" href="hcatalog_integration.html"/>
+
+
+ <script src="https://cdnjs.cloudflare.com/ajax/libs/modernizr/2.6.2/modernizr.min.js"></script>
+
+</head>
+
+<body class="wy-body-for-nav" role="document">
+
+ <div class="wy-grid-for-nav">
+
+
+ <nav data-toggle="wy-nav-shift" class="wy-nav-side">
+ <div class="wy-side-nav-search">
+ <a href="index.html" class="fa fa-home"> Apache Tajo</a>
+ <div role="search">
+ <form id ="rtd-search-form" class="wy-form" action="search.html" method="get">
+ <input type="text" name="q" placeholder="Search docs" />
+ <input type="hidden" name="check_keywords" value="yes" />
+ <input type="hidden" name="area" value="default" />
+ </form>
+</div>
+ </div>
+
+ <div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="main navigation">
+
+
+ <ul class="current">
+<li class="toctree-l1"><a class="reference internal" href="introduction.html">Introduction</a></li>
+<li class="toctree-l1"><a class="reference internal" href="getting_started.html">Getting Started</a><ul>
+<li class="toctree-l2"><a class="reference internal" href="getting_started.html#prerequisites">Prerequisites</a></li>
+<li class="toctree-l2"><a class="reference internal" href="getting_started.html#dowload-and-unpack-the-source-code">Dowload and unpack the source code</a></li>
+<li class="toctree-l2"><a class="reference internal" href="getting_started.html#build-source-code">Build source code</a></li>
+<li class="toctree-l2"><a class="reference internal" href="getting_started.html#setting-up-a-local-tajo-cluster">Setting up a local Tajo cluster</a></li>
+<li class="toctree-l2"><a class="reference internal" href="getting_started.html#first-query-execution">First query execution</a></li>
+</ul>
+</li>
+<li class="toctree-l1"><a class="reference internal" href="configuration.html">Configuration</a><ul>
+<li class="toctree-l2"><a class="reference internal" href="configuration/preliminary.html">Preliminary</a></li>
+<li class="toctree-l2"><a class="reference internal" href="configuration/cluster_setup.html">Cluster Setup</a></li>
+<li class="toctree-l2"><a class="reference internal" href="configuration/tajo_master_configuration.html">Tajo Master Configuration</a></li>
+<li class="toctree-l2"><a class="reference internal" href="configuration/worker_configuration.html">Worker Configuration</a></li>
+<li class="toctree-l2"><a class="reference internal" href="configuration/catalog_configuration.html">Catalog Configuration</a></li>
+<li class="toctree-l2"><a class="reference internal" href="configuration/ha_configuration.html">High Availability for TajoMaster</a></li>
+<li class="toctree-l2"><a class="reference internal" href="configuration/service_config_defaults.html">Cluster Service Configuration Defaults</a></li>
+<li class="toctree-l2"><a class="reference internal" href="configuration/tajo-site-xml.html">The tajo-site.xml File</a></li>
+<li class="toctree-l2"><a class="reference internal" href="configuration/catalog-site-xml.html">The catalog-site.xml File</a></li>
+</ul>
+</li>
+<li class="toctree-l1"><a class="reference internal" href="tsql.html">Tajo Shell (TSQL)</a><ul>
+<li class="toctree-l2"><a class="reference internal" href="tsql/meta_command.html">Meta Commands</a></li>
+<li class="toctree-l2"><a class="reference internal" href="tsql/dfs_command.html">Executing HDFS commands</a></li>
+<li class="toctree-l2"><a class="reference internal" href="tsql/variables.html">Session Variables</a></li>
+<li class="toctree-l2"><a class="reference internal" href="tsql/admin_command.html">Administration Commands</a></li>
+<li class="toctree-l2"><a class="reference internal" href="tsql/intro.html">Introducing to TSQL</a></li>
+<li class="toctree-l2"><a class="reference internal" href="tsql/single_command.html">Executing a single command</a></li>
+<li class="toctree-l2"><a class="reference internal" href="tsql/execute_file.html">Executing Queries from Files</a></li>
+<li class="toctree-l2"><a class="reference internal" href="tsql/background_command.html">Executing as background process</a></li>
+</ul>
+</li>
+<li class="toctree-l1"><a class="reference internal" href="sql_language.html">SQL Language</a><ul>
+<li class="toctree-l2"><a class="reference internal" href="sql_language/data_model.html">Data Model</a></li>
+<li class="toctree-l2"><a class="reference internal" href="sql_language/ddl.html">Data Definition Language</a></li>
+<li class="toctree-l2"><a class="reference internal" href="sql_language/insert.html">INSERT (OVERWRITE) INTO</a></li>
+<li class="toctree-l2"><a class="reference internal" href="sql_language/queries.html">Queries</a></li>
+<li class="toctree-l2"><a class="reference internal" href="sql_language/sql_expression.html">SQL Expressions</a></li>
+<li class="toctree-l2"><a class="reference internal" href="sql_language/predicates.html">Predicates</a></li>
+</ul>
+</li>
+<li class="toctree-l1"><a class="reference internal" href="time_zone.html">Time Zone</a><ul>
+<li class="toctree-l2"><a class="reference internal" href="time_zone.html#server-cluster-time-zone">Server Cluster Time Zone</a></li>
+<li class="toctree-l2"><a class="reference internal" href="time_zone.html#table-time-zone">Table Time Zone</a></li>
+<li class="toctree-l2"><a class="reference internal" href="time_zone.html#client-time-zone">Client Time Zone</a></li>
+<li class="toctree-l2"><a class="reference internal" href="time_zone.html#time-zone-id">Time Zone ID</a></li>
+<li class="toctree-l2"><a class="reference internal" href="time_zone.html#examples-of-time-zone">Examples of Time Zone</a></li>
+</ul>
+</li>
+<li class="toctree-l1"><a class="reference internal" href="functions.html">Functions</a><ul>
+<li class="toctree-l2"><a class="reference internal" href="functions/math_func_and_operators.html">Math Functions and Operators</a></li>
+<li class="toctree-l2"><a class="reference internal" href="functions/string_func_and_operators.html">String Functions and Operators</a></li>
+<li class="toctree-l2"><a class="reference internal" href="functions/datetime_func_and_operators.html">DateTime Functions and Operators</a></li>
+<li class="toctree-l2"><a class="reference internal" href="functions/network_func_and_operators.html">Network Functions and Operators</a></li>
+</ul>
+</li>
+<li class="toctree-l1"><a class="reference internal" href="table_management.html">Table Management</a><ul>
+<li class="toctree-l2"><a class="reference internal" href="table_management/table_overview.html">Overview of Tajo Tables</a></li>
+<li class="toctree-l2"><a class="reference internal" href="table_management/file_formats.html">File Formats</a></li>
+<li class="toctree-l2"><a class="reference internal" href="table_management/compression.html">Compression</a></li>
+</ul>
+</li>
+<li class="toctree-l1"><a class="reference internal" href="table_partitioning.html">Table Partitioning</a><ul>
+<li class="toctree-l2"><a class="reference internal" href="partitioning/intro_to_partitioning.html">Introduction to Partitioning</a></li>
+<li class="toctree-l2"><a class="reference internal" href="partitioning/column_partitioning.html">Column Partitioning</a></li>
+<li class="toctree-l2"><a class="reference internal" href="partitioning/range_partitioning.html">Range Partitioning</a></li>
+<li class="toctree-l2"><a class="reference internal" href="partitioning/hash_partitioning.html">Hash Partitioning</a></li>
+</ul>
+</li>
+<li class="toctree-l1"><a class="reference internal" href="index_overview.html">Index (Experimental Feature)</a><ul>
+<li class="toctree-l2"><a class="reference internal" href="index/types.html">Index Types</a></li>
+<li class="toctree-l2"><a class="reference internal" href="index/how_to_use.html">How to use index?</a></li>
+<li class="toctree-l2"><a class="reference internal" href="index/future_work.html">Future Works</a></li>
+</ul>
+</li>
+<li class="toctree-l1"><a class="reference internal" href="backup_and_restore.html">Backup and Restore</a><ul>
+<li class="toctree-l2"><a class="reference internal" href="backup_and_restore/catalog.html">Backup and Restore Catalog</a></li>
+</ul>
+</li>
+<li class="toctree-l1"><a class="reference internal" href="hcatalog_integration.html">HCatalog Integration</a></li>
+<li class="toctree-l1 current"><a class="current reference internal" href="">HBase Integration</a><ul>
+<li class="toctree-l2"><a class="reference internal" href="#create-table">CREATE TABLE</a></li>
+<li class="toctree-l2"><a class="reference internal" href="#drop-table">DROP TABLE</a></li>
+<li class="toctree-l2"><a class="reference internal" href="#insert-overwrite-into">INSERT (OVERWRITE) INTO</a></li>
+<li class="toctree-l2"><a class="reference internal" href="#usage">Usage</a></li>
+</ul>
+</li>
+<li class="toctree-l1"><a class="reference internal" href="swift_integration.html">OpenStack Swift Integration</a><ul>
+<li class="toctree-l2"><a class="reference internal" href="swift_integration.html#swift-configuration">Swift configuration</a></li>
+<li class="toctree-l2"><a class="reference internal" href="swift_integration.html#hadoop-configurations">Hadoop configurations</a></li>
+<li class="toctree-l2"><a class="reference internal" href="swift_integration.html#tajo-configuration">Tajo configuration</a></li>
+<li class="toctree-l2"><a class="reference internal" href="swift_integration.html#querying-on-swift">Querying on Swift</a></li>
+</ul>
+</li>
+<li class="toctree-l1"><a class="reference internal" href="jdbc_driver.html">Tajo JDBC Driver</a><ul>
+<li class="toctree-l2"><a class="reference internal" href="jdbc_driver.html#how-to-get-jdbc-driver">How to get JDBC driver</a></li>
+<li class="toctree-l2"><a class="reference internal" href="jdbc_driver.html#setting-the-classpath">Setting the CLASSPATH</a></li>
+<li class="toctree-l2"><a class="reference internal" href="jdbc_driver.html#an-example-jdbc-client">An Example JDBC Client</a></li>
+<li class="toctree-l2"><a class="reference internal" href="jdbc_driver.html#faq">FAQ</a></li>
+</ul>
+</li>
+<li class="toctree-l1"><a class="reference internal" href="tajo_client_api.html">Tajo Client API</a></li>
+<li class="toctree-l1"><a class="reference internal" href="faq.html">FAQ</a></li>
+</ul>
+
+
+ </div>
+
+ </nav>
+
+ <section data-toggle="wy-nav-shift" class="wy-nav-content-wrap">
+
+
+ <nav class="wy-nav-top" role="navigation" aria-label="top navigation">
+ <i data-toggle="wy-nav-top" class="fa fa-bars"></i>
+ <a href="index.html">Apache Tajo</a>
+ </nav>
+
+
+
+ <div class="wy-nav-content">
+ <div class="rst-content">
+ <div role="navigation" aria-label="breadcrumbs navigation">
+ <ul class="wy-breadcrumbs">
+ <li><a href="index.html">Docs</a> »</li>
+
+ <li>HBase Integration</li>
+ <li class="wy-breadcrumbs-aside">
+
+ <a href="_sources/hbase_integration.txt" rel="nofollow"> View page source</a>
+
+ </li>
+ </ul>
+ <hr/>
+</div>
+ <div role="main">
+
+ <div class="section" id="hbase-integration">
+<h1>HBase Integration<a class="headerlink" href="#hbase-integration" title="Permalink to this headline">¶</a></h1>
+<p>Apache Tajo⢠storage supports integration with Apache HBaseâ¢.
+This integration allows Tajo to access all tables used in Apache HBase.</p>
+<p>In order to use this feature, you need to build add some configs into <tt class="docutils literal"><span class="pre">conf/tajo-env.sh</span></tt> and then add some properties into a table create statement.</p>
+<p>This section describes how to setup HBase integration.</p>
+<p>First, you need to set your HBase home directory to the environment variable <tt class="docutils literal"><span class="pre">HBASE_HOME</span></tt> in conf/tajo-env.sh as follows:</p>
+<div class="highlight-python"><div class="highlight"><pre>export HBASE_HOME=/path/to/your/hbase/directory
+</pre></div>
+</div>
+<p>If you set the directory, Tajo will add HBase library file to classpath.</p>
+<div class="section" id="create-table">
+<h2>CREATE TABLE<a class="headerlink" href="#create-table" title="Permalink to this headline">¶</a></h2>
+<p><em>Synopsis</em></p>
+<div class="highlight-sql"><div class="highlight"><pre><span class="k">CREATE</span> <span class="p">[</span><span class="k">EXTERNAL</span><span class="p">]</span> <span class="k">TABLE</span> <span class="p">[</span><span class="n">IF</span> <span class="k">NOT</span> <span class="k">EXISTS</span><span class="p">]</span> <span class="o"><</span><span class="k">table_name</span><span class="o">></span> <span class="p">[(</span><span class="o"><</span><span class="k">column_name</span><span class="o">></span> <span class="o"><</span><span class="n">data_type</span><span class="o">></span><span class="p">,</span> <span class="p">...</span> <span class="p">)]</span>
+<span class="k">USING</span> <span class="n">hbase</span>
+<span class="k">WITH</span> <span class="p">(</span><span class="s1">'table'</span><span class="o">=</span><span class="s1">'<hbase_table_name>'</span>
+<span class="p">,</span> <span class="s1">'columns'</span><span class="o">=</span><span class="s1">':key,<column_family_name>:<qualifier_name>, ...'</span>
+<span class="p">,</span> <span class="s1">'hbase.zookeeper.quorum'</span><span class="o">=</span><span class="s1">'<zookeeper_address>'</span>
+<span class="p">,</span> <span class="s1">'hbase.zookeeper.property.clientPort'</span><span class="o">=</span><span class="s1">'<zookeeper_client_port>'</span>
+<span class="p">)</span>
+</pre></div>
+</div>
+<p>Options</p>
+<ul class="simple">
+<li><tt class="docutils literal"><span class="pre">table</span></tt> : Set hbase origin table name. If you want to create an external table, the table must exists on HBase. The other way, if you want to create a managed table, the table must doesn’t exist on HBase.</li>
+<li><tt class="docutils literal"><span class="pre">columns</span></tt> : :key means HBase row key. The number of columns entry need to equals to the number of Tajo table column</li>
+<li><tt class="docutils literal"><span class="pre">hbase.zookeeper.quorum</span></tt> : Set zookeeper quorum address. You can use different zookeeper cluster on the same Tajo database. If you don’t set the zookeeper address, Tajo will refer the property of hbase-site.xml file.</li>
+<li><tt class="docutils literal"><span class="pre">hbase.zookeeper.property.clientPort</span></tt> : Set zookeeper client port. If you don’t set the port, Tajo will refer the property of hbase-site.xml file.</li>
+</ul>
+<p><tt class="docutils literal"><span class="pre">IF</span> <span class="pre">NOT</span> <span class="pre">EXISTS</span></tt> allows <tt class="docutils literal"><span class="pre">CREATE</span> <span class="pre">[EXTERNAL]</span> <span class="pre">TABLE</span></tt> statement to avoid an error which occurs when the table does not exist.</p>
+</div>
+<div class="section" id="drop-table">
+<h2>DROP TABLE<a class="headerlink" href="#drop-table" title="Permalink to this headline">¶</a></h2>
+<p><em>Synopsis</em></p>
+<div class="highlight-sql"><div class="highlight"><pre><span class="k">DROP</span> <span class="k">TABLE</span> <span class="p">[</span><span class="n">IF</span> <span class="k">EXISTS</span><span class="p">]</span> <span class="o"><</span><span class="k">table_name</span><span class="o">></span> <span class="p">[</span><span class="n">PURGE</span><span class="p">]</span>
+</pre></div>
+</div>
+<p><tt class="docutils literal"><span class="pre">IF</span> <span class="pre">EXISTS</span></tt> allows <tt class="docutils literal"><span class="pre">DROP</span> <span class="pre">TABLE</span></tt> statement to avoid an error which occurs when the table does not exist. <tt class="docutils literal"><span class="pre">DROP</span> <span class="pre">TABLE</span></tt> statement removes a table from Tajo catalog, but it does not remove the contents on HBase cluster. If <tt class="docutils literal"><span class="pre">PURGE</span></tt> option is given, <tt class="docutils literal"><span class="pre">DROP</span> <span class="pre">TABLE</span></tt> statement will eliminate the entry in the catalog as well as the contents on HBase cluster.</p>
+</div>
+<div class="section" id="insert-overwrite-into">
+<h2>INSERT (OVERWRITE) INTO<a class="headerlink" href="#insert-overwrite-into" title="Permalink to this headline">¶</a></h2>
+<p>INSERT OVERWRITE statement overwrites a table data of an existing table. Tajo’s INSERT OVERWRITE statement follows <tt class="docutils literal"><span class="pre">INSERT</span> <span class="pre">INTO</span> <span class="pre">SELECT</span></tt> statement of SQL. The examples are as follows:</p>
+<div class="highlight-sql"><div class="highlight"><pre><span class="c1">-- when a target table schema and output schema are equivalent to each other</span>
+<span class="k">INSERT</span> <span class="n">OVERWRITE</span> <span class="k">INTO</span> <span class="n">t1</span> <span class="k">SELECT</span> <span class="n">l_orderkey</span><span class="p">,</span> <span class="n">l_partkey</span><span class="p">,</span> <span class="n">l_quantity</span> <span class="k">FROM</span> <span class="n">lineitem</span><span class="p">;</span>
+<span class="c1">-- or</span>
+<span class="k">INSERT</span> <span class="n">OVERWRITE</span> <span class="k">INTO</span> <span class="n">t1</span> <span class="k">SELECT</span> <span class="o">*</span> <span class="k">FROM</span> <span class="n">lineitem</span><span class="p">;</span>
+
+<span class="c1">-- when the output schema are smaller than the target table schema</span>
+<span class="k">INSERT</span> <span class="n">OVERWRITE</span> <span class="k">INTO</span> <span class="n">t1</span> <span class="k">SELECT</span> <span class="n">l_orderkey</span> <span class="k">FROM</span> <span class="n">lineitem</span><span class="p">;</span>
+
+<span class="c1">-- when you want to specify certain target columns</span>
+<span class="k">INSERT</span> <span class="n">OVERWRITE</span> <span class="k">INTO</span> <span class="n">t1</span> <span class="p">(</span><span class="n">col1</span><span class="p">,</span> <span class="n">col3</span><span class="p">)</span> <span class="k">SELECT</span> <span class="n">l_orderkey</span><span class="p">,</span> <span class="n">l_quantity</span> <span class="k">FROM</span> <span class="n">lineitem</span><span class="p">;</span>
+</pre></div>
+</div>
+<div class="admonition note">
+<p class="first admonition-title">Note</p>
+<p class="last">If you don’t set row key option, You are never able to use your table data. Because Tajo need to have some key columns for sorting before creating result data.</p>
+</div>
+</div>
+<div class="section" id="usage">
+<h2>Usage<a class="headerlink" href="#usage" title="Permalink to this headline">¶</a></h2>
+<p>In order to create a new HBase table which is to be managed by Tajo, use the USING clause on CREATE TABLE:</p>
+<div class="highlight-sql"><div class="highlight"><pre><span class="k">CREATE</span> <span class="k">EXTERNAL</span> <span class="k">TABLE</span> <span class="n">blog</span> <span class="p">(</span><span class="n">rowkey</span> <span class="nb">text</span><span class="p">,</span> <span class="n">author</span> <span class="nb">text</span><span class="p">,</span> <span class="n">register_date</span> <span class="nb">text</span><span class="p">,</span> <span class="n">title</span> <span class="nb">text</span><span class="p">)</span>
+<span class="k">USING</span> <span class="n">hbase</span> <span class="k">WITH</span> <span class="p">(</span>
+ <span class="s1">'table'</span><span class="o">=</span><span class="s1">'blog'</span>
+ <span class="p">,</span> <span class="s1">'columns'</span><span class="o">=</span><span class="s1">':key,info:author,info:date,content:title'</span><span class="p">);</span>
+</pre></div>
+</div>
+<p>After executing the command above, you should be able to see the new table in the HBase shell:</p>
+<div class="highlight-sql"><div class="highlight"><pre>$ hbase shell
+create 'blog', {NAME=>'info'}, {NAME=>'content'}
+put 'blog', 'hyunsik-02', 'content:title', 'Getting started with Tajo on your desktop'
+put 'blog', 'hyunsik-02', 'info:author', 'Hyunsik Choi'
+put 'blog', 'hyunsik-02', 'info:date', '2014-12-03'
+put 'blog', 'blrunner-01', 'content:title', 'Apache Tajo: A Big Data Warehouse System on Hadoop'
+put 'blog', 'blrunner-01', 'info:author', 'Jaehwa Jung'
+put 'blog', 'blrunner-01', 'info:date', '2014-10-31'
+put 'blog', 'jhkim-01', 'content:title', 'APACHE TAJO⢠v0.9 HAS ARRIVED!'
+put 'blog', 'jhkim-01', 'info:author', 'Jinho Kim'
+put 'blog', 'jhkim-01', 'info:date', '2014-10-22'
+</pre></div>
+</div>
+<p>And then create the table and query the table meta data with <tt class="docutils literal"><span class="pre">\d</span></tt> option:</p>
+<div class="highlight-sql"><div class="highlight"><pre>default> \d blog;
+
+table name: default.blog
+table path:
+store type: HBASE
+number of rows: unknown
+volume: 0 B
+Options:
+ 'columns'=':key,info:author,info:date,content:title'
+ 'table'='blog'
+
+schema:
+rowkey TEXT
+author TEXT
+register_date TEXT
+title TEXT
+</pre></div>
+</div>
+<p>And then query the table as follows:</p>
+<div class="highlight-sql"><div class="highlight"><pre>default> SELECT * FROM blog;
+rowkey, author, register_date, title
+-------------------------------
+blrunner-01, Jaehwa Jung, 2014-10-31, Apache Tajo: A Big Data Warehouse System on Hadoop
+hyunsik-02, Hyunsik Choi, 2014-12-03, Getting started with Tajo on your desktop
+jhkim-01, Jinho Kim, 2014-10-22, APACHE TAJO⢠v0.9 HAS ARRIVED!
+
+default> SELECT * FROM blog WHERE rowkey = 'blrunner-01';
+Progress: 100%, response time: 2.043 sec
+rowkey, author, register_date, title
+-------------------------------
+blrunner-01, Jaehwa Jung, 2014-10-31, Apache Tajo: A Big Data Warehouse System on Hadoop
+</pre></div>
+</div>
+<p>Here’s how to insert data the HBase table:</p>
+<div class="highlight-sql"><div class="highlight"><pre><span class="k">CREATE</span> <span class="k">TABLE</span> <span class="n">blog_backup</span><span class="p">(</span><span class="n">rowkey</span> <span class="nb">text</span><span class="p">,</span> <span class="n">author</span> <span class="nb">text</span><span class="p">,</span> <span class="n">register_date</span> <span class="nb">text</span><span class="p">,</span> <span class="n">title</span> <span class="nb">text</span><span class="p">)</span>
+<span class="k">USING</span> <span class="n">hbase</span> <span class="k">WITH</span> <span class="p">(</span>
+ <span class="s1">'table'</span><span class="o">=</span><span class="s1">'blog_backup'</span>
+ <span class="p">,</span> <span class="s1">'columns'</span><span class="o">=</span><span class="s1">':key,info:author,info:date,content:title'</span><span class="p">);</span>
+<span class="k">INSERT</span> <span class="n">OVERWRITE</span> <span class="k">INTO</span> <span class="n">blog_backup</span> <span class="k">SELECT</span> <span class="o">*</span> <span class="k">FROM</span> <span class="n">blog</span><span class="p">;</span>
+</pre></div>
+</div>
+<p>Use HBase shell to verify that the data actually got loaded:</p>
+<div class="highlight-sql"><div class="highlight"><pre>hbase(main):004:0> scan 'blog_backup'
+ ROW COLUMN+CELL
+ blrunner-01 column=content:title, timestamp=1421227531054, value=Apache Tajo: A Big Data Warehouse System on Hadoop
+ blrunner-01 column=info:author, timestamp=1421227531054, value=Jaehwa Jung
+ blrunner-01 column=info:date, timestamp=1421227531054, value=2014-10-31
+ hyunsik-02 column=content:title, timestamp=1421227531054, value=Getting started with Tajo on your desktop
+ hyunsik-02 column=info:author, timestamp=1421227531054, value=Hyunsik Choi
+ hyunsik-02 column=info:date, timestamp=1421227531054, value=2014-12-03
+ jhkim-01 column=content:title, timestamp=1421227531054, value=APACHE TAJO\xE2\x84\xA2 v0.9 HAS ARRIVED!
+ jhkim-01 column=info:author, timestamp=1421227531054, value=Jinho Kim
+ jhkim-01 column=info:date, timestamp=1421227531054, value=2014-10-22
+3 row(s) in 0.0470 seconds
+</pre></div>
+</div>
+</div>
+</div>
+
+
+ </div>
+ <footer>
+
+ <div class="rst-footer-buttons" role="navigation" aria-label="footer navigation">
+
+ <a href="swift_integration.html" class="btn btn-neutral float-right" title="OpenStack Swift Integration"/>Next <span class="fa fa-arrow-circle-right"></span></a>
+
+
+ <a href="hcatalog_integration.html" class="btn btn-neutral" title="HCatalog Integration"><span class="fa fa-arrow-circle-left"></span> Previous</a>
+
+ </div>
+
+
+ <hr/>
+
+ <div role="contentinfo">
+ <p>
+ © Copyright 2014, Apache Tajo Team.
+ </p>
+ </div>
+
+ <a href="https://github.com/snide/sphinx_rtd_theme">Sphinx theme</a> provided by <a href="https://readthedocs.org">Read the Docs</a>
+</footer>
+ </div>
+ </div>
+
+ </section>
+
+ </div>
+
+
+
+
+
+ <script type="text/javascript">
+ var DOCUMENTATION_OPTIONS = {
+ URL_ROOT:'./',
+ VERSION:'0.11.0',
+ COLLAPSE_INDEX:false,
+ FILE_SUFFIX:'.html',
+ HAS_SOURCE: true
+ };
+ </script>
+ <script type="text/javascript" src="_static/jquery.js"></script>
+ <script type="text/javascript" src="_static/underscore.js"></script>
+ <script type="text/javascript" src="_static/doctools.js"></script>
+
+
+
+
+
+ <script type="text/javascript" src="_static/js/theme.js"></script>
+
+
+
+
+ <script type="text/javascript">
+ jQuery(function () {
+ SphinxRtdTheme.StickyNav.enable();
+ });
+ </script>
+
+
+</body>
+</html>
\ No newline at end of file
Added: tajo/site/docs/devel/swift_integration.html
URL: http://svn.apache.org/viewvc/tajo/site/docs/devel/swift_integration.html?rev=1665129&view=auto
==============================================================================
--- tajo/site/docs/devel/swift_integration.html (added)
+++ tajo/site/docs/devel/swift_integration.html Mon Mar 9 05:32:04 2015
@@ -0,0 +1,356 @@
+
+
+<!DOCTYPE html>
+<!--[if IE 8]><html class="no-js lt-ie9" lang="en" > <![endif]-->
+<!--[if gt IE 8]><!--> <html class="no-js" lang="en" > <!--<![endif]-->
+<head>
+ <meta charset="utf-8">
+ <meta name="viewport" content="width=device-width, initial-scale=1.0">
+
+ <title>OpenStack Swift Integration — Apache Tajo 0.11.0 documentation</title>
+
+
+
+
+
+
+ <link href='https://fonts.googleapis.com/css?family=Lato:400,700|Roboto+Slab:400,700|Inconsolata:400,700' rel='stylesheet' type='text/css'>
+
+
+
+
+
+
+
+
+
+ <link rel="stylesheet" href="_static/css/theme.css" type="text/css" />
+
+
+
+ <link rel="top" title="Apache Tajo 0.11.0 documentation" href="index.html"/>
+ <link rel="next" title="Tajo JDBC Driver" href="jdbc_driver.html"/>
+ <link rel="prev" title="HBase Integration" href="hbase_integration.html"/>
+
+
+ <script src="https://cdnjs.cloudflare.com/ajax/libs/modernizr/2.6.2/modernizr.min.js"></script>
+
+</head>
+
+<body class="wy-body-for-nav" role="document">
+
+ <div class="wy-grid-for-nav">
+
+
+ <nav data-toggle="wy-nav-shift" class="wy-nav-side">
+ <div class="wy-side-nav-search">
+ <a href="index.html" class="fa fa-home"> Apache Tajo</a>
+ <div role="search">
+ <form id ="rtd-search-form" class="wy-form" action="search.html" method="get">
+ <input type="text" name="q" placeholder="Search docs" />
+ <input type="hidden" name="check_keywords" value="yes" />
+ <input type="hidden" name="area" value="default" />
+ </form>
+</div>
+ </div>
+
+ <div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="main navigation">
+
+
+ <ul class="current">
+<li class="toctree-l1"><a class="reference internal" href="introduction.html">Introduction</a></li>
+<li class="toctree-l1"><a class="reference internal" href="getting_started.html">Getting Started</a><ul>
+<li class="toctree-l2"><a class="reference internal" href="getting_started.html#prerequisites">Prerequisites</a></li>
+<li class="toctree-l2"><a class="reference internal" href="getting_started.html#dowload-and-unpack-the-source-code">Dowload and unpack the source code</a></li>
+<li class="toctree-l2"><a class="reference internal" href="getting_started.html#build-source-code">Build source code</a></li>
+<li class="toctree-l2"><a class="reference internal" href="getting_started.html#setting-up-a-local-tajo-cluster">Setting up a local Tajo cluster</a></li>
+<li class="toctree-l2"><a class="reference internal" href="getting_started.html#first-query-execution">First query execution</a></li>
+</ul>
+</li>
+<li class="toctree-l1"><a class="reference internal" href="configuration.html">Configuration</a><ul>
+<li class="toctree-l2"><a class="reference internal" href="configuration/preliminary.html">Preliminary</a></li>
+<li class="toctree-l2"><a class="reference internal" href="configuration/cluster_setup.html">Cluster Setup</a></li>
+<li class="toctree-l2"><a class="reference internal" href="configuration/tajo_master_configuration.html">Tajo Master Configuration</a></li>
+<li class="toctree-l2"><a class="reference internal" href="configuration/worker_configuration.html">Worker Configuration</a></li>
+<li class="toctree-l2"><a class="reference internal" href="configuration/catalog_configuration.html">Catalog Configuration</a></li>
+<li class="toctree-l2"><a class="reference internal" href="configuration/ha_configuration.html">High Availability for TajoMaster</a></li>
+<li class="toctree-l2"><a class="reference internal" href="configuration/service_config_defaults.html">Cluster Service Configuration Defaults</a></li>
+<li class="toctree-l2"><a class="reference internal" href="configuration/tajo-site-xml.html">The tajo-site.xml File</a></li>
+<li class="toctree-l2"><a class="reference internal" href="configuration/catalog-site-xml.html">The catalog-site.xml File</a></li>
+</ul>
+</li>
+<li class="toctree-l1"><a class="reference internal" href="tsql.html">Tajo Shell (TSQL)</a><ul>
+<li class="toctree-l2"><a class="reference internal" href="tsql/meta_command.html">Meta Commands</a></li>
+<li class="toctree-l2"><a class="reference internal" href="tsql/dfs_command.html">Executing HDFS commands</a></li>
+<li class="toctree-l2"><a class="reference internal" href="tsql/variables.html">Session Variables</a></li>
+<li class="toctree-l2"><a class="reference internal" href="tsql/admin_command.html">Administration Commands</a></li>
+<li class="toctree-l2"><a class="reference internal" href="tsql/intro.html">Introducing to TSQL</a></li>
+<li class="toctree-l2"><a class="reference internal" href="tsql/single_command.html">Executing a single command</a></li>
+<li class="toctree-l2"><a class="reference internal" href="tsql/execute_file.html">Executing Queries from Files</a></li>
+<li class="toctree-l2"><a class="reference internal" href="tsql/background_command.html">Executing as background process</a></li>
+</ul>
+</li>
+<li class="toctree-l1"><a class="reference internal" href="sql_language.html">SQL Language</a><ul>
+<li class="toctree-l2"><a class="reference internal" href="sql_language/data_model.html">Data Model</a></li>
+<li class="toctree-l2"><a class="reference internal" href="sql_language/ddl.html">Data Definition Language</a></li>
+<li class="toctree-l2"><a class="reference internal" href="sql_language/insert.html">INSERT (OVERWRITE) INTO</a></li>
+<li class="toctree-l2"><a class="reference internal" href="sql_language/queries.html">Queries</a></li>
+<li class="toctree-l2"><a class="reference internal" href="sql_language/sql_expression.html">SQL Expressions</a></li>
+<li class="toctree-l2"><a class="reference internal" href="sql_language/predicates.html">Predicates</a></li>
+</ul>
+</li>
+<li class="toctree-l1"><a class="reference internal" href="time_zone.html">Time Zone</a><ul>
+<li class="toctree-l2"><a class="reference internal" href="time_zone.html#server-cluster-time-zone">Server Cluster Time Zone</a></li>
+<li class="toctree-l2"><a class="reference internal" href="time_zone.html#table-time-zone">Table Time Zone</a></li>
+<li class="toctree-l2"><a class="reference internal" href="time_zone.html#client-time-zone">Client Time Zone</a></li>
+<li class="toctree-l2"><a class="reference internal" href="time_zone.html#time-zone-id">Time Zone ID</a></li>
+<li class="toctree-l2"><a class="reference internal" href="time_zone.html#examples-of-time-zone">Examples of Time Zone</a></li>
+</ul>
+</li>
+<li class="toctree-l1"><a class="reference internal" href="functions.html">Functions</a><ul>
+<li class="toctree-l2"><a class="reference internal" href="functions/math_func_and_operators.html">Math Functions and Operators</a></li>
+<li class="toctree-l2"><a class="reference internal" href="functions/string_func_and_operators.html">String Functions and Operators</a></li>
+<li class="toctree-l2"><a class="reference internal" href="functions/datetime_func_and_operators.html">DateTime Functions and Operators</a></li>
+<li class="toctree-l2"><a class="reference internal" href="functions/network_func_and_operators.html">Network Functions and Operators</a></li>
+</ul>
+</li>
+<li class="toctree-l1"><a class="reference internal" href="table_management.html">Table Management</a><ul>
+<li class="toctree-l2"><a class="reference internal" href="table_management/table_overview.html">Overview of Tajo Tables</a></li>
+<li class="toctree-l2"><a class="reference internal" href="table_management/file_formats.html">File Formats</a></li>
+<li class="toctree-l2"><a class="reference internal" href="table_management/compression.html">Compression</a></li>
+</ul>
+</li>
+<li class="toctree-l1"><a class="reference internal" href="table_partitioning.html">Table Partitioning</a><ul>
+<li class="toctree-l2"><a class="reference internal" href="partitioning/intro_to_partitioning.html">Introduction to Partitioning</a></li>
+<li class="toctree-l2"><a class="reference internal" href="partitioning/column_partitioning.html">Column Partitioning</a></li>
+<li class="toctree-l2"><a class="reference internal" href="partitioning/range_partitioning.html">Range Partitioning</a></li>
+<li class="toctree-l2"><a class="reference internal" href="partitioning/hash_partitioning.html">Hash Partitioning</a></li>
+</ul>
+</li>
+<li class="toctree-l1"><a class="reference internal" href="index_overview.html">Index (Experimental Feature)</a><ul>
+<li class="toctree-l2"><a class="reference internal" href="index/types.html">Index Types</a></li>
+<li class="toctree-l2"><a class="reference internal" href="index/how_to_use.html">How to use index?</a></li>
+<li class="toctree-l2"><a class="reference internal" href="index/future_work.html">Future Works</a></li>
+</ul>
+</li>
+<li class="toctree-l1"><a class="reference internal" href="backup_and_restore.html">Backup and Restore</a><ul>
+<li class="toctree-l2"><a class="reference internal" href="backup_and_restore/catalog.html">Backup and Restore Catalog</a></li>
+</ul>
+</li>
+<li class="toctree-l1"><a class="reference internal" href="hcatalog_integration.html">HCatalog Integration</a></li>
+<li class="toctree-l1"><a class="reference internal" href="hbase_integration.html">HBase Integration</a><ul>
+<li class="toctree-l2"><a class="reference internal" href="hbase_integration.html#create-table">CREATE TABLE</a></li>
+<li class="toctree-l2"><a class="reference internal" href="hbase_integration.html#drop-table">DROP TABLE</a></li>
+<li class="toctree-l2"><a class="reference internal" href="hbase_integration.html#insert-overwrite-into">INSERT (OVERWRITE) INTO</a></li>
+<li class="toctree-l2"><a class="reference internal" href="hbase_integration.html#usage">Usage</a></li>
+</ul>
+</li>
+<li class="toctree-l1 current"><a class="current reference internal" href="">OpenStack Swift Integration</a><ul>
+<li class="toctree-l2"><a class="reference internal" href="#swift-configuration">Swift configuration</a></li>
+<li class="toctree-l2"><a class="reference internal" href="#hadoop-configurations">Hadoop configurations</a></li>
+<li class="toctree-l2"><a class="reference internal" href="#tajo-configuration">Tajo configuration</a></li>
+<li class="toctree-l2"><a class="reference internal" href="#querying-on-swift">Querying on Swift</a></li>
+</ul>
+</li>
+<li class="toctree-l1"><a class="reference internal" href="jdbc_driver.html">Tajo JDBC Driver</a><ul>
+<li class="toctree-l2"><a class="reference internal" href="jdbc_driver.html#how-to-get-jdbc-driver">How to get JDBC driver</a></li>
+<li class="toctree-l2"><a class="reference internal" href="jdbc_driver.html#setting-the-classpath">Setting the CLASSPATH</a></li>
+<li class="toctree-l2"><a class="reference internal" href="jdbc_driver.html#an-example-jdbc-client">An Example JDBC Client</a></li>
+<li class="toctree-l2"><a class="reference internal" href="jdbc_driver.html#faq">FAQ</a></li>
+</ul>
+</li>
+<li class="toctree-l1"><a class="reference internal" href="tajo_client_api.html">Tajo Client API</a></li>
+<li class="toctree-l1"><a class="reference internal" href="faq.html">FAQ</a></li>
+</ul>
+
+
+ </div>
+
+ </nav>
+
+ <section data-toggle="wy-nav-shift" class="wy-nav-content-wrap">
+
+
+ <nav class="wy-nav-top" role="navigation" aria-label="top navigation">
+ <i data-toggle="wy-nav-top" class="fa fa-bars"></i>
+ <a href="index.html">Apache Tajo</a>
+ </nav>
+
+
+
+ <div class="wy-nav-content">
+ <div class="rst-content">
+ <div role="navigation" aria-label="breadcrumbs navigation">
+ <ul class="wy-breadcrumbs">
+ <li><a href="index.html">Docs</a> »</li>
+
+ <li>OpenStack Swift Integration</li>
+ <li class="wy-breadcrumbs-aside">
+
+ <a href="_sources/swift_integration.txt" rel="nofollow"> View page source</a>
+
+ </li>
+ </ul>
+ <hr/>
+</div>
+ <div role="main">
+
+ <div class="section" id="openstack-swift-integration">
+<h1>OpenStack Swift Integration<a class="headerlink" href="#openstack-swift-integration" title="Permalink to this headline">¶</a></h1>
+<p>Tajo supports OpenStack Swift as one of the underlying storage types.
+In Tajo, Swift objects are represented and recognized by the same URI format as in Hadoop.</p>
+<p>You don’t need to run Hadoop to run Tajo on Swift, but need to configure it.
+You will also need to configure Swift and Tajo.</p>
+<p>For details, please see the following sections.</p>
+<div class="section" id="swift-configuration">
+<h2>Swift configuration<a class="headerlink" href="#swift-configuration" title="Permalink to this headline">¶</a></h2>
+<p>This step is not mandatory, but is strongly recommended to configure the Swift’s proxy-server with <tt class="docutils literal"><span class="pre">list_endpoints</span></tt> for better performance.
+More information is available <a class="reference external" href="http://docs.openstack.org/developer/swift/middleware.html#module-swift.common.middleware.list_endpoints">here</a>.</p>
+</div>
+<div class="section" id="hadoop-configurations">
+<h2>Hadoop configurations<a class="headerlink" href="#hadoop-configurations" title="Permalink to this headline">¶</a></h2>
+<p>You need to configure Hadoop to specify how to access Swift objects.
+Here is an example of <tt class="docutils literal"><span class="pre">${HADOOP_HOME}/etc/hadoop/core-site.xml</span></tt>.</p>
+<div class="section" id="common-configurations">
+<h3>Common configurations<a class="headerlink" href="#common-configurations" title="Permalink to this headline">¶</a></h3>
+<div class="highlight-xml"><div class="highlight"><pre><span class="nt"><property></span>
+ <span class="nt"><name></span>fs.swift.impl<span class="nt"></name></span>
+ <span class="nt"><value></span>org.apache.hadoop.fs.swift.snative.SwiftNativeFileSystem<span class="nt"></value></span>
+ <span class="nt"><description></span>File system implementation for Swift<span class="nt"></description></span>
+<span class="nt"></property></span>
+<span class="nt"><property></span>
+ <span class="nt"><name></span>fs.swift.blocksize<span class="nt"></name></span>
+ <span class="nt"><value></span>131072<span class="nt"></value></span>
+ <span class="nt"><description></span>Split size in KB<span class="nt"></description></span>
+<span class="nt"></property></span>
+</pre></div>
+</div>
+</div>
+<div class="section" id="configurations-per-provider">
+<h3>Configurations per provider<a class="headerlink" href="#configurations-per-provider" title="Permalink to this headline">¶</a></h3>
+<div class="highlight-xml"><div class="highlight"><pre><span class="nt"><property></span>
+ <span class="nt"><name></span>fs.swift.service.${PROVIDER}.auth.url<span class="nt"></name></span>
+ <span class="nt"><value></span>http://127.0.0.1/v2.0/tokens<span class="nt"></value></span>
+ <span class="nt"><description></span>Keystone authenticaiton URL<span class="nt"></description></span>
+<span class="nt"></property></span>
+<span class="nt"><property></span>
+ <span class="nt"><name></span>fs.swift.service.${PROVIDER}.auth.endpoint.prefix<span class="nt"></name></span>
+ <span class="nt"><value></span>/endpoints/AUTH_<span class="nt"></value></span>
+ <span class="nt"><description></span>Keystone endpoints prefix<span class="nt"></description></span>
+<span class="nt"></property></span>
+<span class="nt"><property></span>
+ <span class="nt"><name></span>fs.swift.service.${PROVIDER}.http.port<span class="nt"></name></span>
+ <span class="nt"><value></span>8080<span class="nt"></value></span>
+ <span class="nt"><description></span>HTTP port<span class="nt"></description></span>
+<span class="nt"></property></span>
+<span class="nt"><property></span>
+ <span class="nt"><name></span>fs.swift.service.${PROVIDER}.region<span class="nt"></name></span>
+ <span class="nt"><value></span>regionOne<span class="nt"></value></span>
+ <span class="nt"><description></span>Region name<span class="nt"></description></span>
+<span class="nt"></property></span>
+<span class="nt"><property></span>
+ <span class="nt"><name></span>fs.swift.service.${PROVIDER}.tenant<span class="nt"></name></span>
+ <span class="nt"><value></span>demo<span class="nt"></value></span>
+ <span class="nt"><description></span>Tenant name<span class="nt"></description></span>
+<span class="nt"></property></span>
+<span class="nt"><property></span>
+ <span class="nt"><name></span>fs.swift.service.${PROVIDER}.username<span class="nt"></name></span>
+ <span class="nt"><value></span>tajo<span class="nt"></value></span>
+<span class="nt"></property></span>
+<span class="nt"><property></span>
+ <span class="nt"><name></span>fs.swift.service.${PROVIDER}.password<span class="nt"></name></span>
+ <span class="nt"><value></span>tajo_password<span class="nt"></value></span>
+<span class="nt"></property></span>
+<span class="nt"><property></span>
+ <span class="nt"><name></span>fs.swift.service.${PROVIDER}.location-aware<span class="nt"></name></span>
+ <span class="nt"><value></span>true<span class="nt"></value></span>
+ <span class="nt"><description></span>Flag to enable the location-aware computing<span class="nt"></description></span>
+<span class="nt"></property></span>
+</pre></div>
+</div>
+</div>
+</div>
+<div class="section" id="tajo-configuration">
+<h2>Tajo configuration<a class="headerlink" href="#tajo-configuration" title="Permalink to this headline">¶</a></h2>
+<p>Finally, you need to configure the classpath of Tajo by adding the following line to <tt class="docutils literal"><span class="pre">${TAJO_HOME}/conf/tajo-evn.sh</span></tt>.</p>
+<div class="highlight-sh"><div class="highlight"><pre><span class="nb">export </span><span class="nv">TAJO_CLASSPATH</span><span class="o">=</span><span class="nv">$HADOOP_HOME</span>/share/hadoop/tools/lib/hadoop-openstack-x.x.x.jar
+</pre></div>
+</div>
+</div>
+<div class="section" id="querying-on-swift">
+<h2>Querying on Swift<a class="headerlink" href="#querying-on-swift" title="Permalink to this headline">¶</a></h2>
+<p>Given a provider name <em>tajo</em> and a Swift container name <em>demo</em>, you can create a Tajo table with data on Swift as follows.</p>
+<div class="highlight-sql"><div class="highlight"><pre><span class="k">default</span><span class="o">></span> <span class="k">create</span> <span class="k">external</span> <span class="k">table</span> <span class="n">swift_table</span> <span class="p">(</span><span class="n">id</span> <span class="n">int32</span><span class="p">,</span> <span class="n">name</span> <span class="nb">text</span><span class="p">,</span> <span class="n">score</span> <span class="nb">float</span><span class="p">,</span> <span class="k">type</span> <span class="nb">text</span><span class="p">)</span> <span class="k">using</span> <span class="nb">text</span> <span class="k">with</span> <span class="p">(</span><span class="s1">'text.delimiter'</span><span class="o">=</span><span class="s1">'|'</span><span class="p">)</span> <span class="k">location</span> <span class="s1">'swift://demo.tajo/test.tbl'</span><span class="p">;</span>
+</pre></div>
+</div>
+<p>Once a table is created, you can execute any SQL queries on that table as other tables stored on HDFS.
+For query execution details, please refer to <a class="reference internal" href="sql_language.html"><em>SQL Language</em></a>.</p>
+</div>
+</div>
+
+
+ </div>
+ <footer>
+
+ <div class="rst-footer-buttons" role="navigation" aria-label="footer navigation">
+
+ <a href="jdbc_driver.html" class="btn btn-neutral float-right" title="Tajo JDBC Driver"/>Next <span class="fa fa-arrow-circle-right"></span></a>
+
+
+ <a href="hbase_integration.html" class="btn btn-neutral" title="HBase Integration"><span class="fa fa-arrow-circle-left"></span> Previous</a>
+
+ </div>
+
+
+ <hr/>
+
+ <div role="contentinfo">
+ <p>
+ © Copyright 2014, Apache Tajo Team.
+ </p>
+ </div>
+
+ <a href="https://github.com/snide/sphinx_rtd_theme">Sphinx theme</a> provided by <a href="https://readthedocs.org">Read the Docs</a>
+</footer>
+ </div>
+ </div>
+
+ </section>
+
+ </div>
+
+
+
+
+
+ <script type="text/javascript">
+ var DOCUMENTATION_OPTIONS = {
+ URL_ROOT:'./',
+ VERSION:'0.11.0',
+ COLLAPSE_INDEX:false,
+ FILE_SUFFIX:'.html',
+ HAS_SOURCE: true
+ };
+ </script>
+ <script type="text/javascript" src="_static/jquery.js"></script>
+ <script type="text/javascript" src="_static/underscore.js"></script>
+ <script type="text/javascript" src="_static/doctools.js"></script>
+
+
+
+
+
+ <script type="text/javascript" src="_static/js/theme.js"></script>
+
+
+
+
+ <script type="text/javascript">
+ jQuery(function () {
+ SphinxRtdTheme.StickyNav.enable();
+ });
+ </script>
+
+
+</body>
+</html>
\ No newline at end of file