You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@trafodion.apache.org by db...@apache.org on 2016/03/15 21:20:00 UTC
[1/7] incubator-trafodion git commit: Publication of new
documentation.
Repository: incubator-trafodion
Updated Branches:
refs/heads/master 87182d621 -> 88de28e15
http://git-wip-us.apache.org/repos/asf/incubator-trafodion/blob/223db1e4/docs/load_transform/src/asciidoc/_chapters/about.adoc
----------------------------------------------------------------------
diff --git a/docs/load_transform/src/asciidoc/_chapters/about.adoc b/docs/load_transform/src/asciidoc/_chapters/about.adoc
index 4ef4bd0..4a1a1bf 100644
--- a/docs/load_transform/src/asciidoc/_chapters/about.adoc
+++ b/docs/load_transform/src/asciidoc/_chapters/about.adoc
@@ -32,17 +32,19 @@ The information herein is complementary to the following Trafodion documentation
* http://trafodion.apache.org/docs/odb/index.html[Trafodion odb User Guide]
== Intended Audience
-This guide targets anyone wanting to load data into a Trafodion database. You need to have the
-following skill sets to make full of the information in this guide:
+This guide targets anyone wanting to load data into a Trafodion database.
+
+You need to have skills in the following areas to make full use of the information in this guide:
* SQL DDL and DML.
-* Install and configure Linux software.
+* Installion and configuration of Linux software.
* Trafodion administration.
* Depending on your data source, Java and/or Hadoop ecosystem usage.
== New and Changed Information
This is a new guide.
+<<<
== Notation Conventions
This list summarizes the notation conventions for syntax presentation in this manual.
http://git-wip-us.apache.org/repos/asf/incubator-trafodion/blob/223db1e4/docs/load_transform/src/asciidoc/_chapters/bulk_load.adoc
----------------------------------------------------------------------
diff --git a/docs/load_transform/src/asciidoc/_chapters/bulk_load.adoc b/docs/load_transform/src/asciidoc/_chapters/bulk_load.adoc
index 03f1385..f17f718 100644
--- a/docs/load_transform/src/asciidoc/_chapters/bulk_load.adoc
+++ b/docs/load_transform/src/asciidoc/_chapters/bulk_load.adoc
@@ -46,30 +46,31 @@ You copy data between two Trafodion tables by using the appropriate SELECT state
=== Example
```
+LOAD INTO target_table SELECT * FROM source_table WHERE custkey >= 1000 ;
```
[[bulk-load-data-from-hdfs-files]]
== Load Data From HDFS Files
-You can copy your data into an HDFS folder and then create an external Hive tables with the correct fields that
-you point to the HDFS folder containing the data. You can also specify a WHERE clause on the source data as a filter.
+You copy your data (local or remote) into an HDFS folder. Then, you create an external Hive table (with correct fields) that point to the HDFS folder containing the data.
+You may also specify a WHERE clause on the source data as a filter, if needed.
See the https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-ExternalTables[External Tables]
page on the https://cwiki.apache.org/confluence/display/Hive[Hive Wiki] for more information.
-When Hive tables are accessed through Trafodion columns with integral, string, and char types are accessible.
+Trafodion can access columns in Hive tables having integer, string and char types.
See the https://cwiki.apache.org/confluence/display/Hive/LanguageManual+Types[LanguageManual Types]
page on the https://cwiki.apache.org/confluence/display/Hive[Hive Wiki] for the data types available in Hive.
Overall, you do the following:
-1. Export the data on the remote cluster.
-2. Transport files to Trafodion cluster via FTP, scp, or some other method.
+1. Export the data on the local or remote cluster.
+2. If applicable, transport files to Trafodion cluster via FTP, scp, or some other method.
3. Use LOAD referencing HIVE external tables.
=== Example
-
-NEED A DESCRIPTION OF THE USE CASE, THE LOAD FILE, AND ITS LAYOUT.
+You have a customer-demographics in a text file, which you need to load into Trafodion.
+The columns are separated by `|`.
Do the following:
@@ -141,12 +142,12 @@ Task: POPULATE INDEX Status: Ended ET: 00:00:05.262
[[bulk-load-data-from-hive-tables]]
== Load Data From Hive Tables
-You can import data from Hive using the TrafCI or sqlci command interface. Do the following:
+You can import data from Hive using the trafci or sqlci command interface. Do the following:
1. Set these required Control Query Defaults (CQDs) to improve load performance:
+
```
-CQD HIVE_MAX_STRING_LENGTH '1000'; // if the widest column is 1KB
+CQD HIVE_MAX_STRING_LENGTH '1000'; -- if the widest column is 1KB
```
+
This setting is required if there are time-related column types in the target Trafodion table.
@@ -162,10 +163,10 @@ LOAD WITH NO POPULATE INDEXES INTO trafodion.sch.demo SELECT * FROM hive.hive.de
```
See the http://trafodion.incubator.apache.org/docs/sql_reference/index.html#load_statement[Trafodion SQL Reference Manual].
-for the completed syntax of the LOAD statement.
+for the complete syntax of the LOAD statement.
-If multiple invocations of the LOAD statement are used to incrementally load sets of data to a single target table, then several HFiles
-for each partition of the target table are created. This causes inefficient access during SELECT queries and may also cause a compaction
+If you use multiple LOAD statements to incrementally load sets of data into a single target table, then several HFiles are created
+for each partition of the target table. This causes inefficient access during SELECT queries and may also cause a compaction
to be triggered based on the policies configured in the HBase settings.
To avoid this issue, it is good practice to perform a major compaction on a table that has been the target of more than two LOAD statements
@@ -184,8 +185,6 @@ NOTE: The `major_compact` command returns immediately since it's not waited. Typ
>> CQD HIVE_MAX_STRING_LENGTH '1000' ;
>> CQD ALLOW_INCOMPATIBLE_ASSIGNMENT 'on' ;
>> LOAD WITH NO POPULATE INDEXES INTO trafodion.sch.demo SELECT * FROM hive.hive.demo ;
-
-<NEED MORE HERE, INCLUDING CREATING THE TRAFODION TABLE AND WHAT TO DO ABOUT INDEXES>
```
[[bulk-load-data-from-external-databases]]
@@ -243,7 +242,8 @@ export JAVA_OPTIONS=-Dmapred.child.java.opts=\-Djava.security.egd=file:/dev/uran
==== List All Oracle Tables
```
-sqoop list-tables --driver oracle.jdbc.OracleDriver --connect jdbc:oracle:thin:@<Oracle host name>:<port>/<database>
+sqoop list-tables --driver oracle.jdbc.OracleDriver
+--connect jdbc:oracle:thin:@<Oracle host name>:<port>/<database>
--username <user-name> --password <password>
```
@@ -279,9 +279,12 @@ which does not exist in the data itself.
=== Example
-ADD EXAMPLE HERE.
-
```
+sqoop import --connect jdbc:oracle:thin:@localhost:1521/orcl
+--username trafdemo --password traf123 --table CUSTOMER
+--split-by CUSTNUM --hive-import --create-hive-table
+--hive-table customers --hive-overwrite --null-string ''
+--null-non-string '' --hive-drop-import-delims--verbose
```
http://git-wip-us.apache.org/repos/asf/incubator-trafodion/blob/223db1e4/docs/load_transform/src/asciidoc/_chapters/introduction.adoc
----------------------------------------------------------------------
diff --git a/docs/load_transform/src/asciidoc/_chapters/introduction.adoc b/docs/load_transform/src/asciidoc/_chapters/introduction.adoc
index 2bbc3e1..05e0cd5 100644
--- a/docs/load_transform/src/asciidoc/_chapters/introduction.adoc
+++ b/docs/load_transform/src/asciidoc/_chapters/introduction.adoc
@@ -36,19 +36,21 @@ There are two methods used to load data into a Trafodion table. Both methods can
| *Bulk Load* | Large data volumes +
Stage data and load in the batches | Trafodion Bulk Loader
| *Trickle Load* | Small data volumes +
-Insert data at it arrives | odb tool +
-ETL tool or custom ODBC/JDBC application
+Insert data as it arrives | ETL tool +
+Custom ODBC/JDBC application +
+User-Defined Functions +
+odb Tool
|===
These two methods use four types of SQL insert statements
* *Bulk Load*
-** http://trafodion.apache.org/docs/sql_reference/index.html#load_statement[LOAD]
+** http://trafodion.incubator.apache.org/docs/sql_reference/index.html#load_statement[LOAD]
* *Trickle Load*
** http://trafodion.incubator.apache.org/docs/sql_reference/index.html#insert_statement[INSERT]
-** http://trafodion.apache.org/docs/sql_reference/index.html#upsert_statement[UPSERT]
-** http://trafodion.apache.org/docs/sql_reference/index.html#upsert_statement[UPSERT USING LOAD]
+** http://trafodion.incubator.apache.org/docs/sql_reference/index.html#upsert_statement[UPSERT]
+** http://trafodion.incubator.apache.org/docs/sql_reference/index.html#upsert_statement[UPSERT USING LOAD]
The http://trafodion.incubator.apache.org/docs/sql_reference/index.html[Trafodion SQL Reference Manual]
provides syntax descriptions for these statements.
@@ -62,6 +64,7 @@ The data source defines what type of load approach and method you use:
** _Native HBase Tables_: Direct load.
** _Disparate Data Source_: Write Java/C++ UDF to read data from source and pass rows to LOAD.
+<<<
* *Trickle Load* (odb utility)
** _Text Files_: Direct access
** _pipes_: Via `stdin`
@@ -72,6 +75,7 @@ For more information, refer to:
* <<bulk-load,Bulk Load>>
* <<trickle-load, Trickle Load>>
+<<<
[[introduction-insert-types]]
=== Insert Types
http://git-wip-us.apache.org/repos/asf/incubator-trafodion/blob/223db1e4/docs/load_transform/src/asciidoc/_chapters/tables_indexes.adoc
----------------------------------------------------------------------
diff --git a/docs/load_transform/src/asciidoc/_chapters/tables_indexes.adoc b/docs/load_transform/src/asciidoc/_chapters/tables_indexes.adoc
index 71ea7f7..a9f8f78 100644
--- a/docs/load_transform/src/asciidoc/_chapters/tables_indexes.adoc
+++ b/docs/load_transform/src/asciidoc/_chapters/tables_indexes.adoc
@@ -60,6 +60,7 @@ The number of partitions must also be specified during table creation. You choos
cluster and the expected size of the table. A salted table can split if more data is added to it than initially estimated. If this
happens, then more than one partition having rows with the same salt value, which may result in suboptimal execution plans for the table.
+<<<
You can also choose not to salt Trafodion tables. This is similar to range partitioning in a traditional database. The number of partitions
grows with the size of the table, and range boundaries are determined by HBase based on the specified split policy.
@@ -71,18 +72,11 @@ when compared to a Hive table with equivalent data since HBase stores the key se
HBase provides several types of encoding to avoid storing the same key value to disk for every column in the row. HBase also supports various
types of compression of the entire data block, regardless whether it is encoded or not.
-
See http://hbase.apache.org/book.html#compression[Appendix E: Compression and Data Block Encoding In HBase] in the
-http://hbase.apache.org/book.html[Apache HBase Reference Guide] for a comparison of various compression and encoding algorithms.
-
-For a discussion on the space savings achieved by various types of encoding and compression schemes, please see the
-https://blogs.apache.org/hbase/category/General["The Effect of ColumnFamily, RowKey and KeyValue Design on HFile Size"] HBase blog entry,
-dated Friday April 11, 2014.
-
-The performance of an algorithm is also a relevant factor when deciding on the type of compression. For more information, see slides 6, 7,
-and 8 of Govind Kamat's and Sumeet Singh's presentation,
-http://www.slideshare.net/Hadoop_Summit/kamat-singh-june27425pmroom210cv2["Compression Options in Hadoop - A Tale of Tradeoffs"].
-
+http://hbase.apache.org/book.html[Apache HBase Reference Guide] for a comparison of various compression and encoding algorithms. Use the
+information in the http://hbase.apache.org/book.html#data.block.encoding.types[Which Compressor or Data Block Encoder To Use] section to
+determine the best compression technique for your tables.
+<<<
[[create-trafodion-tables-and-indexes]]
== Create Tables and Indexes
@@ -204,5 +198,3 @@ using ON EXISTING COLUMNS. Note that using ON NECESSARY COLUMNS will only identi
requested by the optimizer but do not exist. The current implementation of automation does not know which existing histograms might be stale.
-
-LocalWords: c7
http://git-wip-us.apache.org/repos/asf/incubator-trafodion/blob/223db1e4/docs/load_transform/src/asciidoc/_chapters/trickle_load.adoc
----------------------------------------------------------------------
diff --git a/docs/load_transform/src/asciidoc/_chapters/trickle_load.adoc b/docs/load_transform/src/asciidoc/_chapters/trickle_load.adoc
index 3d3bd1b..4991619 100644
--- a/docs/load_transform/src/asciidoc/_chapters/trickle_load.adoc
+++ b/docs/load_transform/src/asciidoc/_chapters/trickle_load.adoc
@@ -472,14 +472,14 @@ In addition, you need to convert the date format and replace all occurrences of
The following map file accomplishes these goals:
-***
+```
$ cat test/load_map/ml1.map +
-# Map file to load TRAFODION.MFTEST.FRIENDS from friends.dat +
-ID:seq:1 # Inserts into ID column a sequence starting from 1 +
-NAME:4:REPLACE:Lucia:Lucy # Loads field #4 into NAME and replace all occurrences of Lucia with Lucy +
-AGE:2 # Loads field #2 (they start from zero) into AGE +
+# Map file to load TRAFODION.MFTEST.FRIENDS from friends.dat
+ID:seq:1 # Inserts into ID column a sequence starting from 1
+NAME:4:REPLACE:Lucia:Lucy # Loads field #4 into NAME and replace all occurrences of Lucia with Lucy
+AGE:2 # Loads field #2 (they start from zero) into AGE
BDATE:6:DCONV:d.b.y # Loads field #6 into BDATE converting date format from dd mmm yyyy
-***
+```
Load as follows:
http://git-wip-us.apache.org/repos/asf/incubator-trafodion/blob/223db1e4/docs/load_transform/src/asciidoc/_chapters/troubleshoot.adoc
----------------------------------------------------------------------
diff --git a/docs/load_transform/src/asciidoc/_chapters/troubleshoot.adoc b/docs/load_transform/src/asciidoc/_chapters/troubleshoot.adoc
index 112b0ee..77624ec 100644
--- a/docs/load_transform/src/asciidoc/_chapters/troubleshoot.adoc
+++ b/docs/load_transform/src/asciidoc/_chapters/troubleshoot.adoc
@@ -112,8 +112,8 @@ you can specify the default attribute, USE_HIVE_SOURCE. This causes the Hive tab
NOTE: The name of the Hive table must use the Trafodion table name as its prefix.
For example, if the Trafodion table is TRAFODION.SCH.DEMO, then the Hive
-table name can be DEMO_SRC. In this case, set the attribute as follows:
-+
+table name can be DEMO_SRC. In this case, set the attribute as follows: +
+ +
```
CONTROL QUERY DEFAULT USE_HIVE_SOURCE '_SRC' ;
CREATE INDEX demo_ix ON sch.demo(name) ;
http://git-wip-us.apache.org/repos/asf/incubator-trafodion/blob/223db1e4/docs/src/site/markdown/documentation.md
----------------------------------------------------------------------
diff --git a/docs/src/site/markdown/documentation.md b/docs/src/site/markdown/documentation.md
index 720f8cf..a3848eb 100644
--- a/docs/src/site/markdown/documentation.md
+++ b/docs/src/site/markdown/documentation.md
@@ -16,40 +16,46 @@ This page provides links to the per-release Trafodion documentation.
# Latest
-Document | Formats
------------------------------------------------|-----------------------------------
-Trafodion Client Installation Guide | [Web Book](docs/client_install/index.html),[PDF](docs/client_install/Trafodion_Client_Installation_Guide.pdf)
-Trafodion Command Interface Guide | [Web Book](docs/command_interface/index.html),[PDF](docs/command_interface/Trafodion_Command_Interface_Guide.pdf)
-Trafodion Database Connectivity Services Guide | [Web Book](docs/dcs_reference/index.html),[API](docs/dcs_reference/apidocs/index.html)
-Trafodion Messages Guide | [Web Book](docs/messages_guide/index.html),[PDF](docs/messages_guide/Trafodion_Messages_Guide.pdf)
-Trafodion odb User Guide | [Web Book](docs/odb/index.html),[PDF](docs/odb/Trafodion_odb_User_Guide.pdf)
-Trafodion Provisioning Guide | [Web Book](docs/provisioning_guide/index.html),[PDF](docs/provisioning_guide/Trafodion_Provisioning_Guide.pdf)
-Trafodion REST Server Reference Guide | [Web Book](docs/rest_reference/index.html),[API](docs/rest_reference/apidocs/index.html)
-Trafodion SQL Reference Manual | [Web Book](docs/sql_reference/index.html),[PDF](docs/sql_reference/Trafodion_SQL_Reference_Manual.pdf)
+Document | Formats
+------------------------------------------------------|-----------------------------------
+Trafodion Client Installation Guide | [Web Book](docs/client_install/index.html),[PDF](docs/client_install/Trafodion_Client_Installation_Guide.pdf)
+Trafodion Command Interface Guide | [Web Book](docs/command_interface/index.html),[PDF](docs/command_interface/Trafodion_Command_Interface_Guide.pdf)
+Trafodion Control Query Default (CQD) Reference Guide | [Web Book](docs/cqd_reference/index.html),[PDF](docs/cqd_interface/Trafodion_CQD_Reference_Guide.pdf)
+Trafodion Database Connectivity Services Guide | [Web Book](docs/dcs_reference/index.html),[API](docs/dcs_reference/apidocs/index.html)
+Trafodion Load and Transform Guide | [Web Book](docs/load_transform/index.html),[PDF](docs/load_transform/Trafodion_Load_Transform_Guide.pdf)
+Trafodion Messages Guide | [Web Book](docs/messages_guide/index.html),[PDF](docs/messages_guide/Trafodion_Messages_Guide.pdf)
+Trafodion odb User Guide | [Web Book](docs/odb/index.html),[PDF](docs/odb/Trafodion_odb_User_Guide.pdf)
+Trafodion Provisioning Guide | [Web Book](docs/provisioning_guide/index.html),[PDF](docs/provisioning_guide/Trafodion_Provisioning_Guide.pdf)
+Trafodion REST Server Reference Guide | [Web Book](docs/rest_reference/index.html),[API](docs/rest_reference/apidocs/index.html)
+Trafodion SQL Reference Manual | [Web Book](docs/sql_reference/index.html),[PDF](docs/sql_reference/Trafodion_SQL_Reference_Manual.pdf)
# 2.0.0 (In Development)
-Document | Formats
------------------------------------------------|-----------------------------------
-Trafodion Client Installation Guide | [Web Book](docs/2.0.0/client_install/index.html),[PDF](docs/2.0.0/client_install/Trafodion_Client_Installation_Guide.pdf)
-Trafodion Command Interface Guide | [Web Book](docs/2.0.0/command_interface/index.html),[PDF](docs/2.0.0/command_interface/Trafodion_Command_Interface_Guide.pdf)
-Trafodion Database Connectivity Services Guide | [Web Book](docs/2.0.0/dcs_reference/index.html),[API](docs/2.0.0/dcs_reference/apidocs/index.html)
-Trafodion Messages Guide | [Web Book](docs/2.0.0/messages_guide/index.html),[PDF](docs/2.0.0/messages_guide/Trafodion_Messages_Guide.pdf)
-Trafodion odb User Guide | [Web Book](docs/2.0.0/odb/index.html),[PDF](docs/2.0.0/odb/Trafodion_odb_User_Guide.pdf)
-Trafodion Provisioning Guide | [Web Book](docs/2.0.0/provisioning_guide/index.html),[PDF](docs/2.0.0/provisioning_guide/Trafodion_Provisioning_Guide.pdf)
-Trafodion REST Server Reference Guide | [Web Book](docs/2.0.0/rest_reference/index.html),[API](docs/2.0.0/rest_reference/apidocs/index.html)
-Trafodion SQL Reference Manual | [Web Book](docs/2.0.0/sql_reference/index.html),[PDF](docs/2.0.0/sql_reference/Trafodion_SQL_Reference_Manual.pdf)
+Document | Formats
+------------------------------------------------------|-----------------------------------
+Trafodion Client Installation Guide | [Web Book](docs/2.0.0/client_install/index.html),[PDF](docs/2.0.0/client_install/Trafodion_Client_Installation_Guide.pdf)
+Trafodion Command Interface Guide | [Web Book](docs/2.0.0/command_interface/index.html),[PDF](docs/2.0.0/command_interface/Trafodion_Command_Interface_Guide.pdf)
+Trafodion Control Query Default (CQD) Reference Guide | [Web Book](docs/2.0.0/cqd_reference/index.html),[PDF](docs/2.0.0/cqd_interface/Trafodion_CQD_Reference_Guide.pdf)
+Trafodion Database Connectivity Services Guide | [Web Book](docs/2.0.0/dcs_reference/index.html),[API](docs/2.0.0/dcs_reference/apidocs/index.html)
+Trafodion Load and Transform Guide | [Web Book](docs/2.0.0/load_transform/index.html),[PDF](docs/2.0.0/load_transform/Trafodion_Load_Transform_Guide.pdf)
+Trafodion Messages Guide | [Web Book](docs/2.0.0/messages_guide/index.html),[PDF](docs/2.0.0/messages_guide/Trafodion_Messages_Guide.pdf)
+Trafodion odb User Guide | [Web Book](docs/2.0.0/odb/index.html),[PDF](docs/2.0.0/odb/Trafodion_odb_User_Guide.pdf)
+Trafodion Provisioning Guide | [Web Book](docs/2.0.0/provisioning_guide/index.html),[PDF](docs/2.0.0/provisioning_guide/Trafodion_Provisioning_Guide.pdf)
+Trafodion REST Server Reference Guide | [Web Book](docs/2.0.0/rest_reference/index.html),[API](docs/2.0.0/rest_reference/apidocs/index.html)
+Trafodion SQL Reference Manual | [Web Book](docs/2.0.0/sql_reference/index.html),[PDF](docs/2.0.0/sql_reference/Trafodion_SQL_Reference_Manual.pdf)
# 1.3.0
-Document | Formats
------------------------------------------------|-----------------------------------
-Trafodion Client Installation Guide | [Web Book](docs/1.3.0/client_install/index.html),[PDF](docs/1.3.0/client_install/Trafodion_Client_Installation_Guide.pdf)
-Trafodion Command Interface Guide | [Web Book](docs/1.3.0/command_interface/index.html),[PDF](docs/1.3.0/command_interface/Trafodion_Command_Interface_Guide.pdf)
-Trafodion Database Connectivity Services Guide | [Web Book](docs/1.3.0/dcs_reference/index.html),[API](docs/1.3.0/dcs_reference/apidocs/index.html)
-Trafodion Messages Guide | [Web Book](docs/1.3.0/messages_guide/index.html),[PDF](docs/1.3.0/messages_guide/Trafodion_Messages_Guide.pdf)
-Trafodion odb User Guide | [Web Book](docs/1.3.0/odb/index.html),[PDF](docs/1.3.0/odb/Trafodion_odb_User_Guide.pdf)
-Trafodion Provisioning Guide | [Web Book](docs/1.3.0/provisioning_guide/index.html),[PDF](docs/1.3.0/provisioning_guide/Trafodion_Provisioning_Guide.pdf)
-Trafodion REST Server Reference Guide | [Web Book](docs/1.3.0/rest_reference/index.html),[API](docs/1.3.0/rest_reference/apidocs/index.html)
-Trafodion SQL Reference Manual | [Web Book](docs/1.3.0/sql_reference/index.html),[PDF](docs/1.3.0/sql_reference/Trafodion_SQL_Reference_Manual.pdf)
+Document | Formats
+------------------------------------------------------|-----------------------------------
+Trafodion Client Installation Guide | [Web Book](docs/1.3.0/client_install/index.html),[PDF](docs/1.3.0/client_install/Trafodion_Client_Installation_Guide.pdf)
+Trafodion Command Interface Guide | [Web Book](docs/1.3.0/command_interface/index.html),[PDF](docs/1.3.0/command_interface/Trafodion_Command_Interface_Guide.pdf)
+Trafodion Control Query Default (CQD) Reference Guide | [Web Book](docs/1.3.0/cqd_reference/index.html),[PDF](docs/1.3.0/cqd_interface/Trafodion_CQD_Reference_Guide.pdf)
+Trafodion Database Connectivity Services Guide | [Web Book](docs/1.3.0/dcs_reference/index.html),[API](docs/1.3.0/dcs_reference/apidocs/index.html)
+Trafodion Load and Transform Guide | [Web Book](docs/1.3.0/load_transform/index.html),[PDF](docs/1.3.0/load_transform/Trafodion_Load_Transform_Guide.pdf)
+Trafodion Messages Guide | [Web Book](docs/1.3.0/messages_guide/index.html),[PDF](docs/1.3.0/messages_guide/Trafodion_Messages_Guide.pdf)
+Trafodion odb User Guide | [Web Book](docs/1.3.0/odb/index.html),[PDF](docs/1.3.0/odb/Trafodion_odb_User_Guide.pdf)
+Trafodion Provisioning Guide | [Web Book](docs/1.3.0/provisioning_guide/index.html),[PDF](docs/1.3.0/provisioning_guide/Trafodion_Provisioning_Guide.pdf)
+Trafodion REST Server Reference Guide | [Web Book](docs/1.3.0/rest_reference/index.html),[API](docs/1.3.0/rest_reference/apidocs/index.html)
+Trafodion SQL Reference Manual | [Web Book](docs/1.3.0/sql_reference/index.html),[PDF](docs/1.3.0/sql_reference/Trafodion_SQL_Reference_Manual.pdf)
http://git-wip-us.apache.org/repos/asf/incubator-trafodion/blob/223db1e4/docs/src/site/markdown/logo.md
----------------------------------------------------------------------
diff --git a/docs/src/site/markdown/logo.md b/docs/src/site/markdown/logo.md
index f621a28..7dacd5f 100644
--- a/docs/src/site/markdown/logo.md
+++ b/docs/src/site/markdown/logo.md
@@ -44,7 +44,7 @@ when referencing our project. Download as needed.
#### [trafodion-logo.jpg][3]
-[][3]
+[][3]
[3]: images/logos/trafodion-logo.jpg
http://git-wip-us.apache.org/repos/asf/incubator-trafodion/blob/223db1e4/docs/src/site/resources/images/logos/pb-Trafodion.png
----------------------------------------------------------------------
diff --git a/docs/src/site/resources/images/logos/pb-Trafodion.png b/docs/src/site/resources/images/logos/pb-Trafodion.png
deleted file mode 100644
index 67643a4..0000000
Binary files a/docs/src/site/resources/images/logos/pb-Trafodion.png and /dev/null differ
http://git-wip-us.apache.org/repos/asf/incubator-trafodion/blob/223db1e4/docs/src/site/resources/images/logos/pb-trafodion.png
----------------------------------------------------------------------
diff --git a/docs/src/site/resources/images/logos/pb-trafodion.png b/docs/src/site/resources/images/logos/pb-trafodion.png
new file mode 100644
index 0000000..67643a4
Binary files /dev/null and b/docs/src/site/resources/images/logos/pb-trafodion.png differ
http://git-wip-us.apache.org/repos/asf/incubator-trafodion/blob/223db1e4/docs/src/site/site.xml
----------------------------------------------------------------------
diff --git a/docs/src/site/site.xml b/docs/src/site/site.xml
index cf130d6..27b0f16 100644
--- a/docs/src/site/site.xml
+++ b/docs/src/site/site.xml
@@ -206,6 +206,7 @@
<item href="faq.html" name="FAQ"/>
<item href="release-notes.html" name="Release Notes"/>
<item href="https://cwiki.apache.org/confluence/display/TRAFODION/Apache+Trafodion+Home" name="Wiki"/>
+ <item name="Videos" href="https://www.youtube.com/playlist?list=PLyyCp3HBB8Tv9696OnG0_mmgF5tfHeyOi"/>
<item href="management.html" name="Managing Trafodion"/>
<!-- item href="new-features.html" name="New Features"/-->
<!-- item href="roadmap.html" name="Roadmap"/-->
http://git-wip-us.apache.org/repos/asf/incubator-trafodion/blob/223db1e4/pom.xml
----------------------------------------------------------------------
diff --git a/pom.xml b/pom.xml
index f723602..9647b11 100644
--- a/pom.xml
+++ b/pom.xml
@@ -131,16 +131,18 @@
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
</properties>
- <modules>
+ <!--modules>
<module>docs/client_install</module>
<module>dcs</module>
<module>core/rest</module>
<module>docs/command_interface</module>
+ <module>docs/cqd_reference</module>
+ <module>docs/load_transform</module>
<module>docs/provisioning_guide</module>
<module>docs/messages_guide</module>
<module>docs/odb_user</module>
<module>docs/sql_reference</module>
- </modules>
+ </modules-->
<build>
<resources>
[3/7] incubator-trafodion git commit: Fixed change in the main
pom.xml.
Posted by db...@apache.org.
Fixed change in the main pom.xml.
Project: http://git-wip-us.apache.org/repos/asf/incubator-trafodion/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-trafodion/commit/18f983ba
Tree: http://git-wip-us.apache.org/repos/asf/incubator-trafodion/tree/18f983ba
Diff: http://git-wip-us.apache.org/repos/asf/incubator-trafodion/diff/18f983ba
Branch: refs/heads/master
Commit: 18f983ba2a330f62d73b8fca1ae8df0dc9c46783
Parents: 223db1e
Author: Gunnar Tapper <gt...@esgyn.local>
Authored: Fri Mar 11 23:19:35 2016 -0700
Committer: Gunnar Tapper <gt...@esgyn.local>
Committed: Fri Mar 11 23:19:35 2016 -0700
----------------------------------------------------------------------
pom.xml | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-trafodion/blob/18f983ba/pom.xml
----------------------------------------------------------------------
diff --git a/pom.xml b/pom.xml
index 9647b11..7e2bf84 100644
--- a/pom.xml
+++ b/pom.xml
@@ -131,7 +131,7 @@
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
</properties>
- <!--modules>
+ <modules>
<module>docs/client_install</module>
<module>dcs</module>
<module>core/rest</module>
@@ -142,7 +142,7 @@
<module>docs/messages_guide</module>
<module>docs/odb_user</module>
<module>docs/sql_reference</module>
- </modules-->
+ </modules>
<build>
<resources>
[5/7] incubator-trafodion git commit: Merge remote-tracking branch
'origin/master' into TRAFODION-1851
Posted by db...@apache.org.
Merge remote-tracking branch 'origin/master' into TRAFODION-1851
Project: http://git-wip-us.apache.org/repos/asf/incubator-trafodion/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-trafodion/commit/ea2c1a0e
Tree: http://git-wip-us.apache.org/repos/asf/incubator-trafodion/tree/ea2c1a0e
Diff: http://git-wip-us.apache.org/repos/asf/incubator-trafodion/diff/ea2c1a0e
Branch: refs/heads/master
Commit: ea2c1a0e8af295aebb8898dd8fc2a54aecb92699
Parents: 8c33e75 87182d6
Author: Gunnar Tapper <gt...@esgyn.local>
Authored: Tue Mar 15 12:41:58 2016 -0600
Committer: Gunnar Tapper <gt...@esgyn.local>
Committed: Tue Mar 15 12:41:58 2016 -0600
----------------------------------------------------------------------
.rat-excludes | 1 +
core/sqf/monitor/linux/clio.cxx | 7 +
core/sqf/monitor/linux/monitor.cxx | 6 +
core/sqf/monitor/linux/shell.cxx | 6 +
core/sqf/src/seabed/src/msmon.cpp | 6 +
.../transactional/SplitBalanceHelper.java | 8 +-
core/sqf/src/tm/Makefile | 3 +
core/sqf/src/tm/tm.cpp | 33 +-
core/sqf/src/tm/tmaudit.cpp | 18 +-
core/sqf/src/tm/tmauditobj.cpp | 6 +-
core/sqf/src/tm/tmddlrequests.cpp | 12 +-
core/sqf/src/tm/tmlib.cpp | 13 +-
core/sqf/src/tm/tmpoolelement.h | 2 +-
core/sqf/src/tm/tmrecov.cpp | 3 +-
core/sqf/src/tm/tmregisterregion.cpp | 4 +-
core/sqf/src/tm/tmrmhbase.cpp | 3 -
core/sqf/src/tm/tmrmtse.cpp | 3 +-
core/sqf/src/tm/tmtime.h | 3 +-
core/sqf/src/tm/tmxarmmain.cpp | 2 +-
core/sqf/src/tm/tmxatxn.cpp | 4 +-
core/sqf/src/tm/tools/Makefile | 3 +
core/sqf/src/tm/tools/dtmci.cpp | 14 +-
core/sqf/src/tm/xarmapi.cpp | 5 +-
core/sqf/src/tm/xatmapi.cpp | 7 +-
core/sql/exp/exp_datetime.cpp | 1228 +++++++----------
core/sql/exp/exp_datetime.h | 186 ++-
core/sql/exp/exp_function.cpp | 28 +-
core/sql/generator/GenItemFunc.cpp | 134 +-
core/sql/optimizer/BindItemExpr.cpp | 584 ++++-----
core/sql/optimizer/ItemCache.cpp | 5 +-
core/sql/optimizer/ItemExpr.cpp | 52 +-
core/sql/optimizer/ItemFunc.h | 53 +-
core/sql/optimizer/QRDescGenerator.cpp | 19 +-
core/sql/optimizer/Stats.h | 8 +-
core/sql/optimizer/SynthType.cpp | 148 +--
core/sql/parser/ParKeyWords.cpp | 6 +-
core/sql/parser/sqlparser.y | 74 +-
core/sql/regress/compGeneral/DIFF006.KNOWN.SB | 4 -
.../sql/regress/compGeneral/DIFF006.KNOWN.SB.OS | 2 +-
core/sql/regress/compGeneral/EXPECTED006 | 1235 ------------------
core/sql/regress/compGeneral/EXPECTED006.SB | 19 +-
core/sql/regress/compGeneral/TEST006 | 12 +-
core/sql/regress/core/EXPECTED001.SB | 42 +-
core/sql/regress/core/TEST001 | 2 +
core/sql/regress/seabase/EXPECTED030 | 550 ++++++++
core/sql/regress/seabase/TEST030 | 110 ++
core/sql/sqlcomp/CmpSeabaseDDLauth.cpp | 86 +-
core/sql/sqlcomp/CmpSeabaseDDLroutine.cpp | 6 +-
dcs/Makefile | 2 +
dcs/pom.xml | 6 +-
install/installer/checkHBase | 37 +
install/installer/parseHBaseSite.py | 25 +-
install/installer/traf_add_sudoAccess | 3 +-
install/installer/traf_cloudera_mods98 | 40 +-
install/installer/traf_config_check | 48 +-
install/installer/traf_config_setup | 3 +-
install/installer/traf_getHadoopNodes | 21 +-
install/installer/traf_hortonworks_mods98 | 68 +-
install/installer/traf_package_setup | 2 +-
install/installer/trafodion_install | 88 +-
licenses/lic-dcs-bin | 651 ++++++++-
licenses/not-dcs-bin | 20 +
62 files changed, 2767 insertions(+), 3012 deletions(-)
----------------------------------------------------------------------
[4/7] incubator-trafodion git commit: Modified the SQL Reference
Manual to reference the new CQD Manual.
Posted by db...@apache.org.
Modified the SQL Reference Manual to reference the new CQD Manual.
Project: http://git-wip-us.apache.org/repos/asf/incubator-trafodion/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-trafodion/commit/8c33e750
Tree: http://git-wip-us.apache.org/repos/asf/incubator-trafodion/tree/8c33e750
Diff: http://git-wip-us.apache.org/repos/asf/incubator-trafodion/diff/8c33e750
Branch: refs/heads/master
Commit: 8c33e750f680d2bdf8c3f9e5ff6b5baabeedee70
Parents: 18f983b
Author: Gunnar Tapper <gt...@esgyn.local>
Authored: Fri Mar 11 23:56:32 2016 -0700
Committer: Gunnar Tapper <gt...@esgyn.local>
Committed: Fri Mar 11 23:56:32 2016 -0700
----------------------------------------------------------------------
.../src/asciidoc/_chapters/about.adoc | 4 +-
.../src/asciidoc/_chapters/cqds.adoc | 329 -------------------
.../src/asciidoc/_chapters/introduction.adoc | 11 +-
.../src/asciidoc/_chapters/sql_statements.adoc | 6 +-
docs/sql_reference/src/asciidoc/index.adoc | 1 -
5 files changed, 11 insertions(+), 340 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-trafodion/blob/8c33e750/docs/sql_reference/src/asciidoc/_chapters/about.adoc
----------------------------------------------------------------------
diff --git a/docs/sql_reference/src/asciidoc/_chapters/about.adoc b/docs/sql_reference/src/asciidoc/_chapters/about.adoc
index de3dcb8..0cc63d6 100644
--- a/docs/sql_reference/src/asciidoc/_chapters/about.adoc
+++ b/docs/sql_reference/src/asciidoc/_chapters/about.adoc
@@ -87,9 +87,7 @@ Trafodion SQL statements.
Management System (RMS) and describes the RMS counters that are returned.
| <<OLAP_Functions,OLAP Functions>> | Describes specific on line analytical processing functions.
| <<Reserved_Words,Appendix A: Reserved Words>> | Lists the words that are reserved in Trafodion SQL.
-| <<Control_Query_Default,Appendix B: Control Query Default (CQD) Attributes>> |
-Describes the Control Query Default (CQD) attributes that are supported in a Trafodion SQL environment.
-| <<Limits,Appendix C: Limits>> | Describes limits in Trafodion SQL.
+| <<Limits,Appendix B: Limits>> | Describes limits in Trafodion SQL.
|===
http://git-wip-us.apache.org/repos/asf/incubator-trafodion/blob/8c33e750/docs/sql_reference/src/asciidoc/_chapters/cqds.adoc
----------------------------------------------------------------------
diff --git a/docs/sql_reference/src/asciidoc/_chapters/cqds.adoc b/docs/sql_reference/src/asciidoc/_chapters/cqds.adoc
deleted file mode 100644
index c96d125..0000000
--- a/docs/sql_reference/src/asciidoc/_chapters/cqds.adoc
+++ /dev/null
@@ -1,329 +0,0 @@
-////
-/**
-* @@@ START COPYRIGHT @@@
-*
-* Licensed to the Apache Software Foundation (ASF) under one
-* or more contributor license agreements. See the NOTICE file
-* distributed with this work for additional information
-* regarding copyright ownership. The ASF licenses this file
-* to you under the Apache License, Version 2.0 (the
-* "License"); you may not use this file except in compliance
-* with the License. You may obtain a copy of the License at
-*
-* http://www.apache.org/licenses/LICENSE-2.0
-*
-* Unless required by applicable law or agreed to in writing,
-* software distributed under the License is distributed on an
-* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-* KIND, either express or implied. See the License for the
-* specific language governing permissions and limitations
-* under the License.
-*
-* @@@ END COPYRIGHT @@@
- */
-////
-
-<<<
-<<<
-[[cqd_attributes]]
-= Control Query Default (CQD) Attributes
-
-This section describes CQDs that are used to override system-level
-default settings.
-
-[[hive_environment_cqds]]
-== Hive Environment CQDs
-
-[[hive_max_string_length]]
-=== HIVE_MAX_STRING_LENGTH
-
-[cols="25%h,75%"]
-|===
-| Category | Hive
-| Description | Maximum supported string length for the _string_ data type in Hive. All
-string columns in Hive tables get converted to VARCHAR(_n_ BYTES)
-CHARACTER SET UTF8, with _n_ being the value of this CQD.
-| Values | The default value is 32000.
-|===
-
-<<<
-[[managing_histograms]]
-== Managing Histograms
-
-[[cache_histograms_refresh_interval]]
-=== CACHE_HISTOGRAMS_REFRESH_INTERVAL
-
-[cols="25%h,75%"]
-|===
-| Category | Histograms
-| Description | Defines the time interval after which timestamps for cached histograms
-are checked to be refreshed.
-| Values | Unsigned integer. Unit is seconds. +
- +
-The default value is '3600' (1 hour).
-| Usage | Histogram statistics are cached so that the compiler can avoid access to
-the metadata tables, thereby reducing compile times. The time stamp of
-the tables are checked against those of the cached histograms at an
-interval specified by this CQD, in order to see if the cached histograms
-need to be refreshed. +
- +
-You can increase the interval to reduce the impact on compile times as
-long as you do not need to obtain fresh statistics more frequently in
-order to improve query performance. It may
- +
-be that the default interval is too long and you would rather refresh
-the statistics more frequently than the default one hour, in order to
-improve query performance at the cost of increased compile times.
- +
-This setting depends on how frequently you are updating statistics on
-tables. There is no point in refreshing statistics frequently when
-statistics are not being updated during that time. On the other hand if
-you are updating statistics, or generating them for the first time on
-freshly loaded tables frequently enough, and you want these to be picked
-up immediately by the compiler because you have seen this to have a
-dramatic impact on plan quality, then you can make the refresh more
-frequent.
-| Production Usage | Not applicable.
-| Impact | Longer histogram refresh intervals can improve compile times. However,
-the longer the refresh interval the more obsolete the histograms. That
-could result in poor performance for queries that could leverage
-recently updated statistics.
-| Level | System or Service Not applicable
-| Addressing the Real Problem | Not applicable.
-|===
-
-<<<
-[[hist_no_stats_refresh_interval]]
-=== HIST_NO_STATS_REFRESH_INTERVAL
-
-[cols="25%h,75%"]
-|===
-| Category | Histograms
-| Description | Defines the time interval after which the fake histograms in the cache should be refreshed unconditionally.
-| Values | Integer. Unit is seconds. +
- +
-The default value is '3600' (1 hour).
-| Usage | Histogram statistics are "fake" when update statistics is not being run,
-but instead the customer is updating the histogram tables directly with
-statistics to guide the optimizer. This may be done if the data in the
-table is very volatile (such as for temporary tables), update statistics
-is not possible because of constant flush and fill of the table
-occurring, and statistics are manually set to provide some guidance to
-the optimizer to generate a good plan.
- +
-If these fake statistics are updated constantly to reflect the data
-churn, this default can be set to 0. This would ensure that the
-histograms with fake statistics are not cached, and are always
-refreshed. If these fake statistics are set and not touched again, then
-this interval could be set very high.
-| Production Usage | Not applicable.
-| Impact | Setting a high interval improves compilation time. However, if statistics are being updated, the compiler may be working with obsolete
-histogram statistics, potentially resulting in poorer plans.
-| Level | Service.
-| Conflicts/Synergies | Not applicable.
-| Addressing the Real Problem | Not applicable.
-|===
-
-[[hist_prefetch]]
-=== HIST_PREFETCH
-
-[cols="25%h,75%"]
-|===
-| Category | Histograms
-| Description | Influences the compiler to pre-fetch the histograms and save them in cache.
-| Values | 'ON' Pre-fetches the histograms. +
-'OFF' Does not pre-fetch the histograms. +
- +
- The default value is 'ON'.
-| Usage | You may want to turn this off if you don't want to pre-fetch a large number of histograms, many of which may not be used.
-| Production Usage | Not applicable.
-| Impact | Though it makes compilation time faster, it may result in the histogram cache to be filled with histograms that may never be used.
-| Level | System or Service.
-| Conflicts/Synergies | Use this CQD with CACHE_HISTOGRAMS. If CACHE_HISTOGRAMS is OFF, then this CQD has no effect.
-| Addressing the Real Problem | Not applicable.
-|===
-
-[[hist_rowcount_requiring_stats]]
-=== HIST_ROWCOUNT_REQUIRING_STATS
-
-[cols="25%h,75%"]
-|===
-| Category | Histograms
-| Description | Specifies the minimum row count for which the optimizer needs histograms, in order to compute better cardinality estimates. The
-optimizer does not issue any missing statistics warnings for tables whose size is smaller than the value of this CQD.
-| Values | Integer. +
- +
-The default value is '50000'.
-| Usage | Use this CQD to reduce the number of statistics warnings.
-| Production Usage | Not applicable.
-| Impact | Missing statistics warnings are not displayed for smaller tables, which in most cases don't impact plan quality much.
-However, there may be some exceptions where missing statistics on small tables could result in less than optimal plans.
-| Level | System
-| Conflicts/Synergies | Use this CQD with HIST_MISSING_STATS_WARNING_LEVEL. If the warning level CQD is 0, then this CQD does not have any effect. Also, for tables
-having fewer rows than set in this CQD, no warnings are displayed irrespective of the warning level.
-| Addressing the Real Problem | Not applicable.
-|===
-
-<<<
-[[optimizer]]
-== Optimizer
-
-[[join_order_by_user]]
-=== JOIN_ORDER_BY_USER
-
-[cols="25%h,75%"]
-|===
-| Category | Influencing Query Plans
-| Description | Enables or disables the join order in which the optimizer joins the tables to be the sequence of the tables in the FROM clause of the query.
-| Values | 'ON' Join order is forced. +
- +
-'OFF' Join order is decided by the optimizer. +
- +
-The default value is 'OFF'.
-| Usage | When set to ON, the optimizer considers only execution plans that have the join order matching the sequence of the tables in the FROM clause.
-| Production Usage | This setting is to be used only for forcing a desired join order that was not generated by default by the optimizer. It can be used as a
-workaround for query plans with inefficient join order.
-| Impact | Because you are in effect forcing the optimizer to use a plan that joins the table in the order specified in the FROM clause,
-the plan generated may not be the optimal one.
-| Level | Query
-| Conflicts/Synergies | Not applicable.
-| Addressing the Real Problem | Not applicable.
-|===
-
-[[mdam_scan_method]]
-=== MDAM_SCAN_METHOD
-
-[cols="25%h,75%"]
-|===
-| Category | Influencing Query Plans
-| Description | Enables or disables the Multi-Dimensional Access Method.
-| Values | 'ON' MDAM is considered. +
-'OFF' MDAM is disabled. +
- +
-The default value is 'ON'.
-| Usage | In certain situations, the optimizer might choose MDAM inappropriately, causing poor performance.
-In such situations you may want to turn MDAM OFF for the query it is effecting.
-| Production Usage | Not applicable.
-| Impact | Table scans with predicates on non-leading clustering key column(s) could benefit from MDAM access method if
-the leading column(s) has a small number of distinct values. Turning MDAM off results in a longer scan time for such queries.
-| Level | Set this CQD at the query level when MDAM is not working efficiently for a specific query.
-However, there may be cases (usually a defect) where a larger set of queries is being negatively impacted by MDAM.
-In those cases you may want to set it at the service or system level.
-| Conflicts/Synergies | Not applicable.
-| Addressing the Real Problem | Not applicable.
-|===
-
-[[subquery_unnesting]]
-=== SUBQUERY_UNNESTING
-
-[cols="25%h,75%"]
-|===
-| Category | Influencing Query Plans
-| Description | Controls the optimizer's ability to transform nested sub-queries into regular join trees.
-| Values | 'ON' Subquery un-nesting is considered. +
-'OFF' Subquery un-nesting is disabled. +
- +
-The default value is 'ON'.
-| Usage | Use this control to disable subquery un-nesting in the rare situation when un-nesting results in an inefficient query execution plan.
-| Production usage | Not applicable.
-| Impact | In general, subquery un-nesting results in more efficient execution plans for queries with nested sub-queries.
-Use only as a workaround for observed problems due to un-nesting.
-| Level | Query
-| Conflicts/Synergies | Not applicable.
-| Addressing the Real Problem | Not applicable.
-|===
-
-<<<
-[[managing_schemas]]
-== Managing Schemas
-
-[[schema]]
-=== SCHEMA
-
-[cols="25%h,75%"]
-|===
-| Category | Schema controls
-| Description | Sets the default schema for the session.
-| Values | SQL identifier. +
- +
-The default is SEABASE.
-| Usage | A SET SCHEMA statement, or a CONTROL QUERY DEFAULT SCHEMA statement, can be used to override the default schema name.
-| Production Usage | It is a convenience so you do not have to type in two-part names.
-| Impact | Not applicable.
-| Level | Any.
-| Conflicts/Synergies | Alternately you can use the SET SCHEMA statement.
-| Addressing the Real Problem | Not applicable.
-|===
-
-<<<
-[[transaction_control_and_locking]]
-== Transaction Control and Locking
-
-[[block_to_prevent_halloween]]
-=== BLOCK_TO_PREVENT_HALLOWEEN
-
-[cols="25%h,75%"]
-|===
-| Category | Runtime controls
-| Description | A self-referencing insert is one which inserts into a target table and
-also scans from the same target table as part of the query that produces
-rows to be inserted. Inconsistent results are produced by the insert
-statement if the statement scans rows which have been inserted by the
-same statement. This is sometimes called the "Halloween problem."
-Trafodion prevents the Halloween problem using one of two methods: 1)
-the blocking method uses a SORT operation to ensure all rows have been
-scanned before any are inserted, or 2) the disk process (ESAM) locks
-method tracks the rows which have already been inserted and the SCAN
-operator skips these rows. +
- +
-The compiler chooses the blocking method in cases in which static
-analysis of the plan indicates that the disk process locks method cannot
-be used. However, the compiler does not evaluate one condition that
-would prevent the use of the disk process locks method: the AUTOCOMMIT
-setting in which the statement is executed. Instead the compiler assumes
-that the statement is executed with the default setting for AUTOCOMMIT,
-'ON'. If AUTOCOMMIT is set to 'OFF' and self-referencing insert
-statement which uses the disk process locks method is executed, then a
-runtime error (SQLCODE 8107) is raised. +
- +
-This CQD is used to force the compiler to use the blocking method to
-prevent error 8107.
-| Values | 'OFF' The compiler is free to choose which method to use to prevent the Halloween problem. +
-'ON' The compiler is forced to use the blocking method. +
- +
-The default value is 'ON'.
-| Usage | Change this default to 'ON' if error 8107 is raised for a self-referencing insert statement which is
-executed in a session with AUTOCOMMIT set to 'OFF'.
-| Production Usage | Not applicable.
-| Impact | Using the 'ON' value in conditions that require it allows successful completion of the insert statement.
-Using the 'ON' value when not required can decrease performance of some self-referencing insert statements.
-| Level | If self-referencing insert statements which execute with AUTOCOMMIT 'OFF' can be restricted to a service level,
-then this default should be set to 'ON' only for that service level. Otherwise the setting should be made for the system.
-| Conflicts/Synergies | Not applicable.
-| Addressing the Real Problem | Not applicable.
-|===
-
-<<<
-[[upd_ordered]]
-=== UPD_ORDERED
-
-[cols="25%h,75%"]
-|===
-| Category | Influencing Query Plans
-| Description | Controls whether rows should be inserted, updated, or deleted in clustering key order.
-| Values | 'ON' The optimizer generates and considers plans where the rows are inserted, updated, or deleted in clustering key order. +
-'OFF' The optimizer does not generate plans where the rows must be inserted, updated, or deleted in clustering key order.
- +
-The default value is 'ON'.
-| Usage | Inserting, updating or deleting rows in the clustering key order is most efficient and highly recommended. Turning this CQD OFF may result in
-saving the data sorting cost but at the expense of having less efficient random I/O Insert/Update/Delete operations. +
- +
-If you know that the data is already sorted in clustering key order, or is mostly in clustering key order,
-so that it would not result in random I/O, you could set this CQD to OFF.
-| Production Usage | Not applicable.
-| Impact | If turned OFF, the system may perform large number of inefficient Random I/Os when performing Insert/Update/Delete operations.
-| Level | Query
-| Conflicts/Synergies | Not applicable.
-| Addressing the Real Problem | Not applicable.
-|===
-
http://git-wip-us.apache.org/repos/asf/incubator-trafodion/blob/8c33e750/docs/sql_reference/src/asciidoc/_chapters/introduction.adoc
----------------------------------------------------------------------
diff --git a/docs/sql_reference/src/asciidoc/_chapters/introduction.adoc b/docs/sql_reference/src/asciidoc/_chapters/introduction.adoc
index ea73f2a..2badda6 100644
--- a/docs/sql_reference/src/asciidoc/_chapters/introduction.adoc
+++ b/docs/sql_reference/src/asciidoc/_chapters/introduction.adoc
@@ -71,7 +71,8 @@ You can use Trafodion SQL statements to read, update, and create HBase tables.
* <<trafodion_sql_tables_versus_native_hbase_tables,Trafodion SQL Tables Versus Native HBase Tables>>
* <<supported_sql_statements_with_hbase_tables,Supported SQL Statements With HBase Tables>>
-For a list of Control Query Default (CQD) settings for the HBase environment, see <<HBase_Environment_CQDs,HBase Environment CQDs>>.
+For a list of Control Query Default (CQD) settings for the HBase environment, see the
+http://trafodion.incubator.apache.org/docs/cqd_reference/index.hmtl[Trafodion Control Query Default (CQD) Reference Guide].
[[ways_to_access_hbase_tables]]
=== Ways to Access HBase Tables
@@ -223,7 +224,8 @@ You can use Trafodion SQL statements to access Hive tables.
* <<type_mapping_from_hive_to_trafodion_sql,Type Mapping From Hive to Trafodion SQL>>
* <<supported_sql_statements_with_hive_tables,Supported SQL Statements With Hive Tables>>
-For a list of Control Query Default (CQD) settings for the Hive environment, see <<Hive_Environment_CQDs,Hive Environment CQDs>>.
+For a list of Control Query Default (CQD) settings for the Hive environment, see the
+http://trafodion.incubator.apache.org/docs/cqd_reference/index.hmtl[Trafodion Control Query Default (CQD) Reference Guide].
[[ansi_names_for_hive_tables]]
=== ANSI Names for Hive Tables
@@ -239,7 +241,7 @@ This example should work if a Hive table named `T` has already been defined:
```
set schema hive.hive;
-cqd hive_max_string_length '20'; -- creates a more readable display
+CQD HIVE_MAX_STRING_LENGTH '20'; -- creates a more readable display
select * from t; -- implicit table name
set schema trafodion.seabase;
@@ -267,7 +269,8 @@ Trafodion performs the following data-type mappings:
| `timestamp` | `timestamp(6)`^2^
|===
-1. The value `_n_` is determined by `CQD HIVE_MAX_STRING_LENGTH`. See <<hive_environment_cqds,Hive Environment CQDs>>.
+1. The value `_n_` is determined by `CQD HIVE_MAX_STRING_LENGTH`. See the
+http://trafodion.incubator.apache.org/docs/cqd_reference/index.hmtl[Trafodion Control Query Default (CQD) Reference Guide].
2. Hive supports timestamps with nanosecond resolution (precision of 9). Trafodion SQL supports only microsecond resolution (precision 6).
[[supported_sql_statements_with_hive_tables]]
http://git-wip-us.apache.org/repos/asf/incubator-trafodion/blob/8c33e750/docs/sql_reference/src/asciidoc/_chapters/sql_statements.adoc
----------------------------------------------------------------------
diff --git a/docs/sql_reference/src/asciidoc/_chapters/sql_statements.adoc b/docs/sql_reference/src/asciidoc/_chapters/sql_statements.adoc
index 80ce893..2a33dcc 100644
--- a/docs/sql_reference/src/asciidoc/_chapters/sql_statements.adoc
+++ b/docs/sql_reference/src/asciidoc/_chapters/sql_statements.adoc
@@ -1086,7 +1086,7 @@ control-default-option is:
* `_attribute_`
+
is a character string that represents an attribute name. For descriptions of these attributes,
-see <<Control_Query_Default_Attributes,Control Query Default (CQD) Attributes>>.
+see the http://trafodion.incubator.apache.org/docs/cqd_reference/index.html[Trafodion Control Query Default (CQD) Reference Guide].
* `_attr-value_`
+
@@ -7444,8 +7444,8 @@ see the <<control_query default_statement,Control Query Default Statement>>.
* `_attribute-name_[, MATCH {FULL | PARTIAL }]`
+
displays only the defaults that match, either fully or partially, the _attribute_ used in CONTROL QUERY DEFAULT
-statements. The match is not case-sensitive. For descriptions of these attributes, see
-<<control_query_default_attributes,CONTROL QUERY (CQD) DEFAULT Attributes>>.
+statements. The match is not case-sensitive. For descriptions of these attributes, see the
+http://trafodion.incubator.apache.org/docs/cqd_reference/index.html[Trafodion Control Query Default (CQD) Reference Guide].
+
MATCH FULL specifies that _attribute-name_ must be the same as the attribute name used in a control query default
statement. match partial specifies that _attribute-name_ must be included in the attribute name used in a control
http://git-wip-us.apache.org/repos/asf/incubator-trafodion/blob/8c33e750/docs/sql_reference/src/asciidoc/index.adoc
----------------------------------------------------------------------
diff --git a/docs/sql_reference/src/asciidoc/index.adoc b/docs/sql_reference/src/asciidoc/index.adoc
index f874749..0d8f81e 100644
--- a/docs/sql_reference/src/asciidoc/index.adoc
+++ b/docs/sql_reference/src/asciidoc/index.adoc
@@ -72,7 +72,6 @@ include::asciidoc/_chapters/sql_functions_and_expressions.adoc[]
include::asciidoc/_chapters/olap_functions.adoc[]
include::asciidoc/_chapters/runtime_stats.adoc[]
include::asciidoc/_chapters/reserved_words.adoc[]
-include::asciidoc/_chapters/cqds.adoc[]
include::asciidoc/_chapters/limits.adoc[]
[6/7] incubator-trafodion git commit: Incorporated review changes
from Dave Birdsall. Thanks!
Posted by db...@apache.org.
Incorporated review changes from Dave Birdsall. Thanks!
Also fixed too-dumb-to-live issues in the Win-ODBC chapter in the Client
Installation Guide.
Project: http://git-wip-us.apache.org/repos/asf/incubator-trafodion/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-trafodion/commit/6ad54f0b
Tree: http://git-wip-us.apache.org/repos/asf/incubator-trafodion/tree/6ad54f0b
Diff: http://git-wip-us.apache.org/repos/asf/incubator-trafodion/diff/6ad54f0b
Branch: refs/heads/master
Commit: 6ad54f0b25f96970828d7f934cbd6ba64fe32b1b
Parents: ea2c1a0
Author: Gunnar Tapper <gt...@esgyn.local>
Authored: Tue Mar 15 12:42:30 2016 -0600
Committer: Gunnar Tapper <gt...@esgyn.local>
Committed: Tue Mar 15 12:42:30 2016 -0600
----------------------------------------------------------------------
.../src/asciidoc/_chapters/odbc_windows.adoc | 10 +++++-----
.../src/asciidoc/_chapters/histograms.adoc | 12 +++++-------
.../src/asciidoc/_chapters/operational_controls.adoc | 12 ++++++------
.../src/asciidoc/_chapters/query_execution.adoc | 14 +++++++-------
docs/load_transform/src/asciidoc/_chapters/about.adoc | 2 +-
.../src/asciidoc/_chapters/bulk_load.adoc | 4 ++--
6 files changed, 26 insertions(+), 28 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-trafodion/blob/6ad54f0b/docs/client_install/src/asciidoc/_chapters/odbc_windows.adoc
----------------------------------------------------------------------
diff --git a/docs/client_install/src/asciidoc/_chapters/odbc_windows.adoc b/docs/client_install/src/asciidoc/_chapters/odbc_windows.adoc
index 8c20cfd..739d0e6 100644
--- a/docs/client_install/src/asciidoc/_chapters/odbc_windows.adoc
+++ b/docs/client_install/src/asciidoc/_chapters/odbc_windows.adoc
@@ -24,7 +24,7 @@
These installation instructions apply to Trafodion Release 1.3.*_n_* and subsequent updates of the product until otherwise indicated.
== Installation Package
-The installer file for the Trafodion ODBC Driver for Windows is packaged within the Trafodion clients tar file, `clients-1.3.*_n_*.tar.gz`, on the
+The installer file for the Trafodion ODBC Driver for Windows is packaged within the Trafodion clients tar file, `Trafodion Clients`, on the
Trafodion downloads site http://trafodion.incubator.apache.org/download.html. Use this installer file to install the Trafodion ODBC Driver for Windows:
[cols="25%l,25%,50%",options="header"]
@@ -73,10 +73,10 @@ NOTE: To install the driver on your PC, you must be logged on with a user ID tha
==== Task 1: Download the ODBC client installation file
1. Create a download folder on the client workstation.
-2. Open a Web browser and navigate to the Trafodion downloads site http://trafodion.incubator.apache.com/download.html
-3. Click on the `clients-1.3.*_n_*.tar.gz` link to start downloading the Trafodion clients tar file to your workstation.
-4. Place the `clients-1.3.*_n_*.tar.gz file` into the download folder.
-5. Extract the contents of the `clients-1.3.*_n_*.tar.gz` file by using an extract tool like WinZip. The package file contains the
+2. Open a Web browser and navigate to the Trafodion downloads site http://trafodion.incubator.apache.org/download.html
+3. Click on the `Trafodion Clients` link to start downloading the Trafodion clients tar file to your workstation.
+4. Place the `*.tar.gz file` into the download folder.
+5. Extract the contents of the `*.tar.gz` file by using an extract tool like WinZip. The package file contains the
`TFODBC64-1.3.*_n_*.msi` installer file, which is extracted to the clients sub-directory.
==== Task 2: Start the InstallShield wizard
http://git-wip-us.apache.org/repos/asf/incubator-trafodion/blob/6ad54f0b/docs/cqd_reference/src/asciidoc/_chapters/histograms.adoc
----------------------------------------------------------------------
diff --git a/docs/cqd_reference/src/asciidoc/_chapters/histograms.adoc b/docs/cqd_reference/src/asciidoc/_chapters/histograms.adoc
index 2126f53..7086d90 100644
--- a/docs/cqd_reference/src/asciidoc/_chapters/histograms.adoc
+++ b/docs/cqd_reference/src/asciidoc/_chapters/histograms.adoc
@@ -53,7 +53,7 @@ make the refresh more frequent.
| *Impact* | Longer histogram refresh intervals can improve compile times. However, longer refresh intervals yield more obsolete
the histograms. More obsolete histograms may result in poor performance for queries that could leverage recently updated statistics.
| *Level* | System or Service.
-| *Conflicts/Synergies* | Frequency of update statistics run either using MAINTAIN.
+| *Conflicts/Synergies* | Frequency of update statistics run using MAINTAIN.
| *Real Problem Addressed* | Not applicable.
| *Introduced In Release* | Trafodion 1.3.0.
| *Deprecated In Release* | Not applicable.
@@ -67,19 +67,18 @@ the histograms. More obsolete histograms may result in poor performance for quer
|===
| *Description* | Controls the level of missing statistics warnings that should be displayed. The warnings impacted are 6007, 6008, 6010, and 6011.
| *Values* |
-*'0'*: Display no warnings. The Update Statistics Automation Server still continues to record missing column statistics. +
+*'0'*: Display no warnings. +
+
*'1'*: Display only missing single column statistics warnings. These include 6008 and 6011. +
+
*'2'*: Display all missing single and multi-column statistics warnings for scans only. +
+
-*'3'*: Display all missing single and multi-column statistics warnings for scans and join operators only.
-The Update Statistics Automation Server continues to record missing column statistics. +
+*'3'*: Display all missing single and multi-column statistics warnings for scans and join operators only. +
+
*'4'*: Display all missing single and multi-column statistics warnings. +
+
The default value is *'4'*.
-| *Usage* | The Update Statistics Automation Server (USAS) captures these warnings and gathers appropriate statistics.
+| *Usage* |
If you do not want to see these warnings, then change the setting to '0'. If you want to track the warnings, then you have a
choice of which warnings you want to track. Each setting gives you the ability to filter the warnings seen for missing single
or multi-column statistics for join or scan operations. This controls the resulting number of warning messages. +
@@ -99,8 +98,7 @@ to get several warnings. Not all warnings may contribute to plan improvements. T
based on the search path, some of which may not even impact the plan quality. Also, the cost of gathering statistics on those columns
may not bring commensurate benefit to a large number of queries.
| *Level* | System.
-| *Conflicts/Synergies* | The USAS does capture these warnings and gathers statistics felt appropriate
-for query performance. If you are using USAS, then you may set the warning level to 1 or 0.
+| *Conflicts/Synergies* | Not applicable.
| *Real Problem Addressed* | Not applicable.
| *Introduced In Release* | Trafodion 1.3.0.
| *Deprecated In Release* | Not applicable.
http://git-wip-us.apache.org/repos/asf/incubator-trafodion/blob/6ad54f0b/docs/cqd_reference/src/asciidoc/_chapters/operational_controls.adoc
----------------------------------------------------------------------
diff --git a/docs/cqd_reference/src/asciidoc/_chapters/operational_controls.adoc b/docs/cqd_reference/src/asciidoc/_chapters/operational_controls.adoc
index 50d95a2..8d077cf 100644
--- a/docs/cqd_reference/src/asciidoc/_chapters/operational_controls.adoc
+++ b/docs/cqd_reference/src/asciidoc/_chapters/operational_controls.adoc
@@ -114,13 +114,13 @@ are cached and larger scans are not cached to avoid cache trashing.
*> 0*: The cached Hive meta-data is valid only for <value> seconds. +
+
Default: *'0'*.
-| *Usage* | Use of value -1 when the Hive tables are read-only and no updates are possible to avoid reading the
-Hive metadata when the same table is referenced in queries again. +
+| *Usage* | Use a value of -1 when the Hive tables are read-only. This allows Trafodion to avoid repeatedly reading the
+metadata when the table is referenced in multiple queries. +
+
-Use of value 0 when frequent update of Hive tables are possible. +
+Use a value of 0 when updates to the Hive tables are likely to be frequent. +
+
-Use of a positive value when when infrequent update of Hive tables can take place and the value can be set so that it is impossible
-for the Hive tables to be updated within next <value> seconds.
+Use a positive value _n_ to cause Trafodion to re-read metadata after n seconds has elapsed.
+Use this option when updates to the Hive table happen at least _n_ seconds apart.
| *Production Usage* | Consult with the Trafodion community.
| *Impact* | Compilation time.
| *Level* | Session.
@@ -200,7 +200,7 @@ If the plan object is not produced due to a text cache miss, then the plan is st
Default: *'OFF'*.
| *Usage* | When this CQD is ON during a bulk load, then any indexes on a table are maintained incrementally.
New rows are added to the base table and all the indexes in HFiles and then during LOAD COMPLETE phase all new files are moved to HBase.
-Indexes are not offline. However, it causes the index to be inconsistent with the base table if any of the new rows have the same key value as an existing.
+Indexes are not offline. However, it causes the index to be inconsistent with the base table if any of the new rows have the same key value as an existing row.
Change the default to ON only when certain that new rows do not have a conflict with existing rows in table.
| *Production Usage* | Yes.
| *Impact* | Bulk load into tables with index is faster, when the attribute is set to ON.
http://git-wip-us.apache.org/repos/asf/incubator-trafodion/blob/6ad54f0b/docs/cqd_reference/src/asciidoc/_chapters/query_execution.adoc
----------------------------------------------------------------------
diff --git a/docs/cqd_reference/src/asciidoc/_chapters/query_execution.adoc b/docs/cqd_reference/src/asciidoc/_chapters/query_execution.adoc
index c2553f1..eae1ff3 100644
--- a/docs/cqd_reference/src/asciidoc/_chapters/query_execution.adoc
+++ b/docs/cqd_reference/src/asciidoc/_chapters/query_execution.adoc
@@ -71,7 +71,7 @@ data blocks cached earlier thereby impacting the performance of the random reads
in such a case. +
+
Set the CQD <<hbase-region-server-max-heap-size,HBASE_REGION_SERVER_MAX_HEAP_SIZE>> value to reflect the amount of java heap size reserved for the region servers.
-This CQD is used by the Trafodion Optimizer to evaluate if the block cache to be turned off.
+This CQD is used by the Trafodion Optimizer to evaluate if the block cache should be turned off.
| *Production Usage* | Leave the setting to be 'SYSTEM' when HBase is configured to use LRU block cache. If needed,
you can override this settings with 'ON' or 'OFF'. +
+
@@ -162,8 +162,8 @@ if you think that you need to use this CQD.
|===
<<<
-[[hbase-rowset-vssb-opt]]
-== HBASE_ROWSET_VSSB_OPT
+[[hbase-rowset-vsbb-opt]]
+== HBASE_ROWSET_VSBB_OPT
[cols="25%h,75%"]
|===
@@ -187,8 +187,8 @@ If possible, then the query plan involves VSBB operators. The Virtual Sequential
|===
<<<
-[[hbase-rowset-vssb-size]]
-== HBASE_ROWSET_VSSB_SIZE
+[[hbase-rowset-vsbb-size]]
+== HBASE_ROWSET_VSBB_SIZE
[cols="25%h,75%"]
|===
@@ -201,7 +201,7 @@ The default value is *'1024'*.
the queue to IUD (INSERT,UPDATE,DELETE) operator is filled up in the data flow architecture of Trafodion. +
+
You can adjust the maximum size to suit your application needs and thus tune it to perform optimally.
-| *Production Usage* | Yes. You can disable this feature by setting the HBASE_ROWSET_VSSB_OPT CQD to 'OFF'.
+| *Production Usage* | Yes. You can disable this feature by setting the HBASE_ROWSET_VSBB_OPT CQD to 'OFF'.
| *Impact* | The performance of your application may be affected by setting this CQD too low.
| *Level* | Query.
| *Conflicts/Synergies* | Not applicable.
@@ -216,7 +216,7 @@ You can adjust the maximum size to suit your application needs and thus tune it
[cols="25%h,75%"]
|===
-| *Description* | Enables Trafodion to leverage the HBase small scanner optimization. This optimization reduces I/O usage with up to 66%
+| *Description* | Enables Trafodion to leverage the HBase small scanner optimization. This optimization reduces I/O usage up to 66%
and enables non-blocking reads for higher concurrency support. When a scan is known to require less than a HBASE BLOCK SIZE (default is 64K),
then enabling the HBase small scanner optimization increases performance.
| *Values* |
http://git-wip-us.apache.org/repos/asf/incubator-trafodion/blob/6ad54f0b/docs/load_transform/src/asciidoc/_chapters/about.adoc
----------------------------------------------------------------------
diff --git a/docs/load_transform/src/asciidoc/_chapters/about.adoc b/docs/load_transform/src/asciidoc/_chapters/about.adoc
index 4a1a1bf..f15fd04 100644
--- a/docs/load_transform/src/asciidoc/_chapters/about.adoc
+++ b/docs/load_transform/src/asciidoc/_chapters/about.adoc
@@ -37,7 +37,7 @@ This guide targets anyone wanting to load data into a Trafodion database.
You need to have skills in the following areas to make full use of the information in this guide:
* SQL DDL and DML.
-* Installion and configuration of Linux software.
+* Installation and configuration of Linux software.
* Trafodion administration.
* Depending on your data source, Java and/or Hadoop ecosystem usage.
http://git-wip-us.apache.org/repos/asf/incubator-trafodion/blob/6ad54f0b/docs/load_transform/src/asciidoc/_chapters/bulk_load.adoc
----------------------------------------------------------------------
diff --git a/docs/load_transform/src/asciidoc/_chapters/bulk_load.adoc b/docs/load_transform/src/asciidoc/_chapters/bulk_load.adoc
index f17f718..46593b5 100644
--- a/docs/load_transform/src/asciidoc/_chapters/bulk_load.adoc
+++ b/docs/load_transform/src/asciidoc/_chapters/bulk_load.adoc
@@ -52,8 +52,8 @@ LOAD INTO target_table SELECT * FROM source_table WHERE custkey >= 1000 ;
[[bulk-load-data-from-hdfs-files]]
== Load Data From HDFS Files
-You copy your data (local or remote) into an HDFS folder. Then, you create an external Hive table (with correct fields) that point to the HDFS folder containing the data.
-You may also specify a WHERE clause on the source data as a filter, if needed.
+You copy your data (local or remote) into an HDFS folder. Then, you create an external Hive table (with correct fields) that points
+to the HDFS folder containing the data. You may also specify a WHERE clause on the source data as a filter, if needed.
See the https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-ExternalTables[External Tables]
page on the https://cwiki.apache.org/confluence/display/Hive[Hive Wiki] for more information.
[2/7] incubator-trafodion git commit: Publication of new
documentation.
Posted by db...@apache.org.
Publication of new documentation.
Added Load and Transform Guide and CQD Reference
Guide. Added a link to the Trafodion videos under the Project tab. Renamed
misnamed logo file, which corrects the broken link on the logo page.
Project: http://git-wip-us.apache.org/repos/asf/incubator-trafodion/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-trafodion/commit/223db1e4
Tree: http://git-wip-us.apache.org/repos/asf/incubator-trafodion/tree/223db1e4
Diff: http://git-wip-us.apache.org/repos/asf/incubator-trafodion/diff/223db1e4
Branch: refs/heads/master
Commit: 223db1e413dda706a2ab42b9b11175d954195ced
Parents: 1eb6051
Author: Gunnar Tapper <gt...@esgyn.local>
Authored: Fri Mar 11 23:16:54 2016 -0700
Committer: Gunnar Tapper <gt...@esgyn.local>
Committed: Fri Mar 11 23:16:54 2016 -0700
----------------------------------------------------------------------
docs/cqd_reference/pom.xml | 8 +-
.../src/asciidoc/_chapters/debugging.adoc | 95 ++++++
.../src/asciidoc/_chapters/histograms.adoc | 38 +--
.../src/asciidoc/_chapters/introduction.adoc | 13 +-
.../_chapters/operational_controls.adoc | 138 +++++++-
.../src/asciidoc/_chapters/query_execution.adoc | 239 ++++++++++++++
.../src/asciidoc/_chapters/query_plans.adoc | 330 +++++++++++++++----
.../asciidoc/_chapters/runtime_controls.adoc | 4 +-
.../src/asciidoc/_chapters/schema_controls.adoc | 24 +-
.../asciidoc/_chapters/table_definition.adoc | 142 ++++++++
.../src/asciidoc/_chapters/transactions.adoc | 4 +-
.../src/asciidoc/_chapters/update_stats.adoc | 31 +-
docs/cqd_reference/src/asciidoc/index.adoc | 3 +
docs/load_transform/pom.xml | 8 +-
.../src/asciidoc/_chapters/about.adoc | 8 +-
.../src/asciidoc/_chapters/bulk_load.adoc | 37 ++-
.../src/asciidoc/_chapters/introduction.adoc | 14 +-
.../src/asciidoc/_chapters/tables_indexes.adoc | 18 +-
.../src/asciidoc/_chapters/trickle_load.adoc | 12 +-
.../src/asciidoc/_chapters/troubleshoot.adoc | 4 +-
docs/src/site/markdown/documentation.md | 66 ++--
docs/src/site/markdown/logo.md | 2 +-
.../resources/images/logos/pb-Trafodion.png | Bin 49089 -> 0 bytes
.../resources/images/logos/pb-trafodion.png | Bin 0 -> 49089 bytes
docs/src/site/site.xml | 1 +
pom.xml | 6 +-
26 files changed, 1049 insertions(+), 196 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-trafodion/blob/223db1e4/docs/cqd_reference/pom.xml
----------------------------------------------------------------------
diff --git a/docs/cqd_reference/pom.xml b/docs/cqd_reference/pom.xml
index 9fbb88c..2075c15 100644
--- a/docs/cqd_reference/pom.xml
+++ b/docs/cqd_reference/pom.xml
@@ -236,15 +236,15 @@
- target/docs/<version>/<document> contains the PDF version and the web book. The web book is named index.html
-->
<!-- Copy the PDF file to its target directories -->
- <copy file="${basedir}/target/index.pdf" tofile="${basedir}/../target/docs/cqd_reference_guide/Trafodion_CQD_Reference_Guide.pdf" />
- <copy file="${basedir}/target/index.pdf" tofile="${basedir}/../target/docs/${project.version}/cqd_reference_guide/Trafodion_CQD_Reference_Guide.pdf" />
+ <copy file="${basedir}/target/index.pdf" tofile="${basedir}/../target/docs/cqd_reference/Trafodion_CQD_Reference_Guide.pdf" />
+ <copy file="${basedir}/target/index.pdf" tofile="${basedir}/../target/docs/${project.version}/cqd_reference/Trafodion_CQD_Reference_Guide.pdf" />
<!-- Copy the Web Book files to their target directories -->
- <copy todir="${basedir}/../target/docs/cqd_reference_guide">
+ <copy todir="${basedir}/../target/docs/cqd_reference">
<fileset dir="${basedir}/target/site">
<include name="**/*.*"/> <!--All sub-directories, too-->
</fileset>
</copy>
- <copy todir="${basedir}/../target/docs/${project.version}/cqd_reference_guide">
+ <copy todir="${basedir}/../target/docs/${project.version}/cqd_reference">
<fileset dir="${basedir}/target/site">
<include name="**/*.*"/> <!--All sub-directories, too-->
</fileset>
http://git-wip-us.apache.org/repos/asf/incubator-trafodion/blob/223db1e4/docs/cqd_reference/src/asciidoc/_chapters/debugging.adoc
----------------------------------------------------------------------
diff --git a/docs/cqd_reference/src/asciidoc/_chapters/debugging.adoc b/docs/cqd_reference/src/asciidoc/_chapters/debugging.adoc
new file mode 100644
index 0000000..3d50322
--- /dev/null
+++ b/docs/cqd_reference/src/asciidoc/_chapters/debugging.adoc
@@ -0,0 +1,95 @@
+////
+/**
+* @@@ START COPYRIGHT @@@
+*
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing,
+* software distributed under the License is distributed on an
+* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+* KIND, either express or implied. See the License for the
+* specific language governing permissions and limitations
+* under the License.
+*
+* @@@ END COPYRIGHT @@@
+ */
+////
+
+[[debugging]]
+= Debugging
+
+This section describes CQDs that are used for debugging controls.
+
+[[udr-debug-flags]]
+== UDR_DEBUG_FLAGS
+
+[cols="25%h,75%"]
+|===
+| *Description* | Used when debugging user-defined functions (UDFs).
+| *Values* |
+*0-n* +
+ +
+Default: *'0'*.
+| *Usage* |
+See See https://cwiki.apache.org/confluence/display/TRAFODION/Tutorial%3A+The+object-oriented+UDF+interface#Tutorial:Theobject-orientedUDFinterface-DebuggingUDFcode[UDF Tutorial].
+| *Production Usage* | Consult the Trafodion community.
+| *Impact* | Not applicable.
+| *Level* | Session.
+| *Conflicts/Synergies* | Not applicable.
+| *Real Problem Addressed* | Not applicable.
+| *Introduced In Release* | Trafodion 1.3.0.
+| *Deprecated In Release* | Not applicable.
+|===
+
+<<<
+[[udr_jvm_debug_port]]
+== UDR_JVM_DEBUG_PORT
+
+[cols="25%h,75%"]
+|===
+| *Description* | Used when debugging user-defined functions (UDFs).
+| *Values* |
+*0-n* +
+ +
+Default: *'0'*.
+| *Usage* |
+See See https://cwiki.apache.org/confluence/display/TRAFODION/Tutorial%3A+The+object-oriented+UDF+interface#Tutorial:Theobject-orientedUDFinterface-DebuggingUDFcode[UDF Tutorial].
+| *Production Usage* | Consult the Trafodion community.
+| *Impact* | Not applicable.
+| *Level* | Session.
+| *Conflicts/Synergies* | Not applicable.
+| *Real Problem Addressed* | Not applicable.
+| *Introduced In Release* | Trafodion 1.3.0.
+| *Deprecated In Release* | Not applicable.
+|===
+
+<<<
+[[udr-jvm-debug-timeout]]
+== UDR_JVM_DEBUG_TIMEOUT
+
+[cols="25%h,75%"]
+|===
+| *Description* | Used when debugging user-defined functions (UDFs).
+| *Values* |
+*0-n* +
+ +
+Default: *'0'*.
+| *Usage* |
+See See https://cwiki.apache.org/confluence/display/TRAFODION/Tutorial%3A+The+object-oriented+UDF+interface#Tutorial:Theobject-orientedUDFinterface-DebuggingUDFcode[UDF Tutorial].
+| *Production Usage* | Consult the Trafodion community.
+| *Impact* | Not applicable.
+| *Level* | Session.
+| *Conflicts/Synergies* | Not applicable.
+| *Real Problem Addressed* | Not applicable.
+| *Introduced In Release* | Trafodion 1.3.0.
+| *Deprecated In Release* | Not applicable.
+|===
+
http://git-wip-us.apache.org/repos/asf/incubator-trafodion/blob/223db1e4/docs/cqd_reference/src/asciidoc/_chapters/histograms.adoc
----------------------------------------------------------------------
diff --git a/docs/cqd_reference/src/asciidoc/_chapters/histograms.adoc b/docs/cqd_reference/src/asciidoc/_chapters/histograms.adoc
index 53d1539..2126f53 100644
--- a/docs/cqd_reference/src/asciidoc/_chapters/histograms.adoc
+++ b/docs/cqd_reference/src/asciidoc/_chapters/histograms.adoc
@@ -33,9 +33,8 @@ This section describes CQDs that are used to manage histograms.
[cols="25%h,75%"]
|===
-| *Category* | Manage Histograms
-| *Description* | Defines the time interval after which timestamps for cached histograms are checked to be refreshed.
-| *Values* | Unsigned integer Unit is seconds. +
+| *Description* | Defines the time interval after which timestamps for cached histograms are checked for refresh processing.
+| *Values* | Unsigned integer in seconds. +
+
The default value is *'3600'* (1 hour).
| *Usage* | Histogram statistics are cached so that the compiler can avoid access to the metadata tables, thereby reducing compile times.
@@ -51,10 +50,10 @@ being updated during that time. On the other hand, if you are updating statistic
enough, and you want these to be picked up immediately by the compiler because you have seen this to have a dramatic impact on plan quality, then you can
make the refresh more frequent.
| *Production Usage* | Not applicable.
-| *Impact* | Longer histogram refresh intervals can improve compile times. However, the longer the refresh interval the more obsolete
-the histograms. That could result in poor performance for queries that could leverage recently updated statistics.
+| *Impact* | Longer histogram refresh intervals can improve compile times. However, longer refresh intervals yield more obsolete
+the histograms. More obsolete histograms may result in poor performance for queries that could leverage recently updated statistics.
| *Level* | System or Service.
-| *Conflicts/Synergies* | Frequency of update statistics run either using MAINTAIN or using Update Statistics Automation Server.
+| *Conflicts/Synergies* | Frequency of update statistics run either using MAINTAIN.
| *Real Problem Addressed* | Not applicable.
| *Introduced In Release* | Trafodion 1.3.0.
| *Deprecated In Release* | Not applicable.
@@ -66,22 +65,21 @@ the histograms. That could result in poor performance for queries that could lev
[cols="25%h,75%"]
|===
-| *Category* | Manage Histograms
-| *Description* | Controls the level of missing statistics warnings that should be displayed. The warnings impacted are 6007, 6008, 6010 and 6011.
+| *Description* | Controls the level of missing statistics warnings that should be displayed. The warnings impacted are 6007, 6008, 6010, and 6011.
| *Values* |
-*'0'*: Display no warnings. Update Statistics Automation Server still continues to record missing column statistics. +
+*'0'*: Display no warnings. The Update Statistics Automation Server still continues to record missing column statistics. +
+
*'1'*: Display only missing single column statistics warnings. These include 6008 and 6011. +
+
*'2'*: Display all missing single and multi-column statistics warnings for scans only. +
+
*'3'*: Display all missing single and multi-column statistics warnings for scans and join operators only.
-Update Statistics Automation Server continues to record missing column statistics. +
+The Update Statistics Automation Server continues to record missing column statistics. +
+
*'4'*: Display all missing single and multi-column statistics warnings. +
+
The default value is *'4'*.
-| *Usage* | Update Statistics Automation Server captures these warnings and gathers appropriate statistics.
+| *Usage* | The Update Statistics Automation Server (USAS) captures these warnings and gathers appropriate statistics.
If you do not want to see these warnings, then change the setting to '0'. If you want to track the warnings, then you have a
choice of which warnings you want to track. Each setting gives you the ability to filter the warnings seen for missing single
or multi-column statistics for join or scan operations. This controls the resulting number of warning messages. +
@@ -101,8 +99,8 @@ to get several warnings. Not all warnings may contribute to plan improvements. T
based on the search path, some of which may not even impact the plan quality. Also, the cost of gathering statistics on those columns
may not bring commensurate benefit to a large number of queries.
| *Level* | System.
-| *Conflicts/Synergies* | Update Statistics Automation Server does capture these warnings and gathers statistics felt appropriate
-for query performance. If you are using USAS then you may set the warning level to low or 0.
+| *Conflicts/Synergies* | The USAS does capture these warnings and gathers statistics felt appropriate
+for query performance. If you are using USAS, then you may set the warning level to 1 or 0.
| *Real Problem Addressed* | Not applicable.
| *Introduced In Release* | Trafodion 1.3.0.
| *Deprecated In Release* | Not applicable.
@@ -114,9 +112,8 @@ for query performance. If you are using USAS then you may set the warning level
[cols="25%h,75%"]
|===
-| *Category* | Manage Histograms
| *Description* | Defines the time interval after which the fake histograms in the cache should be refreshed unconditionally.
-| *Values* | Unsigned integer Unit is seconds. +
+| *Values* | Unsigned integer. Unit is seconds. +
+
The default value is *'3600'* (1 hour).
| *Usage* | Histogram statistics are "fake" when update statistics is not being run, but instead the customer is
@@ -143,7 +140,6 @@ then the compiler may be working with obsolete histogram statistics, potentially
[cols="25%h,75%"]
|===
-| *Category* | Manage Histograms
| *Description* | Influences the compiler to pre-fetch the histograms and save them in cache.
| *Values* |
*'ON'*: Pre-fetches the histograms. +
@@ -154,7 +150,7 @@ The default value is *'ON'*.
| *Production Usage* | Not applicable.
| *Impact* | Though it makes compilation time faster, it may result in the histogram cache to be filled with histograms that may never be used.
| *Level* | System or Service.
-| *Conflicts/Synergies* | Use this CQD with CACHE_HISTOGRAMS. If CACHE_HISTOGRAMS is OFF, then this CQD has no effect. $$$
+| *Conflicts/Synergies* | Use this CQD with <<cache-histograms,CACHE_HISTOGRAMS>>. If CACHE_HISTOGRAMS is OFF, then this CQD has no effect.
| *Real Problem Addressed* | Not applicable.
| *Introduced In Release* | Trafodion 1.3.0.
| *Deprecated In Release* | Not applicable.
@@ -166,9 +162,8 @@ The default value is *'ON'*.
[cols="25%h,75%"]
|===
-| *Category* | Manage Histograms
-| *Description* | Specifies the minimum row count for which the optimizer needs histograms, in order to compute better cardinality estimates.
-The optimizer does not issue any missing statistics warnings for tables whose size is smaller than the value of this CQD.
+| *Description* | Specifies the minimum row count for which the Trafodion Optimizer needs histograms, in order to compute better cardinality estimates.
+The Optimizer does not issue any missing statistics warnings for tables whose size is smaller than the value of this CQD.
| *Values* | Integer. +
+
The default value is *'50000'*.
@@ -190,7 +185,6 @@ then this CQD does not have any effect. Also, for tables having fewer rows than
[cols="25%h,75%"]
|===
-| *Category* | Manage Histograms
| *Description* | Enables the Compile Time Stats feature. Compile Time Stats are produced during query plan generation by executing a
subset of the query on a subset of data to gather more accurate cardinality estimations.
| *Values* |
@@ -208,7 +202,7 @@ It can also be disabled if the extra collection of statistics seems to be advers
| *Impact* | The feature improves cardinality estimates for Scan operators thus improving the plan quality.
However, it can also increase the compile time.
| *Level* | Any.
-| *Conflicts/Synergies* | In order to use the feature in its default form, sample tables should exist in `public_access_schema`. $$$
+| *Conflicts/Synergies* | In order to use the feature in its default form, sample tables should exist in `public_access_schema`.
| *Real Problem Addressed* | Not applicable.
| *Introduced In Release* | Trafodion 1.3.0.
| *Deprecated In Release* | Not applicable.
http://git-wip-us.apache.org/repos/asf/incubator-trafodion/blob/223db1e4/docs/cqd_reference/src/asciidoc/_chapters/introduction.adoc
----------------------------------------------------------------------
diff --git a/docs/cqd_reference/src/asciidoc/_chapters/introduction.adoc b/docs/cqd_reference/src/asciidoc/_chapters/introduction.adoc
index 5ba441e..fe3be20 100644
--- a/docs/cqd_reference/src/asciidoc/_chapters/introduction.adoc
+++ b/docs/cqd_reference/src/asciidoc/_chapters/introduction.adoc
@@ -27,7 +27,7 @@
= Introduction
Refer to the
-http://trafodion.apache.org/docs/sql_reference/index.html#control_query_cancel_statement[Trafofion SQL Reference Manual]
+http://trafodion.apache.org/docs/sql_reference/index.html#control_query_cancel_statement[Trafodion SQL Reference Manual]
full documentation of the CQD (CONTROL QUERY DEFAULT) statement.
The CONTROL QUERY DEFAULT statement changes the default settings for the current process. You can execute
@@ -71,22 +71,13 @@ CONTROL QUERY DEFAULT HBASE_MAX_COLUMN_NAME_LENGTH RESET;
```
<<<
-[[introduction-cqd-descriptin]]
+[[introduction-cqd-descriptions]]
== CQD Descriptions
The following information is provided for each CQD:
[cols="25%h,75%"]
|===
-| *Category* | Identifies one of these categories with which the CQD is associated: +
- +
-• Influence Query Plans +
-• Manage Histograms +
-• Transaction control and locking +
-• Run time controls +
-• Schema controls +
-• Update Statistics and Reorg +
-• Operational controls
| *Description* | Describes the purpose of the CQD.
| *Values* | Identifies this information: +
+
http://git-wip-us.apache.org/repos/asf/incubator-trafodion/blob/223db1e4/docs/cqd_reference/src/asciidoc/_chapters/operational_controls.adoc
----------------------------------------------------------------------
diff --git a/docs/cqd_reference/src/asciidoc/_chapters/operational_controls.adoc b/docs/cqd_reference/src/asciidoc/_chapters/operational_controls.adoc
index aad0e7f..50d95a2 100644
--- a/docs/cqd_reference/src/asciidoc/_chapters/operational_controls.adoc
+++ b/docs/cqd_reference/src/asciidoc/_chapters/operational_controls.adoc
@@ -33,15 +33,15 @@ This section describes CQDs that are used for operational controls.
[cols="25%h,75%"]
|===
-| *Category* | Operational Controls
| *Description* | Indicates whether a warning should be issued when a query is retried, in case a failed query is automatically retried.
-| *Values* | *'ON'* or *'OFF'* +
+| *Values* |
+*'ON'* or *'OFF'* +
+
The default value is *'OFF'*.
-| *Usage* | There are certain cases, such as when a node failure occurs, where a query may fail midstream. In many of such failure scenarios,
-if the query has not returned any data, then it is retried automatically. +
+| *Usage* | There are certain cases, such as when a node failure occurs, where a query may fail midstream. The query is usually
+retried automatically if it has not returned any data. +
+
-When such retries happen, you may want to see a warning that an automatic retry took place. That would be a reason to turn this on.
+When such retries happen, then you may want to see a warning that an automatic retry took place. That would be a reason to turn this on.
| *Production Usage* | Not applicable.
| *Impact* | You get a warning message every time a query is automatically retried due to a failure. When there is a node failure,
then a large number of queries may be impacted. Therefore, you need to assess if you want to see a flood of warnings. The warning is returned after the query completes.
@@ -53,12 +53,89 @@ then a large number of queries may be impacted. Therefore, you need to assess if
|===
<<<
+[[explain-description-column-size]]
+== EXPLAIN_DESCRIPTION_COLUMN_SIZE
+
+[cols="25%h,75%"]
+|===
+| *Description* | Specifies maximum length of DESCRIPTION column for EXPLAIN virtual tables.
+| *Values* |
+Positive integer > 10,000 +
+ +
+Default: -1, indicating a maximum size of 10,000.
+| *Usage* | Query-plan information for a SQL DML statement is stored temporarily in the Explain virtual table.
+For large queries or queries with complex predicates, the default size of 10 KB may be insufficient to describe certain nodes in the query plan.
+Specifying a larger value for this CQD allows more bytes to be stored in the description column. +
+ +
+Change this setting if you see explain plan being undesirably truncated only.
+| *Production Usage* | Yes.
+| *Impact* | The explain plan truncation is reduced or removed.
+| *Level* | System.
+| *Conflicts/Synergies* | Not applicable.
+| *Real Problem Addressed* | Not applicable.
+| *Introduced In Release* | Trafodion 1.3.0.
+| *Deprecated In Release* | Not applicable.
+|===
+
+<<<
+[[hbase-region-server-max-heap-size]]
+== HBASE_REGION_SERVER_MAX_HEAP_SIZE
+
+[cols="25%h,75%"]
+|===
+| *Description* | Enables Trafodion to use HBase block cache in an optimal manner. Defines the maximum Java
+heap size (-Xmx option) the HBase Region Server are assigned, in MB units.
+| *Values* |
+Positive integer. +
+ +
+Default: *'1024'*.
+| *Usage* | If the HBase Region servers are configured with a maximum Java heap size different than 1 GB, then
+set this attribute so that Trafodion is aware of the actual maximum heap size of the Region Servers.
+| *Production Usage* | Yes.
+| *Impact* | When set correctly, this CQD ensures that HBase block cache are be used optimally. Small scans
+are cached and larger scans are not cached to avoid cache trashing.
+| *Level* | System.
+| *Conflicts/Synergies* | Not applicable.
+| *Real Problem Addressed* | Not applicable.
+| *Introduced In Release* | Trafodion 1.3.0.
+| *Deprecated In Release* | Not applicable.
+|===
+
+<<<
+[[hive-metadata-refresh-interval]]
+== HIVE_METADATA_REFRESH_INTERVAL
+
+[cols="25%h,75%"]
+|===
+| *Description* | Controls the metadata cache for Hive tables.
+| *Values* |
+*-1*: Never invalidate any cache entries. +
+*0*: Always check the latest meta-data from Hive. +
+*> 0*: The cached Hive meta-data is valid only for <value> seconds. +
+ +
+Default: *'0'*.
+| *Usage* | Use of value -1 when the Hive tables are read-only and no updates are possible to avoid reading the
+Hive metadata when the same table is referenced in queries again. +
+ +
+Use of value 0 when frequent update of Hive tables are possible. +
+ +
+Use of a positive value when when infrequent update of Hive tables can take place and the value can be set so that it is impossible
+for the Hive tables to be updated within next <value> seconds.
+| *Production Usage* | Consult with the Trafodion community.
+| *Impact* | Compilation time.
+| *Level* | Session.
+| *Conflicts/Synergies* | Not applicable.
+| *Real Problem Addressed* | Not applicable.
+| *Introduced In Release* | Trafodion 1.3.0.
+| *Deprecated In Release* | Not applicable.
+|===
+
+<<<
[[query-cache]]
== QUERY_CACHE
[cols="25%h,75%"]
|===
-| *Category* | Operational Controls $$$
| *Description* | Attempts to reduce compilation times by storing and reusing previously compiled query plans.
It maximizes the chances of plan reuse by parameterizing literals in equality predicates. Two equality predicates, "col = val1" and "col = val2",
are considered to match if their selectivities match. +
@@ -108,3 +185,52 @@ If the plan object is not produced due to a text cache miss, then the plan is st
| *Introduced In Release* | Trafodion 1.3.0.
| *Deprecated In Release* | Not applicable.
|===
+
+<<<
+[[traf-load-allow-risky-index-maintenance]]
+== TRAF_LOAD_ALLOW_RISKY_INDEX_MAINTENANCE
+
+[cols="25%h,75%"]
+|===
+| *Description* | Allows incremental index maintenance during bulk load.
+| *Values* |
+*'ON'*: Incremental index maintenance enabled. +
+*'OFF'*: Incremental index maintenance disabled. +
+ +
+Default: *'OFF'*.
+| *Usage* | When this CQD is ON during a bulk load, then any indexes on a table are maintained incrementally.
+New rows are added to the base table and all the indexes in HFiles and then during LOAD COMPLETE phase all new files are moved to HBase.
+Indexes are not offline. However, it causes the index to be inconsistent with the base table if any of the new rows have the same key value as an existing.
+Change the default to ON only when certain that new rows do not have a conflict with existing rows in table.
+| *Production Usage* | Yes.
+| *Impact* | Bulk load into tables with index is faster, when the attribute is set to ON.
+| *Level* | System.
+| *Conflicts/Synergies* | Not applicable.
+| *Real Problem Addressed* | Not applicable.
+| *Introduced In Release* | Trafodion 1.3.0.
+| *Deprecated In Release* | Not applicable.
+|===
+
+<<<
+[[traf-load-flush-size-in-kb]]
+== TRAF_LOAD_FLUSH_SIZE_IN_KB
+
+[cols="25%h,75%"]
+|===
+| *Description* | Specifies the flush size used by bulk load when writing to HFiles.
+| *Values* |
+Positive integer. +
+ +
+Default: *'2014'*.
+| *Usage* | If the system is not memory constrained, then specifying a larger value may make the write phase of the LOAD statement to proceed faster. +
+ +
+Consider using lower values if the table has several indexes and is memory constrained.
+| *Production Usage* | Yes.
+| *Impact* | Affects memory usage patterns and write performance of LOAD.
+| *Level* | System.
+| *Conflicts/Synergies* | Not applicable.
+| *Real Problem Addressed* | Not applicable.
+| *Introduced In Release* | Trafodion 1.3.0.
+| *Deprecated In Release* | Not applicable.
+|===
+
http://git-wip-us.apache.org/repos/asf/incubator-trafodion/blob/223db1e4/docs/cqd_reference/src/asciidoc/_chapters/query_execution.adoc
----------------------------------------------------------------------
diff --git a/docs/cqd_reference/src/asciidoc/_chapters/query_execution.adoc b/docs/cqd_reference/src/asciidoc/_chapters/query_execution.adoc
new file mode 100644
index 0000000..c2553f1
--- /dev/null
+++ b/docs/cqd_reference/src/asciidoc/_chapters/query_execution.adoc
@@ -0,0 +1,239 @@
+////
+/**
+* @@@ START COPYRIGHT @@@
+*
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing,
+* software distributed under the License is distributed on an
+* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+* KIND, either express or implied. See the License for the
+* specific language governing permissions and limitations
+* under the License.
+*
+* @@@ END COPYRIGHT @@@
+ */
+////
+
+[[query-execution]]
+= Query Execution
+
+This section describes CQDs that are used to influence query execution.
+
+[[hbase-async-operation]]
+== HBASE_ASYNC_OPERATION
+
+[cols="25%h,75%"]
+|===
+| *Description* | Allows index maintenance to be performed concurrently with base table operation.
+| *Values* | *'ON'* Index maintenance is allowed. +
+*'OFF'* Index maintenance is not allowed. +
+ +
+The default value is *'ON'*.
+| *Usage* | HBase `put` operations are blocking. When the table has one or more indexes,
+then the insert/update/delete (IUD) operation response time is improved by executing the index maintenance
+operations concurrently with the base table operation: the put operations to these HBase tables are executed
+in different threads.
+| *Production Usage* | Yes. It is 'ON' by default. This feature can be disabled by setting this CQD to 'OFF'.
+| *Impact* | IUD operations on tables with one or more indexes can become slower.
+| *Level* | Query.
+| *Conflicts/Synergies* | Not applicable.
+| *Real Problem Addressed* | Not applicable.
+| *Introduced In Release* | Trafodion 1.3.0.
+| *Deprecated In Release* | Not applicable.
+|===
+
+<<<
+[[hbase-cache-blocks]]
+== HBASE_CACHE_BLOCKS
+
+[cols="25%h,75%"]
+|===
+| *Description* | Influences HBase to retain the data blocks in memory after they are read.
+| *Values* |
+*'ON'/'OFF'/'SYSTEM'* +
+ +
+The default value is *'SYSTEM'*.
+| *Usage* | HBase maintains the block cache structure to retain the data blocks in memory after they are read.
+In LRU block cache configuration, the amount of block cache retained in memory is proportional to the amount of reserved maximum
+Java heap size of the region server. LRU Block Cache is the default in HBase. +
+ +
+The Trafodion Optimizer determines whether a sequential scan of the HBase table in a query would cause the full eviction of the
+data blocks cached earlier thereby impacting the performance of the random reads. The cache blocks option is turned off for the table
+in such a case. +
+ +
+Set the CQD <<hbase-region-server-max-heap-size,HBASE_REGION_SERVER_MAX_HEAP_SIZE>> value to reflect the amount of java heap size reserved for the region servers.
+This CQD is used by the Trafodion Optimizer to evaluate if the block cache to be turned off.
+| *Production Usage* | Leave the setting to be 'SYSTEM' when HBase is configured to use LRU block cache. If needed,
+you can override this settings with 'ON' or 'OFF'. +
+ +
+With other HBase configurations, you need to set HBASE_CACHE_BLOCKS to 'ON' or 'OFF' based on your application needs.
+| *Impact* | Automatically retains the random read performance.
+| *Level* | Query.
+| *Conflicts/Synergies* | Not applicable.
+| *Real Problem Addressed* | Not applicable.
+| *Introduced In Release* | Trafodion 1.3.0.
+| *Deprecated In Release* | Not applicable.
+|===
+
+<<<
+[[hbase-filter-preds]]
+== HBASE_FILTER_PREDS
+
+[cols="25%h,75%"]
+|===
+| *Description* | Allows push down of predicates to HBase Region Servers using HBase filters and optimize the columns retrieved
+from Region Servers. Only supported for NON ALIGN FORMAT tables.
+| *Values* |
+*'OFF'*: Predicates are never pushed down. +
+*'ON'*: A first implementation targeted for deprecation is enabled. Support simple predicate formed by a combination of AND only.
+Could be counter-productive when applied on nullable columns. +
+*'1'*: Same as *'ON'*. +
+*'2'*: Full feature is enabled. +
+ +
+An explain plan can show whether predicates are successfully pushed down to the Region Servers and what columns are really retrieved. +
+ +
+The default value is *'OFF'*.
+| *Usage* | Used to improve performance by reducing the number of columns retrieved to a strict minimum
+and filter out rows as early as possible.
+| *Production Usage* | Please consult the Trafodion community.
+| *Impact* | Using this CQD increases the amount of work done in the HBase Region Servers.
+| *Level* | System or Session.
+| *Conflicts/Synergies* | Not applicable.
+| *Real Problem Addressed* | Not applicable.
+| *Introduced In Release* | Trafodion 2.0.0.
+| *Deprecated In Release* | Not applicable.
+|===
+
+<<<
+[[hbase-hash2-partitioning]]
+== HBASE_HASH2_PARTITIONING
+
+[cols="25%h,75%"]
+|===
+| *Description* | Treat salted Trafodion tables as hash-partitioned on the salt columns.
+| *Values* |
+*'OFF'*: Salted Trafodion tables are not hash-partitioned on the salt columns. +
+*'ON'*: Salted Trafodion tables are hash-partitioned on the salt columns. +
+ +
+The default value is *'ON'*.
+| *Usage* | If, for any reason, there are issues with parallel plans on salted tables (especially with data skew) then try setting this CQD to OFF.
+| *Production Usage* | Yes.
+| *Impact* | Not applicable.
+| *Level* | System or Session.
+| *Conflicts/Synergies* | Not applicable.
+| *Real Problem Addressed* | Not applicable.
+| *Introduced In Release* | Trafodion 2.0.0.
+| *Deprecated In Release* | Not applicable.
+|===
+
+<<<
+[[hbase_num_cache_rows_max]]
+== HBASE_NUM_CACHE_ROWS_MAX
+
+[cols="25%h,75%"]
+|===
+| *Description* | Determines the number of rows obtained from HBase in one RPC call to the HBase Region Server in a sequential scan operation,
+| *Values* |
+Numeric value. +
+ +
+The default value is *'10000'*.
+| *Usage* | This CQD can be used to tune the query to perform optimally by reducing the number of interactions to the HBase Region Servers during
+a sequential scan of a table. +
+ +
+You need to consider how soon the maximum number of rows are materialized on the Region Servers. When filtering is pushed down to Region Servers,
+then it can take a longer time depending upon the query and the predicates involved. This can result in HBase scanner timeouts.
+| *Production Usage* | Use the default setting and reduce the value to avoid HBase scanner timeouts. Consult with the Trafodion community
+if you think that you need to use this CQD.
+| *Impact* | Not applicable.
+| *Level* | Query.
+| *Conflicts/Synergies* | Not applicable.
+| *Real Problem Addressed* | Not applicable.
+| *Introduced In Release* | Trafodion 1.3.0.
+| *Deprecated In Release* | Not applicable.
+|===
+
+<<<
+[[hbase-rowset-vssb-opt]]
+== HBASE_ROWSET_VSSB_OPT
+
+[cols="25%h,75%"]
+|===
+| *Description* | Allows INSERT, UPDATE, and DELETE (IUD) operations to be performed as an HBase batch `put` operation.
+| *Values* |
+*'ON'*: Perform IUD operations as an HBase batch `put` operation. +
+*'OFF'*: Do not perform IUD operations as an HBase batch `put` operation. +
+ +
+The default value is *'ON'*.
+| *Usage* | When IUD operation involves multiple tuples, then the Trafodion Optimizer evaluates whether these operations
+can be done in a batch manner at the HBase level thereby reducing the network interactions between the client applications and the HBase Region Servers. +
+ +
+If possible, then the query plan involves VSBB operators. The Virtual Sequential Block Buffer(VSBB) name is retained in Trafodion though it is unrelated to HBase.
+| *Production Usage* | Yes.
+| *Impact* | IUD operations can become slower if this CQD is set to 'OFF'.
+| *Level* | Query.
+| *Conflicts/Synergies* | Not applicable.
+| *Real Problem Addressed* | Not applicable.
+| *Introduced In Release* | Trafodion 1.3.0.
+| *Deprecated In Release* | Not applicable.
+|===
+
+<<<
+[[hbase-rowset-vssb-size]]
+== HBASE_ROWSET_VSSB_SIZE
+
+[cols="25%h,75%"]
+|===
+| *Description* | Determines the maximum number of rows in a batch `put` operation to HBase.
+| *Values* |
+Numeric value.
+ +
+The default value is *'1024'*.
+| *Usage* | The Trafodion execution engine already adjusts the number of rows in a batch depending upon how fast
+the queue to IUD (INSERT,UPDATE,DELETE) operator is filled up in the data flow architecture of Trafodion. +
+ +
+You can adjust the maximum size to suit your application needs and thus tune it to perform optimally.
+| *Production Usage* | Yes. You can disable this feature by setting the HBASE_ROWSET_VSSB_OPT CQD to 'OFF'.
+| *Impact* | The performance of your application may be affected by setting this CQD too low.
+| *Level* | Query.
+| *Conflicts/Synergies* | Not applicable.
+| *Real Problem Addressed* | Not applicable.
+| *Introduced In Release* | Trafodion 1.3.0.
+| *Deprecated In Release* | Not applicable.
+|===
+
+<<<
+[[hbase-small-scanner]]
+== HBASE_SMALL_SCANNER
+
+[cols="25%h,75%"]
+|===
+| *Description* | Enables Trafodion to leverage the HBase small scanner optimization. This optimization reduces I/O usage with up to 66%
+and enables non-blocking reads for higher concurrency support. When a scan is known to require less than a HBASE BLOCK SIZE (default is 64K),
+then enabling the HBase small scanner optimization increases performance.
+| *Values* |
+*'OFF'*: Never use the HBase small scanner optimization. +
+*'SYSTEM'*: Only enable the HBase small scanner optimization when the Trafodion Compiler determines that the scan size will fit in the table's HBASE BLOCK SIZE +
+*'ON'*: Enable the HBase small scanner optimization regardless of the size of scan. +
+ +
+The default value is *'OFF'*.
+| *Usage* | Consider using this CQD to improve the performance of your queries.
+| *Production Usage* | Consult with the Trafodion community.
+| *Impact* | The performance of small scan may increase by 1.4x. This CQD can be very useful for MDAM scans.
+| *Level* | System or Session.
+| *Conflicts/Synergies* | MDAM performance may be improved by 1.4x when correctly picking HBase block size so that each MDAM scan operation fit within a HBASE BLOCK SIZE boundary. +
+ +
+If you enable small scanner on large size scan incorrectly, then you are likely to see a 6% performance decrease. The returned results will still be correct.
+| *Real Problem Addressed* | Not applicable.
+| *Introduced In Release* | Trafodion 2.0.0.
+| *Deprecated In Release* | Not applicable.
+|===
+
http://git-wip-us.apache.org/repos/asf/incubator-trafodion/blob/223db1e4/docs/cqd_reference/src/asciidoc/_chapters/query_plans.adoc
----------------------------------------------------------------------
diff --git a/docs/cqd_reference/src/asciidoc/_chapters/query_plans.adoc b/docs/cqd_reference/src/asciidoc/_chapters/query_plans.adoc
index 33bc8d6..c72aa46 100644
--- a/docs/cqd_reference/src/asciidoc/_chapters/query_plans.adoc
+++ b/docs/cqd_reference/src/asciidoc/_chapters/query_plans.adoc
@@ -23,8 +23,8 @@
*/
////
-[[influence-query-plans]]
-= Influence Query Plans
+[[query-plans]]
+= Query Plans
This section describes CQDs that are used to influence query plans.
@@ -33,24 +33,29 @@ This section describes CQDs that are used to influence query plans.
[cols="25%h,75%"]
|===
-| *Category* | Influence Query Plans
-| *Description* | Defines the minimum size for query operator parallelism.
-The optimizer may choose an adaptive segment size that is equal to, or the multiple of,
-the value of this CQD, depending on the maximum estimated resource consumed by any single
-operator in the query. The optimizer may also decide to run the query with no parallelism
-if the resource consumption estimate is very low.
-| *Values* | Unsigned Integer. +
+| *Description* | Defines the minimum size of the adaptive segment; that is, the number of processors
+available for query operator parallelism. The optimizer may choose an adaptive-segment size that is equal to, or the multiple of,
+the value of this CQD depending on the maximum estimated resource consumed by any single operator in the query.
+The optimizer may also decide to run the query with no parallelism if the resource consumption estimate is very low.
+| *Values* |
+Unsigned Integer. +
+ +
The default value is *'16'*.
-| *Usage* | For clusters running at higher levels of concurrency with
-workloads that include a large number of small queries, reducing the default degree of
-parallelism may help achieve higher throughput.
+| *Usage* | For systems running at higher levels of concurrency with workloads that include a large
+number of small queries, reducing the default degree of parallelism may help achieve higher throughput. +
+ +
+With the default of 16, for 32-node systems, adaptive segmentation can use two 16-node virtual segments to execute queries that
+do not require a degree of parallelism of 32. This default setting can, for example, be changed to 8 for a 16-node system,
+to allow adaptive segmentation to leverage a lower degree of parallelism.
+
| *Production Usage* | Not applicable.
| *Impact* | Lowering the value of this CQD can increase the throughput of
high-concurrency small-query workloads, but has the potential disadvantage of increasing the
-elapsed time for some of the longer running queries.
-| *Level* | System.
+elapsed time for some of the longer running queries that leverage adaptive segmentation.
+| *Level* | System. There may be scenarios where you want to influence the degree of adaptive segmentation
+parallelism only for a certain set of queries and use it at the service level.
| *Conflicts/Synergies* | Not applicable.
-| *Real Problem Sddressed* | Not applicable.
+| *Real Problem Addressed* | Not applicable.
| *Introduced In Release* | Trafodion 1.3.0.
| *Deprecated In Release* | Not applicable.
|===
@@ -61,19 +66,68 @@ elapsed time for some of the longer running queries.
[cols="25%h,75%"]
|===
-| *Category* | Influence Query Plans
-| *Description* | Determines if Hash Join is considered by the optimizer to generate an execution plan.
-| *Values* | *'ON'* Hash Join is considered. +
-'OFF Hash Join is disabled. +
+| *Description* | Determines whether the Trafodion Optimizer considers Hash Join when generating an execution plan.
+| *Values* |
+*'ON'* Hash Join is considered. +
+*'OFF'* Hash Join is not considered. +
+
The default value is *'ON'*.
-| *Usage* | Use this CQD when you want to force a query plan not to use any Hash Joins.
+| *Usage* | Use this CQD when you want to force the optimizer to generate a query plan that does not use any Hash Joins.
| *Production Usage* | Hash Join is an important join implementation strategy for most complex queries.
-It is highly recommended not to turn Hash Join OFF. It should only be used to force a query plan for a particular query on an exception basis.
-| *Impact* | Turning Hash Join OFF may result in very inefficient query plans with expensive nested joins or sorts for merge joins.
+It is highly recommended that you do not turn HASH_JOINS OFF; that is, this CQD should be used to force a query plan for a particular query on an exception basis only.
+| *Impact* | Turning HASH_JOINS OFF may result in very inefficient query plans that use expensive nested joins or sorts for merge joins.
| *Level* | Query.
-| *Conflicts/Synergies* | Avoid turning all the three join implementations OFF (Hash Joins, Nested Joins, and Merge Joins).
-This may result in the compiler failing to generate query plans.
+| *Conflicts/Synergies* | If you turn all three join implementations OFF (Hash Joins, Nested Joins, and Merge Joins), then the compiler may fail to generate query plans.
+| *Real Problem Addressed* | Not applicable.
+| *Introduced In Release* | Trafodion 1.3.0.
+| *Deprecated In Release* | Not applicable.
+|===
+
+<<<
+[[hbase-coprocessors]]
+== HBASE_COPROCESSORS
+
+[cols="25%h,75%"]
+|===
+| *Description* | Allow HBase coprocessors to be used when computing aggregates.
+| *Values* |
+*'ON'*: Use HBase coprocessors. +
+*'OFF'*: Do not use HBase coprocessors. +
+ +
+The default value is *'ON'*.
+| *Usage* | Enables Trafodion to use HBase coprocessors to do early aggregation and filtering
+at the HBase Region Server level. This CQD does not affect Transaction coprocessors used by Trafodion. +
+ +
+As of Release 2.0 only COUNT(*) queries will be affected by this attribute.
+| *Production Usage* | Yes.
+| *Impact* | Network traffic between Region Server and Trafodion processes is reduced but
+the Region Server can become very busy when aggregating over large tables.
+| *Level* | Query.
+| *Conflicts/Synergies* | Not applicable.
+| *Real Problem Addressed* | Not applicable.
+| *Introduced In Release* | Trafodion 1.3.0.
+| *Deprecated In Release* | Not applicable.
+|===
+
+<<<
+[[hive-num-esps-per-datanode]]
+== HIVE_NUM_ESPS_PER_DATANODE
+
+[cols="25%h,75%"]
+|===
+| *Description* | Determines number of ESP processes used to scan a Hive table, per HDFS DataNode.
+| *Values* |
+Positive integer. +
+ +
+The default value is *'2'*.
+| *Usage* | Use this CQD to increase or decrease the number of scanners that process a single Hive table.
+If a Hive scan is found to be the bottleneck for a particular query, then increasing this attribute to; for example, 4 or higher will help.
+On the other hand decreasing the attribute to 1 could help with concurrency.
+| *Production Usage* | Yes.
+| *Impact* | Controls number of ESPs and, therefore, affects query execution time and system workload.
+| *Level* | Query.
+| *Conflicts/Synergies* | The CQD <<hive-min-bytes-per-esp-partition,HIVE_MIN_BYTES_PER_ESP_PARTITION>> (default = 67108864) may need to be adjusted downward
+when this attribute is used to increase the parallelism of scanning smaller Hive tables.
| *Real Problem Addressed* | Not applicable.
| *Introduced In Release* | Trafodion 1.3.0.
| *Deprecated In Release* | Not applicable.
@@ -85,10 +139,10 @@ This may result in the compiler failing to generate query plans.
[cols="25%h,75%"]
|===
-| *Category* | Influence Query Plans
| *Description* | Enables or disables the join order in which the optimizer joins the tables to be the sequence of the
tables in the FROM clause of the query.
-| *Values* | *'ON'* Join order is forced. +
+| *Values* |
+*'ON'* Join order is forced. +
*'OFF'* Join order is decided by the optimizer. +
+
The default value is *'OFF'*.
@@ -105,6 +159,67 @@ specified in the FROM clause, the plan generated may not be the optimal one.
| *Deprecated In Release* | Not applicable.
|===
+<<<
+[[mc-skew-sensitivity-threshold]]
+== MC_SKEW_SENSITIVITY_THRESHOLD
+
+[cols="25%h,75%"]
+|===
+| *Description* | Define the multi-column skew sensitivity threshold T used by multi-column skew-insensitive hash join (Skew Buster). +
+ +
+Let `f` be the occurrence frequency of a skew value `v`, `DoP` be the degree of parallelism of a hash join operator, and `RC` be the row count of
+the source data (for example, fact table) where the skew originates. +
+ +
+The hash join will run in the anti-skew mode for `v` if `f >= T * DoP / RC`.
+| *Values* |
+*< 0*: Disable the multi-column skew buster. +
+*>= 0*: Define the threshold T. +
+ +
+Default value: *0.1*.
+| *Usage* | Use of a *negative value* to disable multi-column anti-skew hash joins. This may slow down query performance when
+multi-column skew values are present in the fact table. +
+ +
+A value of *0* treats every multi-column value as skew values. This may increase network traffic since skewed values are broadcasted from the inner
+side child of the hash join to all join processes. +
+ +
+A value *greater than 0* selects those multi-column values as skewed values if their occurrence frequencies are high enough.
+| *Production Usage* | Consult the Trafodion community.
+| *Impact* | This CQD impacts runtime performance.
+| *Level* | Session.
+| *Conflicts/Synergies* | Not applicable.
+| *Real Problem Addressed* | Not applicable.
+| *Introduced In Release* | Trafodion 1.3.0.
+| *Deprecated In Release* | Not applicable.
+|===
+
+<<<
+[[mdam-no-stats-positions-threshold]]
+== MDAM_NO_STATS_POSITIONS_THRESHOLD
+
+[cols="25%h,75%"]
+|===
+| *Description* | This CQD effects the behavior of the query optimizer when there are no statistics available for a query
+having range predicates on key columns. The Trafodion Optimizer calculates the worst case number of seeks that the MDAM access method
+would do if chosen for the query. +
+ +
+If this number is greater than the value of MDAM_NO_STATS_POSITIONS_THRESHOLD, then MDAM is not considered for the query execution plan.
+| *Values* |
+Any integer greater than equal to zero. +
+ +
+Default value: *10*.
+| *Usage* | In certain situations, queries on tables lacking statistics may not be optimal because MDAM was not chosen.
+Increasing the value for this CQD allows MDAM to be chosen in more cases. On the other hand, if the value is made too high and the worst case
+scenario actually occurs, an MDAM plan may perform poorly.
+| *Production Usage* | Consult the Trafodion community.
+| *Impact* | Table scans on tables lacking statistics may improve by varying the value of this CQD.
+Results vary depending on the actual data in the table and the semantics of the query.
+| *Level* | Query.
+| *Conflicts/Synergies* | If <<mdam-scan-method,MDAM_SCAN_METHOD>> is set to *'OFF'*, then this CQD has no effect.
+| *Real Problem Addressed* | Perform UPDATE STATISTICS on the table (at the very least on key columns) to obtain statistics.
+| *Introduced In Release* | Trafodion 1.3.0.
+| *Deprecated In Release* | Not applicable.
+|===
+
<<<
[[mdam-scan-method]]
@@ -112,15 +227,15 @@ specified in the FROM clause, the plan generated may not be the optimal one.
[cols="25%h,75%"]
|===
-| *Category* | Influence Query Plans
-| *Description* | Enables or disables the Multi-Dimensional Access Method.
-| *Values* | *'ON'* MDAM is considered. +
+| *Description* | Enables or disables the Multi-Dimensional Access Method (MDAM).
+| *Values* |
+*'ON'* MDAM is considered. +
*'OFF'* MDAM is disabled. +
+
The default value is *'ON'*.
-| *Usage* | In certain situations, the optimizer might choose MDAM inappropriately, causing poor performance.
+| *Usage* | In certain situations, the Trafodion Optimizer might choose MDAM inappropriately, causing poor performance.
In such situations you may want to turn MDAM OFF for the query it is effecting.
-| *Production Usage* | Not applicable.
+| *Production Usage* | Yes.
| *Impact* | Table scans with predicates on non-leading clustering key column(s) could benefit from MDAM access
method if the leading column(s) has a small number of distinct values. Turning MDAM off results in a longer scan time for such queries.
| *Level* | Set this CQD at the query level when MDAM is not working efficiently for a specific query. However,
@@ -138,24 +253,26 @@ to set it at the service or system level.
[cols="25%h,75%"]
|===
-| *Category* | Influence Query Plans
| *Description* | Determines if Merge Join is considered by the optimizer to generate an execution plan.
-| *Values* | *'ON'* Merge Join is considered. +
-*'OFF'* Merge Join is disabled. +
+| *Values* |
+*'ON'*: Merge Join is considered. +
+*'OFF'*: Merge Join is disabled. +
+
The default value is *'ON'*.
| *Usage* | Use this CQD when you want to force a query plan not to use Merge Joins. This is useful as a workaround
-for query plans with very expensive sorts for Merge Joins. Turning Merge Join OFF also has the advantage of reducing the query compile time.
+for query plans with very expensive sorts for Merge Joins. Turning MERGE_JOINS OFF also has the advantage of reducing the query compile time.
| *Production Usage* | Merge Join is an efficient join implementation strategy if the physical schema was designed to take advantage
of it. For example, large tables are physically ordered based on the most frequently joined column(s).
-| *Impact* | Turning Merge Join OFF may result in the optimizer not considering potentially efficient query plans,
-for queries with large joins on tables that are physically ordered by the join column(s). Turning Merge Join ON causes an increase in compile
+| *Impact* | Turning MERGE_JOINS OFF may result in the optimizer not considering potentially efficient query plans,
+for queries with large joins on tables that are physically ordered by the join column(s). +
+ +
+Turning MERGE_JOINS ON causes an increase in compile
time because the optimizer now has to consider many more join options.
| *Level* | Set this CQD at the query level when a Merge Join is not working efficiently for a specific query. However,
there may be cases (usually a defect) where a larger set of queries is being negatively impacted by Merge Joins. In those cases you may want to
set it at the service or system level.
| *Conflicts/Synergies* | Avoid turning all the three join implementations OFF (Hash Joins, Nested Joins, and Merge Joins).
-This may result in the compiler failing to generate query plans.
+This may result in the Trafodion Compiler failing to generate query plans.
| *Real Problem Addressed* | Not applicable.
| *Introduced In Release* | Trafodion 1.3.0.
| *Deprecated In Release* | Not applicable.
@@ -167,24 +284,24 @@ This may result in the compiler failing to generate query plans.
[cols="25%h,75%"]
|===
-| *Category* | Influence Query Plans
| *Description* | Determines if Nested Join is considered by the optimizer to generate an execution plan.
-| *Values* | *'ON'* Nested Join is considered. +
-*'OFF'* Nested Join is disabled. +
+| *Values* |
+*'ON'*: Nested Join is considered. +
+*'OFF'*: Nested Join is disabled. +
+
The default value is *'ON'*.
| *Usage* | Use this CQD when you want to force a query plan not to use Nested Joins. This is useful
as a workaround for query plans with very expensive Nested Joins, which may occur if the optimizer fails to estimate the
cost of a Nested Join correctly.
| *Production Usage* | Nested Join is an important join implementation strategy for many complex queries. It is
-recommended not to turn Nested Join OFF. It should only be used to force a query plan for a particular query on an exception basis.
-| *Impact* | Turning Nested Join OFF may result in inefficient query plans for certain type of queries,
+recommended not to turn NESTED_JOINS OFF. It should only be used to force a query plan for a particular query on an exception basis.
+| *Impact* | Turning NESTED_JOINS OFF may result in inefficient query plans for certain type of queries,
such as light workloads and star join queries.
| *Level* | Query.
| *Conflicts/Synergies* | Avoid turning all the three join implementations OFF (Hash Joins, Nested Joins, and Merge Joins).
This may result in the compiler failing to generate query plans.
| *Real Problem Addressed* | The problem of inefficient Nested Joins can be better handled using a higher degree of query plan
-robustness as set by the ROBUST_QUERY_OPTIMIZATION CQD.
+robustness as set by the <<robust-query-optimization,ROBUST_QUERY_OPTIMIZATION>> CQD.
| *Introduced In Release* | Trafodion 1.3.0.
| *Deprecated In Release* | Not applicable.
|===
@@ -195,11 +312,12 @@ robustness as set by the ROBUST_QUERY_OPTIMIZATION CQD.
[cols="25%h,75%"]
|===
-| *Category* | Influence Query Plans
| *Description* | Controls the optimizer resources and time spent for optimizing a query plan, with level 0 indicating
the least amount of optimization effort and level 5 indicating the most. Lower optimization levels produce lower plan quality with
minimal compile time, while higher optimization levels cause the compiler to spend more compilation time to produce better plan quality.
-| *Values* | '0', '2', '3', '5'
+| *Values* |
+'0', '2', '3', '5' +
+ +
The default value is *'3'*.
| *Usage* | Reduce the optimization level when compile time is longer than desired and queries have relatively small
execution cost and are simple in structure.
@@ -219,8 +337,7 @@ Increasing the optimization level over the system default may result in very hig
[cols="25%h,75%"]
|===
-| *Category* | Influence Query Plans
-| *Description* | Controls the maximum number of parallel ESPs that work on a particular operation, like a join.
+| *Description* | Controls the maximum number of parallel ESPs that work on a particular operation; for example, a join.
| *Values* | Unsigned Integer: The maximum number of ESPs that should be used for a particular operation.
The value must be less than the number of nodes in the cluster. +
+
@@ -245,9 +362,9 @@ query workloads, but has the potential disadvantage of increasing the elapsed ti
[cols="25%h,75%"]
|===
-| *Category* | Influence Query Plans
| *Description* | Influences the optimizer to choose other types of joins over nested joins, by making a nested join plan relatively more expensive.
-| *Values* | Any positive fractional value. +
+| *Values* |
+Any positive fractional value. +
+
The default value is *'1.2'*.
| *Usage* | Review <<robust-query-optimization,ROBUST_QUERY_OPTIMIZATION>> before considering the use of this CQD. +
@@ -261,7 +378,7 @@ may be used to influence the optimizer to consider another join instead, such as
+
NESTED_JOINS OFF could turn nested joins off completely. However, there are many cases where nested joins do provide better performance than hash
joins, and turning them off completely may negatively impact the performance of queries that can do a lot better with nested joins.
-| *Production Usage* | Not applicable.
+| *Production Usage* | Consult with the Trafodion community.
| *Impact* | Specifying a risk premium insures against nested joins being chosen when they should not have been. However,
this can also result in nested joins not being chosen where the cardinality estimation was in fact accurate and a nested join could have
performed better. Therefore, this setting should be used with care in order to get robustness with a net gain in performance.
@@ -281,9 +398,9 @@ addressing nested join issues and need to use this setting independent of that C
[cols="25%h,75%"]
|===
-| *Category* | Influence Query Plans
| *Description* | Influences the optimizer to choose a parallel plan over a serial plan, by making a serial plan relatively more expensive.
-| *Values* | Any positive fractional value. +
+| *Values* |
+Any positive fractional value. +
+
The default value is *'1.2'*.
| *Usage* | Review <<robust-query-optimization,ROBUST_QUERY_OPTIMIZATION>> before considering the use of this CQD. +
@@ -294,7 +411,7 @@ allowed to win over competing parallel plans. +
+
If it is determined that the optimizer is using serial plans often enough where these plans are resulting in poor performance, then this CQD
may be used to influence the optimizer to consider parallel plans instead in some of those cases. +
-| *Production Usage* | Not applicable.
+| *Production Usage* | Consult with the Trafodion community.
| *Impact* | Specifying a risk premium insures against serial plans being chosen when they should not have been.
However, this can also result in serial plans not being chosen where the cardinality estimation was in fact accurate and a serial plan
could have performed better. Therefore, this setting should be used with care in order to get robustness with a net gain in performance.
@@ -309,12 +426,37 @@ addressing serial plan issues and need to use this setting independent of that C
|===
<<<
+[[risk-premium-serial-scaleback-maxcard-threshold]]
+== RISK_PREMIUM_SERIAL_SCALEBACK_MAXCARD_THRESHOLD
+
+[cols="25%h,75%"]
+|===
+| *Description* | Defines the minimal estimated max cardinality or row count of any relational operators in a query above
+which the risk premium for serial plan is applied. +
+ +
+A serial query plan is favored by the Trafodion Compiler when it estimates the query reads and processes small amount of data.
+The estimation error could become large when some operator is calculated to produce many rows yielding a non-optimal serial plan.
+This CQD helps prevent utilizing serial plan in such cases.
+| *Values* | An unsigned integer value.
+| *Usage* | Adjust this CQD when necessary only. +
+ +
+Use of a value smaller than the default (10,000) to penalize more serial plans or favor more parallel plans for operators produce less number of rows.
+Otherwise, use of a larger value.
+| *Production Usage* | Consult with the Trafodion community.
+| *Impact* | This CQD impacts plan quality.
+| *Level* | Session.
+| *Conflicts/Synergies* | Not applicable.
+| *Real Problem Addressed* | Not applicable.
+| *Introduced In Release* | Trafodion 1.3.0.
+| *Deprecated In Release* | Not applicable.
+|===
+
+<<<
[[robust-query-optimization]]
== ROBUST_QUERY_OPTIMIZATION
[cols="25%h,75%a"]
|===
-| *Category* | Influence Query Plans
| *Description* | Provides a simpler way to influence the optimizer's choice of query plans. The optimizer chooses query plans
based on cardinality estimates (the number of result rows estimated at each step of a query execution plan). Actual cardinalities encountered
at query execution often differ from estimates. The optimizer considers risky choices and exacts risk premiums before it chooses a plan that
@@ -397,7 +539,6 @@ optimizer may warn about. However, cardinality underestimations may still happen
[cols="25%h,75%"]
|===
-| *Category* | Influence Query Plans
| *Description* | Turns on the reporting of anti-skew join plan details in EXPLAIN or EXPLAIN OPTIONS 'f'.
| *Values* |
*'OFF'*: Disables the use of SKEW_EXPLAIN. +
@@ -420,7 +561,6 @@ The default value is *'OFF'*.
[cols="25%h,75%"]
|===
-| *Category* | Influence Query Plans
| *Description* | The optimizer looks for skewed values and address that skew if the number of rows in the table exceeds this threshold.
| *Values* |
*'n'*: where n is the number of rows +
@@ -460,7 +600,6 @@ is relevant if this threshold allows a skew to be detected only.
[cols="25%h,75%"]
|===
-| *Category* | Influence Query Plans
| *Description* | Used to specify a threshold that determines whether a value in the join column is considered skewed.
| *Values* |
*'-1'*: Disables the use of skew buster. +
@@ -489,12 +628,85 @@ However, there may be design opportunities that could help address the problem a
|===
<<<
+[[subquery_unnesting]]
+== SUBQUERY_UNNESTING
+
+[cols="25%h,75%"]
+|===
+| *Description* | Allows correlated subqueries in a SQL statement to be unnested, so that they can be executed efficiently.
+| *Values* |
+*'ON'*: Unnesting enabled. +
+*'OFF'*: Unnesting disabled. +
+ +
+The default value is *'ON'*.
+| *Usage* | Turn this CQD OFF when (in rare cases) unnesting a correlated subquery causes performance to degrade.
+If this attribute has to be turned OFF, then that could indicate a bug in the Trafodion Optimizer. Consult with the Trafodion community.
+| *Production Usage* | Yes.
+| *Impact* | Turn OFF with caution at a system level, as other queries which rely on un-nesting could be adversely impacted.
+| *Level* | Query.
+| *Conflicts/Synergies* | Not applicable.
+| *Real Problem Addressed* | Not applicable.
+| *Introduced In Release* | Trafodion 1.3.0.
+| *Deprecated In Release* | Not applicable.
+|===
+
+<<<
+[[traf-allow-esp-colocation]]
+== TRAF_ALLOW_ESP_COLOCATION
+
+[cols="25%h,75%"]
+|===
+| *Description* | Enables or disables whether ESP are colocated with HBase Region Servers, which minimizes the inter-node
+network traffic between the ESP processes and the HBase Region Servers.
+| *Values* |
+*'ON'*: Colocation enabled. +
+*'OFF'*: Colocation disabled. +
+ +
+The default value is *'OFF'*.
+| *Usage* | Enable the feature when each region server serves approximately equal amount of data, and/or reducing network traffic is important.
+| *Production Usage* | Consult with the Trafodion community.
+| *Impact* | Plan quality.
+| *Level* | Session.
+| *Conflicts/Synergies* | Not applicable.
+| *Real Problem Addressed* | Not applicable.
+| *Introduced In Release* | Trafodion 1.3.0.
+| *Deprecated In Release* | Not applicable.
+|===
+
+<<<
+[[traf-upsert-with-insert-default-semantics]]
+== TRAF_UPSERT_WITH_INSERT_DEFAULT_SEMANTICS
+
+[cols="25%h,75%"]
+|===
+| *Description* | Enables population of omitted columns in an UPSERT statement with default values when the table is created in aligned-row format mode.
+| *Values* |
+*'ON'*: Default population enabled. +
+*'OFF'*: Default population disabled. +
+ +
+The default value is *'ON'*.
+| *Usage* | When a column with default value is omitted in an UPSERT statement of a table in aligned row format mode,
+then the statement is transformed to merge. +
+ +
+If the row already exists, then the omitted columns are populated with values from the existing row. +
+ +
+If the row doesn't exist, then the omitted columns are populated with default values. This default behavior can be changed by setting this CQD to 'ON',
+which improves the performance of the UPSERT statements with omitted default value columns.
+| *Production Usage* | Yes.
+| *Impact* | Improved upsert performance of aligned row format tables.
+| *Level* | Query.
+| *Conflicts/Synergies* | Not applicable.
+| *Real Problem Addressed* | Not applicable.
+| *Introduced In Release* | Trafodion 1.3.0.
+| *Deprecated In Release* | Not applicable.
+|===
+
+<<<
[[upd-ordered]]
== UPD_ORDERED
[cols="25%h,75%"]
|===
-| *Category* | Influence Query Plans
| *Description* | Controls whether rows should be inserted, updated, or deleted in clustering key order.
| *Values* |
*'ON'*: The optimizer generates and considers plans where the rows are inserted, updated, or deleted in clustering key order. +
http://git-wip-us.apache.org/repos/asf/incubator-trafodion/blob/223db1e4/docs/cqd_reference/src/asciidoc/_chapters/runtime_controls.adoc
----------------------------------------------------------------------
diff --git a/docs/cqd_reference/src/asciidoc/_chapters/runtime_controls.adoc b/docs/cqd_reference/src/asciidoc/_chapters/runtime_controls.adoc
index 6f1c7c7..4196115 100644
--- a/docs/cqd_reference/src/asciidoc/_chapters/runtime_controls.adoc
+++ b/docs/cqd_reference/src/asciidoc/_chapters/runtime_controls.adoc
@@ -33,7 +33,6 @@ This section describes CQDs that are used for runtime controls.
[cols="25%h,75%"]
|===
-| *Category* | Runtime Controls
| *Description* | Ensures that all parts of the query plan are executed but no rows are returned by the query.
| *Values* | *'ON'* or *'OFF'*. +
+
@@ -56,9 +55,8 @@ is the same as when you do not use the [LAST 0] clause in the query.
[cols="25%h,75%"]
|===
-| *Category* | Runtime Controls
| *Description* | Use to limit the amount of CPU time that a query is allowed to use in any one server process (MXESP)
-including the NDCS server (MXOSRVR). Also includes the CPU time the query spends in the disk process (ESAM). $$$ +
+including the NDCS server (MXOSRVR). Also includes the CPU time the query spends in the disk process (ESAM). +
+
If a query exceeds the limit, then an error is raised and the query is terminated. This is a way to limit the impact on the
system of a poorly written or badly optimized query.
http://git-wip-us.apache.org/repos/asf/incubator-trafodion/blob/223db1e4/docs/cqd_reference/src/asciidoc/_chapters/schema_controls.adoc
----------------------------------------------------------------------
diff --git a/docs/cqd_reference/src/asciidoc/_chapters/schema_controls.adoc b/docs/cqd_reference/src/asciidoc/_chapters/schema_controls.adoc
index 0112b9a..0f84bb0 100644
--- a/docs/cqd_reference/src/asciidoc/_chapters/schema_controls.adoc
+++ b/docs/cqd_reference/src/asciidoc/_chapters/schema_controls.adoc
@@ -28,12 +28,34 @@
This section describes CQDs that are used for schema controls.
+[[catalog]]
+== Catalog
+
+[cols="25%h,75%"]
+|===
+| *Description* | Specifies the default catalog name for all DDL and DML statements.
+| *Values* | Any valid ANSI name, including delimited names. +
+ +
+The default is *'TRAFODION'*.
+| *Usage* | Trafodion tables must be in a catalog called TRAFODION. If you mostly access Hive or native HBase tables,
+then the catalog could be changed to HIVE or HBASE respectively. +
+ +
+The default setting is overridden by any catalog name specified in a SQL statement.
+| *Production Usage* | Yes.
+| *Impact* | Not applicable.
+| *Level* | Any.
+| *Conflicts/Synergies* | Alternately you can use the SET CATALOG statement.
+| *Real Problem Addressed* | Not applicable.
+| *Introduced In Release* | Trafodion 1.3.0.
+| *Deprecated In Release* | Not applicable.
+|===
+
+<<<
[[schema]]
== Schema
[cols="25%h,75%"]
|===
-| *Category* | Influencing Query Plans
| *Description* | Sets the default schema for the session.
| *Values* | SQL identifier. +
+
http://git-wip-us.apache.org/repos/asf/incubator-trafodion/blob/223db1e4/docs/cqd_reference/src/asciidoc/_chapters/table_definition.adoc
----------------------------------------------------------------------
diff --git a/docs/cqd_reference/src/asciidoc/_chapters/table_definition.adoc b/docs/cqd_reference/src/asciidoc/_chapters/table_definition.adoc
new file mode 100644
index 0000000..0a93dce
--- /dev/null
+++ b/docs/cqd_reference/src/asciidoc/_chapters/table_definition.adoc
@@ -0,0 +1,142 @@
+////
+/**
+* @@@ START COPYRIGHT @@@
+*
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing,
+* software distributed under the License is distributed on an
+* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+* KIND, either express or implied. See the License for the
+* specific language governing permissions and limitations
+* under the License.
+*
+* @@@ END COPYRIGHT @@@
+ */
+////
+
+[[table_definition]]
+= Table Definition
+
+This section describes CQDs that are used for table definition
+
+[[auto-query-retry-warnings]]
+== ALLOW_NULLABLE_UNIQUE_KEY_CONSTRAINT
+
+[cols="25%h,75%"]
+|===
+| *Description* | Allow Trafodion tables to be created with NULLABLE columns in the PRIMARY or STORE BY key.
+| *Values* | *'ON'* or *'OFF'* +
+ +
+The default value is *'OFF'*.
+| *Usage* | Allows NULLABLE columns to be included in the PRIMARY or STORE BY key for Trafodion tables.
+This CQD must be set prior to creating the table. It is not necessary to specify this CQD during DML operations on the table. +
+ +
+When performing UPDATE STATISTICS with SAMPLE option on such tables, this CQD must be set so that an appropriate sample table can be created.
+| *Production Usage* | Please consult with the Trafodion community.
+| *Impact* | Two bytes are added to the key for each nullable column.
+| *Level* | Session.
+| *Conflicts/Synergies* | Not applicable.
+| *Real Problem Addressed* | Not applicable.
+| *Introduced In Release* | Trafodion 1.3.0.
+| *Deprecated In Release* | Not applicable.
+|===
+
+<<<
+[[hbase-block-size]]
+== HBASE_BLOCK_SIZE
+
+[cols="25%h,75%"]
+|===
+| *Description* | Allow Trafodion tables to be created with specified HBase block size.
+| *Values* | Positive integer. +
+ +
+The default value is *'65536'*.
+| *Usage* | The value of this attribute is passed on to HBase when a Trafodion table is created in HBase.
+See the https://hbase.apache.org/book.html[Apache HBase(TM) Reference Guide] for usage information .
+| *Production Usage* | Yes.
+| *Impact* | The impact depends on the type of table access. Choose a block size that is appropriate
+for how the table is primarily accessed.
+| *Level* | System.
+| *Conflicts/Synergies* | Not applicable.
+| *Real Problem Addressed* | Not applicable.
+| *Introduced In Release* | Trafodion 1.3.0.
+| *Deprecated In Release* | Not applicable.
+|===
+
+<<<
+[[hive-default-charset]]
+== HIVE_DEFAULT_CHARSET
+
+[cols="25%h,75%"]
+|===
+| *Description* | Defines what character set the columns of Hive tables should have.
+| *Values* |
+*'ISO88591'* or *'UTF8'* +
+ +
+Default: *'UTF8'*
+| *Usage* | Set this to ISO88591 when reading from Hive tables with ISO8859-1 data.
+| *Production Usage* | Yes.
+| *Impact* | Not applicable.
+| *Level* | System.
+| *Conflicts/Synergies* | Not applicable.
+| *Real Problem Addressed* | Not applicable.
+| *Introduced In Release* | Trafodion 1.3.0.
+| *Deprecated In Release* | Not applicable.
+|===
+
+<<<
+[[hive-file-charset]]
+== HIVE_FILE_CHARSET
+
+[cols="25%h,75%"]
+|===
+| *Description* | For certain character sets that are not supported in Trafodion, you can specify the character set here
+causing Trafodion to automatically convert the data to the character set specified in the HIVE_DEFAULT_CHARSET CQD. +
+ +
+This feature is currently supported only for *'GBK'*, and only if HIVE_DEFAULT_CHARSET is set to *'UTF8'*.
+| *Values* |
+empty or *'GBK'* +
+ +
+Default: empty
+| *Usage* | Leave this blank, unless you want to access GBK data in Hive tables.
+| *Production Usage* | Yes.
+| *Impact* | Not applicable.
+| *Level* | System.
+| *Conflicts/Synergies* | Only applicable if HIVE_DEFAULT_CHARSET is set to *'UTF8'*.
+| *Real Problem Addressed* | Not applicable.
+| *Introduced In Release* | Trafodion 1.3.0.
+| *Deprecated In Release* | Not applicable.
+|===
+
+<<<
+[[hive-max-string-length]]
+== HIVE_MAX_STRING_LENGTH
+
+[cols="25%h,75%"]
+|===
+| *Description* | Hive columns of type STRING have a maximum length in Trafodion, which you can specify with this CQD. +
+ +
+*NOTE* For UTF-8 data, this length is specified in bytes, not UTF-8 characters.
+| *Values* |
+1-n. +
+ +
+Default: 32000.
+| *Usage* | Set this to the lowest possible value to improve system performance.
+| *Production Usage* | Yes.
+| *Impact* | Not applicable.
+| *Level* | System.
+| *Conflicts/Synergies* | Not applicable.
+| *Real Problem Addressed* | Not applicable.
+| *Introduced In Release* | Trafodion 1.3.0.
+| *Deprecated In Release* | Not applicable.
+|===
+
http://git-wip-us.apache.org/repos/asf/incubator-trafodion/blob/223db1e4/docs/cqd_reference/src/asciidoc/_chapters/transactions.adoc
----------------------------------------------------------------------
diff --git a/docs/cqd_reference/src/asciidoc/_chapters/transactions.adoc b/docs/cqd_reference/src/asciidoc/_chapters/transactions.adoc
index 5f590fb..6a18aab 100644
--- a/docs/cqd_reference/src/asciidoc/_chapters/transactions.adoc
+++ b/docs/cqd_reference/src/asciidoc/_chapters/transactions.adoc
@@ -33,7 +33,6 @@ This section describes CQDs that are used for transaction control and locking.
[cols="25%h,75%"]
|===
-| *Category* | Transaction Control and Locking
| *Description* | Specifies the default transaction isolation level that queries use.
| *Values* | *'READ UNCOMMITTED'*, *'READ COMMITTED'*, *'REPEATABLE READ'*, or *'SERIALIZABLE'*. +
+
@@ -76,7 +75,6 @@ then its value is used as the isolation level for updates and DDL statements. UP
[cols="25%h,75%"]
|===
-| *Category* | Transaction Control and Locking
| *Description* | Specifies the default transaction isolation level for these update operations: INSERT, UPDATE, or DELETE.
| *Values* | *'READ UNCOMMITTED'*, *'READ COMMITTED'*, *'REPEATABLE READ'*, or *'SERIALIZABLE'*. +
+
@@ -86,7 +84,7 @@ The default value is *'READ COMMITTED'* (ANSI).
| *Impact* | When set, this CQD prevents users from doing any of these update operations: INSERT, UPDATE, or DELETE.
| *Level* | Service.
| *Conflicts/Synergies* | Works with the ISOLATION_LEVEL setting. Both settings are READ COMMITTED by default.
-ISOLATION_LEVEL can be set to READ UNCOMMITTED. This CQD still remains READ COMMITTED. You can change it to READ UNCOMMITED to prevent queries
+ISOLATION_LEVEL can be set to READ UNCOMMITTED. This CQD still remains READ COMMITTED. You can change it to READ UNCOMMITTED to prevent queries
running at the service level to not perform any updates.
| *Real Problem Addressed* | Not applicable.
| *Introduced In Release* | Trafodion 1.3.0.
http://git-wip-us.apache.org/repos/asf/incubator-trafodion/blob/223db1e4/docs/cqd_reference/src/asciidoc/_chapters/update_stats.adoc
----------------------------------------------------------------------
diff --git a/docs/cqd_reference/src/asciidoc/_chapters/update_stats.adoc b/docs/cqd_reference/src/asciidoc/_chapters/update_stats.adoc
index 51c0204..9695e3f 100644
--- a/docs/cqd_reference/src/asciidoc/_chapters/update_stats.adoc
+++ b/docs/cqd_reference/src/asciidoc/_chapters/update_stats.adoc
@@ -31,7 +31,6 @@
[cols="25%h,75%"]
|===
-| *Category* | Update Statistics and Reorg
| *Description* | When performing update statistics with the NECESSARY keyword or with automation, this is
the number of minutes that are allowed to have elapsed since a histogram was marked as read for it to be regenerated. +
+
@@ -48,7 +47,7 @@ However, if a table is not used frequently, then this CQD influences how frequen
+
A smaller setting reduces the number of histograms being updated if there are many tables that have not been used within that interval.
A larger setting updates histogram for many more tables that are not being accessed that often.
-| *Production Usage* | Not applicable.
+| *Production Usage* | Consult the Trafodion community.
| *Impact* | Influences the number of histograms that need to be regenerated and therefore the time it takes for
update statistics automation to regenerate histograms for all the tables that so qualify.
| *Level* | System.
@@ -67,7 +66,6 @@ This CQD qualifies the histogram to be regenerated; it is the maximum time since
[cols="25%h,75%"]
|===
-| *Category* | Update Statistics and Reorg
| *Description* | Sets the minimum rows that need to be in a table before sampling is used to update statistics for that table.
If a table has a fewer rows than the value of this CQD, then the SAMPLE option is silently ignored when performing update statistics.
| *Values* | 1 through max unsigned integer. +
@@ -76,7 +74,7 @@ The default value is 10000.
| *Usage* | Influences for what tables sampling is used for histogram statistics generation. If the setting is smaller,
then more tables qualify for sampling. If the setting is larger, then fewer tables qualify for sampling. Sampling can result in faster update
statistics run times. But for smaller tables, it can also result in poor histogram accuracy which could result in poor query plans.
-| *Production Usage* | Not applicable.
+| *Production Usage* | Consult the Trafodion community.
| *Impact* | Setting this CQD to a smaller value means that sampling is used for tables with fewer rows, when the SAMPLE option is
specified as part of update statistics. This can result in less accurate histograms and poor query plans, because the sample size may be too small to
generate good estimates for histograms. +
@@ -90,3 +88,28 @@ However, these tables may also have more accurate histograms.
| *Deprecated In Release* | Not applicable.
|===
+<<<
+[[ustat-min-rowcount-for-low-sample]]
+== USTAT_MIN_ROWCOUNT_FOR_LOW_SAMPLE
+
+[cols="25%h,75%"]
+|===
+| *Description* | Defines the behavior of the UPDATE STATISTICS utility. It places a lower limit on the number of sample rows that will be used when sampling. +
+ +
+If the number of rows in the table is less than this value and sampling is used, then the sample size used will be that specified by
+the <<hist-default-sample-min,HIST_DEFAULT_SAMPLE_MIN>> CQD. (which defaults to 10,000).
+| *Values* | This CQD may take on any integer value greater than zero. +
+ +
+The default for this CQD is *1,000,000*.
+| *Usage* | Prevents accidental use of too-small samples when generating statistics on tables. If sample sizes are too small,
+then histogram statistics will be less accurate, leading to potentially less efficient query plans.
+| *Production Usage* | Consult the Trafodion community.
+| *Impact* | Setting this CQD to lower values may result in smaller sample sizes for small tables.
+This may slightly improve UPDATE STATISTICS run time, but at the cost of potentially less efficient queries.
+| *Level* | System.
+| *Conflicts/Synergies* | <<hist-default-sample-min,HIST_DEFAULT_SAMPLE_MIN>> effects the behavior of this CQD.
+| *Real Problem Addressed* | Not applicable.
+| *Introduced In Release* | Trafodion 1.3.0.
+| *Deprecated In Release* | Not applicable.
+|===
+
http://git-wip-us.apache.org/repos/asf/incubator-trafodion/blob/223db1e4/docs/cqd_reference/src/asciidoc/index.adoc
----------------------------------------------------------------------
diff --git a/docs/cqd_reference/src/asciidoc/index.adoc b/docs/cqd_reference/src/asciidoc/index.adoc
index f67e482..9de403c 100644
--- a/docs/cqd_reference/src/asciidoc/index.adoc
+++ b/docs/cqd_reference/src/asciidoc/index.adoc
@@ -61,9 +61,12 @@ specific language governing permissions and limitations under the License.
include::asciidoc/_chapters/about.adoc[]
include::asciidoc/_chapters/introduction.adoc[]
include::asciidoc/_chapters/query_plans.adoc[]
+include::asciidoc/_chapters/query_execution.adoc[]
include::asciidoc/_chapters/histograms.adoc[]
include::asciidoc/_chapters/transactions.adoc[]
include::asciidoc/_chapters/runtime_controls.adoc[]
include::asciidoc/_chapters/schema_controls.adoc[]
+include::asciidoc/_chapters/table_definition.adoc[]
include::asciidoc/_chapters/update_stats.adoc[]
include::asciidoc/_chapters/operational_controls.adoc[]
+include::asciidoc/_chapters/debugging.adoc[]
http://git-wip-us.apache.org/repos/asf/incubator-trafodion/blob/223db1e4/docs/load_transform/pom.xml
----------------------------------------------------------------------
diff --git a/docs/load_transform/pom.xml b/docs/load_transform/pom.xml
index ef00fd5..d54b0e8 100644
--- a/docs/load_transform/pom.xml
+++ b/docs/load_transform/pom.xml
@@ -234,15 +234,15 @@
- target/docs/<version>/<document> contains the PDF version and the web book. The web book is named index.html
-->
<!-- Copy the PDF file to its target directories -->
- <copy file="${basedir}/target/index.pdf" tofile="${basedir}/../target/docs/load_transform_guide/Trafodion_Load_Transform_Guide.pdf" />
- <copy file="${basedir}/target/index.pdf" tofile="${basedir}/../target/docs/${project.version}/load_transform_guide/Trafodion_Load_Transform_Guide.pdf" />
+ <copy file="${basedir}/target/index.pdf" tofile="${basedir}/../target/docs/load_transform/Trafodion_Load_Transform_Guide.pdf" />
+ <copy file="${basedir}/target/index.pdf" tofile="${basedir}/../target/docs/${project.version}/load_transform/Trafodion_Load_Transform_Guide.pdf" />
<!-- Copy the Web Book files to their target directories -->
- <copy todir="${basedir}/../target/docs/load_transform_guide">
+ <copy todir="${basedir}/../target/docs/load_transform">
<fileset dir="${basedir}/target/site">
<include name="**/*.*"/> <!--All sub-directories, too-->
</fileset>
</copy>
- <copy todir="${basedir}/../target/docs/${project.version}/load_transform_guide">
+ <copy todir="${basedir}/../target/docs/${project.version}/load_transform">
<fileset dir="${basedir}/target/site">
<include name="**/*.*"/> <!--All sub-directories, too-->
</fileset>
[7/7] incubator-trafodion git commit: Merge [TRAFODION-1851] PR 379
Two new guides and other documentation
Posted by db...@apache.org.
Merge [TRAFODION-1851] PR 379 Two new guides and other documentation
Project: http://git-wip-us.apache.org/repos/asf/incubator-trafodion/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-trafodion/commit/88de28e1
Tree: http://git-wip-us.apache.org/repos/asf/incubator-trafodion/tree/88de28e1
Diff: http://git-wip-us.apache.org/repos/asf/incubator-trafodion/diff/88de28e1
Branch: refs/heads/master
Commit: 88de28e1546a10a85939cb634cbfb8b163f655a5
Parents: 87182d6 6ad54f0
Author: Dave Birdsall <db...@apache.org>
Authored: Tue Mar 15 20:19:02 2016 +0000
Committer: Dave Birdsall <db...@apache.org>
Committed: Tue Mar 15 20:19:02 2016 +0000
----------------------------------------------------------------------
.../src/asciidoc/_chapters/odbc_windows.adoc | 10 +-
docs/cqd_reference/pom.xml | 8 +-
.../src/asciidoc/_chapters/debugging.adoc | 95 ++++++
.../src/asciidoc/_chapters/histograms.adoc | 38 +--
.../src/asciidoc/_chapters/introduction.adoc | 13 +-
.../_chapters/operational_controls.adoc | 138 +++++++-
.../src/asciidoc/_chapters/query_execution.adoc | 239 ++++++++++++++
.../src/asciidoc/_chapters/query_plans.adoc | 330 +++++++++++++++----
.../asciidoc/_chapters/runtime_controls.adoc | 4 +-
.../src/asciidoc/_chapters/schema_controls.adoc | 24 +-
.../asciidoc/_chapters/table_definition.adoc | 142 ++++++++
.../src/asciidoc/_chapters/transactions.adoc | 4 +-
.../src/asciidoc/_chapters/update_stats.adoc | 31 +-
docs/cqd_reference/src/asciidoc/index.adoc | 3 +
docs/load_transform/pom.xml | 8 +-
.../src/asciidoc/_chapters/about.adoc | 8 +-
.../src/asciidoc/_chapters/bulk_load.adoc | 37 ++-
.../src/asciidoc/_chapters/introduction.adoc | 14 +-
.../src/asciidoc/_chapters/tables_indexes.adoc | 18 +-
.../src/asciidoc/_chapters/trickle_load.adoc | 12 +-
.../src/asciidoc/_chapters/troubleshoot.adoc | 4 +-
.../src/asciidoc/_chapters/about.adoc | 4 +-
.../src/asciidoc/_chapters/cqds.adoc | 329 ------------------
.../src/asciidoc/_chapters/introduction.adoc | 11 +-
.../src/asciidoc/_chapters/sql_statements.adoc | 6 +-
docs/sql_reference/src/asciidoc/index.adoc | 1 -
docs/src/site/markdown/documentation.md | 66 ++--
docs/src/site/markdown/logo.md | 2 +-
.../resources/images/logos/pb-Trafodion.png | Bin 49089 -> 0 bytes
.../resources/images/logos/pb-trafodion.png | Bin 0 -> 49089 bytes
docs/src/site/site.xml | 1 +
pom.xml | 2 +
32 files changed, 1062 insertions(+), 540 deletions(-)
----------------------------------------------------------------------