You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hawq.apache.org by yo...@apache.org on 2016/09/29 17:22:15 UTC
[01/50] [abbrv] incubator-hawq-docs git commit: updating readme for
new book config location [HAWQ-1027]
Repository: incubator-hawq-docs
Updated Branches:
refs/heads/master 52ce16bfb -> c3d7896c3
updating readme for new book config location [HAWQ-1027]
Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/commit/6c5a29da
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/tree/6c5a29da
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/diff/6c5a29da
Branch: refs/heads/master
Commit: 6c5a29da88d207d1181ae7f88e0a2faa1dc74fcf
Parents: bec5e3e
Author: David Yozie <yo...@apache.org>
Authored: Mon Aug 29 10:06:02 2016 -0700
Committer: David Yozie <yo...@apache.org>
Committed: Mon Aug 29 10:07:38 2016 -0700
----------------------------------------------------------------------
README.md | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/6c5a29da/README.md
----------------------------------------------------------------------
diff --git a/README.md b/README.md
index 3ba3492..8a794a7 100644
--- a/README.md
+++ b/README.md
@@ -17,7 +17,7 @@ This document contains instructions for building the local Apache HAWQ (incubati
Bookbinder is meant to be used from within a project called a **book**. The book includes a configuration file that describes which documentation repositories to use as source materials. Bookbinder provides a set of scripts to aggregate those repositories and publish them to various locations.
-For Apache HAWQ (incubating), a preconfigured **book** is provided in the directory `/hawq-book`. You can use this configuration to build HTML for Apache HAWQ (incubating) on your local system.
+For Apache HAWQ (incubating), a preconfigured **book** is provided in a separate branch named `book`. You can use this configuration to build HTML for Apache HAWQ (incubating) on your local system.
<a name="prereq"></a>
## Prerequisites
@@ -27,10 +27,10 @@ For Apache HAWQ (incubating), a preconfigured **book** is provided in the direct
<a name="building"></a>
## Building the Documentation
-1. Begin by moving or copying the `/hawq-book directory` to a directory that is parallel to `incubator-hawq/docs-apache-hawq-md`. For example:
+1. Begin by cloning the `book` branch of this repository to a new directory that is parallel to `apache/incubator-hawq-docs`. For example:
- $ cd /repos/incubator-hawq/docs-apache-hawq-md
- $ cp -r hawq-book ..
+ $ cd /repos/apache/incubatoro-hawq-docs
+ $ git clone --branch book git@github.com:apache/incubator-hawq-docs.git ../hawq-book
$ cd ../hawq-book
2. The GemFile in the book directory already defines the `gem "bookbindery"` dependency. Make sure you are in the relocated book directory and enter:
[02/50] [abbrv] incubator-hawq-docs git commit: updating book build
instructions to correct paths
Posted by yo...@apache.org.
updating book build instructions to correct paths
Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/commit/7f23af61
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/tree/7f23af61
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/diff/7f23af61
Branch: refs/heads/master
Commit: 7f23af6195d2530daab6e8861a77b3a497f1ad1c
Parents: 6c5a29d
Author: David Yozie <yo...@apache.org>
Authored: Mon Aug 29 10:15:30 2016 -0700
Committer: David Yozie <yo...@apache.org>
Committed: Mon Aug 29 10:15:30 2016 -0700
----------------------------------------------------------------------
README.md | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/7f23af61/README.md
----------------------------------------------------------------------
diff --git a/README.md b/README.md
index 8a794a7..ed629f3 100644
--- a/README.md
+++ b/README.md
@@ -27,10 +27,10 @@ For Apache HAWQ (incubating), a preconfigured **book** is provided in a separate
<a name="building"></a>
## Building the Documentation
-1. Begin by cloning the `book` branch of this repository to a new directory that is parallel to `apache/incubator-hawq-docs`. For example:
+1. Begin by cloning the `book` branch of this repository to a new directory that is parallel to `asf/incubator-hawq-docs`. For example:
- $ cd /repos/apache/incubatoro-hawq-docs
- $ git clone --branch book git@github.com:apache/incubator-hawq-docs.git ../hawq-book
+ $ cd /repos/asf/incubator-hawq-docs
+ $ git clone --branch book http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs.git ../hawq-book
$ cd ../hawq-book
2. The GemFile in the book directory already defines the `gem "bookbindery"` dependency. Make sure you are in the relocated book directory and enter:
[12/50] [abbrv] incubator-hawq-docs git commit: enhance description
of ssh-exkeys -p option
Posted by yo...@apache.org.
enhance description of ssh-exkeys -p option
Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/commit/2a0b4a91
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/tree/2a0b4a91
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/diff/2a0b4a91
Branch: refs/heads/master
Commit: 2a0b4a910bf7baf62888b9d0aa37e6173ea07324
Parents: 9185772
Author: Lisa Owen <lo...@pivotal.io>
Authored: Tue Sep 13 16:17:04 2016 -0700
Committer: Lisa Owen <lo...@pivotal.io>
Committed: Tue Sep 13 16:17:04 2016 -0700
----------------------------------------------------------------------
reference/cli/admin_utilities/hawqssh-exkeys.html.md.erb | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/2a0b4a91/reference/cli/admin_utilities/hawqssh-exkeys.html.md.erb
----------------------------------------------------------------------
diff --git a/reference/cli/admin_utilities/hawqssh-exkeys.html.md.erb b/reference/cli/admin_utilities/hawqssh-exkeys.html.md.erb
index a75f813..2567faf 100644
--- a/reference/cli/admin_utilities/hawqssh-exkeys.html.md.erb
+++ b/reference/cli/admin_utilities/hawqssh-exkeys.html.md.erb
@@ -46,7 +46,7 @@ The `hawq ssh-exkeys` utility performs key exchange using the following steps:
<dd>Specifies a single host name (or host address) that will participate in the SSH key exchange. You can use the `-h` option multiple times to specify multiple host names and host addresses.</dd>
<dt>-p \<password\> </dt>
-<dd>Specifies the password used to log in to the hosts. The hosts should share the same password.</dd>
+<dd>Specifies the password used to log in to the hosts. The hosts should share the same password. This option is useful when invoking `hawq ssh-exkeys` in a script.</dd>
<dt>-\\\-version </dt>
<dd>Displays the version of this utility.</dd>
[41/50] [abbrv] incubator-hawq-docs git commit: Merge branch
'feature/misc-updates-lo2' of
https://github.com/lisakowen/incubator-hawq-docs into develop
Posted by yo...@apache.org.
Merge branch 'feature/misc-updates-lo2' of https://github.com/lisakowen/incubator-hawq-docs into develop
Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/commit/29a7f42b
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/tree/29a7f42b
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/diff/29a7f42b
Branch: refs/heads/master
Commit: 29a7f42b24ad75b6a4bb91078645566cac70ea36
Parents: 697fed8 2cc9b02
Author: David Yozie <yo...@apache.org>
Authored: Tue Sep 27 14:01:27 2016 -0700
Committer: David Yozie <yo...@apache.org>
Committed: Tue Sep 27 14:01:27 2016 -0700
----------------------------------------------------------------------
admin/RecommendedMonitoringTasks.html.md.erb | 6 ++----
admin/monitor.html.md.erb | 2 +-
clientaccess/roles_privs.html.md.erb | 2 +-
datamgmt/BasicDataOperations.html.md.erb | 12 +++++++-----
datamgmt/ConcurrencyControl.html.md.erb | 7 -------
datamgmt/Transactions.html.md.erb | 10 +++-------
datamgmt/about_statistics.html.md.erb | 8 +++-----
...timizing-data-load-and-query-performance.html.md.erb | 6 +++---
ddl/ddl-partition.html.md.erb | 4 ++--
ddl/ddl-storage.html.md.erb | 9 +++------
reference/catalog/pg_class.html.md.erb | 2 +-
reference/catalog/pg_index.html.md.erb | 2 +-
reference/cli/admin_utilities/hawqload.html.md.erb | 2 +-
reference/cli/client_utilities/vacuumdb.html.md.erb | 2 ++
reference/guc/guc_category-list.html.md.erb | 2 +-
reference/guc/parameter_definitions.html.md.erb | 6 +-----
reference/sql/ALTER-TABLE.html.md.erb | 2 +-
reference/sql/BEGIN.html.md.erb | 2 +-
reference/sql/COPY.html.md.erb | 2 +-
reference/sql/CREATE-EXTERNAL-TABLE.html.md.erb | 2 +-
reference/sql/CREATE-TABLE.html.md.erb | 4 ++--
reference/sql/CREATE-VIEW.html.md.erb | 2 +-
reference/sql/DROP-TABLE.html.md.erb | 2 +-
reference/sql/GRANT.html.md.erb | 2 +-
reference/sql/PREPARE.html.md.erb | 2 +-
reference/sql/VACUUM.html.md.erb | 10 +++++++---
reference/toolkit/hawq_toolkit.html.md.erb | 2 +-
27 files changed, 50 insertions(+), 64 deletions(-)
----------------------------------------------------------------------
[11/50] [abbrv] incubator-hawq-docs git commit: fix threshold
misspellings in content and guc
Posted by yo...@apache.org.
fix threshold misspellings in content and guc
Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/commit/aa361f20
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/tree/aa361f20
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/diff/aa361f20
Branch: refs/heads/master
Commit: aa361f20e3f14485dfcb9d3a38da9370e211df3b
Parents: fa5ed53
Author: Lisa Owen <lo...@pivotal.io>
Authored: Tue Sep 13 12:07:26 2016 -0700
Committer: Lisa Owen <lo...@pivotal.io>
Committed: Tue Sep 13 12:07:26 2016 -0700
----------------------------------------------------------------------
admin/ClusterExpansion.html.md.erb | 4 ++--
admin/ambari-admin.html.md.erb | 6 +++---
reference/guc/parameter_definitions.html.md.erb | 4 ++--
3 files changed, 7 insertions(+), 7 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/aa361f20/admin/ClusterExpansion.html.md.erb
----------------------------------------------------------------------
diff --git a/admin/ClusterExpansion.html.md.erb b/admin/ClusterExpansion.html.md.erb
index dcd96df..87b811c 100644
--- a/admin/ClusterExpansion.html.md.erb
+++ b/admin/ClusterExpansion.html.md.erb
@@ -179,10 +179,10 @@ For example purposes in this procedure, we are adding a new node named `sdw4`.
14. To maintain optimal cluster performance, rebalance HDFS data by running the following command:
15.
```shell
- $ sudo -u hdfs hdfs balancer -threshold threshhold_value
+ $ sudo -u hdfs hdfs balancer -threshold threshold_value
```
- where *threshhold\_value* represents how much a DataNode's disk usage, in percentage, can differ from overall disk usage in the cluster. Adjust the threshold value according to the needs of your production data and disk. The smaller the value, the longer the rebalance time.
+ where *threshold\_value* represents how much a DataNode's disk usage, in percentage, can differ from overall disk usage in the cluster. Adjust the threshold value according to the needs of your production data and disk. The smaller the value, the longer the rebalance time.
>
**Note:** If you do not specify a threshold, then a default value of 20 is used. If the balancer detects that a DataNode is using less than a 20% difference of the cluster's overall disk usage, then data on that node will not be rebalanced. For example, if disk usage across all DataNodes in the cluster is 40% of the cluster's total disk-storage capacity, then the balancer script ensures that a DataNode's disk usage is between 20% and 60% of that DataNode's disk-storage capacity. DataNodes whose disk usage falls within that percentage range will not be rebalanced.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/aa361f20/admin/ambari-admin.html.md.erb
----------------------------------------------------------------------
diff --git a/admin/ambari-admin.html.md.erb b/admin/ambari-admin.html.md.erb
index 16513d4..ddd986a 100644
--- a/admin/ambari-admin.html.md.erb
+++ b/admin/ambari-admin.html.md.erb
@@ -408,7 +408,7 @@ This alert is triggered when any of the HAWQ Segments fail to register with the
* Percent HAWQ Segment Status Available:
This Alert monitors the percentage of HAWQ segments available versus total segments.
- Alerts for **WARN**, and **CRITICAL** are displayed when the number of unresponsive HAWQ segments in the cluster is greater than the specified threshhold. Otherwise, the status will show as **OK**.
+ Alerts for **WARN**, and **CRITICAL** are displayed when the number of unresponsive HAWQ segments in the cluster is greater than the specified threshold. Otherwise, the status will show as **OK**.
* PXF Process Alerts:
PXF Process alerts are triggered when a PXF process on a node is down or not responding on the network. If PXF Alerts are enabled, the Alert status is shown on the PXF Status page.
@@ -423,10 +423,10 @@ To customize the interval, perform the following steps:
3. Enter a number for how often to check status for the selected Alert, then click **Save**. The interval must be specified in whole minutes.
-#### Setting the Available HAWQ Segment Threshhold
+#### Setting the Available HAWQ Segment Threshold
HAWQ monitors the percentage of available HAWQ segments and can send an alert when a specified percent of unresponsive segments is reached.
-To set the threshhold for the unresponsive segments that will trigger an alert:
+To set the threshold for the unresponsive segments that will trigger an alert:
1. Click on **Percent HAWQ Segments Available**.
2. Click **Edit**. Enter the percentage of total segments to create a **Warning** alert (default is 10 percent of the total segments) or **Critical** alert (default is 25 percent of total segments).
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/aa361f20/reference/guc/parameter_definitions.html.md.erb
----------------------------------------------------------------------
diff --git a/reference/guc/parameter_definitions.html.md.erb b/reference/guc/parameter_definitions.html.md.erb
index 3b2157a..d314298 100644
--- a/reference/guc/parameter_definitions.html.md.erb
+++ b/reference/guc/parameter_definitions.html.md.erb
@@ -110,7 +110,7 @@ Descriptions of the HAWQ server configuration parameters listed alphabetically.
- **[gp\_autostats\_mode](../../reference/guc/parameter_definitions.html#gp_autostats_mode)**
-- **[gp\_autostats\_on\_change\_threshhold](../../reference/guc/parameter_definitions.html#topic_imj_zhf_gw)**
+- **[gp\_autostats\_on\_change\_threshold](../../reference/guc/parameter_definitions.html#topic_imj_zhf_gw)**
- **[gp\_backup\_directIO](../../reference/guc/parameter_definitions.html#gp_backup_directIO)**
@@ -1111,7 +1111,7 @@ Automatic statistics collection is triggered if data is inserted directly in a l
</tbody>
</table>
-## <a name="topic_imj_zhf_gw"></a>gp\_autostats\_on\_change\_threshhold
+## <a name="topic_imj_zhf_gw"></a>gp\_autostats\_on\_change\_threshold
Specifies the threshold for automatic statistics collection when `gp_autostats_mode` is set to `on_change`. When a triggering table operation affects a number of rows exceeding this threshold, `ANALYZE `is added and statistics are collected for the table.
[43/50] [abbrv] incubator-hawq-docs git commit: clarify wording for
disabling passwd-based auth
Posted by yo...@apache.org.
clarify wording for disabling passwd-based auth
Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/commit/51592812
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/tree/51592812
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/diff/51592812
Branch: refs/heads/master
Commit: 51592812b88f5078d7092b9b4a00a9dc169e8a71
Parents: 433de8e
Author: Lisa Owen <lo...@pivotal.io>
Authored: Mon Sep 12 12:07:28 2016 -0700
Committer: Lisa Owen <lo...@pivotal.io>
Committed: Wed Sep 28 09:34:02 2016 -0700
----------------------------------------------------------------------
admin/ClusterExpansion.html.md.erb | 4 ++--
admin/ambari-admin.html.md.erb | 4 ++--
2 files changed, 4 insertions(+), 4 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/51592812/admin/ClusterExpansion.html.md.erb
----------------------------------------------------------------------
diff --git a/admin/ClusterExpansion.html.md.erb b/admin/ClusterExpansion.html.md.erb
index f7d3856..d74881e 100644
--- a/admin/ClusterExpansion.html.md.erb
+++ b/admin/ClusterExpansion.html.md.erb
@@ -20,7 +20,7 @@ There are several recommendations to keep in mind when modifying the size of you
## <a id="task_hawq_expand"></a>Adding a New Node to an Existing HAWQ Cluster
-The following procedure describes the steps required to add a node to an existing HAWQ cluster. First ensure that the new node has been configured per the instructions in [Apache HAWQ System Requirements](/20/requirements/system-requirements.html) and [Select HAWQ Host Machines](/20/install/select-hosts.html).
+The following procedure describes the steps required to add a node to an existing HAWQ cluster. First ensure that the new node has been configured per the instructions found in [Apache HAWQ System Requirements](/20/requirements/system-requirements.html) and [Select HAWQ Host Machines](/20/install/select-hosts.html).
For example purposes in this procedure, we are adding a new node named `sdw4`.
@@ -71,7 +71,7 @@ For example purposes in this procedure, we are adding a new node named `sdw4`.
$ hawq ssh-exkeys -e hawq_hosts -x new_hosts
```
- 8. (Optional) Turn off temporary password-based authentication as described in [Apache HAWQ System Requirements](/20/requirements/system-requirements.html#topic_pwdlessssh).
+ 8. (Optional) If you enabled temporary password-based authentication while preparing/configuring your new HAWQ host system, turn off password-based authentication as described in [Apache HAWQ System Requirements](/20/requirements/system-requirements.html#topic_pwdlessssh).
8. After setting up passwordless ssh, you can execute the following hawq command to check the target machine's configuration.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/51592812/admin/ambari-admin.html.md.erb
----------------------------------------------------------------------
diff --git a/admin/ambari-admin.html.md.erb b/admin/ambari-admin.html.md.erb
index 5c98dc0..6b5f1e1 100644
--- a/admin/ambari-admin.html.md.erb
+++ b/admin/ambari-admin.html.md.erb
@@ -160,7 +160,7 @@ There are several recommendations to keep in mind when modifying the size of you
- If you are using hash tables, consider updating the `default_hash_table_bucket_number` server configuration parameter to a larger value after expanding the cluster but before redistributing the hash tables.
### Procedure
-First ensure that the new node(s) has been configured per the instructions in [Apache HAWQ System Requirements](/20/requirements/system-requirements.html) and [Select HAWQ Host Machines](/20/install/select-hosts.html).
+First ensure that the new node(s) has been configured per the instructions found in [Apache HAWQ System Requirements](/20/requirements/system-requirements.html) and [Select HAWQ Host Machines](/20/install/select-hosts.html).
1. If you have any user-defined function (UDF) libraries installed in your existing HAWQ cluster, install them on the new node(s) that you want to add to the HAWQ cluster.
2. Access the Ambari web console at http://ambari.server.hostname:8080, and login as the "admin" user. \(The default password is also "admin".\)
@@ -204,7 +204,7 @@ First ensure that the new node(s) has been configured per the instructions in [A
**Note:** The redistribution of table data can take a significant amount of time.
22. (Optional.) If you changed the **Exchange SSH Keys** property value before adding the host(s), change the value back to `false` after Ambari exchanges keys with the new hosts. This prevents Ambari from exchanging keys with all hosts every time the HAWQ master is started or restarted.
-23. (Optional.) Turn off temporary password-based authentication as described in [Apache HAWQ System Requirements](/20/requirements/system-requirements.html#topic_pwdlessssh).
+23. (Optional.) If you enabled temporary password-based authentication while preparing/configuring your HAWQ host systems, turn off password-based authentication as described in [Apache HAWQ System Requirements](/20/requirements/system-requirements.html#topic_pwdlessssh).
#### <a id="manual-config-steps"></a>Manually Updating the HAWQ Configuration
If you need to expand your HAWQ cluster without restarting the HAWQ service, follow these steps to manually apply the new HAWQ configuration. (Use these steps *instead* of following Step 7 in the above procedure.):
[24/50] [abbrv] incubator-hawq-docs git commit: use colon before table
Posted by yo...@apache.org.
use colon before table
Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/commit/c617b1c1
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/tree/c617b1c1
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/diff/c617b1c1
Branch: refs/heads/master
Commit: c617b1c107f725852e3dd880a382bd2d6eb1d2a8
Parents: f6382c2
Author: Lisa Owen <lo...@pivotal.io>
Authored: Mon Sep 19 10:57:08 2016 -0700
Committer: Lisa Owen <lo...@pivotal.io>
Committed: Mon Sep 19 10:57:08 2016 -0700
----------------------------------------------------------------------
install/aws-config.html.md.erb | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/c617b1c1/install/aws-config.html.md.erb
----------------------------------------------------------------------
diff --git a/install/aws-config.html.md.erb b/install/aws-config.html.md.erb
index fb2a623..32675fe 100644
--- a/install/aws-config.html.md.erb
+++ b/install/aws-config.html.md.erb
@@ -73,7 +73,7 @@ Ensure your Ambari and HAWQ master instances are each assigned a public IP addre
A security group is a set of rules that control network traffic to and from your HAWQ instance. One or more rules may be associated with a security group, and one or more security groups may be associated with an instance.
-To configure HAWQ communication between nodes in the HAWQ cluster, include and open the following ports in the appropriate security group for the HAWQ master and segment nodes.
+To configure HAWQ communication between nodes in the HAWQ cluster, include and open the following ports in the appropriate security group for the HAWQ master and segment nodes:
| Port | Application |
|-------|-------------------------------------|
[39/50] [abbrv] incubator-hawq-docs git commit:
VSEGMENT_RESOURCE_QUOTA to VSEG_RESOURCE_QUOTA
Posted by yo...@apache.org.
VSEGMENT_RESOURCE_QUOTA to VSEG_RESOURCE_QUOTA
Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/commit/697fed81
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/tree/697fed81
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/diff/697fed81
Branch: refs/heads/master
Commit: 697fed818c4b4046988794621ec9bbffaa825f4e
Parents: 37857ea
Author: Lisa Owen <lo...@pivotal.io>
Authored: Tue Sep 27 08:42:46 2016 -0700
Committer: Lisa Owen <lo...@pivotal.io>
Committed: Tue Sep 27 08:42:46 2016 -0700
----------------------------------------------------------------------
reference/sql/ALTER-RESOURCE-QUEUE.html.md.erb | 4 ++--
reference/sql/CREATE-RESOURCE-QUEUE.html.md.erb | 4 ++--
2 files changed, 4 insertions(+), 4 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/697fed81/reference/sql/ALTER-RESOURCE-QUEUE.html.md.erb
----------------------------------------------------------------------
diff --git a/reference/sql/ALTER-RESOURCE-QUEUE.html.md.erb b/reference/sql/ALTER-RESOURCE-QUEUE.html.md.erb
index 16e4411..e1a31db 100644
--- a/reference/sql/ALTER-RESOURCE-QUEUE.html.md.erb
+++ b/reference/sql/ALTER-RESOURCE-QUEUE.html.md.erb
@@ -17,7 +17,7 @@ where \<queue\_attribute\> is:
[CORE_LIMIT_CLUSTER=<percentage>]
[ACTIVE_STATEMENTS=<integer>]
[ALLOCATION_POLICY='even']
- [VSEGMENT_RESOURCE_QUOTA='mem:<memory_units>']
+ [VSEG_RESOURCE_QUOTA='mem:<memory_units>']
[RESOURCE_OVERCOMMIT_FACTOR=<double>]
[NVSEG_UPPER_LIMIT=<integer>]
[NVSEG_LOWER_LIMIT=<integer>]
@@ -79,7 +79,7 @@ Setting the allocation policy to `even` means resources are always evenly dispat
For each query resource allocation request, the HAWQ resource mananger determines the minimum and maximum size of a virtual segment based on multiple factors including query cost, user configuration, table properties, and so on. For example, a hash distributed table requires fixed size of virtual segments. With an even allocation policy, the HAWQ resource manager uses the minimum virtual segment size requirement and evenly dispatches resources to each query resource allocation request in the resource queue.</dd>
<dt>VSEG\_RESOURCE\_QUOTA='mem:{128mb | 256mb | 512mb | 1024mb | 2048mb | 4096mb | 8192mb | 16384mb | 1gb | 2gb | 4gb | 8gb | 16gb}'</dt>
-<dd>Optional. This quota defines how resources are split across multiple virtual segments. For example, when the HAWQ resource manager determines that 256GB memory and 128 vcores should be allocated to the current resource queue, there are multiple solutions on how to divide the resources across virtual segments. For example, you could use a) 2GB/1 vcore \* 128 virtual segments or b) 1GB/0.5 vcore \* 256 virtual segments. Therefore, you can use this attribute to make the HAWQ resource manager calculate the number of virtual segments based on how to divide the memory. For example, if `VSEGMENT_RESOURCE_QUOTA``='mem:512mb'`, then the resource queue will use 512MB/0.25 vcore \* 512 virtual segments. The default value is '`mem:256mb`'.
+<dd>Optional. This quota defines how resources are split across multiple virtual segments. For example, when the HAWQ resource manager determines that 256GB memory and 128 vcores should be allocated to the current resource queue, there are multiple solutions on how to divide the resources across virtual segments. For example, you could use a) 2GB/1 vcore \* 128 virtual segments or b) 1GB/0.5 vcore \* 256 virtual segments. Therefore, you can use this attribute to make the HAWQ resource manager calculate the number of virtual segments based on how to divide the memory. For example, if `VSEG_RESOURCE_QUOTA``='mem:512mb'`, then the resource queue will use 512MB/0.25 vcore \* 512 virtual segments. The default value is '`mem:256mb`'.
**Note:** To avoid resource fragmentation, make sure that the segment resource capacity configured for HAWQ (in HAWQ Standalone mode: `hawq_rm_memory_limit_perseg`; in YARN mode: `yarn.nodemanager.resource.memory-mb` must be a multiple of the resource quotas for all virtual segments and CPU to memory ratio must be a multiple of the amount configured for `yarn.scheduler.minimum-allocation-mb`.</dd>
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/697fed81/reference/sql/CREATE-RESOURCE-QUEUE.html.md.erb
----------------------------------------------------------------------
diff --git a/reference/sql/CREATE-RESOURCE-QUEUE.html.md.erb b/reference/sql/CREATE-RESOURCE-QUEUE.html.md.erb
index 2b16652..8fa6e89 100644
--- a/reference/sql/CREATE-RESOURCE-QUEUE.html.md.erb
+++ b/reference/sql/CREATE-RESOURCE-QUEUE.html.md.erb
@@ -18,7 +18,7 @@ where \<queue\_attribute\> is:
����CORE_LIMIT_CLUSTER=<percentage>
����[ACTIVE_STATEMENTS=<integer>]
[ALLOCATION_POLICY='even']
- [VSEGMENT_RESOURCE_QUOTA='mem:<memory_units>']
+ [VSEG_RESOURCE_QUOTA='mem:<memory_units>']
[RESOURCE_OVERCOMMIT_FACTOR=<double>]
[NVSEG_UPPER_LIMIT=<integer>]
[NVSEG_LOWER_LIMIT=<integer>]
@@ -77,7 +77,7 @@ Setting the allocation policy to `even` means resources are always evenly dispat
For each query resource allocation request, the HAWQ resource manager determines the minimum and maximum size of a virtual segment based on multiple factors including query cost, user configuration, table properties, and so on. For example, a hash distributed table requires fixed size of virtual segments. With an even allocation policy, the HAWQ resource manager uses the minimum virtual segment size requirement and evenly dispatches resources to each query resource allocation request in the resource queue.</dd>
<dt>VSEG\_RESOURCE\_QUOTA='mem:{128mb | 256mb | 512mb | 1024mb | 2048mb | 4096mb | 8192mb | 16384mb | 1gb | 2gb | 4gb | 8gb | 16gb}' </dt>
-<dd>Optional. This quota defines how resources are split across multiple virtual segments. For example, when the HAWQ resource manager determines that 256GB memory and 128 vcores should be allocated to the current resource queue, there are multiple solutions on how to divide the resources across virtual segments. For example, you could use a) 2GB/1 vcore \* 128 virtual segments or b) 1GB/0.5 vcore \* 256 virtual segments. Therefore, you can use this attribute to make the HAWQ resource manager calculate the number of virtual segments based on how to divide the memory. For example, if `VSEGMENT_RESOURCE_QUOTA``='mem:512mb'`, then the resource queue will use 512MB/0.25 vcore \* 512 virtual segments. The default value is '`mem:256mb`'.
+<dd>Optional. This quota defines how resources are split across multiple virtual segments. For example, when the HAWQ resource manager determines that 256GB memory and 128 vcores should be allocated to the current resource queue, there are multiple solutions on how to divide the resources across virtual segments. For example, you could use a) 2GB/1 vcore \* 128 virtual segments or b) 1GB/0.5 vcore \* 256 virtual segments. Therefore, you can use this attribute to make the HAWQ resource manager calculate the number of virtual segments based on how to divide the memory. For example, if `VSEG_RESOURCE_QUOTA``='mem:512mb'`, then the resource queue will use 512MB/0.25 vcore \* 512 virtual segments. The default value is '`mem:256mb`'.
**Note:** To avoid resource fragmentation, make sure that the segment resource capacity configured for HAWQ (in HAWQ Standalone mode: `hawq_rm_memory_limit_perseg`; in YARN mode: `yarn.nodemanager.resource.memory-mb` must be a multiple of the resource quotas for all virtual segments and CPU to memory ratio must be a multiple of the amount configured for `yarn.scheduler.minimum-allocation-mb`.</dd>
[09/50] [abbrv] incubator-hawq-docs git commit: enhance passwordless
ssh instructions
Posted by yo...@apache.org.
enhance passwordless ssh instructions
- include passwdless ssh instructions in system requirement section
- add appropriate xrefs to these instructions where appropriate
(expanding cluster, ambari mgmt)
- include cloud.cfg config info in AWS doc
Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/commit/87930225
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/tree/87930225
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/diff/87930225
Branch: refs/heads/master
Commit: 8793022527080ca80198ba43e2f75550c3f157f1
Parents: fa5ed53
Author: Lisa Owen <lo...@pivotal.io>
Authored: Mon Sep 12 11:44:34 2016 -0700
Committer: Lisa Owen <lo...@pivotal.io>
Committed: Mon Sep 12 11:44:34 2016 -0700
----------------------------------------------------------------------
admin/ClusterExpansion.html.md.erb | 4 +-
admin/ambari-admin.html.md.erb | 8 +++-
install/aws-config.html.md.erb | 5 ++-
requirements/system-requirements.html.md.erb | 45 +++++++++++++++++++++++
4 files changed, 57 insertions(+), 5 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/87930225/admin/ClusterExpansion.html.md.erb
----------------------------------------------------------------------
diff --git a/admin/ClusterExpansion.html.md.erb b/admin/ClusterExpansion.html.md.erb
index dcd96df..e72ad14 100644
--- a/admin/ClusterExpansion.html.md.erb
+++ b/admin/ClusterExpansion.html.md.erb
@@ -20,7 +20,7 @@ There are several recommendations to keep in mind when modifying the size of you
## <a id="task_hawq_expand"></a>Adding a New Node to an Existing HAWQ Cluster
-The following procedure describes the steps required to add a node to an existing HAWQ cluster.
+The following procedure describes the steps required to add a node to an existing HAWQ cluster. First ensure that the new node has been configured per the instructions in [Apache HAWQ System Requirements](/20/requirements/system-requirements.html) and [Select HAWQ Host Machines](/20/install/select-hosts.html).
For example purposes in this procedure, we are adding a new node named `sdw4`.
@@ -71,6 +71,8 @@ For example purposes in this procedure, we are adding a new node named `sdw4`.
$ hawq ssh-exkeys -e hawq_hosts -x new_hosts
```
+ 8. (Optional) Turn off temporary password-based authentication as described in [Apache HAWQ System Requirements](/20/requirements/system-requirements.html#topic_pwdlessssh).
+
8. After setting up passwordless ssh, you can execute the following hawq command to check the target machine's configuration.
```shell
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/87930225/admin/ambari-admin.html.md.erb
----------------------------------------------------------------------
diff --git a/admin/ambari-admin.html.md.erb b/admin/ambari-admin.html.md.erb
index 16513d4..54ac91c 100644
--- a/admin/ambari-admin.html.md.erb
+++ b/admin/ambari-admin.html.md.erb
@@ -159,7 +159,9 @@ There are several recommendations to keep in mind when modifying the size of you
- Note that for hash distributed tables, expanding the cluster will not immediately improve performance since hash distributed tables use a fixed number of virtual segments. In order to obtain better performance with hash distributed tables, you must redistribute the table to the updated cluster by either the [ALTER TABLE](/20/reference/sql/ALTER-TABLE.html) or [CREATE TABLE AS](/20/reference/sql/CREATE-TABLE-AS.html) command.
- If you are using hash tables, consider updating the `default_hash_table_bucket_number` server configuration parameter to a larger value after expanding the cluster but before redistributing the hash tables.
-### Procedure
+### Procedure
+First ensure that the new node(s) has been configured per the instructions in [Apache HAWQ System Requirements](/20/requirements/system-requirements.html) and [Select HAWQ Host Machines](/20/install/select-hosts.html).
+
1. If you have any user-defined function (UDF) libraries installed in your existing HAWQ cluster, install them on the new node(s) that you want to add to the HAWQ cluster.
2. Access the Ambari web console at http://ambari.server.hostname:8080, and login as the "admin" user. \(The default password is also "admin".\)
3. Click **HAWQ** in the list of installed services.
@@ -200,7 +202,9 @@ There are several recommendations to keep in mind when modifying the size of you
21. If you are using hash distributed tables and wish to take advantage of the performance benefits of using a larger cluster, redistribute the data in all hash-distributed tables by using either the [ALTER TABLE](/20/reference/sql/ALTER-TABLE.html) or [CREATE TABLE AS](/20/reference/sql/CREATE-TABLE-AS.html) command. You should redistribute the table data if you modified the `default_hash_table_bucket_number` configuration parameter.
**Note:** The redistribution of table data can take a significant amount of time.
-22. (Optional.) If you changed the **Exchange SSH Keys** property value before adding the host(s), change the value back to `false` after Ambari exchanges keys with the new hosts. This prevents Ambari from exchanging keys with all hosts every time the HAWQ master is started or restarted.
+22. (Optional.) If you changed the **Exchange SSH Keys** property value before adding the host(s), change the value back to `false` after Ambari exchanges keys with the new hosts. This prevents Ambari from exchanging keys with all hosts every time the HAWQ master is started or restarted.
+
+23. (Optional.) Turn off temporary password-based authentication as described in [Apache HAWQ System Requirements](/20/requirements/system-requirements.html#topic_pwdlessssh).
#### <a id="manual-config-steps"></a>Manually Updating the HAWQ Configuration
If you need to expand your HAWQ cluster without restarting the HAWQ service, follow these steps to manually apply the new HAWQ configuration. (Use these steps *instead* of following Step 7 in the above procedure.):
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/87930225/install/aws-config.html.md.erb
----------------------------------------------------------------------
diff --git a/install/aws-config.html.md.erb b/install/aws-config.html.md.erb
index 395970b..98aa35a 100644
--- a/install/aws-config.html.md.erb
+++ b/install/aws-config.html.md.erb
@@ -114,10 +114,11 @@ $ ssh -i my-test.pem user1@192.0.2.0
After launching your HAWQ instance, you will connect to and configure the instance. The *Instances* page of the EC2 Console lists the running instances and their associated network access information.
-Before installing HAWQ, set up the EC2 instances as you would local host server machines. Configure the host operating system, configure host network information (for example, update the `/etc/hosts` file), set operating system parameters, and install operating system packages. For information about how to prepare your operating system environment for HAWQ, see [Select HAWQ Host Machines](../install/select-hosts.html).
+Before installing HAWQ, set up the EC2 instances as you would local host server machines. Configure the host operating system, configure host network information (for example, update the `/etc/hosts` file), set operating system parameters, and install operating system packages. For information about how to prepare your operating system environment for HAWQ, see [Apache HAWQ System Requirements](../requirements/system-requirements.html) and [Select HAWQ Host Machines](../install/select-hosts.html).
-?? PACKAGES, KERNEL VERSION, KERNEL SETTINGS, BEST PRACTICES HERE OR WILL THEY BE THE SAME AS FOR PHYSICAL DEVICE??
+###Passwordless SSH Configuration<a id="topic_pwdlessssh_cc"></a>
+HAWQ hosts will be configured during the installation process to use passwordless SSH for intra-cluster communications. Temporary password-based authentication must be enabled on each HAWQ host in preparation for this configuration. Password authentication is typically disabled by default in cloud images. Update the cloud configuration in `/etc/cloud/cloud.cfg` to enable password authentication in your AMI(s). Set `ssh_pwauth: True` in this file.
##References<a id="topic_hgz_zwy_bv"></a>
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/87930225/requirements/system-requirements.html.md.erb
----------------------------------------------------------------------
diff --git a/requirements/system-requirements.html.md.erb b/requirements/system-requirements.html.md.erb
index bc08ef0..3f47092 100644
--- a/requirements/system-requirements.html.md.erb
+++ b/requirements/system-requirements.html.md.erb
@@ -173,6 +173,51 @@ If this system uses YARN for resource management, you would set `yarn.nodemanage
If this system uses the default HAWQ resource manager, you would set `hawq_rm_memory_limit_perseg` = `RAM - NON_HAWQ_MEMORY` = 8 GB - 7GB = 1.
+## <a id="topic_pwdlessssh"></a>Passwordless SSH Configuration
+
+HAWQ hosts will be configured to use passwordless SSH for intra-cluster communications during the installation process. Temporary password-based authentication must be enabled on each HAWQ host in preparation for this configuration.
+
+1. Install the SSH server if not already configured on the HAWQ system:
+
+ ``` shell
+ $ yum list installed | grep openssh-server
+ $ yum -y install openssh-server
+ ```
+
+2. Update the host's SSH configuration to allow password-based authentication. Edit the SSH config file and change the `PasswordAuthentication` configuration value from `no` to `yes`:
+
+ ``` shell
+ $ sudo vi /etc/ssh/sshd_config
+ ```
+
+ ```
+ PasswordAuthentication yes
+ ```
+
+3. Restart SSH:
+
+ ``` shell
+ $ sudo /etc/init.d/sshd restart
+ ```
+
+*After installation is complete*, you may choose to turn off the temporary password-based authentication configured in the previous steps:
+
+1. Open the SSH `/etc/ssh/sshd_config` file in a text editor and update/uncomment the following configuration options.
+
+ ```
+ RSAAuthentication yes
+ PasswordAuthentication no
+ PubkeyAuthentication yes
+ AuthorizedKeyFile .ssh/authorized_keys
+ ```
+
+2. Restart SSH:
+
+ ``` shell
+ $ sudo /etc/init.d/sshd restart
+ ```
+�
+
## <a id="topic_bsm_hhv_2v"></a>Disk Requirements
- 2GB per host for HAWQ installation.�
[37/50] [abbrv] incubator-hawq-docs git commit: Merge branch
'feature/misc-updates-lo2' of github.com:lisakowen/incubator-hawq-docs into
feature/misc-updates-lo2
Posted by yo...@apache.org.
Merge branch 'feature/misc-updates-lo2' of github.com:lisakowen/incubator-hawq-docs into feature/misc-updates-lo2
Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/commit/da84fa8b
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/tree/da84fa8b
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/diff/da84fa8b
Branch: refs/heads/master
Commit: da84fa8b34cd0b06c3d4dd773f8a1c019f9f392e
Parents: 85e8a5d 3f28377
Author: Lisa Owen <lo...@pivotal.io>
Authored: Fri Sep 23 12:02:29 2016 -0700
Committer: Lisa Owen <lo...@pivotal.io>
Committed: Fri Sep 23 12:02:29 2016 -0700
----------------------------------------------------------------------
----------------------------------------------------------------------
[30/50] [abbrv] incubator-hawq-docs git commit: Merge branch
'feature/hawqcheck-hadoopopt' of
https://github.com/lisakowen/incubator-hawq-docs into develop
Posted by yo...@apache.org.
Merge branch 'feature/hawqcheck-hadoopopt' of https://github.com/lisakowen/incubator-hawq-docs into develop
Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/commit/ecb0097b
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/tree/ecb0097b
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/diff/ecb0097b
Branch: refs/heads/master
Commit: ecb0097bcf5914f5f9754bd51615632a2672992b
Parents: d64c0f9 4a61797
Author: David Yozie <yo...@apache.org>
Authored: Tue Sep 20 15:46:21 2016 -0700
Committer: David Yozie <yo...@apache.org>
Committed: Tue Sep 20 15:46:21 2016 -0700
----------------------------------------------------------------------
.../cli/admin_utilities/hawqcheck.html.md.erb | 25 +++++++++++---------
1 file changed, 14 insertions(+), 11 deletions(-)
----------------------------------------------------------------------
[38/50] [abbrv] incubator-hawq-docs git commit: passwdless ssh -
restart cmd change, remove some props
Posted by yo...@apache.org.
passwdless ssh - restart cmd change, remove some props
Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/commit/dd25c6fe
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/tree/dd25c6fe
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/diff/dd25c6fe
Branch: refs/heads/master
Commit: dd25c6fe223ddc0baca9b55ad43dcc9b3820140e
Parents: 831a83e
Author: Lisa Owen <lo...@pivotal.io>
Authored: Tue Sep 27 08:35:02 2016 -0700
Committer: Lisa Owen <lo...@pivotal.io>
Committed: Tue Sep 27 08:35:02 2016 -0700
----------------------------------------------------------------------
requirements/system-requirements.html.md.erb | 19 ++++++++-----------
1 file changed, 8 insertions(+), 11 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/dd25c6fe/requirements/system-requirements.html.md.erb
----------------------------------------------------------------------
diff --git a/requirements/system-requirements.html.md.erb b/requirements/system-requirements.html.md.erb
index 3f47092..835d974 100644
--- a/requirements/system-requirements.html.md.erb
+++ b/requirements/system-requirements.html.md.erb
@@ -178,43 +178,40 @@ If this system uses the default HAWQ resource manager, you would set `hawq_rm_me
HAWQ hosts will be configured to use passwordless SSH for intra-cluster communications during the installation process. Temporary password-based authentication must be enabled on each HAWQ host in preparation for this configuration.
1. Install the SSH server if not already configured on the HAWQ system:
-
+
``` shell
$ yum list installed | grep openssh-server
$ yum -y install openssh-server
```
-
+
2. Update the host's SSH configuration to allow password-based authentication. Edit the SSH config file and change the `PasswordAuthentication` configuration value from `no` to `yes`:
-
+
``` shell
$ sudo vi /etc/ssh/sshd_config
```
-
+
```
PasswordAuthentication yes
```
3. Restart SSH:
-
+
``` shell
- $ sudo /etc/init.d/sshd restart
+ $ sudo service sshd restart
```
*After installation is complete*, you may choose to turn off the temporary password-based authentication configured in the previous steps:
-1. Open the SSH `/etc/ssh/sshd_config` file in a text editor and update/uncomment the following configuration options.
+1. Open the SSH `/etc/ssh/sshd_config` file in a text editor and update the configuration option you enabled in step 2 above:
```
- RSAAuthentication yes
PasswordAuthentication no
- PubkeyAuthentication yes
- AuthorizedKeyFile .ssh/authorized_keys
```
2. Restart SSH:
``` shell
- $ sudo /etc/init.d/sshd restart
+ $ sudo service sshd restart
```
�
[07/50] [abbrv] incubator-hawq-docs git commit: Merge branch
'feature/mdatadir_chgs' of https://github.com/lisakowen/incubator-hawq-docs
into develop
Posted by yo...@apache.org.
Merge branch 'feature/mdatadir_chgs' of https://github.com/lisakowen/incubator-hawq-docs into develop
Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/commit/fa5ed534
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/tree/fa5ed534
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/diff/fa5ed534
Branch: refs/heads/master
Commit: fa5ed534fef7c6faab01d1fb1a68afdf19b4cc6d
Parents: ecc6de2 1ba8a23
Author: David Yozie <yo...@apache.org>
Authored: Thu Sep 1 13:57:38 2016 -0700
Committer: David Yozie <yo...@apache.org>
Committed: Thu Sep 1 13:57:38 2016 -0700
----------------------------------------------------------------------
...acesandHighAvailabilityEnabledHDFS.html.md.erb | 18 ++++++++----------
clientaccess/client_auth.html.md.erb | 14 +++++++-------
clientaccess/disable-kerberos.html.md.erb | 11 +++--------
clientaccess/kerberos.html.md.erb | 18 +++++++++---------
reference/HAWQEnvironmentVariables.html.md.erb | 8 --------
.../cli/admin_utilities/gplogfilter.html.md.erb | 16 ++++++++--------
6 files changed, 35 insertions(+), 50 deletions(-)
----------------------------------------------------------------------
[13/50] [abbrv] incubator-hawq-docs git commit: add new guc
gp_vmem_idle_resource_timeout
Posted by yo...@apache.org.
add new guc gp_vmem_idle_resource_timeout
Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/commit/1212af9a
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/tree/1212af9a
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/diff/1212af9a
Branch: refs/heads/master
Commit: 1212af9a0e228cf5c9f8dfee2128cbadee613487
Parents: 2a0b4a9
Author: Lisa Owen <lo...@pivotal.io>
Authored: Tue Sep 13 16:55:35 2016 -0700
Committer: Lisa Owen <lo...@pivotal.io>
Committed: Tue Sep 13 16:55:35 2016 -0700
----------------------------------------------------------------------
reference/guc/guc_category-list.html.md.erb | 1 +
reference/guc/parameter_definitions.html.md.erb | 11 +++++++++++
2 files changed, 12 insertions(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/1212af9a/reference/guc/guc_category-list.html.md.erb
----------------------------------------------------------------------
diff --git a/reference/guc/guc_category-list.html.md.erb b/reference/guc/guc_category-list.html.md.erb
index dcf31ad..715583b 100644
--- a/reference/guc/guc_category-list.html.md.erb
+++ b/reference/guc/guc_category-list.html.md.erb
@@ -170,6 +170,7 @@ The parameters in this topic control the configuration of the HAWQ array and its
- [gp\_enable\_direct\_dispatch](parameter_definitions.html#gp_enable_direct_dispatch)
- [gp\_segment\_connect\_timeout](parameter_definitions.html#gp_segment_connect_timeout)
- [gp\_set\_proc\_affinity](parameter_definitions.html#gp_set_proc_affinity)
+- [gp\_vmem\_idle\_resource\_timeout](parameter_definitions.html#gp_vmem_idle_resource_timeout)
### <a id="topic52"></a>Fault Operation Parameters
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/1212af9a/reference/guc/parameter_definitions.html.md.erb
----------------------------------------------------------------------
diff --git a/reference/guc/parameter_definitions.html.md.erb b/reference/guc/parameter_definitions.html.md.erb
index 3b2157a..af87b75 100644
--- a/reference/guc/parameter_definitions.html.md.erb
+++ b/reference/guc/parameter_definitions.html.md.erb
@@ -218,6 +218,8 @@ Descriptions of the HAWQ server configuration parameters listed alphabetically.
- **[gp\_statistics\_use\_fkeys](../../reference/guc/parameter_definitions.html#gp_statistics_use_fkeys)**
+- **[gp\_vmem\_idle\_resource\_timeout](../../reference/guc/parameter_definitions.html#gp_vmem_idle_resource_timeout)**
+
- **[gp\_vmem\_protect\_segworker\_cache\_limit](../../reference/guc/parameter_definitions.html#gp_vmem_protect_segworker_cache_limit)**
- **[gp\_workfile\_checksumming](../../reference/guc/parameter_definitions.html#gp_workfile_checksumming)**
@@ -1659,6 +1661,15 @@ When enabled, allows the legacy query optimizer (planner) to use foreign key inf
|-------------|---------|-------------------------|
| Boolean | on | master, session, reload |
+## <a name="gp_vmem_idle_resource_timeout"></a>gp\_vmem\_idle\_resource\_timeout
+
+Sets the time in milliseconds a session can be idle before gangs on the segment databases are released to free up resources.
+
+| Value Range | Default | Set Classifications |
+|----------------|---------|-------------------------|
+| number of milliseconds | 18000 | master, session, reload |
+
+
## <a name="gp_vmem_protect_segworker_cache_limit"></a>gp\_vmem\_protect\_segworker\_cache\_limit
If a query executor process consumes more than this configured amount, then the process will not be cached for use in subsequent queries after the process completes. Systems with lots of connections or idle processes may want to reduce this number to free more memory on the segments. Note that this is a local parameter and must be set for every segment.
[23/50] [abbrv] incubator-hawq-docs git commit: add reverse dns to
Networking Requirements section
Posted by yo...@apache.org.
add reverse dns to Networking Requirements section
Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/commit/f6382c2c
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/tree/f6382c2c
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/diff/f6382c2c
Branch: refs/heads/master
Commit: f6382c2c1aa204ef6e9756ae16e9169b0722a143
Parents: cd4a3db
Author: Lisa Owen <lo...@pivotal.io>
Authored: Mon Sep 19 10:54:55 2016 -0700
Committer: Lisa Owen <lo...@pivotal.io>
Committed: Mon Sep 19 10:54:55 2016 -0700
----------------------------------------------------------------------
requirements/system-requirements.html.md.erb | 5 +----
1 file changed, 1 insertion(+), 4 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/f6382c2c/requirements/system-requirements.html.md.erb
----------------------------------------------------------------------
diff --git a/requirements/system-requirements.html.md.erb b/requirements/system-requirements.html.md.erb
index 35a2ed9..cf9214d 100644
--- a/requirements/system-requirements.html.md.erb
+++ b/requirements/system-requirements.html.md.erb
@@ -186,10 +186,7 @@ If this system uses the default HAWQ resource manager, you would set `hawq_rm_me
- Gigabit Ethernet within the array. For a production cluster, 10 Gigabit Ethernet recommended.
- Dedicated, non-blocking switch.
- Systems with multiple NICs require NIC bonding to utilize all available network bandwidth.
-
-## <a id="topic_rdb_jhv_2v"></a>Networking Requirements
-
-Communications between the HAWQ master and segments require reverse DNS lookup be configured in your cluster network.
+- Communication between the HAWQ master and segments requires reverse DNS lookup be configured in your cluster network.
## <a id="port-req"></a>Port Requirements
Individual PXF plug-ins, which you install after adding the HAWQ and PXF services, require that you Tomcat on the host machine. Tomcat reserves ports 8005, 8080, and 8009.
[48/50] [abbrv] incubator-hawq-docs git commit: changing to use
relative links instead of root links,
to account for different versions of the docs
Posted by yo...@apache.org.
changing to use relative links instead of root links, to account for different versions of the docs
Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/commit/459e3bc7
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/tree/459e3bc7
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/diff/459e3bc7
Branch: refs/heads/master
Commit: 459e3bc7dc45994718960e2f33b09b7969dea1ac
Parents: 97317c4
Author: David Yozie <yo...@apache.org>
Authored: Wed Sep 28 14:33:02 2016 -0700
Committer: David Yozie <yo...@apache.org>
Committed: Wed Sep 28 14:33:02 2016 -0700
----------------------------------------------------------------------
admin/ClusterExpansion.html.md.erb | 10 +++++-----
admin/RecommendedMonitoringTasks.html.md.erb | 4 ++--
admin/ambari-admin.html.md.erb | 10 +++++-----
admin/startstop.html.md.erb | 4 ++--
ddl/ddl-partition.html.md.erb | 2 +-
ddl/ddl-table.html.md.erb | 4 ++--
overview/HAWQArchitecture.html.md.erb | 4 ++--
overview/ManagementTools.html.md.erb | 2 +-
overview/RedundancyFailover.html.md.erb | 2 +-
overview/ResourceManagement.html.md.erb | 2 +-
overview/TableDistributionStorage.html.md.erb | 6 +++---
plext/using_pljava.html.md.erb | 4 ++--
reference/HAWQEnvironmentVariables.html.md.erb | 2 +-
reference/guc/parameter_definitions.html.md.erb | 4 ++--
reference/sql/ALTER-RESOURCE-QUEUE.html.md.erb | 4 ++--
reference/sql/CREATE-RESOURCE-QUEUE.html.md.erb | 4 ++--
resourcemgmt/ConfigureResourceManagement.html.md.erb | 6 +++---
resourcemgmt/HAWQResourceManagement.html.md.erb | 2 +-
resourcemgmt/ResourceQueues.html.md.erb | 8 ++++----
resourcemgmt/YARNIntegration.html.md.erb | 2 +-
resourcemgmt/best-practices.html.md.erb | 4 ++--
troubleshooting/Troubleshooting.html.md.erb | 10 +++++-----
22 files changed, 50 insertions(+), 50 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/459e3bc7/admin/ClusterExpansion.html.md.erb
----------------------------------------------------------------------
diff --git a/admin/ClusterExpansion.html.md.erb b/admin/ClusterExpansion.html.md.erb
index d74881e..d99c760 100644
--- a/admin/ClusterExpansion.html.md.erb
+++ b/admin/ClusterExpansion.html.md.erb
@@ -4,7 +4,7 @@ title: Expanding a Cluster
Apache HAWQ supports dynamic node expansion. You can add segment nodes while HAWQ is running without having to suspend or terminate cluster operations.
-**Note:** This topic describes how to expand a cluster using the command-line interface. If you are using Ambari to manage your HAWQ cluster, see [Expanding the HAWQ Cluster](/20/admin/ambari-admin.html#amb-expand) in [Managing HAWQ Using Ambari](/20/admin/ambari-admin.html)
+**Note:** This topic describes how to expand a cluster using the command-line interface. If you are using Ambari to manage your HAWQ cluster, see [Expanding the HAWQ Cluster](../admin/ambari-admin.html#amb-expand) in [Managing HAWQ Using Ambari](../admin/ambari-admin.html)
## <a id="topic_kkc_tgb_h5"></a>Guidelines for Cluster Expansion
@@ -15,12 +15,12 @@ There are several recommendations to keep in mind when modifying the size of you
- When you add a new node, install both a DataNode and a physical segment on the new node.
- After adding a new node, you should always rebalance HDFS data to maintain cluster performance.
- Adding or removing a node also necessitates an update to the HDFS metadata cache. This update will happen eventually, but can take some time. To speed the update of the metadata cache, execute **`select gp_metadata_cache_clear();`**.
-- Note that for hash distributed tables, expanding the cluster will not immediately improve performance since hash distributed tables use a fixed number of virtual segments. In order to obtain better performance with hash distributed tables, you must redistribute the table to the updated cluster by either the [ALTER TABLE](/20/reference/sql/ALTER-TABLE.html) or [CREATE TABLE AS](/20/reference/sql/CREATE-TABLE-AS.html) command.
+- Note that for hash distributed tables, expanding the cluster will not immediately improve performance since hash distributed tables use a fixed number of virtual segments. In order to obtain better performance with hash distributed tables, you must redistribute the table to the updated cluster by either the [ALTER TABLE](../reference/sql/ALTER-TABLE.html) or [CREATE TABLE AS](../reference/sql/CREATE-TABLE-AS.html) command.
- If you are using hash tables, consider updating the `default_hash_table_bucket_number` server configuration parameter to a larger value after expanding the cluster but before redistributing the hash tables.
## <a id="task_hawq_expand"></a>Adding a New Node to an Existing HAWQ Cluster
-The following procedure describes the steps required to add a node to an existing HAWQ cluster. First ensure that the new node has been configured per the instructions found in [Apache HAWQ System Requirements](/20/requirements/system-requirements.html) and [Select HAWQ Host Machines](/20/install/select-hosts.html).
+The following procedure describes the steps required to add a node to an existing HAWQ cluster. First ensure that the new node has been configured per the instructions found in [Apache HAWQ System Requirements](../requirements/system-requirements.html) and [Select HAWQ Host Machines](../install/select-hosts.html).
For example purposes in this procedure, we are adding a new node named `sdw4`.
@@ -71,7 +71,7 @@ For example purposes in this procedure, we are adding a new node named `sdw4`.
$ hawq ssh-exkeys -e hawq_hosts -x new_hosts
```
- 8. (Optional) If you enabled temporary password-based authentication while preparing/configuring your new HAWQ host system, turn off password-based authentication as described in [Apache HAWQ System Requirements](/20/requirements/system-requirements.html#topic_pwdlessssh).
+ 8. (Optional) If you enabled temporary password-based authentication while preparing/configuring your new HAWQ host system, turn off password-based authentication as described in [Apache HAWQ System Requirements](../requirements/system-requirements.html#topic_pwdlessssh).
8. After setting up passwordless ssh, you can execute the following hawq command to check the target machine's configuration.
@@ -220,7 +220,7 @@ For example purposes in this procedure, we are adding a new node named `sdw4`.
|\> 256 and <= 512|1 \* \#nodes|
|\> 512|512|
-18. If you are using hash distributed tables and wish to take advantage of the performance benefits of using a larger cluster, redistribute the data in all hash-distributed tables by using either the [ALTER TABLE](/20/reference/sql/ALTER-TABLE.html) or [CREATE TABLE AS](/20/reference/sql/CREATE-TABLE-AS.html) command. You should redistribute the table data if you modified the `default_hash_table_bucket_number` configuration parameter.
+18. If you are using hash distributed tables and wish to take advantage of the performance benefits of using a larger cluster, redistribute the data in all hash-distributed tables by using either the [ALTER TABLE](../reference/sql/ALTER-TABLE.html) or [CREATE TABLE AS](../reference/sql/CREATE-TABLE-AS.html) command. You should redistribute the table data if you modified the `default_hash_table_bucket_number` configuration parameter.
**Note:** The redistribution of table data can take a significant amount of time.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/459e3bc7/admin/RecommendedMonitoringTasks.html.md.erb
----------------------------------------------------------------------
diff --git a/admin/RecommendedMonitoringTasks.html.md.erb b/admin/RecommendedMonitoringTasks.html.md.erb
index 3007aee..5083b44 100644
--- a/admin/RecommendedMonitoringTasks.html.md.erb
+++ b/admin/RecommendedMonitoringTasks.html.md.erb
@@ -188,7 +188,7 @@ GROUP BY 1;
</tr>
<tr>
<td>
- <p>Vacuum all system catalogs (tables in the <code>pg_catalog</code> schema) that are approaching <a href="/20/reference/guc/parameter_definitions.html">vacuum_freeze_min_age</a>.</p>
+ <p>Vacuum all system catalogs (tables in the <code>pg_catalog</code> schema) that are approaching <a href="../reference/guc/parameter_definitions.html">vacuum_freeze_min_age</a>.</p>
<p>Recommended frequency: daily</p>
<p>Severity: CRITICAL</p>
</td>
@@ -196,7 +196,7 @@ GROUP BY 1;
<p><p>Vacuum an individual system catalog table:</p>
<pre><code>VACUUM <<i>table</i>>;</code></pre>
</td>
- <td>After the <a href="/20/reference/guc/parameter_definitions.html">vacuum_freeze_min_age</a> value is reached, VACUUM will no longer replace transaction IDs with <code>FrozenXID</code> while scanning a table. Perform vacuum on these tables before the limit is reached.</td>
+ <td>After the <a href="../reference/guc/parameter_definitions.html">vacuum_freeze_min_age</a> value is reached, VACUUM will no longer replace transaction IDs with <code>FrozenXID</code> while scanning a table. Perform vacuum on these tables before the limit is reached.</td>
</tr>
<td>
<p>Update table statistics.</p>
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/459e3bc7/admin/ambari-admin.html.md.erb
----------------------------------------------------------------------
diff --git a/admin/ambari-admin.html.md.erb b/admin/ambari-admin.html.md.erb
index 6b5f1e1..e41adc6 100644
--- a/admin/ambari-admin.html.md.erb
+++ b/admin/ambari-admin.html.md.erb
@@ -8,7 +8,7 @@ Ambari provides an easy interface to perform some of the most common HAWQ and PX
HAWQ supports integration with YARN for global resource management. In a YARN managed environment, HAWQ can request resources (containers) dynamically from YARN, and return resources when HAWQ\u2019s workload is not heavy.
-See also [Integrating YARN with HAWQ](/20/resourcemgmt/YARNIntegration.html) for command-line instructions and additional details about using HAWQ with YARN.
+See also [Integrating YARN with HAWQ](../resourcemgmt/YARNIntegration.html) for command-line instructions and additional details about using HAWQ with YARN.
### When to Perform
@@ -156,11 +156,11 @@ There are several recommendations to keep in mind when modifying the size of you
- When you add a new node, install both a DataNode and a HAWQ segment on the new node.
- After adding a new node, you should always rebalance HDFS data to maintain cluster performance.
- Adding or removing a node also necessitates an update to the HDFS metadata cache. This update will happen eventually, but can take some time. To speed the update of the metadata cache, select the **Service Actions > Clear HAWQ's HDFS Metadata Cache** option in Ambari.
-- Note that for hash distributed tables, expanding the cluster will not immediately improve performance since hash distributed tables use a fixed number of virtual segments. In order to obtain better performance with hash distributed tables, you must redistribute the table to the updated cluster by either the [ALTER TABLE](/20/reference/sql/ALTER-TABLE.html) or [CREATE TABLE AS](/20/reference/sql/CREATE-TABLE-AS.html) command.
+- Note that for hash distributed tables, expanding the cluster will not immediately improve performance since hash distributed tables use a fixed number of virtual segments. In order to obtain better performance with hash distributed tables, you must redistribute the table to the updated cluster by either the [ALTER TABLE](../reference/sql/ALTER-TABLE.html) or [CREATE TABLE AS](../reference/sql/CREATE-TABLE-AS.html) command.
- If you are using hash tables, consider updating the `default_hash_table_bucket_number` server configuration parameter to a larger value after expanding the cluster but before redistributing the hash tables.
### Procedure
-First ensure that the new node(s) has been configured per the instructions found in [Apache HAWQ System Requirements](/20/requirements/system-requirements.html) and [Select HAWQ Host Machines](/20/install/select-hosts.html).
+First ensure that the new node(s) has been configured per the instructions found in [Apache HAWQ System Requirements](../requirements/system-requirements.html) and [Select HAWQ Host Machines](../install/select-hosts.html).
1. If you have any user-defined function (UDF) libraries installed in your existing HAWQ cluster, install them on the new node(s) that you want to add to the HAWQ cluster.
2. Access the Ambari web console at http://ambari.server.hostname:8080, and login as the "admin" user. \(The default password is also "admin".\)
@@ -199,12 +199,12 @@ First ensure that the new node(s) has been configured per the instructions found
19. Consider the impact of rebalancing HDFS to other components, such as HBase, before you complete this step.
<br/><br/>Rebalance your HDFS data by selecting the **HDFS** service and then choosing **Service Actions > Rebalance HDFS**. Follow the Ambari instructions to complete the rebalance action.
20. Speed up the clearing of the metadata cache by first selecting the **HAWQ** service and then selecting **Service Actions > Clear HAWQ's HDFS Metadata Cache**.
-21. If you are using hash distributed tables and wish to take advantage of the performance benefits of using a larger cluster, redistribute the data in all hash-distributed tables by using either the [ALTER TABLE](/20/reference/sql/ALTER-TABLE.html) or [CREATE TABLE AS](/20/reference/sql/CREATE-TABLE-AS.html) command. You should redistribute the table data if you modified the `default_hash_table_bucket_number` configuration parameter.
+21. If you are using hash distributed tables and wish to take advantage of the performance benefits of using a larger cluster, redistribute the data in all hash-distributed tables by using either the [ALTER TABLE](../reference/sql/ALTER-TABLE.html) or [CREATE TABLE AS](../reference/sql/CREATE-TABLE-AS.html) command. You should redistribute the table data if you modified the `default_hash_table_bucket_number` configuration parameter.
**Note:** The redistribution of table data can take a significant amount of time.
22. (Optional.) If you changed the **Exchange SSH Keys** property value before adding the host(s), change the value back to `false` after Ambari exchanges keys with the new hosts. This prevents Ambari from exchanging keys with all hosts every time the HAWQ master is started or restarted.
-23. (Optional.) If you enabled temporary password-based authentication while preparing/configuring your HAWQ host systems, turn off password-based authentication as described in [Apache HAWQ System Requirements](/20/requirements/system-requirements.html#topic_pwdlessssh).
+23. (Optional.) If you enabled temporary password-based authentication while preparing/configuring your HAWQ host systems, turn off password-based authentication as described in [Apache HAWQ System Requirements](../requirements/system-requirements.html#topic_pwdlessssh).
#### <a id="manual-config-steps"></a>Manually Updating the HAWQ Configuration
If you need to expand your HAWQ cluster without restarting the HAWQ service, follow these steps to manually apply the new HAWQ configuration. (Use these steps *instead* of following Step 7 in the above procedure.):
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/459e3bc7/admin/startstop.html.md.erb
----------------------------------------------------------------------
diff --git a/admin/startstop.html.md.erb b/admin/startstop.html.md.erb
index 106fa7d..5c07f96 100644
--- a/admin/startstop.html.md.erb
+++ b/admin/startstop.html.md.erb
@@ -12,7 +12,7 @@ Use the `hawq start `*`object`* and `hawq stop `*`object`* commands to start and
Do not issue a `KILL` command to end any Postgres process. Instead, use the database command `pg_cancel_backend()`.
-For information about [hawq start](/20/reference/cli/admin_utilities/hawqstart.html) and [hawq stop](/20/reference/cli/admin_utilities/hawqstop.html), see the appropriate pages in the HAWQ Management Utility Reference or enter `hawq start -h` or `hawq stop -h` on the command line.
+For information about [hawq start](../reference/cli/admin_utilities/hawqstart.html) and [hawq stop](../reference/cli/admin_utilities/hawqstop.html), see the appropriate pages in the HAWQ Management Utility Reference or enter `hawq start -h` or `hawq stop -h` on the command line.
## <a id="task_g1y_xtm_s5"></a>Initialize HAWQ
@@ -68,7 +68,7 @@ The `hawq restart` command with the appropriate cluster or node command can stop
Reload changes to the HAWQ configuration files without interrupting the system.
-The `hawq stop` command can reload changes to the pg\_hba.conf configuration file and to *runtime* parameters in the hawq-site.xml file and pg\_hba.conf file without service interruption. Active sessions pick up changes when they reconnect to the database. Many server configuration parameters require a full system restart \(`hawq restart cluster`\) to activate. For information about server configuration parameters, see the [Server Configuration Parameter Reference](/20/reference/guc/guc_config.html).
+The `hawq stop` command can reload changes to the pg\_hba.conf configuration file and to *runtime* parameters in the hawq-site.xml file and pg\_hba.conf file without service interruption. Active sessions pick up changes when they reconnect to the database. Many server configuration parameters require a full system restart \(`hawq restart cluster`\) to activate. For information about server configuration parameters, see the [Server Configuration Parameter Reference](../reference/guc/guc_config.html).
- Reload configuration file changes without shutting down the system using the `hawq stop` command:
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/459e3bc7/ddl/ddl-partition.html.md.erb
----------------------------------------------------------------------
diff --git a/ddl/ddl-partition.html.md.erb b/ddl/ddl-partition.html.md.erb
index 39fd43c..f790161 100644
--- a/ddl/ddl-partition.html.md.erb
+++ b/ddl/ddl-partition.html.md.erb
@@ -37,7 +37,7 @@ Not all tables are good candidates for partitioning. If the answer is *yes* to a
Do not create more partitions than are needed. Creating too many partitions can slow down management and maintenance jobs, such as vacuuming, recovering segments, expanding the cluster, checking disk usage, and others.
-Partitioning does not improve query performance unless the query optimizer can eliminate partitions based on the query predicates. Queries that scan every partition run slower than if the table were not partitioned, so avoid partitioning if few of your queries achieve partition elimination. Check the explain plan for queries to make sure that partitions are eliminated. See [Query Profiling](/20/query/query-profiling.html) for more about partition elimination.
+Partitioning does not improve query performance unless the query optimizer can eliminate partitions based on the query predicates. Queries that scan every partition run slower than if the table were not partitioned, so avoid partitioning if few of your queries achieve partition elimination. Check the explain plan for queries to make sure that partitions are eliminated. See [Query Profiling](../query/query-profiling.html) for more about partition elimination.
Be very careful with multi-level partitioning because the number of partition files can grow very quickly. For example, if a table is partitioned by both day and city, and there are 1,000 days of data and 1,000 cities, the total number of partitions is one million. Column-oriented tables store each column in a physical table, so if this table has 100 columns, the system would be required to manage 100 million files for the table.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/459e3bc7/ddl/ddl-table.html.md.erb
----------------------------------------------------------------------
diff --git a/ddl/ddl-table.html.md.erb b/ddl/ddl-table.html.md.erb
index 7120031..62ece36 100644
--- a/ddl/ddl-table.html.md.erb
+++ b/ddl/ddl-table.html.md.erb
@@ -12,7 +12,7 @@ The `CREATE TABLE` command creates a table and defines its structure. When you c
- Any table constraints to limit the data that a column or table can contain. See [Setting Table Constraints](#topic28).
- The distribution policy of the table, which determines how HAWQ divides data is across the segments. See [Choosing the Table Distribution Policy](#topic34).
- The way the table is stored on disk.
-- The table partitioning strategy for large tables, which specifies how the data should be divided. See [Creating and Managing Databases](/20/ddl/ddl-database.html).
+- The table partitioning strategy for large tables, which specifies how the data should be divided. See [Creating and Managing Databases](../ddl/ddl-database.html).
### <a id="topic27"></a>Choosing Column Data Types
@@ -126,7 +126,7 @@ For hash tables, the `SELECT INTO` function always uses random distribution.
`CREATE TABLE`'s optional clause `DISTRIBUTED BY` specifies the distribution policy for a table. The default is a random distribution policy. You can also choose to distribute data as a hash-based policy, where the `bucketnum` attribute sets the number of hash buckets used by a hash-distributed table. HASH distributed tables are created with the number of hash buckets specified by the `default_hash_table_bucket_number` parameter.
-Policies for different application scenarios can be specified to optimize performance. The number of virtual segments used for query execution can now be tuned using the `hawq_rm_nvseg_perquery_limit `and `hawq_rm_nvseg_perquery_perseg_limit` parameters, in connection with the `default_hash_table_bucket_number` parameter, which sets the default `bucketnum`. For more information, see the guidelines for Virtual Segments in the next section and in [Query Performance](/20/query/query-performance.html#topic38).
+Policies for different application scenarios can be specified to optimize performance. The number of virtual segments used for query execution can now be tuned using the `hawq_rm_nvseg_perquery_limit `and `hawq_rm_nvseg_perquery_perseg_limit` parameters, in connection with the `default_hash_table_bucket_number` parameter, which sets the default `bucketnum`. For more information, see the guidelines for Virtual Segments in the next section and in [Query Performance](../query/query-performance.html#topic38).
#### <a id="topic_wff_mqm_gv"></a>Performance Tuning
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/459e3bc7/overview/HAWQArchitecture.html.md.erb
----------------------------------------------------------------------
diff --git a/overview/HAWQArchitecture.html.md.erb b/overview/HAWQArchitecture.html.md.erb
index 84fc3c2..d42d241 100755
--- a/overview/HAWQArchitecture.html.md.erb
+++ b/overview/HAWQArchitecture.html.md.erb
@@ -52,7 +52,7 @@ By default, the interconnect uses UDP \(User Datagram Protocol\) to send message
The HAWQ resource manager obtains resources from YARN and responds to resource requests. Resources are buffered by the HAWQ resource manager to support low latency queries. The HAWQ resource manager can also run in standalone mode. In these deployments, HAWQ manages resources by itself without YARN.
-See [How HAWQ Manages Resources](/20/resourcemgmt/HAWQResourceManagement.html) for more details on HAWQ resource management.
+See [How HAWQ Manages Resources](../resourcemgmt/HAWQResourceManagement.html) for more details on HAWQ resource management.
## <a id="topic_mrl_psq_f5"></a>HAWQ Catalog Service
@@ -62,7 +62,7 @@ The HAWQ catalog service stores all metadata, such as UDF/UDT information, relat
The HAWQ fault tolerance service \(FTS\) is responsible for detecting segment failures and accepting heartbeats from segments.
-See [Understanding the Fault Tolerance Service](/20/admin/FaultTolerance.html) for more information on this service.
+See [Understanding the Fault Tolerance Service](../admin/FaultTolerance.html) for more information on this service.
## <a id="topic_jtc_nkm_g5"></a>HAWQ Dispatcher
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/459e3bc7/overview/ManagementTools.html.md.erb
----------------------------------------------------------------------
diff --git a/overview/ManagementTools.html.md.erb b/overview/ManagementTools.html.md.erb
index 39f072b..0c7439d 100755
--- a/overview/ManagementTools.html.md.erb
+++ b/overview/ManagementTools.html.md.erb
@@ -6,4 +6,4 @@ HAWQ management tools are consolidated into one `hawq` command.
The `hawq` command can init, start and stop each segment separately, and supports dynamic expansion of the cluster.
-See [HAWQ Management Tools Reference](/20/reference/cli/management_tools.html) for a list of all tools available in HAWQ.
+See [HAWQ Management Tools Reference](../reference/cli/management_tools.html) for a list of all tools available in HAWQ.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/459e3bc7/overview/RedundancyFailover.html.md.erb
----------------------------------------------------------------------
diff --git a/overview/RedundancyFailover.html.md.erb b/overview/RedundancyFailover.html.md.erb
index bf439f6..90eec63 100755
--- a/overview/RedundancyFailover.html.md.erb
+++ b/overview/RedundancyFailover.html.md.erb
@@ -13,7 +13,7 @@ HAWQ employs several mechanisms for ensuring high availability. The foremost mec
* Master mirroring. Clusters have a standby master in the event of failure of the primary master.
* Dual clusters. Administrators can create a secondary cluster and synchronizes its data with the primary cluster either through dual ETL or backup and restore mechanisms.
-In addition to high availability managed on the HAWQ level, you can enable high availability in HDFS for HAWQ by implementing the high availability feature for NameNodes. See [HAWQ Filespaces and High Availability Enabled HDFS](/20/admin/HAWQFilespacesandHighAvailabilityEnabledHDFS.html).
+In addition to high availability managed on the HAWQ level, you can enable high availability in HDFS for HAWQ by implementing the high availability feature for NameNodes. See [HAWQ Filespaces and High Availability Enabled HDFS](../admin/HAWQFilespacesandHighAvailabilityEnabledHDFS.html).
## <a id="aboutsegmentfailover"></a>About Segment Fault Tolerance
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/459e3bc7/overview/ResourceManagement.html.md.erb
----------------------------------------------------------------------
diff --git a/overview/ResourceManagement.html.md.erb b/overview/ResourceManagement.html.md.erb
index 5a5adda..8f7e2fd 100755
--- a/overview/ResourceManagement.html.md.erb
+++ b/overview/ResourceManagement.html.md.erb
@@ -11,4 +11,4 @@ HAWQ has the ability to manage resources by using the following mechanisms:
- Dynamic resource allocation at query runtime. HAWQ dynamically allocates resources based on resource queue definitions. HAWQ automatically distributes resources based on running \(or queued\) queries and resource queue capacities.
- Resource limitations on virtual segments and queries. You can configure HAWQ to enforce limits on CPU and memory usage both for virtual segments and the resource queues used by queries.
-For more details on resource management in HAWQ and how it works, see [Managing Resources](/20/resourcemgmt/HAWQResourceManagement.html).
+For more details on resource management in HAWQ and how it works, see [Managing Resources](../resourcemgmt/HAWQResourceManagement.html).
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/459e3bc7/overview/TableDistributionStorage.html.md.erb
----------------------------------------------------------------------
diff --git a/overview/TableDistributionStorage.html.md.erb b/overview/TableDistributionStorage.html.md.erb
index 8bf6542..aa03b59 100755
--- a/overview/TableDistributionStorage.html.md.erb
+++ b/overview/TableDistributionStorage.html.md.erb
@@ -16,7 +16,7 @@ Randomly distributed tables have some benefits over hash distributed tables. For
On the other hand, for some queries, hash distributed tables are faster than randomly distributed tables. For example, hash distributed tables have some performance benefits for some TPC-H queries. You should choose the distribution policy that is best suited for your application's scenario.
-See [Choosing the Table Distribution Policy](/20/ddl/ddl-table.html) for more details.
+See [Choosing the Table Distribution Policy](../ddl/ddl-table.html) for more details.
## Data Locality
@@ -33,9 +33,9 @@ PXF is an extensible framework that allows HAWQ to access data in external
sources as readable or writable HAWQ tables. PXF has built-in connectors for
accessing data inside HDFS files, Hive tables, and HBase tables. PXF also
integrates with HCatalog to query Hive tables directly. See [Working with PXF
-and External Data](/20/pxf/HawqExtensionFrameworkPXF.html) for more
+and External Data](../pxf/HawqExtensionFrameworkPXF.html) for more
details.
Users can create custom PXF connectors to access other parallel data stores or
processing engines. Connectors are Java plug-ins that use the PXF API. For more
-information see [PXF External Tables and API](/20/pxf/PXFExternalTableandAPIReference.html).
+information see [PXF External Tables and API](../pxf/PXFExternalTableandAPIReference.html).
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/459e3bc7/plext/using_pljava.html.md.erb
----------------------------------------------------------------------
diff --git a/plext/using_pljava.html.md.erb b/plext/using_pljava.html.md.erb
index 3cce857..d19fbbe 100644
--- a/plext/using_pljava.html.md.erb
+++ b/plext/using_pljava.html.md.erb
@@ -59,7 +59,7 @@ HAWQ uses the `pljava_classpath` server configuration parameter in place of the
The following server configuration parameters are used by PL/Java in HAWQ. These parameters replace the `pljava.*` parameters that are used in the standard PostgreSQL PL/Java implementation.
-<p class="note"><b>Note:</b> See the <a href="/20/reference/hawq-reference.html">HAWQ Reference</a> for information about HAWQ server configuration parameters.</p>
+<p class="note"><b>Note:</b> See the <a href="../reference/hawq-reference.html">HAWQ Reference</a> for information about HAWQ server configuration parameters.</p>
#### pljava\_classpath
@@ -597,7 +597,7 @@ Main-Class: Example
Specification-Title: "Example"
Specification-Version: "1.0"
Created-By: 1.6.0_35-b10-428-11M3811
-Build-Date: 01/20/2013 10:09 AM
+Build-Date: 01../2013 10:09 AM
```
Compile the Java code:
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/459e3bc7/reference/HAWQEnvironmentVariables.html.md.erb
----------------------------------------------------------------------
diff --git a/reference/HAWQEnvironmentVariables.html.md.erb b/reference/HAWQEnvironmentVariables.html.md.erb
index 8061781..ce21798 100644
--- a/reference/HAWQEnvironmentVariables.html.md.erb
+++ b/reference/HAWQEnvironmentVariables.html.md.erb
@@ -66,7 +66,7 @@ The password used if the server demands password authentication. Use of this env
The name of the password file to use for lookups. If not set, it defaults to `~/.pgpass`.
-See The Password File under�[Configuring Client Authentication](/20/clientaccess/client_auth.html).
+See The Password File under�[Configuring Client Authentication](../clientaccess/client_auth.html).
### <a id="pgoptions"></a>PGOPTIONS
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/459e3bc7/reference/guc/parameter_definitions.html.md.erb
----------------------------------------------------------------------
diff --git a/reference/guc/parameter_definitions.html.md.erb b/reference/guc/parameter_definitions.html.md.erb
index 645335b..2012155 100644
--- a/reference/guc/parameter_definitions.html.md.erb
+++ b/reference/guc/parameter_definitions.html.md.erb
@@ -2458,7 +2458,7 @@ Sets the maximum number of append-only relations that can be written to or loade
## <a name="max_connections"></a>max\_connections
-The maximum number of concurrent connections allowed on master. In a HAWQ system, user client connections go through the HAWQ master instance only. Segment instances should allow 5-10 times the amount as the master. When you increase this parameter, you must increase [max\_prepared\_transactions](#max_prepared_transactions) as well. For more information about limiting concurrent connections, see [Configuring Client Authentication](/20/clientaccess/client_auth.html).
+The maximum number of concurrent connections allowed on master. In a HAWQ system, user client connections go through the HAWQ master instance only. Segment instances should allow 5-10 times the amount as the master. When you increase this parameter, you must increase [max\_prepared\_transactions](#max_prepared_transactions) as well. For more information about limiting concurrent connections, see [Configuring Client Authentication](../../clientaccess/client_auth.html).
Increasing this parameter may cause HAWQ to request more shared memory. See [shared\_buffers](#shared_buffers) for information about HAWQ server instance shared memory buffers.
@@ -2828,7 +2828,7 @@ Specifies the order in which schemas are searched when an object is referenced b
## <a name="seg_max_connections"></a>seg\_max\_connections
-The maximum number of concurrent connections on a segment. In a HAWQ system, user client connections go through the HAWQ master instance only. Segment instances should allow 5-10 times the amount of connections allowed on the master (see [max\_connections](#max_connections).) When you increase this parameter, you must increase [max\_prepared\_transactions](#max_prepared_transactions) as well. For more information about limiting concurrent connections, see [Configuring Client Authentication](/20/clientaccess/client_auth.html).
+The maximum number of concurrent connections on a segment. In a HAWQ system, user client connections go through the HAWQ master instance only. Segment instances should allow 5-10 times the amount of connections allowed on the master (see [max\_connections](#max_connections).) When you increase this parameter, you must increase [max\_prepared\_transactions](#max_prepared_transactions) as well. For more information about limiting concurrent connections, see [Configuring Client Authentication](../../clientaccess/client_auth.html).
Increasing this parameter may cause HAWQ to request more shared memory. See [shared\_buffers](#shared_buffers) for information about HAWQ server instance shared memory buffers.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/459e3bc7/reference/sql/ALTER-RESOURCE-QUEUE.html.md.erb
----------------------------------------------------------------------
diff --git a/reference/sql/ALTER-RESOURCE-QUEUE.html.md.erb b/reference/sql/ALTER-RESOURCE-QUEUE.html.md.erb
index e1a31db..ec051e8 100644
--- a/reference/sql/ALTER-RESOURCE-QUEUE.html.md.erb
+++ b/reference/sql/ALTER-RESOURCE-QUEUE.html.md.erb
@@ -40,9 +40,9 @@ When modifying the resource queue, use the MEMORY\_LIMIT\_CLUSTER and CORE\_LIMI
To modify the role associated with the resource queue, use the [ALTER ROLE](ALTER-ROLE.html) or [CREATE ROLE](CREATE-ROLE.html) command. You can only assign roles to the leaf-level resource queues (resource queues that do not have any children.)
-The default memory allotment can be overridden on a per-query basis by using `hawq_rm_stmt_vseg_memory` and` hawq_rm_stmt_nvseg` configuration parameters. See [Configuring Resource Quotas for Query Statements](/20/resourcemgmt/ConfigureResourceManagement.html#topic_g2p_zdq_15).
+The default memory allotment can be overridden on a per-query basis by using `hawq_rm_stmt_vseg_memory` and` hawq_rm_stmt_nvseg` configuration parameters. See [Configuring Resource Quotas for Query Statements](../../resourcemgmt/ConfigureResourceManagement.html#topic_g2p_zdq_15).
-To see the status of a resource queue, see [Checking Existing Resource Queues](/20/resourcemgmt/ResourceQueues.html#topic_lqy_gls_zt).
+To see the status of a resource queue, see [Checking Existing Resource Queues](../../resourcemgmt/ResourceQueues.html#topic_lqy_gls_zt).
See also [Best Practices for Using Resource Queues](../../bestpractices/managing_resources_bestpractices.html#topic_hvd_pls_wv).
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/459e3bc7/reference/sql/CREATE-RESOURCE-QUEUE.html.md.erb
----------------------------------------------------------------------
diff --git a/reference/sql/CREATE-RESOURCE-QUEUE.html.md.erb b/reference/sql/CREATE-RESOURCE-QUEUE.html.md.erb
index 8fa6e89..8f9fe93 100644
--- a/reference/sql/CREATE-RESOURCE-QUEUE.html.md.erb
+++ b/reference/sql/CREATE-RESOURCE-QUEUE.html.md.erb
@@ -108,11 +108,11 @@ By default, both limits are set to **-1**, which means the limits are disabled.
## <a id="topic1__section5"></a>Notes
-To check on the configuration of a resource queue, you can query the `pg_resqueue` catalog table. To see the runtime status of all resource queues, you can use the `pg_resqueue_status`. See [Checking Existing Resource Queues](/20/resourcemgmt/ResourceQueues.html#topic_lqy_gls_zt).
+To check on the configuration of a resource queue, you can query the `pg_resqueue` catalog table. To see the runtime status of all resource queues, you can use the `pg_resqueue_status`. See [Checking Existing Resource Queues](../../resourcemgmt/ResourceQueues.html#topic_lqy_gls_zt).
`CREATE RESOURCE QUEUE` cannot be run within a transaction.
-To see the status of a resource queue, see [Checking Existing Resource Queues](/20/resourcemgmt/ResourceQueues.html#topic_lqy_gls_zt).
+To see the status of a resource queue, see [Checking Existing Resource Queues](../../resourcemgmt/ResourceQueues.html#topic_lqy_gls_zt).
## <a id="topic1__section6"></a>Examples
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/459e3bc7/resourcemgmt/ConfigureResourceManagement.html.md.erb
----------------------------------------------------------------------
diff --git a/resourcemgmt/ConfigureResourceManagement.html.md.erb b/resourcemgmt/ConfigureResourceManagement.html.md.erb
index cc17e06..23fe860 100644
--- a/resourcemgmt/ConfigureResourceManagement.html.md.erb
+++ b/resourcemgmt/ConfigureResourceManagement.html.md.erb
@@ -81,8 +81,8 @@ In some cases, you may want to specify additional resource quotas on the query s
The following configuration properties allow a user to control resource quotas without altering corresponding resource queues.
-- [hawq\_rm\_stmt\_vseg\_memory](/20/reference/guc/parameter_definitions.html)
-- [hawq\_rm\_stmt\_nvseg](/20/reference/guc/parameter_definitions.html)
+- [hawq\_rm\_stmt\_vseg\_memory](../reference/guc/parameter_definitions.html)
+- [hawq\_rm\_stmt\_nvseg](../reference/guc/parameter_definitions.html)
However, the changed resource quota for the virtual segment cannot exceed the resource queue\u2019s maximum capacity in HAWQ.
@@ -117,4 +117,4 @@ To alleviate the load on NameNode, you can limit V, the number of virtual segmen
- `hawq_rm_nvseg_perquery_limit` limits the maximum number of virtual segments that can be used for one statement execution on a cluster-wide level. The hash buckets defined in `default_hash_table_bucket_number` cannot exceed this number. The default value is 512.
- `default_hash_table_bucket_number` defines the number of buckets used by default when you create a hash table. When you query a hash table, the query's virtual segment resources are fixed and allocated based on the bucket number defined for the table. A best practice is to tune this configuration parameter after you expand the cluster.
-You can also limit the number of virtual segments used by queries when configuring your resource queues. \(See [CREATE RESOURCE QUEUE](/20/reference/sql/CREATE-RESOURCE-QUEUE.html).\) The global configuration parameters are a hard limit, however, and any limits set on the resource queue or on the statement-level cannot be larger than these limits set on the cluster-wide level.
+You can also limit the number of virtual segments used by queries when configuring your resource queues. \(See [CREATE RESOURCE QUEUE](../reference/sql/CREATE-RESOURCE-QUEUE.html).\) The global configuration parameters are a hard limit, however, and any limits set on the resource queue or on the statement-level cannot be larger than these limits set on the cluster-wide level.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/459e3bc7/resourcemgmt/HAWQResourceManagement.html.md.erb
----------------------------------------------------------------------
diff --git a/resourcemgmt/HAWQResourceManagement.html.md.erb b/resourcemgmt/HAWQResourceManagement.html.md.erb
index 095bc9d..dd5c9b3 100644
--- a/resourcemgmt/HAWQResourceManagement.html.md.erb
+++ b/resourcemgmt/HAWQResourceManagement.html.md.erb
@@ -66,4 +66,4 @@ Resource manager adjusts segment localhost original resource capacity from (8192
Resource manager adjusts segment localhost original global resource manager resource capacity from (8192 MB, 5 CORE) to (5120 MB, 5 CORE)
```
-See [Viewing the Database Server Log Files](/20/admin/monitor.html#topic28) for more information on working with HAWQ log files.
+See [Viewing the Database Server Log Files](../admin/monitor.html#topic28) for more information on working with HAWQ log files.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/459e3bc7/resourcemgmt/ResourceQueues.html.md.erb
----------------------------------------------------------------------
diff --git a/resourcemgmt/ResourceQueues.html.md.erb b/resourcemgmt/ResourceQueues.html.md.erb
index ab71547..2c9ea48 100644
--- a/resourcemgmt/ResourceQueues.html.md.erb
+++ b/resourcemgmt/ResourceQueues.html.md.erb
@@ -30,7 +30,7 @@ The HAWQ resource manager follows several principles when allocating resources t
**Enforcing Limits on Resources**
-You can configure HAWQ to enforce limits on resource usage by setting memory and CPU usage limits on both segments and resource queues. See [Configuring Segment Resource Capacity](ConfigureResourceManagement.html) and [Creating Resource Queues](ResourceQueues.html). For some best practices on designing and using resource queues in HAWQ, see [Best Practices for Managing Resources](/20/bestpractices/managing_resources_bestpractices.html).
+You can configure HAWQ to enforce limits on resource usage by setting memory and CPU usage limits on both segments and resource queues. See [Configuring Segment Resource Capacity](ConfigureResourceManagement.html) and [Creating Resource Queues](ResourceQueues.html). For some best practices on designing and using resource queues in HAWQ, see [Best Practices for Managing Resources](../bestpractices/managing_resources_bestpractices.html).
For a high-level overview of how resource management works in HAWQ, see [Managing Resources](HAWQResourceManagement.html).
@@ -68,7 +68,7 @@ postgres=# show hawq_rm_nresqueue_limit;
Use CREATE RESOURCE QUEUE to create a new resource queue. Only a superuser can run this DDL statement.
-Creating a resource queue involves giving it a name, a parent, setting the CPU and memory limits for the queue, and optionally a limit to the number of active statements on the resource queue. See [CREATE RESOURCE QUEUE](/20/reference/sql/CREATE-RESOURCE-QUEUE.html).
+Creating a resource queue involves giving it a name, a parent, setting the CPU and memory limits for the queue, and optionally a limit to the number of active statements on the resource queue. See [CREATE RESOURCE QUEUE](../reference/sql/CREATE-RESOURCE-QUEUE.html).
**Note:** You can only associate roles and queries with leaf-level resource queues. Leaf-level resource queues are resource queues that do not have any children.
@@ -100,7 +100,7 @@ However, when you alter a resource queue, queued resource requests may encounter
To prevent conflicts, HAWQ cancels by default all resource requests that are in conflict with the new resource queue definition. This behavior is controlled by the `hawq_rm_force_alterqueue_cancel_queued_request` server configuration parameter, which is by default set to true \(`on`\). If you set the server configuration parameter `hawq_rm_force_alterqueue_cancel_queued_request` to false, the actions specified in ALTER RESOURCE QUEUE are canceled if the resource manager finds at least one resource request that is in conflict with the new resource definitions supplied in the altering command.
-For more information, see [ALTER RESOURCE QUEUE](/20/reference/sql/ALTER-RESOURCE-QUEUE.html).
+For more information, see [ALTER RESOURCE QUEUE](../reference/sql/ALTER-RESOURCE-QUEUE.html).
**Note:** To change the roles \(users\) assigned to a resource queue, use the ALTER ROLE command.
@@ -159,7 +159,7 @@ FROM pg_resqueue WHERE rsqname='test_queue_1';
test_queue_1 | 9800 | 100 | 50% | 50% | 2 | even | mem:128mb | 0 | 0 | 0 |1
```
-The query displays all the attributes and their values of the selected resource queue. See [CREATE RESOURCE QUEUE](/20/reference/sql/CREATE-RESOURCE-QUEUE.html) for a description of these attributes.
+The query displays all the attributes and their values of the selected resource queue. See [CREATE RESOURCE QUEUE](../reference/sql/CREATE-RESOURCE-QUEUE.html) for a description of these attributes.
You can also check the runtime status of existing resource queues by querying the `pg_resqueue_status` view:
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/459e3bc7/resourcemgmt/YARNIntegration.html.md.erb
----------------------------------------------------------------------
diff --git a/resourcemgmt/YARNIntegration.html.md.erb b/resourcemgmt/YARNIntegration.html.md.erb
index 790c099..6898f6c 100644
--- a/resourcemgmt/YARNIntegration.html.md.erb
+++ b/resourcemgmt/YARNIntegration.html.md.erb
@@ -143,7 +143,7 @@ However, if you had set `yarn.scheduler.minimum-allocation-mb` to 4GB, then it w
**Note:** If you are specifying 1GB or under for `yarn.scheduler.minimum-allocation-mb` in `yarn-site.xml`, then make sure that the property is an equal subdivision of 1GB. For example, 1024, 512.
-See [Handling Segment Resource Fragmentation](/20/troubleshooting/Troubleshooting.html) for general information on resource fragmentation.
+See [Handling Segment Resource Fragmentation](../troubleshooting/Troubleshooting.html) for general information on resource fragmentation.
## <a id="topic_rtd_cjh_15"></a>Enabling YARN Mode in HAWQ
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/459e3bc7/resourcemgmt/best-practices.html.md.erb
----------------------------------------------------------------------
diff --git a/resourcemgmt/best-practices.html.md.erb b/resourcemgmt/best-practices.html.md.erb
index db69871..74bd815 100644
--- a/resourcemgmt/best-practices.html.md.erb
+++ b/resourcemgmt/best-practices.html.md.erb
@@ -6,10 +6,10 @@ When configuring resource management, you can apply certain best practices to en
The following is a list of high-level best practices for optimal resource management:
-- Make sure segments do not have identical IP addresses. See [Segments Do Not Appear in gp\_segment\_configuration](/20/troubleshooting/Troubleshooting.html) for an explanation of this problem.
+- Make sure segments do not have identical IP addresses. See [Segments Do Not Appear in gp\_segment\_configuration](../troubleshooting/Troubleshooting.html) for an explanation of this problem.
- Configure all segments to have the same resource capacity. See [Configuring Segment Resource Capacity](ConfigureResourceManagement.html).
- To prevent resource fragmentation, ensure that your deployment's segment resource capacity \(standalone mode\) or YARN node resource capacity \(YARN mode\) is a multiple of all virtual segment resource quotas. See [Configuring Segment Resource Capacity](ConfigureResourceManagement.html) \(HAWQ standalone mode\) and [Setting HAWQ Segment Resource Capacity in YARN](YARNIntegration.html).
-- Ensure that enough registered segments are available and usable for query resource requests. If the number of unavailable or unregistered segments is higher than a set limit, then query resource requests are rejected. Also ensure that the variance of dispatched virtual segments across physical segments is not greater than the configured limit. See [Rejection of Query Resource Requests](/20/troubleshooting/Troubleshooting.html).
+- Ensure that enough registered segments are available and usable for query resource requests. If the number of unavailable or unregistered segments is higher than a set limit, then query resource requests are rejected. Also ensure that the variance of dispatched virtual segments across physical segments is not greater than the configured limit. See [Rejection of Query Resource Requests](../troubleshooting/Troubleshooting.html).
- Use multiple master and segment temporary directories on separate, large disks (2TB or greater) to load balance writes to temporary files (for example, `/disk1/tmp /disk2/tmp`). For a given query, HAWQ will use a separate temp directory (if available) for each virtual segment to store spill files. Multiple HAWQ sessions will also use separate temp directories where available to avoid disk contention. If you configure too few temp directories, or you place multiple temp directories on the same disk, you increase the risk of disk contention or running out of disk space when multiple virtual segments target the same disk.
- Configure minimum resource levels in YARN, and tune the timeout of when idle resources are returned to YARN. See [Tune HAWQ Resource Negotiations with YARN](YARNIntegration.html).
- Make sure that the property `yarn.scheduler.minimum-allocation-mb` in `yarn-site.xml` is an equal subdivision of 1GB. For example, 1024, 512. See [Setting HAWQ Segment Resource Capacity in YARN](YARNIntegration.html#topic_pzf_kqn_c5).
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/459e3bc7/troubleshooting/Troubleshooting.html.md.erb
----------------------------------------------------------------------
diff --git a/troubleshooting/Troubleshooting.html.md.erb b/troubleshooting/Troubleshooting.html.md.erb
index 3589ce2..2b7414b 100644
--- a/troubleshooting/Troubleshooting.html.md.erb
+++ b/troubleshooting/Troubleshooting.html.md.erb
@@ -21,12 +21,12 @@ A query is not executing as quickly as you would expect. Here is how to investig
2. Are there many failed disks?
2. Check table statistics. Have the tables involved in the query been analyzed?
-3. Check the plan of the query and run [`EXPLAIN ANALYZE`](/20/reference/sql/EXPLAIN.html) to determine the bottleneck.
+3. Check the plan of the query and run [`EXPLAIN ANALYZE`](../reference/sql/EXPLAIN.html) to determine the bottleneck.
Sometimes, there is not enough memory for some operators, such as Hash Join, or spill files are used. If an operator cannot perform all of its work in the memory allocated to it, it caches data on disk in *spill files*. Compared with no spill files, a query will run much slower.
-4. Check data locality statistics using [`EXPLAIN ANALYZE`](/20/reference/sql/EXPLAIN.html). Alternately you can check the logs. Data locality result for every query could also be found in the log of HAWQ. See [Data Locality Statistics](../query/query-performance.html#topic_amk_drc_d5) for information on the statistics.
-5. Check resource queue status. You can query view `pg_resqueue_status` to check if the target queue has already dispatched some resource to the queries, or if the target queue is lacking resources. See [Checking Existing Resource Queues](/20/resourcemgmt/ResourceQueues.html#topic_lqy_gls_zt).
-6. Analyze a dump of the resource manager's status to see more resource queue status. See [Analyzing Resource Manager Status](/20/resourcemgmt/ResourceQueues.html#topic_zrh_pkc_f5).
+4. Check data locality statistics using [`EXPLAIN ANALYZE`](../reference/sql/EXPLAIN.html). Alternately you can check the logs. Data locality result for every query could also be found in the log of HAWQ. See [Data Locality Statistics](../query/query-performance.html#topic_amk_drc_d5) for information on the statistics.
+5. Check resource queue status. You can query view `pg_resqueue_status` to check if the target queue has already dispatched some resource to the queries, or if the target queue is lacking resources. See [Checking Existing Resource Queues](../resourcemgmt/ResourceQueues.html#topic_lqy_gls_zt).
+6. Analyze a dump of the resource manager's status to see more resource queue status. See [Analyzing Resource Manager Status](../resourcemgmt/ResourceQueues.html#topic_zrh_pkc_f5).
## <a id="topic_vm5_znx_15"></a>Rejection of Query Resource Requests
@@ -94,7 +94,7 @@ Different HAWQ resource queues can have different virtual segment resource quota
In standalone mode, the segment resources are all exclusively occupied by HAWQ. Resource fragmentation can occur when segment capacity is not a multiple of a virtual segment resource quota. For example, a segment has 15GB memory capacity, but the virtual segment resource quota is set to 2GB. The maximum possible memory consumption in a segment is 14GB. Therefore, you should configure segment resource capacity as a multiple of all virtual segment resource quotas.
-In YARN mode, resources are allocated from the YARN resource manager. The HAWQ resource manager acquires a YARN container by 1 vcore. For example, if YARN reports that a segment having 64GB memory and 16 vcore is configured for YARN applications, HAWQ requests YARN containers by 4GB memory and 1 vcore. In this manner, HAWQ resource manager acquires YARN containers on demand. If the capacity of the YARN container is not a multiple of the virtual segment resource quota, resource fragmentation may occur. For example, if the YARN container resource capacity is 3GB memory 1 vcore, one segment may have 1 or 3 YARN containers for HAWQ query execution. In this situation, if the virtual segment resource quota is 2GB memory, then HAWQ will always have 1 GB memory that cannot be utilized. Therefore, it is recommended to configure YARN node resource capacity carefully to make YARN container resource quota as a multiple of all virtual segment resource quotas. In addition, make sure your CPU to m
emory ratio is a multiple of the amount configured for `yarn.scheduler.minimum-allocation-mb`. See [Setting HAWQ Segment Resource Capacity in YARN](/20/resourcemgmt/YARNIntegration.html#topic_pzf_kqn_c5) for more information.
+In YARN mode, resources are allocated from the YARN resource manager. The HAWQ resource manager acquires a YARN container by 1 vcore. For example, if YARN reports that a segment having 64GB memory and 16 vcore is configured for YARN applications, HAWQ requests YARN containers by 4GB memory and 1 vcore. In this manner, HAWQ resource manager acquires YARN containers on demand. If the capacity of the YARN container is not a multiple of the virtual segment resource quota, resource fragmentation may occur. For example, if the YARN container resource capacity is 3GB memory 1 vcore, one segment may have 1 or 3 YARN containers for HAWQ query execution. In this situation, if the virtual segment resource quota is 2GB memory, then HAWQ will always have 1 GB memory that cannot be utilized. Therefore, it is recommended to configure YARN node resource capacity carefully to make YARN container resource quota as a multiple of all virtual segment resource quotas. In addition, make sure your CPU to m
emory ratio is a multiple of the amount configured for `yarn.scheduler.minimum-allocation-mb`. See [Setting HAWQ Segment Resource Capacity in YARN](../resourcemgmt/YARNIntegration.html#topic_pzf_kqn_c5) for more information.
If resource fragmentation occurs, queued requests are not processed until either some running queries return resources or the global resource manager provides more resources. If you encounter resource fragmentation, you should double check the configured capacities of the resource queues for any errors. For example, an error might be that the global resource manager container's memory to core ratio is not a multiple of virtual segment resource quota.
[17/50] [abbrv] incubator-hawq-docs git commit: clarify use of hawq
check --hadoop option
Posted by yo...@apache.org.
clarify use of hawq check --hadoop option
Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/commit/0f642f1c
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/tree/0f642f1c
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/diff/0f642f1c
Branch: refs/heads/master
Commit: 0f642f1ce67bd570d2043174bbdaed990c7840bb
Parents: 1661c62
Author: Lisa Owen <lo...@pivotal.io>
Authored: Wed Sep 14 14:11:38 2016 -0700
Committer: Lisa Owen <lo...@pivotal.io>
Committed: Wed Sep 14 14:11:38 2016 -0700
----------------------------------------------------------------------
.../cli/admin_utilities/hawqcheck.html.md.erb | 18 +++++++++---------
1 file changed, 9 insertions(+), 9 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/0f642f1c/reference/cli/admin_utilities/hawqcheck.html.md.erb
----------------------------------------------------------------------
diff --git a/reference/cli/admin_utilities/hawqcheck.html.md.erb b/reference/cli/admin_utilities/hawqcheck.html.md.erb
index 7163218..517004d 100644
--- a/reference/cli/admin_utilities/hawqcheck.html.md.erb
+++ b/reference/cli/admin_utilities/hawqcheck.html.md.erb
@@ -25,9 +25,9 @@ hawq check -?
## <a id="topic1__section3"></a>Description
-The `hawq check` utility determines the platform on which you are running HAWQ and validates various platform-specific configuration settings as well as HAWQ and HDFS-specific configuration settings. In order to perform HAWQ configuration checks, make sure HAWQ has been already started and `hawq config` works. For HDFS checks, you should either set the HADOOP\_HOME environment variable or give the hadoop installation location using `--hadoop` option.
+The `hawq check` utility determines the platform on which you are running HAWQ and validates various platform-specific configuration settings as well as HAWQ and HDFS-specific configuration settings. In order to perform HAWQ configuration checks, make sure HAWQ has been already started and `hawq config` works. For HDFS checks, you should either set the HADOOP\_HOME environment variable or provide the hadoop installation location using `--hadoop` option.
-The `hawq check` utility can use a host file or a file previously created with the `--zipout `option to validate platform settings. If `GPCHECK_ERROR` displays, one or more validation checks failed. You can also use `hawq check` to gather and view platform settings on hosts without running validation checks. When running checks, `hawq check` compares your actual configuration setting with an expected value listed in a config file (`$GPHOME/etc/hawq check.cnf` by default). You must modify your configuration values for "mount.points" and "diskusage.monitor.mounts" to reflect the actual mount points you want to check, as a comma-separated list. Otherwise, the utility only checks the root directory, which may not be helpful.
+The `hawq check` utility can use a host file or a file previously created with the `--zipout `option to validate platform settings. If `GPCHECK_ERROR` displays, one or more validation checks failed. You can also use `hawq check` to gather and view platform settings on hosts without running validation checks. When running checks, `hawq check` compares your actual configuration setting with an expected value listed in a config file (`$GPHOME/etc/hawq_check.cnf` by default). You must modify your configuration values for "mount.points" and "diskusage.monitor.mounts" to reflect the actual mount points you want to check, as a comma-separated list. Otherwise, the utility only checks the root directory, which may not be helpful.
An example is shown below:
@@ -54,7 +54,7 @@ diskusage.monitor.mounts = /,/data1,/data2
<dd>The name of a configuration file to use instead of the default file `$GPHOME/etc/hawq_check.cnf`.</dd>
<dt>-\\\-hadoop, -\\\-hadoop-home \<hadoop\_home\> </dt>
-<dd>Use this option to specify your hadoop installation location so that `hawq check` can validate HDFS settings. This option is not needed if `HADOOP_HOME` environment variable is set.</dd>
+<dd>Use this option to specify the full path to your hadoop installation location so that `hawq check` can validate HDFS settings. This option is not needed if the `HADOOP_HOME` environment variable is set.</dd>
<dt>-\\\-stdout </dt>
<dd>Send collected host information from `hawq check` to standard output. No checks or validations are performed.</dd>
@@ -82,25 +82,25 @@ diskusage.monitor.mounts = /,/data1,/data2
## <a id="topic1__section5"></a>Examples
-Verify and validate the HAWQ platform settings by entering a host file and specifying the hadoop location:
+Verify and validate the HAWQ platform settings by entering a host file and specifying the full hadoop install path:
``` shell
-$ hawq check -f hostfile_hawq_check --hadoop ~/hadoop-2.0.0/
+$ hawq check -f hostfile_hawq_check --hadoop /usr/hdp/version/hadoop
```
Verify and validate the HAWQ platform settings with HDFS HA enabled, YARN HA enabled and Kerberos enabled:
``` shell
-$ hawq check -f hostfile_hawq_check --hadoop ~/hadoop-2.0.0/ --hdfs-ha --yarn-ha --kerberos
+$ hawq check -f hostfile_hawq_check --hadoop /usr/hdp/version/hadoop --hdfs-ha --yarn-ha --kerberos
```
Verify and validate the HAWQ platform settings with HDFS HA enabled, and Kerberos enabled:
``` shell
-$ hawq check -f hostfile_hawq_check --hadoop ~/hadoop-2.0.0/ --hdfs-ha --kerberos
+$ hawq check -f hostfile_hawq_check --hadoop /usr/hdp/version/hadoop --hdfs-ha --kerberos
```
-Save HAWQ platform settings to a zip file, when HADOOP\_HOME environment variable is set:
+Save HAWQ platform settings to a zip file, when the $HADOOP\_HOME environment variable is set:
``` shell
$ hawq check -f hostfile_hawq_check --zipout
@@ -115,7 +115,7 @@ $ hawq check --zipin hawq_check_timestamp.tar.gz
View collected HAWQ platform settings:
``` shell
-$ hawq check -f hostfile_hawq_check --hadoop ~/hadoop-2.0.0/ --stdout
+$ hawq check -f hostfile_hawq_check --hadoop /usr/hdp/version/hadoop --stdout
```
## <a id="topic1__section6"></a>See Also
[29/50] [abbrv] incubator-hawq-docs git commit: Merge branch
'feature/client-loadtools' of
https://github.com/lisakowen/incubator-hawq-docs into develop
Posted by yo...@apache.org.
Merge branch 'feature/client-loadtools' of https://github.com/lisakowen/incubator-hawq-docs into develop
Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/commit/d64c0f91
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/tree/d64c0f91
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/diff/d64c0f91
Branch: refs/heads/master
Commit: d64c0f917e77b3eb95491fa10feb6df22a071255
Parents: c617b1c e429aaa
Author: David Yozie <yo...@apache.org>
Authored: Tue Sep 20 14:21:43 2016 -0700
Committer: David Yozie <yo...@apache.org>
Committed: Tue Sep 20 14:21:43 2016 -0700
----------------------------------------------------------------------
datamgmt/load/client-loadtools.html.md.erb | 58 +++++++++++---------
.../cli/admin_utilities/gpfdist.html.md.erb | 2 +-
2 files changed, 34 insertions(+), 26 deletions(-)
----------------------------------------------------------------------
[08/50] [abbrv] incubator-hawq-docs git commit: document hawq
ssh-exkeys -p option
Posted by yo...@apache.org.
document hawq ssh-exkeys -p option
Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/commit/9185772c
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/tree/9185772c
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/diff/9185772c
Branch: refs/heads/master
Commit: 9185772c59477d4c001851399aa3189eff3d5f81
Parents: fa5ed53
Author: Lisa Owen <lo...@pivotal.io>
Authored: Thu Sep 8 08:58:53 2016 -0700
Committer: Lisa Owen <lo...@pivotal.io>
Committed: Thu Sep 8 08:58:53 2016 -0700
----------------------------------------------------------------------
reference/cli/admin_utilities/hawqssh-exkeys.html.md.erb | 7 +++++--
1 file changed, 5 insertions(+), 2 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/9185772c/reference/cli/admin_utilities/hawqssh-exkeys.html.md.erb
----------------------------------------------------------------------
diff --git a/reference/cli/admin_utilities/hawqssh-exkeys.html.md.erb b/reference/cli/admin_utilities/hawqssh-exkeys.html.md.erb
index aabd81f..a75f813 100644
--- a/reference/cli/admin_utilities/hawqssh-exkeys.html.md.erb
+++ b/reference/cli/admin_utilities/hawqssh-exkeys.html.md.erb
@@ -7,9 +7,9 @@ Exchanges SSH public keys between hosts.
## <a id="topic1__section2"></a>Synopsis
``` pre
-hawq ssh-exkeys -f <hostfile_exkeys> | - h <hostname> [-h <hostname> ...]
+hawq ssh-exkeys -f <hostfile_exkeys> | - h <hostname> [-h <hostname> ...] [-p <password>]
-hawq ssh-exkeys -e <hostfile_exkeys> -x <hostfile_hawqexpand>
+hawq ssh-exkeys -e <hostfile_exkeys> -x <hostfile_hawqexpand> [-p <password>]
hawq ssh-exkeys --version
@@ -45,6 +45,9 @@ The `hawq ssh-exkeys` utility performs key exchange using the following steps:
<dt>-h \<hostname\> </dt>
<dd>Specifies a single host name (or host address) that will participate in the SSH key exchange. You can use the `-h` option multiple times to specify multiple host names and host addresses.</dd>
+<dt>-p \<password\> </dt>
+<dd>Specifies the password used to log in to the hosts. The hosts should share the same password.</dd>
+
<dt>-\\\-version </dt>
<dd>Displays the version of this utility.</dd>
[03/50] [abbrv] incubator-hawq-docs git commit: removing hawq-book
directory from this branch [HAWQ-1027]
Posted by yo...@apache.org.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/ecc6de22/hawq-book/master_middleman/source/subnavs/apache-hawq-nav.erb
----------------------------------------------------------------------
diff --git a/hawq-book/master_middleman/source/subnavs/apache-hawq-nav.erb b/hawq-book/master_middleman/source/subnavs/apache-hawq-nav.erb
deleted file mode 100644
index 29ec8c2..0000000
--- a/hawq-book/master_middleman/source/subnavs/apache-hawq-nav.erb
+++ /dev/null
@@ -1,955 +0,0 @@
-<div id="sub-nav" class="js-sidenav nav-container" role="navigation">
- <a class="sidenav-title" data-behavior="SubMenuMobile"> Doc Index</a>
- <div class="nav-content">
- <ul>
- <li>
- Apache HAWQ (incubating)
- </li>
- <li><a href="/20/requirements/system-requirements.html">System Requirements</a>
- </li>
-<!-- <li class="has_submenu">
- <span>
- Installing Apache HAWQ
- </span>
- <ul>
- <li>
- <a href="/20/install/select-hosts.html">Selecting HAWQ Host Machines</a>
- </li>
- <li>
- <a href="/20/install/install-ambari.html">Installing HAWQ Using Ambari</a>
- </li>
- <li>
- <a href="/20/install/install-cli.html">Installing HAWQ from the Command Line (Optional)</a>
- </li>
- <li>
- <a href="/20/install/aws-config.html">Amazon EC2 Configuration</a>
- </li>
- <li>
- <a href="/hdb/install/install_package_extensions.html">Installing Procedural Languages and Package Extensions for HAWQ</a>
- </li>
- <li>
- <a href="/hdb/install/install_pgcrypto.html">Installing Cryptographic Functions for PostgreSQL (pgcrypto)</a></li>
- <li>
- <a href="/hdb/install/install_pljava.html">Installing PL/Java</a>
- </li>
- <li>
- <a href="/hdb/install/install_plr.html">Installing PL/R</a>
- </li>
- </ul>
- </li>-->
- <li class="has_submenu">
- <span>
- HAWQ System Overview
- </span>
- <ul>
- <li>
- <a href="/20/overview/HAWQOverview.html">What is HAWQ?</a>
- </li>
- <li>
- <a href="/20/overview/HAWQArchitecture.html">HAWQ Architecture</a>
- </li>
- <li>
- <a href="/20/overview/TableDistributionStorage.html">Table Distribution and Storage</a>
- </li>
- <li>
- <a href="/20/overview/ElasticSegments.html">Elastic Query Execution Runtime</a>
- </li>
- <li>
- <a href="/20/overview/ResourceManagement.html">Resource Management</a>
- </li>
- <li>
- <a href="/20/overview/HDFSCatalogCache.html">HDFS Catalog Cache</a>
- </li>
- <li>
- <a href="/20/overview/ManagementTools.html">Management Tools</a>
- </li>
- <li>
- <a href="/20/overview/RedundancyFailover.html">High Availability, Redundancy and Fault Tolerance</a>
- </li>
- </ul>
- </li>
- <li class="has_submenu">
- <span>
- Running a HAWQ Cluster
- </span>
- <ul>
- <li>
- <a href="/20/admin/RunningHAWQ.html">Overview</a>
- </li>
- <li>
- <a href="/20/admin/ambari-admin.html">Managing HAWQ Using Ambari</a>
- </li>
- <li>
- <a href="/20/admin/startstop.html">Starting and Stopping HAWQ</a>
- </li>
- <li>
- <a href="/20/admin/ClusterExpansion.html">Expanding a Cluster</a>
- </li>
- <li>
- <a href="/20/admin/ClusterShrink.html">Removing a Node</a>
- </li>
- <li>
- <a href="/20/admin/BackingUpandRestoringHAWQDatabases.html">Backing Up and Restoring HAWQ</a>
- </li>
- <li>
- <a href="/20/admin/HighAvailability.html">High Availability in HAWQ</a>
- </li>
- <li>
- <a href="/20/admin/MasterMirroring.html">Master Mirroring</a>
- </li>
- <li>
- <a href="/20/admin/HAWQFilespacesandHighAvailabilityEnabledHDFS.html">HAWQ Filespaces and High Availability Enabled HDFS</a>
- </li>
- <li>
- <a href="/20/admin/FaultTolerance.html">Understanding the Fault Tolerance Service</a>
- </li>
- <li>
- <a href="/20/admin/RecommendedMonitoringTasks.html">Recommended Monitoring and Maintenance Tasks</a>
- </li>
- <li>
- <a href="/20/admin/maintain.html">Routine System Maintenance Tasks</a>
- </li>
- <li>
- <a href="/20/admin/monitor.html">Monitoring a HAWQ System</a>
- </li>
- </ul>
- </li>
- <li class="has_submenu">
- <span>
- Managing Resources
- </span>
- <ul>
- <li>
- <a href="/20/resourcemgmt/HAWQResourceManagement.html">How HAWQ Manages Resources</a>
- </li>
- <li>
- <a href="/20/resourcemgmt/best-practices.html">Best Practices for Configuring Resource Management</a>
- </li>
- <li>
- <a href="/20/resourcemgmt/ConfigureResourceManagement.html">Configuring Resource Management</a>
- </li>
- <li>
- <a href="/20/resourcemgmt/YARNIntegration.html">Integrating YARN with HAWQ</a>
- </li>
- <li>
- <a href="/20/resourcemgmt/ResourceQueues.html">Working with Hierarchical Resource Queues</a>
- </li>
- <li>
- <a href="/20/resourcemgmt/ResourceManagerStatus.html">Analyzing Resource Manager Status</a>
- </li>
- </ul>
- </li>
- <li class="has_submenu">
- <span>
- Managing Client Access
- </span>
- <ul>
- <li>
- <a href="/20/clientaccess/client_auth.html">Configuring Client Authentication</a>
- </li>
- <li>
- <a href="/20/clientaccess/ldap.html">Using LDAP Authentication with TLS/SSL</a>
- </li>
- <li>
- <a href="/20/clientaccess/kerberos.html">Using Kerberos Authentication</a>
- </li>
- <li>
- <a href="/20/clientaccess/disable-kerberos.html">Diabling Kerberos Security</a>
- </li>
- <li>
- <a href="/20/clientaccess/roles_privs.html">Managing Roles and Privileges</a>
- </li>
- <li>
- <a href="/20/clientaccess/g-establishing-a-database-session.html">Establishing a Database Session</a>
- </li>
- <li>
- <a href="/20/clientaccess/g-supported-client-applications.html">Supported Client Applications</a>
- </li>
- <li>
- <a href="/20/clientaccess/g-greenplum-database-client-applications.html">HAWQ Client Applications</a>
- </li>
- <li>
- <a href="/20/clientaccess/g-connecting-with-psql.html">Connecting with psql</a>
- </li>
- <li>
- <a href="/20/clientaccess/g-database-application-interfaces.html">ODBC/JDBC Application Interfaces</a>
- </li>
- <li>
- <a href="/20/clientaccess/g-troubleshooting-connection-problems.html">Troubleshooting Connection Problems</a>
- </li>
- </ul>
- </li>
- <li class="has_submenu">
- <span>
- Defining Database Objects
- </span>
- <ul>
- <li>
- <a href="/20/ddl/ddl.html">Overview</a>
- </li>
- <li>
- <a href="/20/ddl/ddl-database.html">Creating and Managing Databases</a>
- </li>
- <li>
- <a href="/20/ddl/ddl-tablespace.html">Creating and Managing Tablespaces</a>
- </li>
- <li>
- <a href="/20/ddl/ddl-schema.html">Creating and Managing Schemas</a>
- </li>
- <li>
- <a href="/20/ddl/ddl-table.html">Creating and Managing Tables</a>
- </li>
- <li>
- <a href="/20/ddl/ddl-storage.html">Choosing the Table Storage Model</a>
- </li>
- <li>
- <a href="/20/ddl/ddl-partition.html">Partitioning Large Tables</a>
- </li>
- <li>
- <a href="/20/ddl/ddl-view.html">Creating and Managing Views</a>
- </li>
- </ul>
- </li>
- <li class="has_submenu">
- <span>
- Using Procedural Languages
- </span>
- <ul>
- <li>
- <a href="/20/plext/UsingProceduralLanguages.html">Using Procedural Languages in HAWQ</a>
- </li>
- <li>
- <a href="/20/plext/using_pljava.html">Using PL/Java</a>
- </li>
- <li>
- <a href="/20/plext/using_plpgsql.html">Using PL/pgSQL</a>
- </li>
- <li>
- <a href="/20/plext/using_plpython.html">Using PL/Python</a>
- </li>
- <li>
- <a href="/20/plext/using_plr.html">Using PL/R</a>
- </li>
- </ul>
- </li>
- <li class="has_submenu"><a href="/20/datamgmt/dml.html">Managing Data</a>
- <ul>
- <li><a href="/20/datamgmt/BasicDataOperations.html">Basic Data Operations</a></li>
- <li><a href="/20/datamgmt/about_statistics.html">About Database Statistics</a></li>
- <li><a href="/20/datamgmt/ConcurrencyControl.html">Concurrency Control</a></li>
- <li><a href="/20/datamgmt/Transactions.html">Working with Transactions</a></li>
- <li class="has_submenu"><a href="/20/datamgmt/load/g-loading-and-unloading-data.html">Loading and Unloading Data</a>
- <ul>
- <li class="has_submenu"><a href="/20/datamgmt/load/g-working-with-file-based-ext-tables.html">Working with File-Based External Tables</a>
- <ul>
- <li><a href="/20/datamgmt/load/g-external-tables.html">Accessing File-Based External Tables</a></li>
- <li><a href="/20/datamgmt/load/g-gpfdist-protocol.html">gpfdist Protocol</a></li>
- <li><a href="/20/datamgmt/load/g-gpfdists-protocol.html">gpfdists Protocol</a></li>
- <li><a href="/20/datamgmt/load/g-handling-errors-ext-table-data.html">Handling Errors in External Table Data</a></li>
- </ul>
- </li>
- <li class="has_submenu"><a href="/20/datamgmt/load/g-using-the-greenplum-parallel-file-server--gpfdist-.html">Using the Greenplum Parallel File Server (gpfdist)</a>
- <ul>
- <li><a href="/20/datamgmt/load/g-about-gpfdist-setup-and-performance.html">About gpfdist Setup and Performance</a></li>
- <li><a href="/20/datamgmt/load/g-controlling-segment-parallelism.html">Controlling Segment Parallelism</a></li>
- <li><a href="/20/datamgmt/load/g-installing-gpfdist.html">Installing gpfdist</a></li>
- <li><a href="/20/datamgmt/load/g-starting-and-stopping-gpfdist.html">Starting and Stopping gpfdist</a></li>
- <li><a href="/20/datamgmt/load/g-troubleshooting-gpfdist.html">Troubleshooting gpfdist</a></li>
- </ul>
- </li>
- <li class="has_submenu"><a href="/20/datamgmt/load/g-creating-and-using-web-external-tables.html">Creating and Using Web External Tables</a>
- <ul>
- <li><a href="/20/datamgmt/load/g-command-based-web-external-tables.html">Command-based Web External Tables</a></li>
- <li><a href="/20/datamgmt/load/g-url-based-web-external-tables.html">URL-based Web External Tables</a></li>
- </ul>
- </li>
- <li><a href="/20/datamgmt/load/g-loading-data-using-an-external-table.html">Loading Data Using an External Table</a></li>
- <li class="has_submenu"><a href="/20/datamgmt/load/g-loading-and-writing-non-hdfs-custom-data.html">Loading and Writing Non-HDFS Custom Data</a>
- <ul>
- <li class="has_submenu"><a href="/20/datamgmt/load/g-using-a-custom-format.html">Using a Custom Format</a>
- <ul>
- <li><a href="/20/datamgmt/load/g-importing-and-exporting-fixed-width-data.html">Importing and Exporting Fixed Width Data</a></li>
- <li><a href="/20/datamgmt/load/g-examples-read-fixed-width-data.html">Examples - Read Fixed-Width Data</a></li>
- </ul>
- </li>
- </ul>
- </li>
- <li><a href="/20/datamgmt/load/creating-external-tables-examples.html">Creating External Tables - Examples</a>
- </li>
- <li class="has_submenu"><a href="/20/datamgmt/load/g-handling-load-errors.html">Handling Load Errors</a>
- <ul>
- <li><a href="/20/datamgmt/load/g-define-an-external-table-with-single-row-error-isolation.html">Define an External Table with Single Row Error Isolation</a></li>
- <li><a href="/20/datamgmt/load/g-create-an-error-table-and-declare-a-reject-limit.html">Capture Row Formatting Errors and Declare a Reject Limit </a></li>
- <li><a href="/20/datamgmt/load/g-identifying-invalid-csv-files-in-error-table-data.html">Identifying Invalid CSV Files in Error Table Data</a></li>
- <li><a href="/20/datamgmt/load/g-moving-data-between-tables.html">Moving Data between Tables</a></li>
- </ul>
- </li>
- <li><a href="/20/datamgmt/load/g-loading-data-with-hawqload.html">Loading Data with hawq load</a></li>
- <li><a href="/20/datamgmt/load/g-loading-data-with-copy.html">Loading Data with COPY</a></li>
- <li><a href="/20/datamgmt/load/g-running-copy-in-single-row-error-isolation-mode.html">Running COPY in Single Row Error Isolation Mode</a></li>
- <li><a href="/20/datamgmt/load/g-optimizing-data-load-and-query-performance.html">Optimizing Data Load and Query Performance</a></li>
- <li class="has_submenu"><a href="/20/datamgmt/load/g-unloading-data-from-greenplum-database.html">Unloading Data from HAWQ</a>
- <ul>
- <li class="has_submenu"><a href="/20/datamgmt/load/g-defining-a-file-based-writable-external-table.html">Defining a File-Based Writable External Table</a>
- <ul>
- <li><a href="/20/datamgmt/load/g-example-greenplum-file-server-gpfdist.html">Example - HAWQ file server (gpfdist)</a></li>
- </ul>
- </li>
- <li class="has_submenu"><a href="/20/datamgmt/load/g-defining-a-command-based-writable-external-web-table.html">Defining a Command-Based Writable External Web Table</a>
- <ul>
- <li><a href="/20/datamgmt/load/g-disabling-execute-for-web-or-writable-external-tables.html">Disabling EXECUTE for Web or Writable External Tables</a></li>
- </ul>
- </li>
- <li><a href="/20/datamgmt/load/g-unloading-data-using-a-writable-external-table.html">Unloading Data Using a Writable External Table</a></li>
- <li><a href="/20/datamgmt/load/g-unloading-data-using-copy.html">Unloading Data Using COPY</a></li>
- </ul>
- </li>
- <li class="has_submenu"><a href="/20/datamgmt/load/g-transforming-xml-data.html">Transforming XML Data</a>
- <ul>
- <li><a href="/20/datamgmt/load/g-determine-the-transformation-schema.html">Determine the Transformation Schema</a></li>
- <li><a href="/20/datamgmt/load/g-write-a-transform.html">Write a Transform</a></li>
- <li><a href="/20/datamgmt/load/g-write-the-gpfdist-configuration.html">Write the gpfdist Configuration</a></li>
- <li><a href="/20/datamgmt/load/g-load-the-data.html">Load the Data</a></li>
- <li class="has_submenu"><a href="/20/datamgmt/load/g-transfer-and-store-the-data.html">Transfer and Store the Data</a>
- <ul>
- <li><a href="/20/datamgmt/load/g-transforming-with-gpload.html">Transforming with GPLOAD</a></li>
- <li><a href="/20/datamgmt/load/g-transforming-with-insert-into-select-from.html">Transforming with INSERT INTO SELECT FROM</a></li>
- <li><a href="/20/datamgmt/load/g-configuration-file-format.html">Configuration File Format</a></li>
- </ul>
- </li>
- <li class="has_submenu"><a href="/20/datamgmt/load/g-xml-transformation-examples.html">XML Transformation Examples</a>
- <ul>
- <li><a href="/20/datamgmt/load/g-example-1-dblp-database-publications-in-demo-directory.html">Command-based Web External Tables</a></li>
- <li><a href="/20/datamgmt/load/g-example-irs-mef-xml-files-in-demo-directory.html">Example using IRS MeF XML Files (In demo Directory)</a></li>
- <li><a href="/20/datamgmt/load/g-example-witsml-files-in-demo-directory.html">Example using WITSML\u2122 Files (In demo Directory)</a></li>
- </ul>
- </li>
- </ul>
- </li>
- <li class="has_submenu"><a href="/20/datamgmt/load/g-formatting-data-files.html">Formatting Data Files</a>
- <ul>
- <li><a href="/20/datamgmt/load/g-formatting-rows.html">Formatting Rows</a></li>
- <li><a href="/20/datamgmt/load/g-formatting-columns.html">Formatting Columns</a></li>
- <li><a href="/20/datamgmt/load/g-representing-null-values.html">Representing NULL Values</a></li>
- <li class="has_submenu"><a href="/20/datamgmt/load/g-escaping.html">Escaping</a>
- <ul>
- <li><a href="/20/datamgmt/load/g-escaping-in-text-formatted-files.html">Escaping in Text Formatted Files</a></li>
- <li><a href="/20/datamgmt/load/g-escaping-in-csv-formatted-files.html">Escaping in CSV Formatted Files</a></li>
- </ul>
- </li>
- <li><a href="/20/datamgmt/load/g-character-encoding.html">Character Encoding</a></li>
- </ul>
- </li>
- </ul>
- </li>
- <li class="has_submenu"><a href="/20/pxf/HawqExtensionFrameworkPXF.html">Working with PXF and External Data</a>
- <ul>
- <li class="has_submenu"><a href="/20/pxf/InstallPXFPlugins.html">Installing PXF Plugins</a>
- <ul>
- <li><a href="/20/pxf/InstallPXFPlugins.html#topic_qsz_wq5_25">Installing the PXF HDFS Plugin</a></li>
- <li><a href="/20/pxf/InstallPXFPlugins.html#topic_s1y_1v5_25">Installing the PXF Hive Plugin</a></li>
- <li><a href="/20/pxf/InstallPXFPlugins.html#topic_std_cv5_25">Installing the PXF HBase Plugin</a></li>
- <li><a href="/20/pxf/InstallPXFPlugins.html#directories_and_logs">Installation and Log File Directories</a></li>
- </ul>
- </li>
- <li class="has_submenu"><a href="/20/pxf/ConfigurePXF.html">Configuring PXF</a>
- <ul>
- <li><a href="/20/pxf/ConfigurePXF.html#settingupthejavaclasspath">Setting up the Java Classpath</a></li>
- <li><a href="/20/pxf/ConfigurePXF.html#settingupthejvmcommandlineoptionsforpxfservice">Setting up the JVM Command Line Options for the PXF Service</a></li>
- <li><a href="/20/pxf/ConfigurePXF.html#topic_i3f_hvm_ss">Using PXF on a Secure HDFS Cluster</a></li>
- <li><a href="/20/pxf/ConfigurePXF.html#credentialsforremoteservices">Credentials for Remote Services</a></li>
- </ul>
- </li>
- <li class="has_submenu"><a href="/20/pxf/HDFSFileDataPXF.html">Accessing HDFS File Data</a>
- <ul>
- <li><a href="/20/pxf/HDFSFileDataPXF.html#installingthepxfhdfsplugin">Prerequisites</a></li>
- <li><a href="/20/pxf/HDFSFileDataPXF.html#syntax1">Syntax</a></li>
- <li><a href="/20/pxf/HDFSFileDataPXF.html#accessingdataonahighavailabilityhdfscluster">Accessing Data on a High Availability HDFS Cluster</a></li>
- <li><a href="/20/pxf/HDFSFileDataPXF.html#recordkeyinkey-valuefileformats">Using a Record Key with Key-Value File Formats</a></li>
- <li><a href="/20/pxf/HDFSFileDataPXF.html#topic_oy3_qwm_ss">Working with Avro Files</a></li>
- </ul>
- </li>
- <li class="has_submenu"><a href="/20/pxf/HivePXF.html">Accessing Hive Data</a>
- <ul>
- <li><a href="/20/pxf/HivePXF.html#installingthepxfhiveplugin">Prerequisites</a></li>
- <li><a href="/20/pxf/HivePXF.html#hivecommandline">Hive Command Line</a></li>
- <li class="has_submenu"><a href="/20/pxf/HivePXF.html#topic_p2s_lvl_25">Using PXF Tables to Query Hive</a>
- <ul>
- <li><a href="/20/pxf/HivePXF.html#syntax2">Syntax Example</a></li>
- <li><a href="/20/pxf/HivePXF.html#topic_b4v_g3n_25">Hive Complex Types</a></li>
- </ul>
- </li>
- <li class="has_submenu"><a href="/20/pxf/HivePXF.html#hcatalog">Using PXF and HCatalog to Query Hive</a>
- <ul>
- <li><a href="/20/pxf/HivePXF.html#topic_j1l_y55_c5">Usage</a></li>
- <li><a href="/20/pxf/HivePXF.html#topic_r5k_pst_25">Limitations</a></li>
- </ul>
- </li>
- <li><a href="/20/pxf/HivePXF.html#partitionfiltering">Partition Filtering</a></li>
- <li><a href="/20/pxf/HivePXF.html#topic_fdm_zrh_1s">Using PXF with Hive Default Partitions</a></li>
- <li><a href="/20/pxf/HivePXF.html#topic_dbb_nz3_ts">Accessing Hive Tables in Parquet Format</a></li>
- </ul>
- </li>
- <li class="has_submenu"><a href="/20/pxf/HBasePXF.html">Accessing HBase Data</a>
- <ul>
- <li><a href="/20/pxf/HBasePXF.html#installingthepxfhbaseplugin">Prerequisites</a></li>
- <li><a href="/20/pxf/HBasePXF.html#syntax3">Syntax</a></li>
- <li><a href="/20/pxf/HBasePXF.html#columnmapping">Column Mapping</a></li>
- <li><a href="/20/pxf/HBasePXF.html#rowkey">Row Key</a></li>
- <li><a href="/20/pxf/HBasePXF.html#directmapping">Direct Mapping</a></li>
- <li><a href="/20/pxf/HBasePXF.html#indirectmappingvialookuptable">Indirect Mapping (via Lookup Table)</a></li>
- </ul>
- </li>
- <li class="has_submenu"><a href="/20/pxf/ReadWritePXF.html">Using Profiles to Read and Write Data</a>
- <ul>
- <li><a href="/20/pxf/ReadWritePXF.html#built-inprofiles">Built-In Profiles</a></li>
- <li><a href="/20/pxf/ReadWritePXF.html#addingandupdatingprofiles">Adding and Updating Profiles</a></li>
- <li><a href="/20/pxf/ReadWritePXF.html#customprofileexample">Custom Profile Example</a></li>
- </ul>
- </li>
- <li class="has_submenu"><a href="/20/pxf/PXFExternalTableandAPIReference.html">PXF External Tables and API</a>
- <ul>
- <li><a href="/20/pxf/PXFExternalTableandAPIReference.html#creatinganexternaltable">Creating an External Table</a></li>
- <li class="has_submenu"><a href="/20/pxf/PXFExternalTableandAPIReference.html#aboutthejavaclassservicesandformats">About the Java Class Services and Formats</a>
- <ul>
- <li><a href="/20/pxf/PXFExternalTableandAPIReference.html#fragmenter">Fragmenter</a></li>
- <li><a href="/20/pxf/PXFExternalTableandAPIReference.html#accessor">Accessor</a></li>
- <li><a href="/20/pxf/PXFExternalTableandAPIReference.html#resolver">Resolver</a></li>
- </ul>
- </li>
- <li><a href="/20/pxf/PXFExternalTableandAPIReference.html#aboutcustomprofiles">About Custom Profiles</a></li>
- <li><a href="/20/pxf/PXFExternalTableandAPIReference.html#aboutqueryfilterpush-down">About Query Filter Push-Down</a></li>
- <li class="has_submenu"><a href="/20/pxf/PXFExternalTableandAPIReference.html#reference">Examples</a>
- <ul>
- <li><a href="/20/pxf/PXFExternalTableandAPIReference.html#externaltableexamples">External Table Examples</a></li>
- <li><a href="/20/pxf/PXFExternalTableandAPIReference.html#pluginexamples">Plugin Examples</a></li>
- </ul>
- </li>
- <li><a href="/20/pxf/PXFExternalTableandAPIReference.html#topic_b44_yw4_c5">Renamed Package Reference</a></li>
- </ul>
- </li>
- <li><a href="/20/pxf/TroubleshootingPXF.html">Troubleshooting PXF</a></li>
- </ul>
- </li>
- <li><a href="/20/datamgmt/HAWQInputFormatforMapReduce.html">HAWQ InputFormat for MapReduce</a></li>
- </ul>
- </li>
- <li class="has_submenu"><a href="/20/query/query.html">Querying Data</a>
- <ul>
- <li><a href="/20/query/HAWQQueryProcessing.html">About HAWQ Query Processing</a></li>
- <li class="has_submenu"><a href="/20/query/gporca/query-gporca-optimizer.html">About GPORCA</a>
- <ul>
- <li><a href="/20/query/gporca/query-gporca-overview.html">Overview of GPORCA</a></li>
- <li><a href="/20/query/gporca/query-gporca-features.html">GPORCA Features and Enhancements</a></li>
- <li><a href="/20/query/gporca/query-gporca-enable.html">Enabling GPORCA</a></li>
- <li><a href="/20/query/gporca/query-gporca-notes.html">Considerations when Using GPORCA</a></li>
- <li><a href="/20/query/gporca/query-gporca-fallback.html">Determining The Query Optimizer In Use</a></li>
- <li><a href="/20/query/gporca/query-gporca-changed.html">Changed Behavior with GPORCA</a></li>
- <li><a href="/20/query/gporca/query-gporca-limitations.html">GPORCA Limitations</a></li>
- </ul>
- </li>
- <li><a href="/20/query/defining-queries.html">Defining Queries</a></li>
- <li><a href="/20/query/functions-operators.html">Using Functions and Operators</a></li>
- <li><a href="/20/query/query-performance.html">Query Performance</a></li>
- <li><a href="/20/query/query-profiling.html">Query Profiling</a></li>
- </ul>
- </li>
- <li class="has_submenu"><a href="/20/bestpractices/HAWQBestPracticesOverview.html">Best Practices</a>
- <ul>
- <li><a href="/20/bestpractices/general_bestpractices.html">HAWQ Best Practices</a></li>
- <li><a href="/20/bestpractices/operating_hawq_bestpractices.html">Operating HAWQ</a></li>
- <li><a href="/20/bestpractices/secure_bestpractices.html">Securing HAWQ</a></li>
- <li><a href="/20/bestpractices/managing_resources_bestpractices.html">Managing Resources</a></li>
- <li><a href="/20/bestpractices/managing_data_bestpractices.html">Managing Data</a></li>
- <li><a href="/20/bestpractices/querying_data_bestpractices.html">Querying Data</a></li>
- </ul>
- </li>
- <li class="has_submenu"><a href="/20/troubleshooting/Troubleshooting.html">Troubleshooting</a>
- <ul>
- <li><a href="/20/troubleshooting/Troubleshooting.html#topic_dwd_rnx_15">Query Performance Issues</a></li>
- <li><a href="/20/troubleshooting/Troubleshooting.html#topic_vm5_znx_15">Rejection of Query Resource Requests</a></li>
- <li><a href="/20/troubleshooting/Troubleshooting.html#topic_qq4_rkl_wv">Queries Cancelled Due to High VMEM Usage</a></li>
- <li><a href="/20/troubleshooting/Troubleshooting.html#topic_hlj_zxx_15">Segments Do Not Appear in gp_segment_configuration</a></li>
- <li><a href="/20/troubleshooting/Troubleshooting.html#topic_mdz_q2y_15">Handling Segment Resource Fragmentation</a></li>
- </ul>
- </li>
- <li class="has_submenu"><a href="/20/reference/hawq-reference.html">HAWQ Reference</a>
- <ul>
- <li class="has_submenu"><a href="/20/reference/HAWQSiteConfig.html">Server Configuration Parameter Reference</a>
- <ul>
- <li><a href="/20/reference/guc/guc_config.html">About Server Configuration Parameters</a></li>
- <li class="has_submenu"><a href="/20/reference/guc/guc_category-list.html">Configuration Parameter Categories</a>
- <ul>
- <li><a href="/20/reference/guc/guc_category-list.html#topic_hfd_1tl_zp">Append-Only Table Parameters</a></li>
- <li><a href="/20/reference/guc/guc_category-list.html#topic39">Client Connection Default Parameters</a></li>
- <li><a href="/20/reference/guc/guc_category-list.html#topic12">Connection and Authentication Parameters</a></li>
- <li><a href="/20/reference/guc/guc_category-list.html#topic47">Database and Tablespace/Filespace Parameters</a></li>
- <li><a href="/20/reference/guc/guc_category-list.html#topic29">Error Reporting and Logging Parameters</a></li>
- <li><a href="/20/reference/guc/guc_category-list.html#topic45">External Table Parameters</a></li>
- <li><a href="/20/reference/guc/guc_category-list.html#topic57">GPORCA Parameters</a></li>
- <li><a href="/20/reference/guc/guc_category-list.html#topic49">HAWQ Array Configuration Parameters</a></li>
- <li><a href="/20/reference/guc/guc_category-list.html#topic56">HAWQ PL/Java Extension Parameters</a></li>
- <li><a href="/20/reference/guc/guc_category-list.html#hawq_resource_management">HAWQ Resource Management Parameters</a></li>
- <li><a href="/20/reference/guc/guc_category-list.html#topic43">Lock Management Parameters</a></li>
- <li><a href="/20/reference/guc/guc_category-list.html#topic48">Past PostgreSQL Version Compatibility Parameters</a></li>
- <li><a href="/20/reference/guc/guc_category-list.html#topic21">Query Tuning Parameters</a></li>
- <li><a href="/20/reference/guc/guc_category-list.html#statistics_collection">Statistics Collection Parameters</a></li>
- <li><a href="/20/reference/guc/guc_category-list.html#topic15">System Resource Consumption Parameters</a></li>
- </ul>
- </li>
- <li class="has_submenu"><a href="/20/reference/guc/parameter_definitions.html">Configuration Parameters</a>
- <ul>
- <li><a href="/20/reference/guc/parameter_definitions.html#add_missing_from">add_missing_from</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#application_name">application_name</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#array_nulls">array_nulls</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#authentication_timeout">authentication_timeout</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#backslash_quote">backslash_quote</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#block_size">block_size</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#bonjour_name">bonjour_name</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#check_function_bodies">check_function_bodies</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#client_encoding">client_encoding</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#client_min_messages">client_min_messages</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#cpu_index_tuple_cost">cpu_index_tuple_cost</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#cpu_operator_cost">cpu_operator_cost</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#cpu_tuple_cost">cpu_tuple_cost</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#cursor_tuple_fraction">cursor_tuple_fraction</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#custom_variable_classes">custom_variable_classes</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#DateStyle">DateStyle</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#db_user_namespace">db_user_namespace</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#deadlock_timeout">deadlock_timeout</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#debug_assertions">debug_assertions</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#debug_pretty_print">debug_pretty_print</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#debug_print_parse">debug_print_parse</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#debug_print_plan">debug_print_plan</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#debug_print_prelim_plan">debug_print_prelim_plan</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#debug_print_rewritten">debug_print_rewritten</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#debug_print_slice_table">debug_print_slice_table</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#topic_fqj_4fd_kv">default_hash_table_bucket_number</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#default_statistics_target">default_statistics_target</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#default_tablespace">default_tablespace</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#default_transaction_isolation">default_transaction_isolation</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#default_transaction_read_only">default_transaction_read_only</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#dfs_url">dfs_url</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#dynamic_library_path">dynamic_library_path</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#effective_cache_size">effective_cache_size</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#enable_bitmapscan">enable_bitmapscan</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#enable_groupagg">enable_groupagg</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#enable_hashagg">enable_hashagg</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#enable_hashjoin">enable_hashjoin</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#enable_indexscan">enable_indexscan</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#enable_mergejoin">enable_mergejoin</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#enable_nestloop">enable_nestloop</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#enable_seqscan">enable_seqscan</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#enable_sort">enable_sort</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#enable_tidscan">enable_tidscan</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#escape_string_warning">escape_string_warning</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#explain_pretty_print">explain_pretty_print</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#extra_float_digits">extra_float_digits</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#from_collapse_limit">from_collapse_limit</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#gp_adjust_selectivity_for_outerjoins">gp_adjust_selectivity_for_outerjoins</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#gp_analyze_relative_error">gp_analyze_relative_error</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#gp_autostats_mode">gp_autostats_mode</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#topic_imj_zhf_gw">gp_autostats_on_change_threshhold</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#gp_backup_directIO">gp_backup_directIO</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#gp_backup_directIO_read_chunk_mb">gp_backup_directIO_read_chunk_mb</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#gp_cached_segworkers_threshold">gp_cached_segworkers_threshold</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#gp_command_count">gp_command_count</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#gp_connections_per_thread">gp_connections_per_thread</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#gp_debug_linger">gp_debug_linger</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#gp_dynamic_partition_pruning">gp_dynamic_partition_pruning</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#gp_enable_agg_distinct">gp_enable_agg_distinct</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#gp_enable_agg_distinct_pruning">gp_enable_agg_distinct_pruning</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#gp_enable_direct_dispatch">gp_enable_direct_dispatch</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#gp_enable_fallback_plan">gp_enable_fallback_plan</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#gp_enable_fast_sri">gp_enable_fast_sri</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#gp_enable_groupext_distinct_gather">gp_enable_groupext_distinct_gather</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#gp_enable_groupext_distinct_pruning">gp_enable_groupext_distinct_pruning</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#gp_enable_multiphase_agg">gp_enable_multiphase_agg</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#gp_enable_predicate_propagation">gp_enable_predicate_propagation</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#gp_enable_preunique">gp_enable_preunique</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#gp_enable_sequential_window_plans">gp_enable_sequential_window_plans</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#gp_enable_sort_distinct">gp_enable_sort_distinct</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#gp_enable_sort_limit">gp_enable_sort_limit</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#gp_external_enable_exec">gp_external_enable_exec</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#gp_external_grant_privileges">gp_external_grant_privileges</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#gp_external_max_segs">gp_external_max_segs</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#gp_filerep_tcp_keepalives_count">gp_filerep_tcp_keepalives_count</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#gp_filerep_tcp_keepalives_idle">gp_filerep_tcp_keepalives_idle</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#gp_filerep_tcp_keepalives_interval">gp_filerep_tcp_keepalives_interval</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#gp_hashjoin_tuples_per_bucket">gp_hashjoin_tuples_per_bucket</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#gp_idf_deduplicate">gp_idf_deduplicate</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#gp_interconnect_fc_method">gp_interconnect_fc_method</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#gp_interconnect_hash_multiplier">gp_interconnect_hash_multiplier</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#gp_interconnect_queue_depth">gp_interconnect_queue_depth</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#gp_interconnect_setup_timeout">gp_interconnect_setup_timeout</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#gp_interconnect_snd_queue_depth">gp_interconnect_snd_queue_depth</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#gp_interconnect_type">gp_interconnect_type</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#gp_log_format">gp_log_format</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#gp_max_csv_line_length">gp_max_csv_line_length</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#gp_max_databases">gp_max_databases</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#gp_max_filespaces">gp_max_filespaces</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#gp_max_packet_size">gp_max_packet_size</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#gp_max_plan_size">gp_max_plan_size</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#gp_max_tablespaces">gp_max_tablespaces</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#gp_motion_cost_per_row">gp_motion_cost_per_row</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#gp_reject_percent_threshold">gp_reject_percent_threshold</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#gp_reraise_signal">gp_reraise_signal</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#gp_role">gp_role</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#gp_safefswritesize">gp_safefswritesize</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#gp_segment_connect_timeout">gp_segment_connect_timeout</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#gp_segments_for_planner">gp_segments_for_planner</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#gp_session_id">gp_session_id</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#gp_set_proc_affinity">gp_set_proc_affinity</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#gp_set_read_only">gp_set_read_only</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#gp_statistics_pullup_from_child_partition">gp_statistics_pullup_from_child_partition</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#gp_statistics_use_fkeys">gp_statistics_use_fkeys</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#gp_vmem_protect_segworker_cache_limit">gp_vmem_protect_segworker_cache_limit</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#gp_workfile_checksumming">gp_workfile_checksumming</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#gp_workfile_compress_algorithm">gp_workfile_compress_algorithm</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#gp_workfile_limit_files_per_query">gp_workfile_limit_files_per_query</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#gp_workfile_limit_per_query">gp_workfile_limit_per_query</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#gp_workfile_limit_per_segment">gp_workfile_limit_per_segment</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#hawq_dfs_url">hawq_dfs_url</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#hawq_global_rm_type">hawq_global_rm_type</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#hawq_master_address_host">hawq_master_address_host</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#hawq_master_address_port">hawq_master_address_port</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#hawq_master_directory">hawq_master_directory</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#hawq_master_temp_directory">hawq_master_temp_directory</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#hawq_re_memory_overcommit_max">hawq_re_memory_overcommit_max</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#hawq_rm_cluster_report">hawq_rm_cluster_report_period</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#hawq_rm_force_alterqueue_cancel_queued_request">hawq_rm_force_alterqueue_cancel_queued_request</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#hawq_rm_master_port">hawq_rm_master_port</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#hawq_rm_memory_limit_perseg">hawq_rm_memory_limit_perseg</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#hawq_rm_min_resource_perseg">hawq_rm_min_resource_perseg</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#hawq_rm_nresqueue_limit">hawq_rm_nresqueue_limit</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#hawq_rm_nslice_perseg_limit">hawq_rm_nslice_perseg_limit</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#hawq_rm_nvcore_limit_perseg">hawq_rm_nvcore_limit_perseg</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#hawq_rm_nvseg_perquery_limit">hawq_rm_nvseg_perquery_limit</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#hawq_rm_nvseg_perquery_perseg_limit">hawq_rm_nvseg_perquery_perseg_limit</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#hawq_rm_nvseg_variance_amon_seg_limit">hawq_rm_nvseg_variance_amon_seg_limit</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#hawq_rm_rejectrequest_nseg_limit">hawq_rm_rejectrequest_nseg_limit</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#hawq_rm_resource_idle_timeout">hawq_rm_resource_idle_timeout</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#hawq_rm_return_percent_on_overcommit">hawq_rm_return_percent_on_overcommit</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#hawq_rm_segment_heartbeat_interval">hawq_rm_segment_heartbeat_interval</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#hawq_rm_segment_port">hawq_rm_segment_port</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#hawq_rm_stmt_nvseg">hawq_rm_stmt_nvseg</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#hawq_rm_stmt_vseg_memory">hawq_rm_stmt_vseg_memory</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#hawq_rm_tolerate_nseg_limit">hawq_rm_tolerate_nseg_limit</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#hawq_rm_yarn_address">hawq_rm_yarn_address</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#hawq_rm_yarn_app_name">hawq_rm_yarn_app_name</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#hawq_rm_yarn_queue_name">hawq_rm_yarn_queue_name</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#hawq_rm_yarn_scheduler_address">hawq_rm_yarn_scheduler_address</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#hawq_segment_address_port">hawq_segment_address_port</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#hawq_segment_directory">hawq_segment_directory</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#hawq_segment_temp_directory">hawq_segment_temp_directory</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#integer_datetimes">integer_datetimes</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#IntervalStyle">IntervalStyle</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#join_collapse_limit">join_collapse_limit</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#krb_caseins_users">krb_caseins_users</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#krb_server_keyfile">krb_server_keyfile</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#krb_srvname">krb_srvname</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#lc_collate">lc_collate</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#lc_ctype">lc_ctype</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#lc_messages">lc_messages</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#lc_monetary">lc_monetary</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#lc_numeric">lc_numeric</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#lc_time">lc_time</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#listen_addresses">listen_addresses</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#local_preload_libraries">local_preload_libraries</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#log_autostats">log_autostats</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#log_connections">log_connections</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#log_disconnections">log_disconnections</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#log_dispatch_stats">log_dispatch_stats</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#log_duration">log_duration</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#log_error_verbosity">log_error_verbosity</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#log_executor_stats">log_executor_stats</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#log_hostname">log_hostname</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#log_min_duration_statement">log_min_duration_statement</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#log_min_error_statement">log_min_error_statement</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#log_min_messages">log_min_messages</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#log_parser_stats">log_parser_stats</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#log_planner_stats">log_planner_stats</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#log_rotation_age">log_rotation_age</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#log_rotation_size">log_rotation_size</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#log_statement">log_statement</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#log_statement_stats">log_statement_stats</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#log_timezone">log_timezone</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#log_truncate_on_rotation">log_truncate_on_rotation</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#max_appendonly_tables">max_appendonly_tables</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#max_connections">max_connections</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#max_files_per_process">max_files_per_process</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#max_fsm_pages">max_fsm_pages</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#max_fsm_relations">max_fsm_relations</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#max_function_args">max_function_args</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#max_identifier_length">max_identifier_length</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#max_index_keys">max_index_keys</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#max_locks_per_transaction">max_locks_per_transaction</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#max_prepared_transactions">max_prepared_transactions</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#max_stack_depth">max_stack_depth</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#optimizer">optimizer</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#optimizer_analyze_root_partition">optimizer_analyze_root_partition</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#optimizer_minidump">optimizer_minidump</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#optimizer_parts_to_force_sort_on_insert">optimizer_parts_to_force_sort_on_insert</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#optimizer_prefer_scalar_dqa_multistage_agg">optimizer_prefer_scalar_dqa_multistage_agg</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#password_encryption">password_encryption</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#pgstat_track_activity_query_size">pgstat_track_activity_query_size</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#pljava_classpath">pljava_classpath</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#pljava_statement_cache_size">pljava_statement_cache_size</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#pljava_release_lingering_savepoints">pljava_release_lingering_savepoints</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#pljava_vmoptions">pljava_vmoptions</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#port">port</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#pxf_enable_stat_collection">pxf_enable_stat_collection</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#pxf_service_address">pxf_service_address</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#pxf_service_port">pxf_service_port</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#pxf_stat_max_fragments">pxf_stat_max_fragments</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#random_page_cost">random_page_cost</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#regex_flavor">regex_flavor</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#runaway_detector_activation_percent">runaway_detector_activation_percent</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#search_path">search_path</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#seg_max_connections">seg_max_connections</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#seq_page_cost">seq_page_cost</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#server_encoding">server_encoding</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#server_version">server_version</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#server_version_num">server_version_num</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#shared_buffers">shared_buffers</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#shared_preload_libraries">shared_preload_libraries</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#ssl">ssl</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#ssl_ciphers">ssl_ciphers</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#standard_conforming_strings">standard_conforming_strings</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#statement_timeout">statement_timeout</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#superuser_reserved_connections">superuser_reserved_connections</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#tcp_keepalives_count">tcp_keepalives_count</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#tcp_keepalives_idle">tcp_keepalives_idle</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#tcp_keepalives_interval">tcp_keepalives_interval</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#temp_buffers">temp_buffers</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#TimeZone">TimeZone</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#timezone_abbreviations">timezone_abbreviations</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#track_activities">track_activities</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#track_counts">track_counts</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#transaction_isolation">transaction_isolation</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#transaction_read_only">transaction_read_only</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#transform_null_equals">transform_null_equals</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#unix_socket_directory">unix_socket_directory</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#unix_socket_group">unix_socket_group</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#unix_socket_permissions">unix_socket_permissions</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#update_process_title">update_process_title</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#vacuum_cost_delay">vacuum_cost_delay</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#vacuum_cost_limit">vacuum_cost_limit</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#vacuum_cost_page_dirty">vacuum_cost_page_dirty</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#vacuum_cost_page_miss">vacuum_cost_page_miss</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#vacuum_freeze_min_age">vacuum_freeze_min_age</a></li>
- <li><a href="/20/reference/guc/parameter_definitions.html#xid_stop_limit">xid_stop_limit</a></li>
- </ul>
- </li>
- <li><a href="/20/reference/HAWQSampleSiteConfig.html">Sample hawq-site.xml Configuration File</a></li>
- </ul>
- </li>
- <li><a href="/20/reference/HDFSConfigurationParameterReference.html">HDFS Configuration Reference</a></li>
- <li><a href="/20/reference/HAWQEnvironmentVariables.html">Environment Variables</a></li>
- <li><a href="/20/reference/CharacterSetSupportReference.html">Character Set Support Reference</a></li>
- <li><a href="/20/reference/HAWQDataTypes.html">Data Types</a></li>
- <li class="has_submenu"><a href="/20/reference/SQLCommandReference.html">SQL Commands</a>
- <ul>
- <li><a href="/20/reference/sql/ABORT.html">ABORT</a></li>
- <li><a href="/20/reference/sql/ALTER-AGGREGATE.html">ALTER AGGREGATE</a></li>
- <li><a href="/20/reference/sql/ALTER-DATABASE.html">ALTER DATABASE</a></li>
- <li><a href="/20/reference/sql/ALTER-FUNCTION.html">ALTER FUNCTION</a></li>
- <li><a href="/20/reference/sql/ALTER-OPERATOR.html">ALTER OPERATOR</a></li>
- <li><a href="/20/reference/sql/ALTER-OPERATOR-CLASS.html">ALTER OPERATOR CLASS</a></li>
- <li><a href="/20/reference/sql/ALTER-RESOURCE-QUEUE.html">ALTER RESOURCE QUEUE</a></li>
- <li><a href="/20/reference/sql/ALTER-ROLE.html">ALTER ROLE</a></li>
- <li><a href="/20/reference/sql/ALTER-TABLE.html">ALTER TABLE</a></li>
- <li><a href="/20/reference/sql/ALTER-TABLESPACE.html">ALTER TABLESPACE</a></li>
- <li><a href="/20/reference/sql/ALTER-TYPE.html">ALTER TYPE</a></li>
- <li><a href="/20/reference/sql/ALTER-USER.html">ALTER USER</a></li>
- <li><a href="/20/reference/sql/ANALYZE.html">ANALYZE</a></li>
- <li><a href="/20/reference/sql/BEGIN.html">BEGIN</a></li>
- <li><a href="/20/reference/sql/CHECKPOINT.html">CHECKPOINT</a></li>
- <li><a href="/20/reference/sql/CLOSE.html">CLOSE</a></li>
- <li><a href="/20/reference/sql/COMMIT.html">COMMIT</a></li>
- <li><a href="/20/reference/sql/COPY.html">COPY</a></li>
- <li><a href="/20/reference/sql/CREATE-AGGREGATE.html">CREATE AGGREGATE</a></li>
- <li><a href="/20/reference/sql/CREATE-DATABASE.html">CREATE DATABASE</a></li>
- <li><a href="/20/reference/sql/CREATE-EXTERNAL-TABLE.html">CREATE EXTERNAL TABLE</a></li>
- <li><a href="/20/reference/sql/CREATE-FUNCTION.html">CREATE FUNCTION</a></li>
- <li><a href="/20/reference/sql/CREATE-GROUP.html">CREATE GROUP</a></li>
- <li><a href="/20/reference/sql/CREATE-LANGUAGE.html">CREATE LANGUAGE</a></li>
- <li><a href="/20/reference/sql/CREATE-OPERATOR.html">CREATE OPERATOR</a></li>
- <li><a href="/20/reference/sql/CREATE-OPERATOR-CLASS.html">CREATE OPERATOR CLASS</a></li>
- <li><a href="/20/reference/sql/CREATE-RESOURCE-QUEUE.html">CREATE RESOURCE QUEUE</a></li>
- <li><a href="/20/reference/sql/CREATE-ROLE.html">CREATE ROLE</a></li>
- <li><a href="/20/reference/sql/CREATE-SCHEMA.html">CREATE SCHEMA</a></li>
- <li><a href="/20/reference/sql/CREATE-SEQUENCE.html">CREATE SEQUENCE</a></li>
- <li><a href="/20/reference/sql/CREATE-TABLE.html">CREATE TABLE</a></li>
- <li><a href="/20/reference/sql/CREATE-TABLE-AS.html">CREATE TABLE AS</a></li>
- <li><a href="/20/reference/sql/CREATE-TABLESPACE.html">CREATE TABLESPACE</a></li>
- <li><a href="/20/reference/sql/CREATE-TYPE.html">CREATE TYPE</a></li>
- <li><a href="/20/reference/sql/CREATE-USER.html">CREATE USER</a></li>
- <li><a href="/20/reference/sql/CREATE-VIEW.html">CREATE VIEW</a></li>
- <li><a href="/20/reference/sql/DEALLOCATE.html">DEALLOCATE</a></li>
- <li><a href="/20/reference/sql/DECLARE.html">DECLARE</a></li>
- <li><a href="/20/reference/sql/DROP-AGGREGATE.html">DROP AGGREGATE</a></li>
- <li><a href="/20/reference/sql/DROP-DATABASE.html">DROP DATABASE</a></li>
- <li><a href="/20/reference/sql/DROP-EXTERNAL-TABLE.html">DROP EXTERNAL TABLE</a></li>
- <li><a href="/20/reference/sql/DROP-FILESPACE.html">DROP FILESPACE</a></li>
- <li><a href="/20/reference/sql/DROP-FUNCTION.html">DROP FUNCTION</a></li>
- <li><a href="/20/reference/sql/DROP-GROUP.html">DROP GROUP</a></li>
- <li><a href="/20/reference/sql/DROP-OPERATOR.html">DROP OPERATOR</a></li>
- <li><a href="/20/reference/sql/DROP-OPERATOR-CLASS.html">DROP OPERATOR CLASS</a></li>
- <li><a href="/20/reference/sql/DROP-OWNED.html">DROP OWNED</a></li>
- <li><a href="/20/reference/sql/DROP-RESOURCE-QUEUE.html">DROP RESOURCE QUEUE</a></li>
- <li><a href="/20/reference/sql/DROP-ROLE.html">DROP ROLE</a></li>
- <li><a href="/20/reference/sql/DROP-SCHEMA.html">DROP SCHEMA</a></li>
- <li><a href="/20/reference/sql/DROP-SEQUENCE.html">DROP SEQUENCE</a></li>
- <li><a href="/20/reference/sql/DROP-TABLE.html">DROP TABLE</a></li>
- <li><a href="/20/reference/sql/DROP-TABLESPACE.html">DROP TABLESPACE</a></li>
- <li><a href="/20/reference/sql/DROP-TYPE.html">DROP TYPE</a></li>
- <li><a href="/20/reference/sql/DROP-USER.html">DROP USER</a></li>
- <li><a href="/20/reference/sql/DROP-VIEW.html">DROP VIEW</a></li>
- <li><a href="/20/reference/sql/END.html">END</a></li>
- <li><a href="/20/reference/sql/EXECUTE.html">EXECUTE</a></li>
- <li><a href="/20/reference/sql/EXPLAIN.html">EXPLAIN</a></li>
- <li><a href="/20/reference/sql/FETCH.html">FETCH</a></li>
- <li><a href="/20/reference/sql/GRANT.html">GRANT</a></li>
- <li><a href="/20/reference/sql/INSERT.html">INSERT</a></li>
- <li><a href="/20/reference/sql/PREPARE.html">PREPARE</a></li>
- <li><a href="/20/reference/sql/REASSIGN-OWNED.html">REASSIGN OWNED</a></li>
- <li><a href="/20/reference/sql/RELEASE-SAVEPOINT.html">RELEASE SAVEPOINT</a></li>
- <li><a href="/20/reference/sql/RESET.html">RESET</a></li>
- <li><a href="/20/reference/sql/REVOKE.html">REVOKE</a></li>
- <li><a href="/20/reference/sql/ROLLBACK.html">ROLLBACK</a></li>
- <li><a href="/20/reference/sql/ROLLBACK-TO-SAVEPOINT.html">ROLLBACK TO SAVEPOINT</a></li>
- <li><a href="/20/reference/sql/SAVEPOINT.html">SAVEPOINT</a></li>
- <li><a href="/20/reference/sql/SELECT.html">SELECT</a></li>
- <li><a href="/20/reference/sql/SELECT-INTO.html">SELECT INTO</a></li>
- <li><a href="/20/reference/sql/SET.html">SET</a></li>
- <li><a href="/20/reference/sql/SET-ROLE.html">SET ROLE</a></li>
- <li><a href="/20/reference/sql/SET-SESSION-AUTHORIZATION.html">SET SESSION AUTHORIZATION</a></li>
- <li><a href="/20/reference/sql/SHOW.html">SHOW</a></li>
- <li><a href="/20/reference/sql/TRUNCATE.html">TRUNCATE</a></li>
- <li><a href="/20/reference/sql/VACUUM.html">VACUUM</a></li>
- </ul>
- </li>
- <li class="has_submenu"><a href="/20/reference/catalog/catalog_ref.html">System Catalog Reference</a>
- <ul>
- <li><a href="/20/reference/catalog/catalog_ref-tables.html">System Tables</a></li>
- <li><a href="/20/reference/catalog/catalog_ref-views.html">System Views</a></li>
- <li class="has_submenu"><a href="/20/reference/catalog/catalog_ref-html.html">System Catalogs Definitions</a>
- <ul>
- <li><a href="/20/reference/catalog/gp_configuration_history.html">gp_configuration_history</a></li>
- <li><a href="/20/reference/catalog/gp_distribution_policy.html">gp_distribution_policy</a></li>
- <li><a href="/20/reference/catalog/gp_global_sequence.html">gp_global_sequence</a></li>
- <li><a href="/20/reference/catalog/gp_master_mirroring.html">gp_master_mirroring</a></li>
- <li><a href="/20/reference/catalog/gp_persistent_database_node.html">gp_persistent_database_node</a></li>
- <li><a href="/20/reference/catalog/gp_persistent_filespace_node.html">gp_persistent_filespace_node</a></li>
- <li><a href="/20/reference/catalog/gp_persistent_relation_node.html">gp_persistent_relation_node</a></li>
- <li><a href="/20/reference/catalog/gp_persistent_relfile_node.html">gp_persistent_relfile_node</a></li>
- <li><a href="/20/reference/catalog/gp_persistent_tablespace_node.html">gp_persistent_tablespace_node</a></li>
- <li><a href="/20/reference/catalog/gp_relfile_node.html">gp_relfile_node</a></li>
- <li><a href="/20/reference/catalog/gp_segment_configuration.html">gp_segment_configuration</a></li>
- <li><a href="/20/reference/catalog/gp_version_at_initdb.html">gp_version_at_initdb</a></li>
- <li><a href="/20/reference/catalog/pg_aggregate.html">pg_aggregate</a></li>
- <li><a href="/20/reference/catalog/pg_am.html">pg_am</a></li>
- <li><a href="/20/reference/catalog/pg_amop.html">pg_amop</a></li>
- <li><a href="/20/reference/catalog/pg_amproc.html">pg_amproc</a></li>
- <li><a href="/20/reference/catalog/pg_appendonly.html">pg_appendonly</a></li>
- <li><a href="/20/reference/catalog/pg_attrdef.html">pg_attrdef</a></li>
- <li><a href="/20/reference/catalog/pg_attribute.html">pg_attribute</a></li>
- <li><a href="/20/reference/catalog/pg_attribute_encoding.html">pg_attribute_encoding</a></li>
- <li><a href="/20/reference/catalog/pg_auth_members.html">pg_auth_members</a></li>
- <li><a href="/20/reference/catalog/pg_authid.html">pg_authid</a></li>
- <li><a href="/20/reference/catalog/pg_cast.html">pg_cast</a></li>
- <li><a href="/20/reference/catalog/pg_class.html">pg_class</a></li>
- <li><a href="/20/reference/catalog/pg_compression.html">pg_compression</a></li>
- <li><a href="/20/reference/catalog/pg_constraint.html">pg_constraint</a></li>
- <li><a href="/20/reference/catalog/pg_conversion.html">pg_conversion</a></li>
- <li><a href="/20/reference/catalog/pg_database.html">pg_database</a></li>
- <li><a href="/20/reference/catalog/pg_depend.html">pg_depend</a></li>
- <li><a href="/20/reference/catalog/pg_description.html">pg_description</a></li>
- <li><a href="/20/reference/catalog/pg_exttable.html">pg_exttable</a></li>
- <li><a href="/20/reference/catalog/pg_filespace.html">pg_filespace</a></li>
- <li><a href="/20/reference/catalog/pg_filespace_entry.html">pg_filespace_entry</a></li>
- <li><a href="/20/reference/catalog/pg_index.html">pg_index</a></li>
- <li><a href="/20/reference/catalog/pg_inherits.html">pg_inherits</a></li>
- <li><a href="/20/reference/catalog/pg_language.html">pg_language</a></li>
- <li><a href="/20/reference/catalog/pg_largeobject.html">pg_largeobject</a></li>
- <li><a href="/20/reference/catalog/pg_listener.html">pg_listener</a></li>
- <li><a href="/20/reference/catalog/pg_locks.html">pg_locks</a></li>
- <li><a href="/20/reference/catalog/pg_namespace.html">pg_namespace</a></li>
- <li><a href="/20/reference/catalog/pg_opclass.html">pg_opclass</a></li>
- <li><a href="/20/reference/catalog/pg_operator.html">pg_operator</a></li>
- <li><a href="/20/reference/catalog/pg_partition.html">pg_partition</a></li>
- <li><a href="/20/reference/catalog/pg_partition_columns.html">pg_partition_columns</a></li>
- <li><a href="/20/reference/catalog/pg_partition_encoding.html">pg_partition_encoding</a></li>
- <li><a href="/20/reference/catalog/pg_partition_rule.html">pg_partition_rule</a></li>
- <li><a href="/20/reference/catalog/pg_partition_templates.html">pg_partition_templates</a></li>
- <li><a href="/20/reference/catalog/pg_partitions.html">pg_partitions</a></li>
- <li><a href="/20/reference/catalog/pg_pltemplate.html">pg_pltemplate</a></li>
- <li><a href="/20/reference/catalog/pg_proc.html">pg_proc</a></li>
- <li><a href="/20/reference/catalog/pg_resqueue.html">pg_resqueue</a></li>
- <li><a href="/20/reference/catalog/pg_resqueue_status.html">pg_resqueue_status</a></li>
- <li><a href="/20/reference/catalog/pg_rewrite.html">pg_rewrite</a></li>
- <li><a href="/20/reference/catalog/pg_roles.html">pg_roles</a></li>
- <li><a href="/20/reference/catalog/pg_shdepend.html">pg_shdepend</a></li>
- <li><a href="/20/reference/catalog/pg_shdescription.html">pg_shdescription</a></li>
- <li><a href="/20/reference/catalog/pg_stat_activity.html">pg_stat_activity</a></li>
- <li><a href="/20/reference/catalog/pg_stat_last_operation.html">pg_stat_last_operation</a></li>
- <li><a href="/20/reference/catalog/pg_stat_last_shoperation.html">pg_stat_last_shoperation</a></li>
- <li><a href="/20/reference/catalog/pg_stat_operations.html">pg_stat_operations</a></li>
- <li><a href="/20/reference/catalog/pg_stat_partition_operations.html">pg_stat_partition_operations</a></li>
- <li><a href="/20/reference/catalog/pg_statistic.html">pg_statistic</a></li>
- <li><a href="/20/reference/catalog/pg_stats.html">pg_stats</a></li>
- <li><a href="/20/reference/catalog/pg_tablespace.html">pg_tablespace</a></li>
- <li><a href="/20/reference/catalog/pg_trigger.html">pg_trigger</a></li>
- <li><a href="/20/reference/catalog/pg_type.html">pg_type</a></li>
- <li><a href="/20/reference/catalog/pg_type_encoding.html">pg_type_encoding</a></li>
- <li><a href="/20/reference/catalog/pg_window.html">pg_window</a></li>
- </ul>
- </li>
- </ul>
- </li>
- <li class="has_submenu"><a href="/20/reference/toolkit/hawq_toolkit.html">The hawq_toolkit Administrative Schema</a>
- <ul>
- <li><a href="/20/reference/toolkit/hawq_toolkit.html#topic2">Checking for Tables that Need Routine Maintenance</a></li>
- <li><a href="/20/reference/toolkit/hawq_toolkit.html#topic16">Viewing HAWQ Server Log Files</a></li>
- <li><a href="/20/reference/toolkit/hawq_toolkit.html#topic38">Checking Database Object Sizes and Disk Space</a></li>
- </ul>
- </li>
- <li class="has_submenu"><a href="/20/reference/cli/management_tools.html">HAWQ Management Tools Reference</a>
- <ul>
- <li><a href="/20/reference/cli/admin_utilities/analyzedb.html">analyzedb</a></li>
- <li><a href="/20/reference/cli/client_utilities/createdb.html">createdb</a></li>
- <li><a href="/20/reference/cli/client_utilities/createuser.html">createuser</a></li>
- <li><a href="/20/reference/cli/client_utilities/dropdb.html">dropdb</a></li>
- <li><a href="/20/reference/cli/client_utilities/dropuser.html">dropuser</a></li>
- <li><a href="/20/reference/cli/admin_utilities/gpfdist.html">gpfdist</a></li>
- <li><a href="/20/reference/cli/admin_utilities/gplogfilter.html">gplogfilter</a></li>
- <li><a href="/20/reference/cli/admin_utilities/hawqactivate.html">hawq activate</a></li>
- <li><a href="/20/reference/cli/admin_utilities/hawqcheck.html">hawq check</a></li>
- <li><a href="/20/reference/cli/admin_utilities/hawqcheckperf.html">hawq checkperf</a></li>
- <li><a href="/20/reference/cli/admin_utilities/hawqconfig.html">hawq config</a></li>
- <li><a href="/20/reference/cli/admin_utilities/hawqextract.html">hawq extract</a></li>
- <li><a href="/20/reference/cli/admin_utilities/hawqfilespace.html">hawq filespace</a></li>
- <li><a href="/20/reference/cli/admin_utilities/hawqinit.html">hawq init</a></li>
- <li><a href="/20/reference/cli/admin_utilities/hawqload.html">hawq load</a></li>
- <li><a href="/20/reference/cli/admin_utilities/hawqregister.html">hawq register</a></li>
- <li><a href="/20/reference/cli/admin_utilities/hawqrestart.html">hawq restart</a></li>
- <li><a href="/20/reference/cli/admin_utilities/hawqscp.html">hawq scp</a></li>
- <li><a href="/20/reference/cli/admin_utilities/hawqssh.html">hawq ssh</a></li>
- <li><a href="/20/reference/cli/admin_utilities/hawqssh-exkeys.html">hawq ssh-exkeys</a></li>
- <li><a href="/20/reference/cli/admin_utilities/hawqstart.html">hawq start</a></li>
- <li><a href="/20/reference/cli/admin_utilities/hawqstate.html">hawq state</a></li>
- <li><a href="/20/reference/cli/admin_utilities/hawqstop.html">hawq stop</a></li>
- <li><a href="/20/reference/cli/client_utilities/pg_dump.html">pg_dump</a></li>
- <li><a href="/20/reference/cli/client_utilities/pg_dumpall.html">pg_dumpall</a></li>
- <li><a href="/20/reference/cli/client_utilities/pg_restore.html">pg_restore</a></li>
- <li><a href="/20/reference/cli/client_utilities/psql.html">psql</a></li>
- <li><a href="/20/reference/cli/client_utilities/vacuumdb.html">vacuumdb</a></li>
- </ul>
- </li>
- </ul>
- </li>
- </ul>
- </div>
-</div>
-<!--end of sub-nav-->
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/ecc6de22/hawq-book/redirects.rb
----------------------------------------------------------------------
diff --git a/hawq-book/redirects.rb b/hawq-book/redirects.rb
deleted file mode 100644
index 7fb53eb..0000000
--- a/hawq-book/redirects.rb
+++ /dev/null
@@ -1,6 +0,0 @@
-r301 %r{.*}, 'http://docs-hdb-staging$&', if: Proc.new {|rack_env|
- rack_env['SERVER_NAME'] == 'docs-hawq-staging'
-}
-rewrite '/', '/20/overview/HAWQOverview.html'
-r301 '/index.html', '/20/overview/HAWQOverview.html'
-
[04/50] [abbrv] incubator-hawq-docs git commit: removing hawq-book
directory from this branch [HAWQ-1027]
Posted by yo...@apache.org.
removing hawq-book directory from this branch [HAWQ-1027]
Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/commit/ecc6de22
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/tree/ecc6de22
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/diff/ecc6de22
Branch: refs/heads/master
Commit: ecc6de22f2ff9eb612718de205fdbffb84ed55a0
Parents: 7f23af6
Author: David Yozie <yo...@apache.org>
Authored: Mon Aug 29 10:18:49 2016 -0700
Committer: David Yozie <yo...@apache.org>
Committed: Mon Aug 29 10:18:49 2016 -0700
----------------------------------------------------------------------
hawq-book/Gemfile | 5 -
hawq-book/Gemfile.lock | 203 ----
hawq-book/config.yml | 22 -
.../master_middleman/source/images/favicon.ico | Bin 1150 -> 0 bytes
.../master_middleman/source/javascripts/book.js | 16 -
.../source/javascripts/waypoints/context.js | 300 ------
.../source/javascripts/waypoints/group.js | 105 --
.../javascripts/waypoints/noframeworkAdapter.js | 213 -----
.../source/javascripts/waypoints/sticky.js | 63 --
.../source/javascripts/waypoints/waypoint.js | 160 ----
.../master_middleman/source/layouts/_title.erb | 6 -
.../patch/dynamic_variable_interpretation.py | 192 ----
.../source/stylesheets/book-styles.css.scss | 3 -
.../stylesheets/partials/_book-base-values.scss | 0
.../source/stylesheets/partials/_book-vars.scss | 19 -
.../source/subnavs/apache-hawq-nav.erb | 955 -------------------
hawq-book/redirects.rb | 6 -
17 files changed, 2268 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/ecc6de22/hawq-book/Gemfile
----------------------------------------------------------------------
diff --git a/hawq-book/Gemfile b/hawq-book/Gemfile
deleted file mode 100644
index f66d333..0000000
--- a/hawq-book/Gemfile
+++ /dev/null
@@ -1,5 +0,0 @@
-source "https://rubygems.org"
-
-gem 'bookbindery'
-
-gem 'libv8', '3.16.14.7'
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/ecc6de22/hawq-book/Gemfile.lock
----------------------------------------------------------------------
diff --git a/hawq-book/Gemfile.lock b/hawq-book/Gemfile.lock
deleted file mode 100644
index 3c483c0..0000000
--- a/hawq-book/Gemfile.lock
+++ /dev/null
@@ -1,203 +0,0 @@
-GEM
- remote: https://rubygems.org/
- specs:
- activesupport (4.2.7.1)
- i18n (~> 0.7)
- json (~> 1.7, >= 1.7.7)
- minitest (~> 5.1)
- thread_safe (~> 0.3, >= 0.3.4)
- tzinfo (~> 1.1)
- addressable (2.4.0)
- ansi (1.5.0)
- bookbindery (9.12.0)
- ansi (~> 1.4)
- css_parser
- elasticsearch
- fog-aws (~> 0.7.1)
- font-awesome-sass
- git (~> 1.2.8)
- middleman (~> 3.4.0)
- middleman-livereload (~> 3.4.3)
- middleman-syntax (~> 2.0)
- nokogiri (= 1.6.7.2)
- puma
- rack-rewrite
- redcarpet (~> 3.2.3)
- rouge (!= 1.9.1)
- therubyracer
- thor
- builder (3.2.2)
- capybara (2.4.4)
- mime-types (>= 1.16)
- nokogiri (>= 1.3.3)
- rack (>= 1.0.0)
- rack-test (>= 0.5.4)
- xpath (~> 2.0)
- chunky_png (1.3.6)
- coffee-script (2.4.1)
- coffee-script-source
- execjs
- coffee-script-source (1.10.0)
- compass (1.0.3)
- chunky_png (~> 1.2)
- compass-core (~> 1.0.2)
- compass-import-once (~> 1.0.5)
- rb-fsevent (>= 0.9.3)
- rb-inotify (>= 0.9)
- sass (>= 3.3.13, < 3.5)
- compass-core (1.0.3)
- multi_json (~> 1.0)
- sass (>= 3.3.0, < 3.5)
- compass-import-once (1.0.5)
- sass (>= 3.2, < 3.5)
- css_parser (1.4.5)
- addressable
- elasticsearch (2.0.0)
- elasticsearch-api (= 2.0.0)
- elasticsearch-transport (= 2.0.0)
- elasticsearch-api (2.0.0)
- multi_json
- elasticsearch-transport (2.0.0)
- faraday
- multi_json
- em-websocket (0.5.1)
- eventmachine (>= 0.12.9)
- http_parser.rb (~> 0.6.0)
- erubis (2.7.0)
- eventmachine (1.2.0.1)
- excon (0.51.0)
- execjs (2.7.0)
- faraday (0.9.2)
- multipart-post (>= 1.2, < 3)
- ffi (1.9.14)
- fog-aws (0.7.6)
- fog-core (~> 1.27)
- fog-json (~> 1.0)
- fog-xml (~> 0.1)
- ipaddress (~> 0.8)
- fog-core (1.42.0)
- builder
- excon (~> 0.49)
- formatador (~> 0.2)
- fog-json (1.0.2)
- fog-core (~> 1.0)
- multi_json (~> 1.10)
- fog-xml (0.1.2)
- fog-core
- nokogiri (~> 1.5, >= 1.5.11)
- font-awesome-sass (4.6.2)
- sass (>= 3.2)
- formatador (0.2.5)
- git (1.2.9.1)
- haml (4.0.7)
- tilt
- hike (1.2.3)
- hooks (0.4.1)
- uber (~> 0.0.14)
- http_parser.rb (0.6.0)
- i18n (0.7.0)
- ipaddress (0.8.3)
- json (1.8.3)
- kramdown (1.12.0)
- libv8 (3.16.14.7)
- listen (3.0.8)
- rb-fsevent (~> 0.9, >= 0.9.4)
- rb-inotify (~> 0.9, >= 0.9.7)
- middleman (3.4.1)
- coffee-script (~> 2.2)
- compass (>= 1.0.0, < 2.0.0)
- compass-import-once (= 1.0.5)
- execjs (~> 2.0)
- haml (>= 4.0.5)
- kramdown (~> 1.2)
- middleman-core (= 3.4.1)
- middleman-sprockets (>= 3.1.2)
- sass (>= 3.4.0, < 4.0)
- uglifier (~> 2.5)
- middleman-core (3.4.1)
- activesupport (~> 4.1)
- bundler (~> 1.1)
- capybara (~> 2.4.4)
- erubis
- hooks (~> 0.3)
- i18n (~> 0.7.0)
- listen (~> 3.0.3)
- padrino-helpers (~> 0.12.3)
- rack (>= 1.4.5, < 2.0)
- thor (>= 0.15.2, < 2.0)
- tilt (~> 1.4.1, < 2.0)
- middleman-livereload (3.4.6)
- em-websocket (~> 0.5.1)
- middleman-core (>= 3.3)
- rack-livereload (~> 0.3.15)
- middleman-sprockets (3.4.2)
- middleman-core (>= 3.3)
- sprockets (~> 2.12.1)
- sprockets-helpers (~> 1.1.0)
- sprockets-sass (~> 1.3.0)
- middleman-syntax (2.1.0)
- middleman-core (>= 3.2)
- rouge (~> 1.0)
- mime-types (3.1)
- mime-types-data (~> 3.2015)
- mime-types-data (3.2016.0521)
- mini_portile2 (2.0.0)
- minitest (5.9.0)
- multi_json (1.12.1)
- multipart-post (2.0.0)
- nokogiri (1.6.7.2)
- mini_portile2 (~> 2.0.0.rc2)
- padrino-helpers (0.12.8)
- i18n (~> 0.6, >= 0.6.7)
- padrino-support (= 0.12.8)
- tilt (~> 1.4.1)
- padrino-support (0.12.8)
- activesupport (>= 3.1)
- puma (3.6.0)
- rack (1.6.4)
- rack-livereload (0.3.16)
- rack
- rack-rewrite (1.5.1)
- rack-test (0.6.3)
- rack (>= 1.0)
- rb-fsevent (0.9.7)
- rb-inotify (0.9.7)
- ffi (>= 0.5.0)
- redcarpet (3.2.3)
- ref (2.0.0)
- rouge (1.11.1)
- sass (3.4.22)
- sprockets (2.12.4)
- hike (~> 1.2)
- multi_json (~> 1.0)
- rack (~> 1.0)
- tilt (~> 1.1, != 1.3.0)
- sprockets-helpers (1.1.0)
- sprockets (~> 2.0)
- sprockets-sass (1.3.1)
- sprockets (~> 2.0)
- tilt (~> 1.1)
- therubyracer (0.12.2)
- libv8 (~> 3.16.14.0)
- ref
- thor (0.19.1)
- thread_safe (0.3.5)
- tilt (1.4.1)
- tzinfo (1.2.2)
- thread_safe (~> 0.1)
- uber (0.0.15)
- uglifier (2.7.2)
- execjs (>= 0.3.0)
- json (>= 1.8.0)
- xpath (2.0.0)
- nokogiri (~> 1.3)
-
-PLATFORMS
- ruby
-
-DEPENDENCIES
- bookbindery
- libv8 (= 3.16.14.7)
-
-BUNDLED WITH
- 1.11.2
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/ecc6de22/hawq-book/config.yml
----------------------------------------------------------------------
diff --git a/hawq-book/config.yml b/hawq-book/config.yml
deleted file mode 100644
index 7240fee..0000000
--- a/hawq-book/config.yml
+++ /dev/null
@@ -1,22 +0,0 @@
-book_repo: incubator-hawq/docs-book-hawq
-
-public_host: http://localhost:9292/
-
-sections:
- - repository:
- name: incubator-hawq/docs-apache-hawq-md
- ref: develop
- directory: 20
- subnav_template: apache-hawq-nav
-
-template_variables:
- use_global_header: true
- global_header_product_href: https://github.com/apache/incubator-hawq
- global_header_product_link_text: Downloads
- support_url: http://mail-archives.apache.org/mod_mbox/incubator-hawq-dev/
- product_url: http://hawq.incubator.apache.org/
- book_title: Apache HAWQ (incubating) Documentation
- support_link: <a href="https://issues.apache.org/jira/browse/HAWQ" target="_blank">Support</a>
- support_call_to_action: <a href="https://issues.apache.org/jira/browse/HAWQ" target="_blank">Need Help?</a>
- product_link: <div class="header-item"><a href="http://hawq.incubator.apache.org/">Back to Apache HAWQ Page</a></div>
- book_title_short: Apache HAWQ (Incubating) Docs
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/ecc6de22/hawq-book/master_middleman/source/images/favicon.ico
----------------------------------------------------------------------
diff --git a/hawq-book/master_middleman/source/images/favicon.ico b/hawq-book/master_middleman/source/images/favicon.ico
deleted file mode 100644
index b2c3a0c..0000000
Binary files a/hawq-book/master_middleman/source/images/favicon.ico and /dev/null differ
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/ecc6de22/hawq-book/master_middleman/source/javascripts/book.js
----------------------------------------------------------------------
diff --git a/hawq-book/master_middleman/source/javascripts/book.js b/hawq-book/master_middleman/source/javascripts/book.js
deleted file mode 100644
index 90879c4..0000000
--- a/hawq-book/master_middleman/source/javascripts/book.js
+++ /dev/null
@@ -1,16 +0,0 @@
-// Declare your book-specific javascript overrides in this file.
-//= require 'waypoints/waypoint'
-//= require 'waypoints/context'
-//= require 'waypoints/group'
-//= require 'waypoints/noframeworkAdapter'
-//= require 'waypoints/sticky'
-
-window.onload = function() {
- Bookbinder.boot();
- var sticky = new Waypoint.Sticky({
- element: document.querySelector('#js-to-top'),
- wrapper: '<div class="sticky-wrapper" />',
- stuckClass: 'sticky',
- offset: 100
- });
-}
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/ecc6de22/hawq-book/master_middleman/source/javascripts/waypoints/context.js
----------------------------------------------------------------------
diff --git a/hawq-book/master_middleman/source/javascripts/waypoints/context.js b/hawq-book/master_middleman/source/javascripts/waypoints/context.js
deleted file mode 100644
index 5e3551b..0000000
--- a/hawq-book/master_middleman/source/javascripts/waypoints/context.js
+++ /dev/null
@@ -1,300 +0,0 @@
-(function() {
- 'use strict'
-
- function requestAnimationFrameShim(callback) {
- window.setTimeout(callback, 1000 / 60)
- }
-
- var keyCounter = 0
- var contexts = {}
- var Waypoint = window.Waypoint
- var oldWindowLoad = window.onload
-
- /* http://imakewebthings.com/waypoints/api/context */
- function Context(element) {
- this.element = element
- this.Adapter = Waypoint.Adapter
- this.adapter = new this.Adapter(element)
- this.key = 'waypoint-context-' + keyCounter
- this.didScroll = false
- this.didResize = false
- this.oldScroll = {
- x: this.adapter.scrollLeft(),
- y: this.adapter.scrollTop()
- }
- this.waypoints = {
- vertical: {},
- horizontal: {}
- }
-
- element.waypointContextKey = this.key
- contexts[element.waypointContextKey] = this
- keyCounter += 1
-
- this.createThrottledScrollHandler()
- this.createThrottledResizeHandler()
- }
-
- /* Private */
- Context.prototype.add = function(waypoint) {
- var axis = waypoint.options.horizontal ? 'horizontal' : 'vertical'
- this.waypoints[axis][waypoint.key] = waypoint
- this.refresh()
- }
-
- /* Private */
- Context.prototype.checkEmpty = function() {
- var horizontalEmpty = this.Adapter.isEmptyObject(this.waypoints.horizontal)
- var verticalEmpty = this.Adapter.isEmptyObject(this.waypoints.vertical)
- if (horizontalEmpty && verticalEmpty) {
- this.adapter.off('.waypoints')
- delete contexts[this.key]
- }
- }
-
- /* Private */
- Context.prototype.createThrottledResizeHandler = function() {
- var self = this
-
- function resizeHandler() {
- self.handleResize()
- self.didResize = false
- }
-
- this.adapter.on('resize.waypoints', function() {
- if (!self.didResize) {
- self.didResize = true
- Waypoint.requestAnimationFrame(resizeHandler)
- }
- })
- }
-
- /* Private */
- Context.prototype.createThrottledScrollHandler = function() {
- var self = this
- function scrollHandler() {
- self.handleScroll()
- self.didScroll = false
- }
-
- this.adapter.on('scroll.waypoints', function() {
- if (!self.didScroll || Waypoint.isTouch) {
- self.didScroll = true
- Waypoint.requestAnimationFrame(scrollHandler)
- }
- })
- }
-
- /* Private */
- Context.prototype.handleResize = function() {
- Waypoint.Context.refreshAll()
- }
-
- /* Private */
- Context.prototype.handleScroll = function() {
- var triggeredGroups = {}
- var axes = {
- horizontal: {
- newScroll: this.adapter.scrollLeft(),
- oldScroll: this.oldScroll.x,
- forward: 'right',
- backward: 'left'
- },
- vertical: {
- newScroll: this.adapter.scrollTop(),
- oldScroll: this.oldScroll.y,
- forward: 'down',
- backward: 'up'
- }
- }
-
- for (var axisKey in axes) {
- var axis = axes[axisKey]
- var isForward = axis.newScroll > axis.oldScroll
- var direction = isForward ? axis.forward : axis.backward
-
- for (var waypointKey in this.waypoints[axisKey]) {
- var waypoint = this.waypoints[axisKey][waypointKey]
- var wasBeforeTriggerPoint = axis.oldScroll < waypoint.triggerPoint
- var nowAfterTriggerPoint = axis.newScroll >= waypoint.triggerPoint
- var crossedForward = wasBeforeTriggerPoint && nowAfterTriggerPoint
- var crossedBackward = !wasBeforeTriggerPoint && !nowAfterTriggerPoint
- if (crossedForward || crossedBackward) {
- waypoint.queueTrigger(direction)
- triggeredGroups[waypoint.group.id] = waypoint.group
- }
- }
- }
-
- for (var groupKey in triggeredGroups) {
- triggeredGroups[groupKey].flushTriggers()
- }
-
- this.oldScroll = {
- x: axes.horizontal.newScroll,
- y: axes.vertical.newScroll
- }
- }
-
- /* Private */
- Context.prototype.innerHeight = function() {
- /*eslint-disable eqeqeq */
- if (this.element == this.element.window) {
- return Waypoint.viewportHeight()
- }
- /*eslint-enable eqeqeq */
- return this.adapter.innerHeight()
- }
-
- /* Private */
- Context.prototype.remove = function(waypoint) {
- delete this.waypoints[waypoint.axis][waypoint.key]
- this.checkEmpty()
- }
-
- /* Private */
- Context.prototype.innerWidth = function() {
- /*eslint-disable eqeqeq */
- if (this.element == this.element.window) {
- return Waypoint.viewportWidth()
- }
- /*eslint-enable eqeqeq */
- return this.adapter.innerWidth()
- }
-
- /* Public */
- /* http://imakewebthings.com/waypoints/api/context-destroy */
- Context.prototype.destroy = function() {
- var allWaypoints = []
- for (var axis in this.waypoints) {
- for (var waypointKey in this.waypoints[axis]) {
- allWaypoints.push(this.waypoints[axis][waypointKey])
- }
- }
- for (var i = 0, end = allWaypoints.length; i < end; i++) {
- allWaypoints[i].destroy()
- }
- }
-
- /* Public */
- /* http://imakewebthings.com/waypoints/api/context-refresh */
- Context.prototype.refresh = function() {
- /*eslint-disable eqeqeq */
- var isWindow = this.element == this.element.window
- /*eslint-enable eqeqeq */
- var contextOffset = isWindow ? undefined : this.adapter.offset()
- var triggeredGroups = {}
- var axes
-
- this.handleScroll()
- axes = {
- horizontal: {
- contextOffset: isWindow ? 0 : contextOffset.left,
- contextScroll: isWindow ? 0 : this.oldScroll.x,
- contextDimension: this.innerWidth(),
- oldScroll: this.oldScroll.x,
- forward: 'right',
- backward: 'left',
- offsetProp: 'left'
- },
- vertical: {
- contextOffset: isWindow ? 0 : contextOffset.top,
- contextScroll: isWindow ? 0 : this.oldScroll.y,
- contextDimension: this.innerHeight(),
- oldScroll: this.oldScroll.y,
- forward: 'down',
- backward: 'up',
- offsetProp: 'top'
- }
- }
-
- for (var axisKey in axes) {
- var axis = axes[axisKey]
- for (var waypointKey in this.waypoints[axisKey]) {
- var waypoint = this.waypoints[axisKey][waypointKey]
- var adjustment = waypoint.options.offset
- var oldTriggerPoint = waypoint.triggerPoint
- var elementOffset = 0
- var freshWaypoint = oldTriggerPoint == null
- var contextModifier, wasBeforeScroll, nowAfterScroll
- var triggeredBackward, triggeredForward
-
- if (waypoint.element !== waypoint.element.window) {
- elementOffset = waypoint.adapter.offset()[axis.offsetProp]
- }
-
- if (typeof adjustment === 'function') {
- adjustment = adjustment.apply(waypoint)
- }
- else if (typeof adjustment === 'string') {
- adjustment = parseFloat(adjustment)
- if (waypoint.options.offset.indexOf('%') > - 1) {
- adjustment = Math.ceil(axis.contextDimension * adjustment / 100)
- }
- }
-
- contextModifier = axis.contextScroll - axis.contextOffset
- waypoint.triggerPoint = elementOffset + contextModifier - adjustment
- wasBeforeScroll = oldTriggerPoint < axis.oldScroll
- nowAfterScroll = waypoint.triggerPoint >= axis.oldScroll
- triggeredBackward = wasBeforeScroll && nowAfterScroll
- triggeredForward = !wasBeforeScroll && !nowAfterScroll
-
- if (!freshWaypoint && triggeredBackward) {
- waypoint.queueTrigger(axis.backward)
- triggeredGroups[waypoint.group.id] = waypoint.group
- }
- else if (!freshWaypoint && triggeredForward) {
- waypoint.queueTrigger(axis.forward)
- triggeredGroups[waypoint.group.id] = waypoint.group
- }
- else if (freshWaypoint && axis.oldScroll >= waypoint.triggerPoint) {
- waypoint.queueTrigger(axis.forward)
- triggeredGroups[waypoint.group.id] = waypoint.group
- }
- }
- }
-
- Waypoint.requestAnimationFrame(function() {
- for (var groupKey in triggeredGroups) {
- triggeredGroups[groupKey].flushTriggers()
- }
- })
-
- return this
- }
-
- /* Private */
- Context.findOrCreateByElement = function(element) {
- return Context.findByElement(element) || new Context(element)
- }
-
- /* Private */
- Context.refreshAll = function() {
- for (var contextId in contexts) {
- contexts[contextId].refresh()
- }
- }
-
- /* Public */
- /* http://imakewebthings.com/waypoints/api/context-find-by-element */
- Context.findByElement = function(element) {
- return contexts[element.waypointContextKey]
- }
-
- window.onload = function() {
- if (oldWindowLoad) {
- oldWindowLoad()
- }
- Context.refreshAll()
- }
-
- Waypoint.requestAnimationFrame = function(callback) {
- var requestFn = window.requestAnimationFrame ||
- window.mozRequestAnimationFrame ||
- window.webkitRequestAnimationFrame ||
- requestAnimationFrameShim
- requestFn.call(window, callback)
- }
- Waypoint.Context = Context
-}())
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/ecc6de22/hawq-book/master_middleman/source/javascripts/waypoints/group.js
----------------------------------------------------------------------
diff --git a/hawq-book/master_middleman/source/javascripts/waypoints/group.js b/hawq-book/master_middleman/source/javascripts/waypoints/group.js
deleted file mode 100644
index 57c3038..0000000
--- a/hawq-book/master_middleman/source/javascripts/waypoints/group.js
+++ /dev/null
@@ -1,105 +0,0 @@
-(function() {
- 'use strict'
-
- function byTriggerPoint(a, b) {
- return a.triggerPoint - b.triggerPoint
- }
-
- function byReverseTriggerPoint(a, b) {
- return b.triggerPoint - a.triggerPoint
- }
-
- var groups = {
- vertical: {},
- horizontal: {}
- }
- var Waypoint = window.Waypoint
-
- /* http://imakewebthings.com/waypoints/api/group */
- function Group(options) {
- this.name = options.name
- this.axis = options.axis
- this.id = this.name + '-' + this.axis
- this.waypoints = []
- this.clearTriggerQueues()
- groups[this.axis][this.name] = this
- }
-
- /* Private */
- Group.prototype.add = function(waypoint) {
- this.waypoints.push(waypoint)
- }
-
- /* Private */
- Group.prototype.clearTriggerQueues = function() {
- this.triggerQueues = {
- up: [],
- down: [],
- left: [],
- right: []
- }
- }
-
- /* Private */
- Group.prototype.flushTriggers = function() {
- for (var direction in this.triggerQueues) {
- var waypoints = this.triggerQueues[direction]
- var reverse = direction === 'up' || direction === 'left'
- waypoints.sort(reverse ? byReverseTriggerPoint : byTriggerPoint)
- for (var i = 0, end = waypoints.length; i < end; i += 1) {
- var waypoint = waypoints[i]
- if (waypoint.options.continuous || i === waypoints.length - 1) {
- waypoint.trigger([direction])
- }
- }
- }
- this.clearTriggerQueues()
- }
-
- /* Private */
- Group.prototype.next = function(waypoint) {
- this.waypoints.sort(byTriggerPoint)
- var index = Waypoint.Adapter.inArray(waypoint, this.waypoints)
- var isLast = index === this.waypoints.length - 1
- return isLast ? null : this.waypoints[index + 1]
- }
-
- /* Private */
- Group.prototype.previous = function(waypoint) {
- this.waypoints.sort(byTriggerPoint)
- var index = Waypoint.Adapter.inArray(waypoint, this.waypoints)
- return index ? this.waypoints[index - 1] : null
- }
-
- /* Private */
- Group.prototype.queueTrigger = function(waypoint, direction) {
- this.triggerQueues[direction].push(waypoint)
- }
-
- /* Private */
- Group.prototype.remove = function(waypoint) {
- var index = Waypoint.Adapter.inArray(waypoint, this.waypoints)
- if (index > -1) {
- this.waypoints.splice(index, 1)
- }
- }
-
- /* Public */
- /* http://imakewebthings.com/waypoints/api/first */
- Group.prototype.first = function() {
- return this.waypoints[0]
- }
-
- /* Public */
- /* http://imakewebthings.com/waypoints/api/last */
- Group.prototype.last = function() {
- return this.waypoints[this.waypoints.length - 1]
- }
-
- /* Private */
- Group.findOrCreate = function(options) {
- return groups[options.axis][options.name] || new Group(options)
- }
-
- Waypoint.Group = Group
-}())
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/ecc6de22/hawq-book/master_middleman/source/javascripts/waypoints/noframeworkAdapter.js
----------------------------------------------------------------------
diff --git a/hawq-book/master_middleman/source/javascripts/waypoints/noframeworkAdapter.js b/hawq-book/master_middleman/source/javascripts/waypoints/noframeworkAdapter.js
deleted file mode 100644
index 99abcb5..0000000
--- a/hawq-book/master_middleman/source/javascripts/waypoints/noframeworkAdapter.js
+++ /dev/null
@@ -1,213 +0,0 @@
-(function() {
- 'use strict'
-
- var Waypoint = window.Waypoint
-
- function isWindow(element) {
- return element === element.window
- }
-
- function getWindow(element) {
- if (isWindow(element)) {
- return element
- }
- return element.defaultView
- }
-
- function classNameRegExp(className) {
- return new RegExp("\\b" + className + "\\b");
- }
-
- function NoFrameworkAdapter(element) {
- this.element = element
- this.handlers = {}
- }
-
- NoFrameworkAdapter.prototype.innerHeight = function() {
- var isWin = isWindow(this.element)
- return isWin ? this.element.innerHeight : this.element.clientHeight
- }
-
- NoFrameworkAdapter.prototype.innerWidth = function() {
- var isWin = isWindow(this.element)
- return isWin ? this.element.innerWidth : this.element.clientWidth
- }
-
- NoFrameworkAdapter.prototype.off = function(event, handler) {
- function removeListeners(element, listeners, handler) {
- for (var i = 0, end = listeners.length - 1; i < end; i++) {
- var listener = listeners[i]
- if (!handler || handler === listener) {
- element.removeEventListener(listener)
- }
- }
- }
-
- var eventParts = event.split('.')
- var eventType = eventParts[0]
- var namespace = eventParts[1]
- var element = this.element
-
- if (namespace && this.handlers[namespace] && eventType) {
- removeListeners(element, this.handlers[namespace][eventType], handler)
- this.handlers[namespace][eventType] = []
- }
- else if (eventType) {
- for (var ns in this.handlers) {
- removeListeners(element, this.handlers[ns][eventType] || [], handler)
- this.handlers[ns][eventType] = []
- }
- }
- else if (namespace && this.handlers[namespace]) {
- for (var type in this.handlers[namespace]) {
- removeListeners(element, this.handlers[namespace][type], handler)
- }
- this.handlers[namespace] = {}
- }
- }
-
- /* Adapted from jQuery 1.x offset() */
- NoFrameworkAdapter.prototype.offset = function() {
- if (!this.element.ownerDocument) {
- return null
- }
-
- var documentElement = this.element.ownerDocument.documentElement
- var win = getWindow(this.element.ownerDocument)
- var rect = {
- top: 0,
- left: 0
- }
-
- if (this.element.getBoundingClientRect) {
- rect = this.element.getBoundingClientRect()
- }
-
- return {
- top: rect.top + win.pageYOffset - documentElement.clientTop,
- left: rect.left + win.pageXOffset - documentElement.clientLeft
- }
- }
-
- NoFrameworkAdapter.prototype.on = function(event, handler) {
- var eventParts = event.split('.')
- var eventType = eventParts[0]
- var namespace = eventParts[1] || '__default'
- var nsHandlers = this.handlers[namespace] = this.handlers[namespace] || {}
- var nsTypeList = nsHandlers[eventType] = nsHandlers[eventType] || []
-
- nsTypeList.push(handler)
- this.element.addEventListener(eventType, handler)
- }
-
- NoFrameworkAdapter.prototype.outerHeight = function(includeMargin) {
- var height = this.innerHeight()
- var computedStyle
-
- if (includeMargin && !isWindow(this.element)) {
- computedStyle = window.getComputedStyle(this.element)
- height += parseInt(computedStyle.marginTop, 10)
- height += parseInt(computedStyle.marginBottom, 10)
- }
-
- return height
- }
-
- NoFrameworkAdapter.prototype.outerWidth = function(includeMargin) {
- var width = this.innerWidth()
- var computedStyle
-
- if (includeMargin && !isWindow(this.element)) {
- computedStyle = window.getComputedStyle(this.element)
- width += parseInt(computedStyle.marginLeft, 10)
- width += parseInt(computedStyle.marginRight, 10)
- }
-
- return width
- }
-
- NoFrameworkAdapter.prototype.scrollLeft = function() {
- var win = getWindow(this.element)
- return win ? win.pageXOffset : this.element.scrollLeft
- }
-
- NoFrameworkAdapter.prototype.scrollTop = function() {
- var win = getWindow(this.element)
- return win ? win.pageYOffset : this.element.scrollTop
- }
-
- NoFrameworkAdapter.prototype.height = function(newHeight) {
- this.element.style.height = newHeight;
- }
-
- NoFrameworkAdapter.prototype.removeClass = function(className) {
- this.element.className = this.element.className.replace(classNameRegExp(className), '');
- }
-
- NoFrameworkAdapter.prototype.toggleClass = function(className, addClass) {
- var check = classNameRegExp(className);
- if (check.test(this.element.className)) {
- if (!addClass) {
- this.removeClass(className);
- }
- } else {
- this.element.className += ' ' + className;
- }
- }
-
- NoFrameworkAdapter.prototype.parent = function() {
- return new NoFrameworkAdapter(this.element.parentNode);
- }
-
- NoFrameworkAdapter.prototype.wrap = function(wrapper) {
- this.element.insertAdjacentHTML('beforebegin', wrapper)
- var wrapperNode = this.element.previousSibling
- this.element.parentNode.removeChild(this.element)
- wrapperNode.appendChild(this.element)
- }
-
- NoFrameworkAdapter.extend = function() {
- var args = Array.prototype.slice.call(arguments)
-
- function merge(target, obj) {
- if (typeof target === 'object' && typeof obj === 'object') {
- for (var key in obj) {
- if (obj.hasOwnProperty(key)) {
- target[key] = obj[key]
- }
- }
- }
-
- return target
- }
-
- for (var i = 1, end = args.length; i < end; i++) {
- merge(args[0], args[i])
- }
- return args[0]
- }
-
- NoFrameworkAdapter.inArray = function(element, array, i) {
- return array == null ? -1 : array.indexOf(element, i)
- }
-
- NoFrameworkAdapter.isEmptyObject = function(obj) {
- /* eslint no-unused-vars: 0 */
- for (var name in obj) {
- return false
- }
- return true
- }
-
- NoFrameworkAdapter.proxy = function(func, obj) {
- return function() {
- return func.apply(obj, arguments);
- }
- }
-
- Waypoint.adapters.push({
- name: 'noframework',
- Adapter: NoFrameworkAdapter
- })
- Waypoint.Adapter = NoFrameworkAdapter
-}())
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/ecc6de22/hawq-book/master_middleman/source/javascripts/waypoints/sticky.js
----------------------------------------------------------------------
diff --git a/hawq-book/master_middleman/source/javascripts/waypoints/sticky.js b/hawq-book/master_middleman/source/javascripts/waypoints/sticky.js
deleted file mode 100644
index 569fcdb..0000000
--- a/hawq-book/master_middleman/source/javascripts/waypoints/sticky.js
+++ /dev/null
@@ -1,63 +0,0 @@
-(function() {
- 'use strict'
-
- var Waypoint = window.Waypoint;
- var adapter = Waypoint.Adapter;
-
- /* http://imakewebthings.com/waypoints/shortcuts/sticky-elements */
- function Sticky(options) {
- this.options = adapter.extend({}, Waypoint.defaults, Sticky.defaults, options)
- this.element = this.options.element
- this.$element = new adapter(this.element)
- this.createWrapper()
- this.createWaypoint()
- }
-
- /* Private */
- Sticky.prototype.createWaypoint = function() {
- var originalHandler = this.options.handler
-
- this.waypoint = new Waypoint(adapter.extend({}, this.options, {
- element: this.wrapper,
- handler: adapter.proxy(function(direction) {
- var shouldBeStuck = this.options.direction.indexOf(direction) > -1
- var wrapperHeight = shouldBeStuck ? this.$element.outerHeight(true) : ''
-
- this.$wrapper.height(wrapperHeight)
- this.$element.toggleClass(this.options.stuckClass, shouldBeStuck)
-
- if (originalHandler) {
- originalHandler.call(this, direction)
- }
- }, this)
- }))
- }
-
- /* Private */
- Sticky.prototype.createWrapper = function() {
- if (this.options.wrapper) {
- this.$element.wrap(this.options.wrapper)
- }
- this.$wrapper = this.$element.parent()
- this.wrapper = this.$wrapper.element
- }
-
- /* Public */
- Sticky.prototype.destroy = function() {
- if (this.$element.parent().element === this.wrapper) {
- this.waypoint.destroy()
- this.$element.removeClass(this.options.stuckClass)
- if (this.options.wrapper) {
- this.$element.unwrap()
- }
- }
- }
-
- Sticky.defaults = {
- wrapper: '<div class="sticky-wrapper" />',
- stuckClass: 'stuck',
- direction: 'down right'
- }
-
- Waypoint.Sticky = Sticky
-}())
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/ecc6de22/hawq-book/master_middleman/source/javascripts/waypoints/waypoint.js
----------------------------------------------------------------------
diff --git a/hawq-book/master_middleman/source/javascripts/waypoints/waypoint.js b/hawq-book/master_middleman/source/javascripts/waypoints/waypoint.js
deleted file mode 100644
index 7f76f1d..0000000
--- a/hawq-book/master_middleman/source/javascripts/waypoints/waypoint.js
+++ /dev/null
@@ -1,160 +0,0 @@
-(function() {
- 'use strict'
-
- var keyCounter = 0
- var allWaypoints = {}
-
- /* http://imakewebthings.com/waypoints/api/waypoint */
- function Waypoint(options) {
- if (!options) {
- throw new Error('No options passed to Waypoint constructor')
- }
- if (!options.element) {
- throw new Error('No element option passed to Waypoint constructor')
- }
- if (!options.handler) {
- throw new Error('No handler option passed to Waypoint constructor')
- }
-
- this.key = 'waypoint-' + keyCounter
- this.options = Waypoint.Adapter.extend({}, Waypoint.defaults, options)
- this.element = this.options.element
- this.adapter = new Waypoint.Adapter(this.element)
- this.callback = options.handler
- this.axis = this.options.horizontal ? 'horizontal' : 'vertical'
- this.enabled = this.options.enabled
- this.triggerPoint = null
- this.group = Waypoint.Group.findOrCreate({
- name: this.options.group,
- axis: this.axis
- })
- this.context = Waypoint.Context.findOrCreateByElement(this.options.context)
-
- if (Waypoint.offsetAliases[this.options.offset]) {
- this.options.offset = Waypoint.offsetAliases[this.options.offset]
- }
- this.group.add(this)
- this.context.add(this)
- allWaypoints[this.key] = this
- keyCounter += 1
- }
-
- /* Private */
- Waypoint.prototype.queueTrigger = function(direction) {
- this.group.queueTrigger(this, direction)
- }
-
- /* Private */
- Waypoint.prototype.trigger = function(args) {
- if (!this.enabled) {
- return
- }
- if (this.callback) {
- this.callback.apply(this, args)
- }
- }
-
- /* Public */
- /* http://imakewebthings.com/waypoints/api/destroy */
- Waypoint.prototype.destroy = function() {
- this.context.remove(this)
- this.group.remove(this)
- delete allWaypoints[this.key]
- }
-
- /* Public */
- /* http://imakewebthings.com/waypoints/api/disable */
- Waypoint.prototype.disable = function() {
- this.enabled = false
- return this
- }
-
- /* Public */
- /* http://imakewebthings.com/waypoints/api/enable */
- Waypoint.prototype.enable = function() {
- this.context.refresh()
- this.enabled = true
- return this
- }
-
- /* Public */
- /* http://imakewebthings.com/waypoints/api/next */
- Waypoint.prototype.next = function() {
- return this.group.next(this)
- }
-
- /* Public */
- /* http://imakewebthings.com/waypoints/api/previous */
- Waypoint.prototype.previous = function() {
- return this.group.previous(this)
- }
-
- /* Private */
- Waypoint.invokeAll = function(method) {
- var allWaypointsArray = []
- for (var waypointKey in allWaypoints) {
- allWaypointsArray.push(allWaypoints[waypointKey])
- }
- for (var i = 0, end = allWaypointsArray.length; i < end; i++) {
- allWaypointsArray[i][method]()
- }
- }
-
- /* Public */
- /* http://imakewebthings.com/waypoints/api/destroy-all */
- Waypoint.destroyAll = function() {
- Waypoint.invokeAll('destroy')
- }
-
- /* Public */
- /* http://imakewebthings.com/waypoints/api/disable-all */
- Waypoint.disableAll = function() {
- Waypoint.invokeAll('disable')
- }
-
- /* Public */
- /* http://imakewebthings.com/waypoints/api/enable-all */
- Waypoint.enableAll = function() {
- Waypoint.invokeAll('enable')
- }
-
- /* Public */
- /* http://imakewebthings.com/waypoints/api/refresh-all */
- Waypoint.refreshAll = function() {
- Waypoint.Context.refreshAll()
- }
-
- /* Public */
- /* http://imakewebthings.com/waypoints/api/viewport-height */
- Waypoint.viewportHeight = function() {
- return window.innerHeight || document.documentElement.clientHeight
- }
-
- /* Public */
- /* http://imakewebthings.com/waypoints/api/viewport-width */
- Waypoint.viewportWidth = function() {
- return document.documentElement.clientWidth
- }
-
- Waypoint.adapters = []
-
- Waypoint.defaults = {
- context: window,
- continuous: true,
- enabled: true,
- group: 'default',
- horizontal: false,
- offset: 0
- }
-
- Waypoint.offsetAliases = {
- 'bottom-in-view': function() {
- return this.context.innerHeight() - this.adapter.outerHeight()
- },
- 'right-in-view': function() {
- return this.context.innerWidth() - this.adapter.outerWidth()
- }
- }
-
- window.Waypoint = Waypoint
-}())
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/ecc6de22/hawq-book/master_middleman/source/layouts/_title.erb
----------------------------------------------------------------------
diff --git a/hawq-book/master_middleman/source/layouts/_title.erb b/hawq-book/master_middleman/source/layouts/_title.erb
deleted file mode 100644
index ea744d9..0000000
--- a/hawq-book/master_middleman/source/layouts/_title.erb
+++ /dev/null
@@ -1,6 +0,0 @@
-<% if current_page.data.title %>
- <h1 class="title-container" <%= current_page.data.dita ? 'style="display: none;"' : '' %>>
- <%= current_page.data.title %>
- </h1>
-<% end %>
-
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/ecc6de22/hawq-book/master_middleman/source/patch/dynamic_variable_interpretation.py
----------------------------------------------------------------------
diff --git a/hawq-book/master_middleman/source/patch/dynamic_variable_interpretation.py b/hawq-book/master_middleman/source/patch/dynamic_variable_interpretation.py
deleted file mode 100644
index 66df9ff..0000000
--- a/hawq-book/master_middleman/source/patch/dynamic_variable_interpretation.py
+++ /dev/null
@@ -1,192 +0,0 @@
-#!/usr/bin/env python
-"""
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements. See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership. The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
-
-"""
-
-__all__ = ["copy_tarballs_to_hdfs", ]
-import os
-import glob
-import re
-import tempfile
-from resource_management.libraries.functions.default import default
-from resource_management.libraries.functions.format import format
-from resource_management.libraries.resources.copy_from_local import CopyFromLocal
-from resource_management.libraries.resources.execute_hadoop import ExecuteHadoop
-from resource_management.core.resources.system import Execute
-from resource_management.core.exceptions import Fail
-from resource_management.core.logger import Logger
-from resource_management.core import shell
-
-"""
-This file provides helper methods needed for the versioning of RPMs. Specifically, it does dynamic variable
-interpretation to replace strings like {{ hdp_stack_version }} where the value of the
-variables cannot be determined ahead of time, but rather, depends on what files are found.
-
-It assumes that {{ hdp_stack_version }} is constructed as ${major.minor.patch.rev}-${build_number}
-E.g., 998.2.2.1.0-998
-Please note that "-${build_number}" is optional.
-"""
-
-# These values must be the suffix of the properties in cluster-env.xml
-TAR_SOURCE_SUFFIX = "_tar_source"
-TAR_DESTINATION_FOLDER_SUFFIX = "_tar_destination_folder"
-
-
-def _get_tar_source_and_dest_folder(tarball_prefix):
- """
- :param tarball_prefix: Prefix of the tarball must be one of tez, hive, mr, pig
- :return: Returns a tuple of (x, y) after verifying the properties
- """
- component_tar_source_file = default("/configurations/cluster-env/%s%s" % (tarball_prefix.lower(), TAR_SOURCE_SUFFIX), None)
- # E.g., /usr/hdp/current/hadoop-client/tez-{{ hdp_stack_version }}.tar.gz
-
- component_tar_destination_folder = default("/configurations/cluster-env/%s%s" % (tarball_prefix.lower(), TAR_DESTINATION_FOLDER_SUFFIX), None)
- # E.g., hdfs:///hdp/apps/{{ hdp_stack_version }}/mapreduce/
-
- if not component_tar_source_file or not component_tar_destination_folder:
- Logger.warning("Did not find %s tar source file and destination folder properties in cluster-env.xml" %
- tarball_prefix)
- return None, None
-
- if component_tar_source_file.find("/") == -1:
- Logger.warning("The tar file path %s is not valid" % str(component_tar_source_file))
- return None, None
-
- if not component_tar_destination_folder.endswith("/"):
- component_tar_destination_folder = component_tar_destination_folder + "/"
-
- if not component_tar_destination_folder.startswith("hdfs://"):
- return None, None
-
- return component_tar_source_file, component_tar_destination_folder
-
-
-def _copy_files(source_and_dest_pairs, file_owner, group_owner, kinit_if_needed):
- """
- :param source_and_dest_pairs: List of tuples (x, y), where x is the source file in the local file system,
- and y is the destination file path in HDFS
- :param file_owner: Owner to set for the file copied to HDFS (typically hdfs account)
- :param group_owner: Owning group to set for the file copied to HDFS (typically hadoop group)
- :param kinit_if_needed: kinit command if it is needed, otherwise an empty string
- :return: Returns 0 if at least one file was copied and no exceptions occurred, and 1 otherwise.
-
- Must kinit before calling this function.
- """
- import params
-
- return_value = 1
- if source_and_dest_pairs and len(source_and_dest_pairs) > 0:
- return_value = 0
- for (source, destination) in source_and_dest_pairs:
- try:
- destination_dir = os.path.dirname(destination)
-
- params.HdfsDirectory(destination_dir,
- action="create",
- owner=file_owner,
- mode=0555
- )
-
- CopyFromLocal(source,
- mode=0444,
- owner=file_owner,
- group=group_owner,
- dest_dir=destination_dir,
- kinnit_if_needed=kinit_if_needed,
- hdfs_user=params.hdfs_user,
- hadoop_bin_dir=params.hadoop_bin_dir,
- hadoop_conf_dir=params.hadoop_conf_dir
- )
- except:
- return_value = 1
- return return_value
-
-
-def copy_tarballs_to_hdfs(tarball_prefix, component_user, file_owner, group_owner):
- """
- :param tarball_prefix: Prefix of the tarball must be one of tez, hive, mr, pig
- :param component_user: User that will execute the Hadoop commands
- :param file_owner: Owner of the files copied to HDFS (typically hdfs account)
- :param group_owner: Group owner of the files copied to HDFS (typically hadoop group)
- :return: Returns 0 on success, 1 if no files were copied, and in some cases may raise an exception.
-
- In order to call this function, params.py must have all of the following,
- hdp_stack_version, kinit_path_local, security_enabled, hdfs_user, hdfs_principal_name, hdfs_user_keytab,
- hadoop_bin_dir, hadoop_conf_dir, and HdfsDirectory as a partial function.
- """
- import params
-
- if not hasattr(params, "hdp_stack_version") or params.hdp_stack_version is None:
- Logger.warning("Could not find hdp_stack_version")
- return 1
-
- component_tar_source_file, component_tar_destination_folder = _get_tar_source_and_dest_folder(tarball_prefix)
- if not component_tar_source_file or not component_tar_destination_folder:
- Logger.warning("Could not retrieve properties for tarball with prefix: %s" % str(tarball_prefix))
- return 1
-
- if not os.path.exists(component_tar_source_file):
- Logger.warning("Could not find file: %s" % str(component_tar_source_file))
- return 1
-
- # Ubuntu returns: "stdin: is not a tty", as subprocess output.
- tmpfile = tempfile.NamedTemporaryFile()
- with open(tmpfile.name, 'r+') as file:
- get_hdp_version_cmd = '/usr/bin/hdp-select versions > %s' % tmpfile.name
- code, stdoutdata = shell.call(get_hdp_version_cmd)
- out = file.read()
- pass
- if code != 0 or out is None:
- Logger.warning("Could not verify HDP version by calling '%s'. Return Code: %s, Output: %s." %
- (get_hdp_version_cmd, str(code), str(out)))
- return 1
-
- hdp_version = out.strip() # this should include the build number
-
- file_name = os.path.basename(component_tar_source_file)
- destination_file = os.path.join(component_tar_destination_folder, file_name)
- destination_file = destination_file.replace("{{ hdp_stack_version }}", hdp_version)
-
- does_hdfs_file_exist_cmd = "fs -ls %s" % destination_file
-
- kinit_if_needed = ""
- if params.security_enabled:
- kinit_if_needed = format("{kinit_path_local} -kt {hdfs_user_keytab} {hdfs_principal_name};")
-
- if kinit_if_needed:
- Execute(kinit_if_needed,
- user=component_user,
- path='/bin'
- )
-
- does_hdfs_file_exist = False
- try:
- ExecuteHadoop(does_hdfs_file_exist_cmd,
- user=component_user,
- logoutput=True,
- conf_dir=params.hadoop_conf_dir,
- bin_dir=params.hadoop_bin_dir
- )
- does_hdfs_file_exist = True
- except Fail:
- pass
-
- if not does_hdfs_file_exist:
- source_and_dest_pairs = [(component_tar_source_file, destination_file), ]
- return _copy_files(source_and_dest_pairs, file_owner, group_owner, kinit_if_needed)
- return 1
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/ecc6de22/hawq-book/master_middleman/source/stylesheets/book-styles.css.scss
----------------------------------------------------------------------
diff --git a/hawq-book/master_middleman/source/stylesheets/book-styles.css.scss b/hawq-book/master_middleman/source/stylesheets/book-styles.css.scss
deleted file mode 100644
index 1236d8e..0000000
--- a/hawq-book/master_middleman/source/stylesheets/book-styles.css.scss
+++ /dev/null
@@ -1,3 +0,0 @@
-* {
- box-sizing: border-box;
-}
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/ecc6de22/hawq-book/master_middleman/source/stylesheets/partials/_book-base-values.scss
----------------------------------------------------------------------
diff --git a/hawq-book/master_middleman/source/stylesheets/partials/_book-base-values.scss b/hawq-book/master_middleman/source/stylesheets/partials/_book-base-values.scss
deleted file mode 100644
index e69de29..0000000
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/ecc6de22/hawq-book/master_middleman/source/stylesheets/partials/_book-vars.scss
----------------------------------------------------------------------
diff --git a/hawq-book/master_middleman/source/stylesheets/partials/_book-vars.scss b/hawq-book/master_middleman/source/stylesheets/partials/_book-vars.scss
deleted file mode 100644
index 4245d57..0000000
--- a/hawq-book/master_middleman/source/stylesheets/partials/_book-vars.scss
+++ /dev/null
@@ -1,19 +0,0 @@
-$navy: #243640;
-$blue1: #2185c5;
-$blue2: #a7cae1;
-$bluegray1: #4b6475;
-$teal1: #03786D;
-$teal2: #00a79d;
-
-$color-accent: $teal1;
-$color-accent-bright: $teal2;
-
-// link colors
-$color-link: $blue1;
-$color-link-border: $blue2;
-
-$color-border-tip: $blue2;
-
-$color-bg-header: $navy;
-$color-bg-dark: $bluegray1;
-
[15/50] [abbrv] incubator-hawq-docs git commit: Merge branch
'feature/sshextkeys-pwd' of https://github.com/lisakowen/incubator-hawq-docs
into develop
Posted by yo...@apache.org.
Merge branch 'feature/sshextkeys-pwd' of https://github.com/lisakowen/incubator-hawq-docs into develop
Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/commit/32ccc733
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/tree/32ccc733
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/diff/32ccc733
Branch: refs/heads/master
Commit: 32ccc733eb4b2c9b32830970aec82daed0a779e5
Parents: aa361f2 2a0b4a9
Author: David Yozie <yo...@apache.org>
Authored: Wed Sep 14 09:26:39 2016 -0700
Committer: David Yozie <yo...@apache.org>
Committed: Wed Sep 14 09:26:39 2016 -0700
----------------------------------------------------------------------
reference/cli/admin_utilities/hawqssh-exkeys.html.md.erb | 7 +++++--
1 file changed, 5 insertions(+), 2 deletions(-)
----------------------------------------------------------------------
[28/50] [abbrv] incubator-hawq-docs git commit: get HAWQ from product
distribution or build from source
Posted by yo...@apache.org.
get HAWQ from product distribution or build from source
Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/commit/e429aaaa
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/tree/e429aaaa
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/diff/e429aaaa
Branch: refs/heads/master
Commit: e429aaaab72edb5f1b10501bfeea6ee1bb477b9e
Parents: 19b9c98
Author: Lisa Owen <lo...@pivotal.io>
Authored: Tue Sep 20 09:03:00 2016 -0700
Committer: Lisa Owen <lo...@pivotal.io>
Committed: Tue Sep 20 09:03:00 2016 -0700
----------------------------------------------------------------------
datamgmt/load/client-loadtools.html.md.erb | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/e429aaaa/datamgmt/load/client-loadtools.html.md.erb
----------------------------------------------------------------------
diff --git a/datamgmt/load/client-loadtools.html.md.erb b/datamgmt/load/client-loadtools.html.md.erb
index af5d4c2..9a83e61 100644
--- a/datamgmt/load/client-loadtools.html.md.erb
+++ b/datamgmt/load/client-loadtools.html.md.erb
@@ -7,14 +7,14 @@ This topic presents the instructions to install the HAWQ Load Tools on your clie
## <a id="installloadrunrhel"></a>RHEL Load Tools
-The RHEL Load Tools are provided in the HAWQ installation package.
+The RHEL Load Tools are provided in a HAWQ distribution.
### <a id="installloadrunux"></a>Installing the RHEL Loader
-1. Download the `hdb-2.0.1.0-nnnnn.tar.gz` installer package from [Pivotal Network](https://network.pivotal.io/products/pivotal-hdb). Make note of the directory to which the file was downloaded.
+1. Download a HAWQ installer package or build HAWQ from source.
-2. Refer to the HAWQ command line install instructions to set up your HDB repository and install the HAWQ binary.
+2. Refer to the HAWQ command line install instructions to set up your package repositories and install the HAWQ binary.
### <a id="installrhelloadabout"></a>About the RHEL Loader Installation
[35/50] [abbrv] incubator-hawq-docs git commit: add missing space
Posted by yo...@apache.org.
add missing space
Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/commit/3f283772
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/tree/3f283772
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/diff/3f283772
Branch: refs/heads/master
Commit: 3f28377290794cb3f6ce64eea8b4094589cd92c1
Parents: b6dd137
Author: Lisa Owen <lo...@pivotal.io>
Authored: Fri Sep 23 11:44:37 2016 -0700
Committer: Lisa Owen <lo...@pivotal.io>
Committed: Fri Sep 23 11:44:37 2016 -0700
----------------------------------------------------------------------
ddl/ddl-storage.html.md.erb | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/3f283772/ddl/ddl-storage.html.md.erb
----------------------------------------------------------------------
diff --git a/ddl/ddl-storage.html.md.erb b/ddl/ddl-storage.html.md.erb
index 3e89d4f..264e552 100644
--- a/ddl/ddl-storage.html.md.erb
+++ b/ddl/ddl-storage.html.md.erb
@@ -62,7 +62,7 @@ The`DROP TABLE`command removes tables from the database. For example:
DROP TABLE mytable;
```
-`DROP TABLE`always removes any indexes, rules, triggers, and constraints that exist for the target table. Specify `CASCADE`to drop a table that is referenced by a view. `CASCADE` removes dependent views.
+`DROP TABLE` always removes any indexes, rules, triggers, and constraints that exist for the target table. Specify `CASCADE`to drop a table that is referenced by a view. `CASCADE` removes dependent views.
To empty a table of rows without removing the table definition, use `TRUNCATE`. For example:
[40/50] [abbrv] incubator-hawq-docs git commit: vacuum - remove
duplicate sentence
Posted by yo...@apache.org.
vacuum - remove duplicate sentence
Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/commit/2cc9b024
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/tree/2cc9b024
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/diff/2cc9b024
Branch: refs/heads/master
Commit: 2cc9b02419ec7ac26a326f5a4f7d115f016ab26f
Parents: da84fa8
Author: Lisa Owen <lo...@pivotal.io>
Authored: Tue Sep 27 13:26:05 2016 -0700
Committer: Lisa Owen <lo...@pivotal.io>
Committed: Tue Sep 27 13:26:05 2016 -0700
----------------------------------------------------------------------
datamgmt/BasicDataOperations.html.md.erb | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/2cc9b024/datamgmt/BasicDataOperations.html.md.erb
----------------------------------------------------------------------
diff --git a/datamgmt/BasicDataOperations.html.md.erb b/datamgmt/BasicDataOperations.html.md.erb
index 6c3323c..66328c7 100644
--- a/datamgmt/BasicDataOperations.html.md.erb
+++ b/datamgmt/BasicDataOperations.html.md.erb
@@ -44,7 +44,7 @@ To insert large amounts of data, use external tables or the `COPY` command. Thes
Only HAWQ system catalog tables use multiple version concurrency control. Deleted or updated data rows in the catalog tables occupy physical space on disk even though new transactions cannot see them. Periodically running the `VACUUM` command removes these expired rows.
-Periodically running the `VACUUM` command on system catalog tables removes these expired rows. The `VACUUM` command also collects table-level statistics such as the number of rows and pages.
+The `VACUUM` command also collects table-level statistics such as the number of rows and pages.
For example:
[10/50] [abbrv] incubator-hawq-docs git commit: clarify wording for
disabling passwd-based auth
Posted by yo...@apache.org.
clarify wording for disabling passwd-based auth
Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/commit/831a83e7
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/tree/831a83e7
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/diff/831a83e7
Branch: refs/heads/master
Commit: 831a83e7ca4364d73a901b4288c5fcbc5e2e949f
Parents: 8793022
Author: Lisa Owen <lo...@pivotal.io>
Authored: Mon Sep 12 12:07:28 2016 -0700
Committer: Lisa Owen <lo...@pivotal.io>
Committed: Mon Sep 12 12:07:28 2016 -0700
----------------------------------------------------------------------
admin/ClusterExpansion.html.md.erb | 4 ++--
admin/ambari-admin.html.md.erb | 4 ++--
2 files changed, 4 insertions(+), 4 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/831a83e7/admin/ClusterExpansion.html.md.erb
----------------------------------------------------------------------
diff --git a/admin/ClusterExpansion.html.md.erb b/admin/ClusterExpansion.html.md.erb
index e72ad14..e335bd5 100644
--- a/admin/ClusterExpansion.html.md.erb
+++ b/admin/ClusterExpansion.html.md.erb
@@ -20,7 +20,7 @@ There are several recommendations to keep in mind when modifying the size of you
## <a id="task_hawq_expand"></a>Adding a New Node to an Existing HAWQ Cluster
-The following procedure describes the steps required to add a node to an existing HAWQ cluster. First ensure that the new node has been configured per the instructions in [Apache HAWQ System Requirements](/20/requirements/system-requirements.html) and [Select HAWQ Host Machines](/20/install/select-hosts.html).
+The following procedure describes the steps required to add a node to an existing HAWQ cluster. First ensure that the new node has been configured per the instructions found in [Apache HAWQ System Requirements](/20/requirements/system-requirements.html) and [Select HAWQ Host Machines](/20/install/select-hosts.html).
For example purposes in this procedure, we are adding a new node named `sdw4`.
@@ -71,7 +71,7 @@ For example purposes in this procedure, we are adding a new node named `sdw4`.
$ hawq ssh-exkeys -e hawq_hosts -x new_hosts
```
- 8. (Optional) Turn off temporary password-based authentication as described in [Apache HAWQ System Requirements](/20/requirements/system-requirements.html#topic_pwdlessssh).
+ 8. (Optional) If you enabled temporary password-based authentication while preparing/configuring your new HAWQ host system, turn off password-based authentication as described in [Apache HAWQ System Requirements](/20/requirements/system-requirements.html#topic_pwdlessssh).
8. After setting up passwordless ssh, you can execute the following hawq command to check the target machine's configuration.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/831a83e7/admin/ambari-admin.html.md.erb
----------------------------------------------------------------------
diff --git a/admin/ambari-admin.html.md.erb b/admin/ambari-admin.html.md.erb
index 54ac91c..96c292f 100644
--- a/admin/ambari-admin.html.md.erb
+++ b/admin/ambari-admin.html.md.erb
@@ -160,7 +160,7 @@ There are several recommendations to keep in mind when modifying the size of you
- If you are using hash tables, consider updating the `default_hash_table_bucket_number` server configuration parameter to a larger value after expanding the cluster but before redistributing the hash tables.
### Procedure
-First ensure that the new node(s) has been configured per the instructions in [Apache HAWQ System Requirements](/20/requirements/system-requirements.html) and [Select HAWQ Host Machines](/20/install/select-hosts.html).
+First ensure that the new node(s) has been configured per the instructions found in [Apache HAWQ System Requirements](/20/requirements/system-requirements.html) and [Select HAWQ Host Machines](/20/install/select-hosts.html).
1. If you have any user-defined function (UDF) libraries installed in your existing HAWQ cluster, install them on the new node(s) that you want to add to the HAWQ cluster.
2. Access the Ambari web console at http://ambari.server.hostname:8080, and login as the "admin" user. \(The default password is also "admin".\)
@@ -204,7 +204,7 @@ First ensure that the new node(s) has been configured per the instructions in [A
**Note:** The redistribution of table data can take a significant amount of time.
22. (Optional.) If you changed the **Exchange SSH Keys** property value before adding the host(s), change the value back to `false` after Ambari exchanges keys with the new hosts. This prevents Ambari from exchanging keys with all hosts every time the HAWQ master is started or restarted.
-23. (Optional.) Turn off temporary password-based authentication as described in [Apache HAWQ System Requirements](/20/requirements/system-requirements.html#topic_pwdlessssh).
+23. (Optional.) If you enabled temporary password-based authentication while preparing/configuring your HAWQ host systems, turn off password-based authentication as described in [Apache HAWQ System Requirements](/20/requirements/system-requirements.html#topic_pwdlessssh).
#### <a id="manual-config-steps"></a>Manually Updating the HAWQ Configuration
If you need to expand your HAWQ cluster without restarting the HAWQ service, follow these steps to manually apply the new HAWQ configuration. (Use these steps *instead* of following Step 7 in the above procedure.):
[26/50] [abbrv] incubator-hawq-docs git commit: fix typo
Posted by yo...@apache.org.
fix typo
Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/commit/23759ec6
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/tree/23759ec6
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/diff/23759ec6
Branch: refs/heads/master
Commit: 23759ec6e5fea9cf27ee1502459cbef38428240e
Parents: 1d78599
Author: Lisa Owen <lo...@pivotal.io>
Authored: Mon Sep 19 14:56:55 2016 -0700
Committer: Lisa Owen <lo...@pivotal.io>
Committed: Mon Sep 19 14:56:55 2016 -0700
----------------------------------------------------------------------
datamgmt/load/client-loadtools.html.md.erb | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/23759ec6/datamgmt/load/client-loadtools.html.md.erb
----------------------------------------------------------------------
diff --git a/datamgmt/load/client-loadtools.html.md.erb b/datamgmt/load/client-loadtools.html.md.erb
index 7ad9ee2..7555621 100644
--- a/datamgmt/load/client-loadtools.html.md.erb
+++ b/datamgmt/load/client-loadtools.html.md.erb
@@ -21,7 +21,7 @@ The RHEL Load Tools are provided in the HAWQ installation package.
The files/directories of interest in a HAWQ RHEL Load Tools installation include:
`bin/` \u2014 data loading command-line tools (`gpfdist` and `hawq load`)
-`greenplum_path.sh0` \u2014 environment set up file
+`greenplum_path.sh` \u2014 environment set up file
### <a id="installloadrhelcfgenv"></a>Configuring the RHEL Load Environment
[50/50] [abbrv] incubator-hawq-docs git commit: Merge branch 'develop'
Posted by yo...@apache.org.
Merge branch 'develop'
Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/commit/c3d7896c
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/tree/c3d7896c
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/diff/c3d7896c
Branch: refs/heads/master
Commit: c3d7896c3c0c2152e30b27f19a413d82e14de699
Parents: 52ce16b 10cde80
Author: David Yozie <yo...@apache.org>
Authored: Thu Sep 29 10:21:52 2016 -0700
Committer: David Yozie <yo...@apache.org>
Committed: Thu Sep 29 10:21:52 2016 -0700
----------------------------------------------------------------------
README.md | 8 +-
admin/ClusterExpansion.html.md.erb | 14 +-
...esandHighAvailabilityEnabledHDFS.html.md.erb | 18 +-
admin/RecommendedMonitoringTasks.html.md.erb | 10 +-
admin/ambari-admin.html.md.erb | 20 +-
admin/monitor.html.md.erb | 2 +-
admin/startstop.html.md.erb | 4 +-
bestpractices/general_bestpractices.html.md.erb | 1 +
clientaccess/client_auth.html.md.erb | 14 +-
clientaccess/disable-kerberos.html.md.erb | 11 +-
...-database-application-interfaces.html.md.erb | 15 +-
.../g-supported-client-applications.html.md.erb | 2 +-
clientaccess/kerberos.html.md.erb | 20 +-
clientaccess/roles_privs.html.md.erb | 2 +-
datamgmt/BasicDataOperations.html.md.erb | 12 +-
datamgmt/ConcurrencyControl.html.md.erb | 7 -
datamgmt/Transactions.html.md.erb | 10 +-
datamgmt/about_statistics.html.md.erb | 8 +-
datamgmt/load/client-loadtools.html.md.erb | 58 +-
...-data-load-and-query-performance.html.md.erb | 6 +-
ddl/ddl-partition.html.md.erb | 6 +-
ddl/ddl-storage.html.md.erb | 9 +-
ddl/ddl-table.html.md.erb | 6 +-
.../source/subnavs/apache-hawq-nav.erb | 955 -------------------
install/aws-config.html.md.erb | 18 +-
overview/HAWQArchitecture.html.md.erb | 4 +-
overview/ManagementTools.html.md.erb | 2 +-
overview/RedundancyFailover.html.md.erb | 2 +-
overview/ResourceManagement.html.md.erb | 2 +-
overview/TableDistributionStorage.html.md.erb | 6 +-
plext/using_pljava.html.md.erb | 4 +-
pxf/HawqExtensionFrameworkPXF.html.md.erb | 4 +
pxf/HivePXF.html.md.erb | 18 +-
pxf/InstallPXFPlugins.html.md.erb | 12 +-
pxf/TroubleshootingPXF.html.md.erb | 71 +-
query/functions-operators.html.md.erb | 2 +-
reference/HAWQDataTypes.html.md.erb | 4 +-
reference/HAWQEnvironmentVariables.html.md.erb | 12 +-
reference/catalog/pg_class.html.md.erb | 2 +-
reference/catalog/pg_index.html.md.erb | 2 +-
.../cli/admin_utilities/gpfdist.html.md.erb | 2 +-
.../cli/admin_utilities/gplogfilter.html.md.erb | 16 +-
.../cli/admin_utilities/hawqcheck.html.md.erb | 25 +-
.../cli/admin_utilities/hawqload.html.md.erb | 2 +-
.../admin_utilities/hawqregister.html.md.erb | 27 +-
.../admin_utilities/hawqssh-exkeys.html.md.erb | 7 +-
.../cli/admin_utilities/hawqstate.html.md.erb | 8 -
.../cli/client_utilities/vacuumdb.html.md.erb | 2 +
reference/guc/guc_category-list.html.md.erb | 3 +-
reference/guc/parameter_definitions.html.md.erb | 25 +-
reference/sql/ALTER-RESOURCE-QUEUE.html.md.erb | 8 +-
reference/sql/ALTER-TABLE.html.md.erb | 2 +-
reference/sql/BEGIN.html.md.erb | 2 +-
reference/sql/COPY.html.md.erb | 2 +-
reference/sql/CREATE-EXTERNAL-TABLE.html.md.erb | 2 +-
reference/sql/CREATE-RESOURCE-QUEUE.html.md.erb | 8 +-
reference/sql/CREATE-TABLE.html.md.erb | 10 +-
reference/sql/CREATE-VIEW.html.md.erb | 2 +-
reference/sql/DROP-TABLE.html.md.erb | 2 +-
reference/sql/GRANT.html.md.erb | 2 +-
reference/sql/PREPARE.html.md.erb | 2 +-
reference/sql/VACUUM.html.md.erb | 10 +-
reference/toolkit/hawq_toolkit.html.md.erb | 2 +-
requirements/system-requirements.html.md.erb | 44 +-
.../ConfigureResourceManagement.html.md.erb | 6 +-
resourcemgmt/HAWQResourceManagement.html.md.erb | 2 +-
resourcemgmt/ResourceQueues.html.md.erb | 8 +-
resourcemgmt/YARNIntegration.html.md.erb | 2 +-
resourcemgmt/best-practices.html.md.erb | 4 +-
troubleshooting/Troubleshooting.html.md.erb | 10 +-
70 files changed, 392 insertions(+), 1238 deletions(-)
----------------------------------------------------------------------
[32/50] [abbrv] incubator-hawq-docs git commit: first pass at
cleaning up update, delete, vacuum references
Posted by yo...@apache.org.
first pass at cleaning up update, delete, vacuum references
Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/commit/ffda5ae3
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/tree/ffda5ae3
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/diff/ffda5ae3
Branch: refs/heads/master
Commit: ffda5ae32c77edae77c8279e855adc09fba2fe33
Parents: 37857ea
Author: Lisa Owen <lo...@pivotal.io>
Authored: Thu Sep 22 14:37:13 2016 -0700
Committer: Lisa Owen <lo...@pivotal.io>
Committed: Thu Sep 22 14:37:13 2016 -0700
----------------------------------------------------------------------
admin/RecommendedMonitoringTasks.html.md.erb | 6 ++----
admin/monitor.html.md.erb | 2 +-
clientaccess/roles_privs.html.md.erb | 2 +-
datamgmt/BasicDataOperations.html.md.erb | 10 ++++++----
datamgmt/Transactions.html.md.erb | 10 ++++------
datamgmt/about_statistics.html.md.erb | 8 +++-----
...optimizing-data-load-and-query-performance.html.md.erb | 6 +++---
ddl/ddl-partition.html.md.erb | 4 ++--
ddl/ddl-storage.html.md.erb | 9 +++------
reference/catalog/pg_class.html.md.erb | 2 +-
reference/catalog/pg_index.html.md.erb | 2 +-
reference/cli/admin_utilities/hawqload.html.md.erb | 2 +-
reference/cli/client_utilities/vacuumdb.html.md.erb | 2 ++
reference/guc/guc_category-list.html.md.erb | 2 +-
reference/guc/parameter_definitions.html.md.erb | 6 +-----
reference/sql/ALTER-TABLE.html.md.erb | 2 +-
reference/sql/BEGIN.html.md.erb | 2 +-
reference/sql/COPY.html.md.erb | 2 +-
reference/sql/CREATE-EXTERNAL-TABLE.html.md.erb | 2 +-
reference/sql/CREATE-TABLE.html.md.erb | 4 ++--
reference/sql/CREATE-VIEW.html.md.erb | 2 +-
reference/sql/DROP-TABLE.html.md.erb | 2 +-
reference/sql/PREPARE.html.md.erb | 2 +-
reference/sql/VACUUM.html.md.erb | 10 +++++++---
reference/toolkit/hawq_toolkit.html.md.erb | 2 +-
25 files changed, 49 insertions(+), 54 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/ffda5ae3/admin/RecommendedMonitoringTasks.html.md.erb
----------------------------------------------------------------------
diff --git a/admin/RecommendedMonitoringTasks.html.md.erb b/admin/RecommendedMonitoringTasks.html.md.erb
index a01ea55..3007aee 100644
--- a/admin/RecommendedMonitoringTasks.html.md.erb
+++ b/admin/RecommendedMonitoringTasks.html.md.erb
@@ -193,7 +193,7 @@ GROUP BY 1;
<p>Severity: CRITICAL</p>
</td>
<td>
- <p><p>Vacuum an individual table:</p>
+ <p><p>Vacuum an individual system catalog table:</p>
<pre><code>VACUUM <<i>table</i>>;</code></pre>
</td>
<td>After the <a href="/20/reference/guc/parameter_definitions.html">vacuum_freeze_min_age</a> value is reached, VACUUM will no longer replace transaction IDs with <code>FrozenXID</code> while scanning a table. Perform vacuum on these tables before the limit is reached.</td>
@@ -224,9 +224,7 @@ GROUP BY 1;
<p>Recommended frequency: weekly, or more often if database objects are created and dropped frequently</p>
</td>
<td>
- <ol>
- <li><code>VACUUM</code> the system tables in each database.</li>
- </ol>
+ <p><code>VACUUM</code> the system tables in each database.</p>
</td>
<td>The optimizer retrieves information from the system tables to create query plans. If system tables and indexes are allowed to become bloated over time, scanning the system tables increases query execution time.</td>
</tr>
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/ffda5ae3/admin/monitor.html.md.erb
----------------------------------------------------------------------
diff --git a/admin/monitor.html.md.erb b/admin/monitor.html.md.erb
index 1e464e2..d1fbf31 100644
--- a/admin/monitor.html.md.erb
+++ b/admin/monitor.html.md.erb
@@ -78,7 +78,7 @@ HAWQ tracks various metadata information in its system catalogs about the object
#### <a id="topic25"></a>Viewing the Last Operation Performed
-You can use the system views *pg\_stat\_operations* and *pg\_stat\_partition\_operations* to look up actions performed on an object, such as a table. For example, to see the actions performed on a table, such as when it was created and when it was last vacuumed and analyzed:
+You can use the system views *pg\_stat\_operations* and *pg\_stat\_partition\_operations* to look up actions performed on an object, such as a table. For example, to see the actions performed on a table, such as when it was created and when it was last analyzed:
```sql
=> SELECT schemaname as schema, objname as table,
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/ffda5ae3/clientaccess/roles_privs.html.md.erb
----------------------------------------------------------------------
diff --git a/clientaccess/roles_privs.html.md.erb b/clientaccess/roles_privs.html.md.erb
index bd2de0a..4503951 100644
--- a/clientaccess/roles_privs.html.md.erb
+++ b/clientaccess/roles_privs.html.md.erb
@@ -16,7 +16,7 @@ In order to bootstrap the HAWQ system, a freshly initialized system always conta
## <a id="topic2"></a>Security Best Practices for Roles and Privileges
-- **Secure the gpadmin system user.** HAWQ requires a UNIX user id to install and initialize the HAWQ system. This system user is referred to as `gpadmin` in the HAWQ documentation. This `gpadmin` user is the default database superuser in HAWQ, as well as the file system owner of the HAWQ installation and its underlying data files. This default administrator account is fundamental to the design of HAWQ. The system cannot run without it, and there is no way to limit the access of this gpadmin user id. Use roles to manage who has access to the database for specific purposes. You should only use the `gpadmin` account for system maintenance tasks such as expansion and upgrade. Anyone who logs on to a HAWQ host as this user id can read, alter or delete any data; including system catalog data and database access rights. Therefore, it is very important to secure the gpadmin user id and only provide access to essential system administrators. Administrators should only log in to HAWQ as `g
padmin` when performing certain system maintenance tasks \(such as upgrade or expansion\). Database users should never log on as `gpadmin`, and ETL or production workloads should never run as `gpadmin`.
+- **Secure the gpadmin system user.** HAWQ requires a UNIX user id to install and initialize the HAWQ system. This system user is referred to as `gpadmin` in the HAWQ documentation. This `gpadmin` user is the default database superuser in HAWQ, as well as the file system owner of the HAWQ installation and its underlying data files. This default administrator account is fundamental to the design of HAWQ. The system cannot run without it, and there is no way to limit the access of this gpadmin user id. Use roles to manage who has access to the database for specific purposes. You should only use the `gpadmin` account for system maintenance tasks such as expansion and upgrade. Anyone who logs on to a HAWQ host as this user id can read, alter or delete any data; specifically system catalog data and database access rights. Therefore, it is very important to secure the gpadmin user id and only provide access to essential system administrators. Administrators should only log in to HAWQ as
`gpadmin` when performing certain system maintenance tasks \(such as upgrade or expansion\). Database users should never log on as `gpadmin`, and ETL or production workloads should never run as `gpadmin`.
- **Assign a distinct role to each user that logs in.** For logging and auditing purposes, each user that is allowed to log in to HAWQ should be given their own database role. For applications or web services, consider creating a distinct role for each application or service. See [Creating New Roles \(Users\)](#topic3).
- **Use groups to manage access privileges.** See [Role Membership](#topic5).
- **Limit users who have the SUPERUSER role attribute.** Roles that are superusers bypass all access privilege checks in HAWQ, as well as resource queuing. Only system administrators should be given superuser rights. See [Altering Role Attributes](#topic4).
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/ffda5ae3/datamgmt/BasicDataOperations.html.md.erb
----------------------------------------------------------------------
diff --git a/datamgmt/BasicDataOperations.html.md.erb b/datamgmt/BasicDataOperations.html.md.erb
index cd2cf16..e50060c 100644
--- a/datamgmt/BasicDataOperations.html.md.erb
+++ b/datamgmt/BasicDataOperations.html.md.erb
@@ -40,16 +40,18 @@ To insert data into a partitioned table, you specify the root partitioned table,
To insert large amounts of data, use external tables or the `COPY` command. These load mechanisms are more efficient than `INSERT` for inserting large quantities of rows. See [Loading and Unloading Data](load/g-loading-and-unloading-data.html#topic1) for more information about bulk data loading.
-## <a id="topic9"></a>Vacuuming the Database
+## <a id="topic9"></a>Vacuuming the System Catalog Tables
-Deleted or updated data rows occupy physical space on disk even though new transactions cannot see them. Periodically running the `VACUUM` command removes these expired rows. For example:
+Only HAWQ system catalog tables use multiple version concurrency control. Deleted or updated data rows in the catalog tables occupy physical space on disk even though new transactions cannot see them. Periodically running the `VACUUM` command removes these expired rows.
+
+Periodically running the `VACUUM` command on system catalog tables removes these expired rows. The `VACUUM` command also collects table-level statistics such as the number of rows and pages.
+
+For example:
``` sql
VACUUM mytable;
```
-The `VACUUM` command collects table-level statistics such as the number of rows and pages. Vacuum all tables after loading data.
-
### <a id="topic10"></a>Configuring the Free Space Map
Expired rows are held in the *free space map*. The free space map must be sized large enough to hold all expired rows in your database. If not, a regular `VACUUM` command cannot reclaim space occupied by expired rows that overflow the free space map.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/ffda5ae3/datamgmt/Transactions.html.md.erb
----------------------------------------------------------------------
diff --git a/datamgmt/Transactions.html.md.erb b/datamgmt/Transactions.html.md.erb
index b5d7e97..a057121 100644
--- a/datamgmt/Transactions.html.md.erb
+++ b/datamgmt/Transactions.html.md.erb
@@ -24,7 +24,7 @@ HAWQ accepts the standard SQL transaction levels as follows:
The following information describes the behavior of the HAWQ transaction levels:
-- **read committed/read uncommitted** \u2014 Provides fast, simple, partial transaction isolation. With read committed and read uncommitted transaction isolation, `SELECT`, `UPDATE`, and `DELETE` transactions operate on a snapshot of the database taken when the query started.
+- **read committed/read uncommitted** \u2014 Provides fast, simple, partial transaction isolation. With read committed and read uncommitted transaction isolation, `SELECT` transactions operate on a snapshot of the database taken when the query started.
A `SELECT` query:
@@ -33,9 +33,9 @@ A `SELECT` query:
- Does not see uncommitted data outside the transaction.
- Can possibly see changes that concurrent transactions made if the concurrent transaction is committed after the initial read in its own transaction.
-Successive `SELECT` queries in the same transaction can see different data if other concurrent transactions commit changes before the queries start. `UPDATE` and `DELETE` commands find only rows committed before the commands started.
+Successive `SELECT` queries in the same transaction can see different data if other concurrent transactions commit changes before the queries start.
-Read committed or read uncommitted transaction isolation allows concurrent transactions to modify or lock a row before `UPDATE` or `DELETE` finds the row. Read committed or read uncommitted transaction isolation may be inadequate for applications that perform complex queries and updates and require a consistent view of the database.
+Read committed or read uncommitted transaction isolation may be inadequate for applications that perform complex queries and require a consistent view of the database.
- **serializable/repeatable read** \u2014 Provides strict transaction isolation in which transactions execute as if they run one after another rather than concurrently. Applications on the serializable or repeatable read level must be designed to retry transactions in case of serialization failures.
@@ -49,9 +49,7 @@ A `SELECT` query:
Successive `SELECT` commands within a single transaction always see the same data.
- `UPDATE`, `DELETE, SELECT FOR UPDATE,` and `SELECT FOR SHARE` commands find only rows committed before the command started. If a concurrent transaction has already updated, deleted, or locked a target row when the row is found, the serializable or repeatable read transaction waits for the concurrent transaction to update the row, delete the row, or roll back.
-
- If the concurrent transaction updates or deletes the row, the serializable or repeatable read transaction rolls back. If the concurrent transaction rolls back, then the serializable or repeatable read transaction updates or deletes the row.
+ `SELECT FOR UPDATE` and `SELECT FOR SHARE` commands find only rows committed before the command started. If a concurrent transaction has already locked a target row when the row is found, the serializable or repeatable read transaction waits for the concurrent transaction to update the row, delete the row, or roll back.
The default transaction isolation level in HAWQ is *read committed*. To change the isolation level for a transaction, declare the isolation level when you `BEGIN` the transaction or use the `SET TRANSACTION` command after the transaction starts.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/ffda5ae3/datamgmt/about_statistics.html.md.erb
----------------------------------------------------------------------
diff --git a/datamgmt/about_statistics.html.md.erb b/datamgmt/about_statistics.html.md.erb
index d4b7665..25b3b24 100644
--- a/datamgmt/about_statistics.html.md.erb
+++ b/datamgmt/about_statistics.html.md.erb
@@ -20,7 +20,7 @@ Calculating statistics consumes time and resources, so HAWQ produces estimates b
### <a id="tablesize"></a>Table Size
-The query planner seeks to minimize the disk I/O and network traffic required to execute a query, using estimates of the number of rows that must be processed and the number of disk pages the query must access. The data from which these estimates are derived are the `pg_class` system table columns `reltuples` and `relpages`, which contain the number of rows and pages at the time a `VACUUM` or `ANALYZE` command was last run. As rows are added or deleted, the numbers become less accurate. However, an accurate count of disk pages is always available from the operating system, so as long as the ratio of `reltuples` to `relpages` does not change significantly, the optimizer can produce an estimate of the number of rows that is sufficiently accurate to choose the correct query execution plan.
+The query planner seeks to minimize the disk I/O and network traffic required to execute a query, using estimates of the number of rows that must be processed and the number of disk pages the query must access. The data from which these estimates are derived are the `pg_class` system table columns `reltuples` and `relpages`, which contain the number of rows and pages at the time a `VACUUM` or `ANALYZE` command was last run. As rows are added, the numbers become less accurate. However, an accurate count of disk pages is always available from the operating system, so as long as the ratio of `reltuples` to `relpages` does not change significantly, the optimizer can produce an estimate of the number of rows that is sufficiently accurate to choose the correct query execution plan.
In append-optimized tables, the number of tuples is kept up-to-date in the system catalogs, so the `reltuples` statistic is not an estimate. Non-visible tuples in the table are subtracted from the total. The `relpages` value is estimated from the append-optimized block sizes.
@@ -122,8 +122,6 @@ If a sample table is created, the number of rows in the sample is calculated to
Running `ANALYZE` with no arguments updates statistics for all tables in the database. This could take a very long time, so it is better to analyze tables selectively after data has changed. You can also analyze a subset of the columns in a table, for example columns used in joins, `WHERE` clauses, `SORT` clauses, `GROUP BY` clauses, or `HAVING` clauses.
-Analyzing a severely bloated table can generate poor statistics if the sample contains empty pages, so it is good practice to vacuum a bloated table before analyzing it.
-
See the SQL Command Reference for details of running the `ANALYZE` command.
Refer to the Management Utility Reference for details of running the `analyzedb` command.
@@ -150,7 +148,7 @@ Set the system default statistics target to a different value by setting the `de
$ hawq config -c default_statistics_target -v 50
```
-The statististics target for individual columns can be set with the `ALTER TABLE` command. For example, some queries can be improved by increasing the target for certain columns, especially columns that have irregular distributions. You can set the target to zero for columns that never contribute to query otpimization. When the target is 0, `ANALYZE` ignores the column. For example, the following `ALTER TABLE` command sets the statistics target for the `notes` column in the `emp` table to zero:
+The statististics target for individual columns can be set with the `ALTER TABLE` command. For example, some queries can be improved by increasing the target for certain columns, especially columns that have irregular distributions. You can set the target to zero for columns that never contribute to query optimization. When the target is 0, `ANALYZE` ignores the column. For example, the following `ALTER TABLE` command sets the statistics target for the `notes` column in the `emp` table to zero:
``` sql
ALTER TABLE emp ALTER COLUMN notes SET STATISTICS 0;
@@ -168,7 +166,7 @@ Automatic statistics collection has three modes:
- `none` disables automatic statistics collection.
- `on_no_stats` triggers an analyze operation for a table with no existing statistics when any of the commands `CREATE TABLE AS SELECT`, `INSERT`, or `COPY` are executed on the table.
-- `on_change` triggers an analyze operation when any of the commands `CREATE TABLE AS SELECT`, `UPDATE`, `DELETE`, `INSERT`, or `COPY` are executed on the table and the number of rows affected exceeds the threshold defined by the `gp_autostats_on_change_threshold` configuration parameter.
+- `on_change` triggers an analyze operation when any of the commands `CREATE TABLE AS SELECT`, `INSERT`, or `COPY` are executed on the table and the number of rows affected exceeds the threshold defined by the `gp_autostats_on_change_threshold` configuration parameter.
The automatic statistics collection mode is set separately for commands that occur within a procedural language function and commands that execute outside of a function:
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/ffda5ae3/datamgmt/load/g-optimizing-data-load-and-query-performance.html.md.erb
----------------------------------------------------------------------
diff --git a/datamgmt/load/g-optimizing-data-load-and-query-performance.html.md.erb b/datamgmt/load/g-optimizing-data-load-and-query-performance.html.md.erb
index 203efc5..ff1c230 100644
--- a/datamgmt/load/g-optimizing-data-load-and-query-performance.html.md.erb
+++ b/datamgmt/load/g-optimizing-data-load-and-query-performance.html.md.erb
@@ -2,9 +2,9 @@
title: Optimizing Data Load and Query Performance
---
-Use the following tips to help optimize your data load and subsequent query performance.
+Use the following tip to help optimize your data load and subsequent query performance.
+
+- Run `ANALYZE` after loading data. If you significantly altered the data in a table, run `ANALYZE` or `VACUUM ANALYZE` (system catalog tables only) to update table statistics for the query optimizer. Current statistics ensure that the optimizer makes the best decisions during query planning and avoids poor performance due to inaccurate or nonexistent statistics.
-- Run `ANALYZE` after loading data. If you significantly altered the data in a table, run `ANALYZE` or `VACUUM ANALYZE` to update table statistics for the query optimizer. Current statistics ensure that the optimizer makes the best decisions during query planning and avoids poor performance due to inaccurate or nonexistent statistics.
-- Run `VACUUM` after load errors. If the load operation does not run in single row error isolation mode, the operation stops at the first error. The target table contains the rows loaded before the error occurred. You cannot access these rows, but they occupy disk space. Use the `VACUUM` command to recover the wasted space.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/ffda5ae3/ddl/ddl-partition.html.md.erb
----------------------------------------------------------------------
diff --git a/ddl/ddl-partition.html.md.erb b/ddl/ddl-partition.html.md.erb
index 42b7ebe..39fd43c 100644
--- a/ddl/ddl-partition.html.md.erb
+++ b/ddl/ddl-partition.html.md.erb
@@ -23,7 +23,7 @@ HAWQ uses the partition criteria defined during table creation to create each pa
The HAWQ system catalog stores partition hierarchy information so that rows inserted into the top-level parent table propagate correctly to the child table partitions. To change the partition design or table structure, alter the parent table using `ALTER TABLE` with the `PARTITION` clause.
-To insert data into a partitioned table, you specify the root partitioned table, the table created with the `CREATE TABLE` command. You also can specify a leaf child table of the partitioned table in an `INSERT` command. An error is returned if the data is not valid for the specified leaf child table. Specifying a child table that is not a leaf child table in the `INSERT` command is not supported. Execution of other DML commands such as `UPDATE` and `DELETE` on any child table of a partitioned table is not supported. These commands must be executed on the root partitioned table, the table created with the `CREATE TABLE` command.
+To insert data into a partitioned table, you specify the root partitioned table, the table created with the `CREATE TABLE` command. You also can specify a leaf child table of the partitioned table in an `INSERT` command. An error is returned if the data is not valid for the specified leaf child table. Specifying a child table that is not a leaf child table in the `INSERT` command is not supported.
## <a id="topic65"></a>Deciding on a Table Partitioning Strategy
@@ -258,7 +258,7 @@ WHERE tablename='sales';
The following table and views show information about partitioned tables.
-- *pg\_partition*- Tracks partitioned tables and their inheritance level relationships.
+- *pg\_partition* - Tracks partitioned tables and their inheritance level relationships.
- *pg\_partition\_templates* - Shows the subpartitions created using a subpartition template.
- *pg\_partition\_columns* - Shows the partition key columns used in a partition design.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/ffda5ae3/ddl/ddl-storage.html.md.erb
----------------------------------------------------------------------
diff --git a/ddl/ddl-storage.html.md.erb b/ddl/ddl-storage.html.md.erb
index 4711d00..3e89d4f 100644
--- a/ddl/ddl-storage.html.md.erb
+++ b/ddl/ddl-storage.html.md.erb
@@ -16,7 +16,6 @@ HAWQ provides storage orientation models of either row-oriented or Parquet table
Row-oriented storage provides the best options for the following situations:
-- **Updates of table data.** Where you load and update the table data frequently.
- **Frequent INSERTs.** Where rows are frequently inserted into the table
- **Number of columns requested in queries.** Where you typically request all or the majority of columns in the `SELECT` list or `WHERE` clause of your queries, choose a row-oriented model.
- **Number of columns in the table.** Row-oriented storage is most efficient when many columns are required at the same time, or when the row-size of a table is relatively small.
@@ -63,12 +62,10 @@ The`DROP TABLE`command removes tables from the database. For example:
DROP TABLE mytable;
```
-To empty a table of rows without removing the table definition, use `DELETE` or `TRUNCATE`. For example:
+`DROP TABLE`always removes any indexes, rules, triggers, and constraints that exist for the target table. Specify `CASCADE`to drop a table that is referenced by a view. `CASCADE` removes dependent views.
-``` sql
-DELETE FROM mytable;
+To empty a table of rows without removing the table definition, use `TRUNCATE`. For example:
+``` sql
TRUNCATE mytable;
```
-
-`DROP TABLE`always removes any indexes, rules, triggers, and constraints that exist for the target table. Specify `CASCADE`to drop a table that is referenced by a view. `CASCADE` removes dependent views.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/ffda5ae3/reference/catalog/pg_class.html.md.erb
----------------------------------------------------------------------
diff --git a/reference/catalog/pg_class.html.md.erb b/reference/catalog/pg_class.html.md.erb
index 5dff1f2..c0244b1 100644
--- a/reference/catalog/pg_class.html.md.erb
+++ b/reference/catalog/pg_class.html.md.erb
@@ -69,7 +69,7 @@ The system catalog table `pg_class` catalogs tables and most everything else tha
<td><code class="ph codeph">relpages</code></td>
<td>integer</td>
<td>�</td>
-<td>Size of the on-disk representation of this table in pages (of 32K each). This is only an estimate used by the planner. It is updated by <code class="ph codeph">VACUUM</code>, <code class="ph codeph">ANALYZE</code>, and a few DDL commands.</td>
+<td>Size of the on-disk representation of this table in pages (of 32K each). This is only an estimate used by the planner. It is updated by <code class="ph codeph">ANALYZE</code>, and a few DDL commands.</td>
</tr>
<tr class="odd">
<td><code class="ph codeph">reltuples</code></td>
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/ffda5ae3/reference/catalog/pg_index.html.md.erb
----------------------------------------------------------------------
diff --git a/reference/catalog/pg_index.html.md.erb b/reference/catalog/pg_index.html.md.erb
index cd1a054..7abe921 100644
--- a/reference/catalog/pg_index.html.md.erb
+++ b/reference/catalog/pg_index.html.md.erb
@@ -14,7 +14,7 @@ The `pg_index` system catalog table contains part of the information about index
| `indnatts` | smallint | � | The number of columns in the index (duplicates pg\_class.relnatts). |
| `indisunique` | boolean | � | If true, this is a unique index. |
| `indisclustered` | boolean | � | If true, the table was last clustered on this index via the `CLUSTER` command. |
-| `indisvalid` | boolean | � | If true, the index is currently valid for queries. False means the index is possibly incomplete: it must still be modified by `INSERT`/`UPDATE` operations, but it cannot safely be used for queries. |
+| `indisvalid` | boolean | � | If true, the index is currently valid for queries. False means the index is possibly incomplete: it must still be modified by `INSERT` operations, but it cannot safely be used for queries. |
| `indkey` | int2vector | pg\_attribute.attnum | This is an array of indnatts values that indicate which table columns this index indexes. For example a value of 1 3 would mean that the first and the third table columns make up the index key. A zero in this array indicates that the corresponding index attribute is an expression over the table columns, rather than a simple column reference. |
| `indclass` | oidvector | pg\_opclass.oid | For each column in the index key this contains the OID of the operator class to use. |
| `indexprs` | text | � | Expression trees (in `nodeToString()` representation) for index attributes that are not simple column references. This is a list with one element for each zero entry in indkey. NULL if all index attributes are simple references. |
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/ffda5ae3/reference/cli/admin_utilities/hawqload.html.md.erb
----------------------------------------------------------------------
diff --git a/reference/cli/admin_utilities/hawqload.html.md.erb b/reference/cli/admin_utilities/hawqload.html.md.erb
index 8d7d984..b9fe441 100644
--- a/reference/cli/admin_utilities/hawqload.html.md.erb
+++ b/reference/cli/admin_utilities/hawqload.html.md.erb
@@ -42,7 +42,7 @@ The client machine where `hawq load` is executed must have the following:
## <a id="topic1__section4"></a>Description
-`hawq load` is a data loading utility that acts as an interface to HAWQ's external table parallel loading feature. Using a load specification defined in a YAML formatted control file, `hawq load` executes a load by invoking the HAWQ parallel file server ([gpfdist](gpfdist.html#topic1)), creating an external table definition based on the source data defined, and executing an `INSERT`, `UPDATE` or `MERGE` operation to load the source data into the target table in the database.
+`hawq load` is a data loading utility that acts as an interface to HAWQ's external table parallel loading feature. Using a load specification defined in a YAML formatted control file, `hawq load` executes a load by invoking the HAWQ parallel file server ([gpfdist](gpfdist.html#topic1)), creating an external table definition based on the source data defined, and executing an `INSERT` operation to load the source data into the target table in the database.
The operation, including any SQL commands specified in the `SQL` collection of the YAML control file (see [Control File Format](#topic1__section7)), are performed as a single transaction to prevent inconsistent data when performing multiple, simultaneous load operations on a target table.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/ffda5ae3/reference/cli/client_utilities/vacuumdb.html.md.erb
----------------------------------------------------------------------
diff --git a/reference/cli/client_utilities/vacuumdb.html.md.erb b/reference/cli/client_utilities/vacuumdb.html.md.erb
index 30617df..cbc37f3 100644
--- a/reference/cli/client_utilities/vacuumdb.html.md.erb
+++ b/reference/cli/client_utilities/vacuumdb.html.md.erb
@@ -4,6 +4,8 @@ title: vacuumdb
Garbage-collects and analyzes a database.
+`vacuumdb` is typically run on system catalog tables. It has no effect when run on HAWQ user tables.
+
## <a id="topic1__section2"></a>Synopsis
``` pre
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/ffda5ae3/reference/guc/guc_category-list.html.md.erb
----------------------------------------------------------------------
diff --git a/reference/guc/guc_category-list.html.md.erb b/reference/guc/guc_category-list.html.md.erb
index 715583b..e77f925 100644
--- a/reference/guc/guc_category-list.html.md.erb
+++ b/reference/guc/guc_category-list.html.md.erb
@@ -344,7 +344,7 @@ These parameters adjust the amount of data sampled by an `ANALYZE` operation. Ad
### <a id="topic_qvz_nz3_yv"></a>Automatic Statistics Collection
-When automatic statistics collection is enabled, you can run `ANALYZE` automatically in the same transaction as an `INSERT`, `UPDATE`, `DELETE`, `COPY` or `CREATE TABLE...AS SELECT` statement when a certain threshold of rows is affected (`on_change`), or when a newly generated table has no statistics (`on_no_stats`). To enable this feature, set the following server configuration parameters in your HAWQ `hawq-site.xml` file by using the `hawq config` utility and restart HAWQ:
+When automatic statistics collection is enabled, you can run `ANALYZE` automatically in the same transaction as an `INSERT`, `COPY` or `CREATE TABLE...AS SELECT` statement when a certain threshold of rows is affected (`on_change`), or when a newly generated table has no statistics (`on_no_stats`). To enable this feature, set the following server configuration parameters in your HAWQ `hawq-site.xml` file by using the `hawq config` utility and restart HAWQ:
- [gp\_autostats\_mode](parameter_definitions.html#gp_autostats_mode)
- [log\_autostats](parameter_definitions.html#log_autostats)
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/ffda5ae3/reference/guc/parameter_definitions.html.md.erb
----------------------------------------------------------------------
diff --git a/reference/guc/parameter_definitions.html.md.erb b/reference/guc/parameter_definitions.html.md.erb
index c792d1b..645335b 100644
--- a/reference/guc/parameter_definitions.html.md.erb
+++ b/reference/guc/parameter_definitions.html.md.erb
@@ -1076,10 +1076,6 @@ The `on_change` option triggers statistics collection only when the number of ro
`CREATE TABLE AS SELECT`
-`UPDATE`
-
-`DELETE`
-
`INSERT`
`COPY`
@@ -2391,7 +2387,7 @@ The maximum value is INT\_MAX/1024. If an invalid value is specified, the defaul
## <a name="log_statement"></a>log\_statement
-Controls which SQL statements are logged. DDL logs all data definition commands like CREATE, ALTER, and DROP commands. MOD logs all DDL statements, plus INSERT, UPDATE, DELETE, TRUNCATE, and COPY FROM. PREPARE and EXPLAIN ANALYZE statements are also logged if their contained command is of an appropriate type.
+Controls which SQL statements are logged. DDL logs all data definition commands like CREATE, ALTER, and DROP commands. MOD logs all DDL statements, plus INSERT, TRUNCATE, and COPY FROM. PREPARE and EXPLAIN ANALYZE statements are also logged if their contained command is of an appropriate type.
<table>
<colgroup>
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/ffda5ae3/reference/sql/ALTER-TABLE.html.md.erb
----------------------------------------------------------------------
diff --git a/reference/sql/ALTER-TABLE.html.md.erb b/reference/sql/ALTER-TABLE.html.md.erb
index 7b1d74d..4303f0c 100644
--- a/reference/sql/ALTER-TABLE.html.md.erb
+++ b/reference/sql/ALTER-TABLE.html.md.erb
@@ -314,7 +314,7 @@ When a column is added with `ADD COLUMN`, all existing rows in the table are ini
You can specify multiple changes in a single `ALTER TABLE` command, which will be done in a single pass over the table.
-The `DROP COLUMN` form does not physically remove the column, but simply makes it invisible to SQL operations. Subsequent insert and update operations in the table will store a null value for the column. Thus, dropping a column is quick but it will not immediately reduce the on-disk size of your table, as the space occupied by the dropped column is not reclaimed. The space will be reclaimed over time as existing rows are updated.
+The `DROP COLUMN` form does not physically remove the column, but simply makes it invisible to SQL operations. Subsequent insert operations in the table will store a null value for the column. Thus, dropping a column is quick but it will not immediately reduce the on-disk size of your table, as the space occupied by the dropped column is not reclaimed.
The fact that `ALTER TYPE` requires rewriting the whole table is sometimes an advantage, because the rewriting process eliminates any dead space in the table. For example, to reclaim the space occupied by a dropped column immediately, the fastest way is: `ALTER TABLE <table> ALTER COLUMN <anycol> TYPE <sametype>;` Where�\<anycol\>�is any remaining table column and�\<sametype\>�is the same type that column already has. This results in no semantically-visible change in the table, but the command forces rewriting, which gets rid of no-longer-useful data.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/ffda5ae3/reference/sql/BEGIN.html.md.erb
----------------------------------------------------------------------
diff --git a/reference/sql/BEGIN.html.md.erb b/reference/sql/BEGIN.html.md.erb
index 5c2a9bb..265e66e 100644
--- a/reference/sql/BEGIN.html.md.erb
+++ b/reference/sql/BEGIN.html.md.erb
@@ -31,7 +31,7 @@ READ UNCOMMITTED </dt>
<dt>READ WRITE
READ ONLY </dt>
-<dd>Determines whether the transaction is read/write or read-only. Read/write is the default. When a transaction is read-only, the following SQL commands are disallowed: `INSERT`, `UPDATE`, `DELETE`, and `COPY FROM` if the table they would write to is not a temporary table; all `CREATE`, `ALTER`, and `DROP` commands; `GRANT`, `REVOKE`, `TRUNCATE`; and `EXPLAIN ANALYZE` and `EXECUTE` if the command they would execute is among those listed.</dd>
+<dd>Determines whether the transaction is read/write or read-only. Read/write is the default. When a transaction is read-only, the following SQL commands are disallowed: `INSERT` and `COPY FROM` if the table they would write to is not a temporary table; all `CREATE`, `ALTER`, and `DROP` commands; `GRANT`, `REVOKE`, `TRUNCATE`; and `EXPLAIN ANALYZE` and `EXECUTE` if the command they would execute is among those listed.</dd>
## <a id="topic1__section5"></a>Notes
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/ffda5ae3/reference/sql/COPY.html.md.erb
----------------------------------------------------------------------
diff --git a/reference/sql/COPY.html.md.erb b/reference/sql/COPY.html.md.erb
index 6069aa5..aaa2270 100644
--- a/reference/sql/COPY.html.md.erb
+++ b/reference/sql/COPY.html.md.erb
@@ -139,7 +139,7 @@ Files named in a `COPY` command are read or written directly by the database ser
`COPY` input and output is affected by `DateStyle`. To ensure portability to other HAWQ installations that might use non-default `DateStyle` settings, `DateStyle` should be set to ISO before using `COPY TO`.
-By default, `COPY` stops operation at the first error. This should not lead to problems in the event of a `COPY TO`, but the target table will already have received earlier rows in a `COPY FROM`. These rows will not be visible or accessible, but they still occupy disk space. This may amount to a considerable amount of wasted disk space if the failure happened well into a large `COPY FROM` operation. You may wish to invoke `VACUUM` to recover the wasted space. Another option would be to use single row error isolation mode to filter out error rows while still loading good rows.
+By default, `COPY` stops operation at the first error. This should not lead to problems in the event of a `COPY TO`, but the target table will already have received earlier rows in a `COPY FROM`. These rows will not be visible or accessible, but they still occupy disk space. This may amount to a considerable amount of wasted disk space if the failure happened well into a large `COPY FROM` operation. You may wish to use single row error isolation mode to filter out error rows while still loading good rows.
COPY supports creating readable foreign tables with error tables. The default for concurrently inserting into the error table is 127.�You can use�error tables with foreign tables under the following circumstances:
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/ffda5ae3/reference/sql/CREATE-EXTERNAL-TABLE.html.md.erb
----------------------------------------------------------------------
diff --git a/reference/sql/CREATE-EXTERNAL-TABLE.html.md.erb b/reference/sql/CREATE-EXTERNAL-TABLE.html.md.erb
index 2b164dc..4eec64b 100644
--- a/reference/sql/CREATE-EXTERNAL-TABLE.html.md.erb
+++ b/reference/sql/CREATE-EXTERNAL-TABLE.html.md.erb
@@ -109,7 +109,7 @@ where \<pxf parameters\> is:
## <a id="topic1__section3"></a>Description
-`CREATE EXTERNAL TABLE` or `CREATE EXTERNAL WEB TABLE` creates a new readable external table definition in HAWQ. Readable external tables are typically used for fast, parallel data loading. Once an external table is defined, you can query its data directly (and in parallel) using SQL commands. For example, you can select, join, or sort external table data. You can also create views for external tables. DML operations (`UPDATE`, `INSERT`, `DELETE`, or` TRUNCATE`) are not allowed on readable external tables.
+`CREATE EXTERNAL TABLE` or `CREATE EXTERNAL WEB TABLE` creates a new readable external table definition in HAWQ. Readable external tables are typically used for fast, parallel data loading. Once an external table is defined, you can query its data directly (and in parallel) using SQL commands. For example, you can select, join, or sort external table data. You can also create views for external tables. DML operations (`INSERT` or `TRUNCATE`) are not allowed on readable external tables.
`CREATE WRITABLE EXTERNAL TABLE` or `CREATE WRITABLE EXTERNAL WEB TABLE` creates a new writable external table definition in HAWQ. Writable external tables are typically used for unloading data from the database into a set of files or named pipes.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/ffda5ae3/reference/sql/CREATE-TABLE.html.md.erb
----------------------------------------------------------------------
diff --git a/reference/sql/CREATE-TABLE.html.md.erb b/reference/sql/CREATE-TABLE.html.md.erb
index 99ff35e..162a438 100644
--- a/reference/sql/CREATE-TABLE.html.md.erb
+++ b/reference/sql/CREATE-TABLE.html.md.erb
@@ -161,7 +161,7 @@ where \<storage\_parameter\> for a partition is:
`CREATE TABLE` creates a new, initially empty table in the current database. The table is owned by the user issuing the command. If a schema name is given then the table is created in the specified schema. Otherwise it is created in the current schema. Temporary tables exist in a special schema, so a schema name may not be given when creating a temporary table. The name of the table must be distinct from the name of any other table, external table, sequence, or view in the same schema.
-The optional constraint clauses specify conditions that new or updated rows must satisfy for an insert or update operation to succeed. A constraint is an SQL object that helps define the set of valid values in the table in various ways. Constraints apply to tables, not to partitions. You cannot add a constraint to a partition or subpartition.
+The optional constraint clauses specify conditions that new rows must satisfy for an insert operation to succeed. A constraint is an SQL object that helps define the set of valid values in the table in various ways. Constraints apply to tables, not to partitions. You cannot add a constraint to a partition or subpartition.
There are two ways to define constraints: table constraints and column constraints. A column constraint is defined as part of a column definition. A table constraint definition is not tied to a particular column, and it can encompass more than one column. Every column constraint can also be written as a table constraint; a column constraint is only a notational convenience for use when the constraint only affects one column.
@@ -213,7 +213,7 @@ Note also that unlike `INHERITS`, copied columns and constraints are not merged
<dd>Specifies if the column is or is not allowed to contain null values. `NULL` is the default.</dd>
<dt>CHECK ( \<expression\> ) </dt>
-<dd>The `CHECK` clause specifies an expression producing a Boolean result which new or updated rows must satisfy for an insert or update operation to succeed. Expressions evaluating to `TRUE` or `UNKNOWN` succeed. Should any row of an insert or update operation produce a `FALSE` result an error exception is raised and the insert or update does not alter the database. A check constraint specified as a column constraint should reference that column's value only, while an expression appearing in a table constraint may reference multiple columns. `CHECK` expressions cannot contain subqueries nor refer to variables other than columns of the current row.</dd>
+<dd>The `CHECK` clause specifies an expression producing a Boolean result which new rows must satisfy for an insert operation to succeed. Expressions evaluating to `TRUE` or `UNKNOWN` succeed. Should any row of an insert operation produce a `FALSE` result an error exception is raised and the insert does not alter the database. A check constraint specified as a column constraint should reference that column's value only, while an expression appearing in a table constraint may reference multiple columns. `CHECK` expressions cannot contain subqueries nor refer to variables other than columns of the current row.</dd>
<dt>WITH ( \<storage\_option\>=\<value\> ) </dt>
<dd>The `WITH` clause can be used to set storage options for the table or its indexes. Note that you can also set storage parameters on a particular partition or subpartition by declaring the `WITH` clause in the partition specification.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/ffda5ae3/reference/sql/CREATE-VIEW.html.md.erb
----------------------------------------------------------------------
diff --git a/reference/sql/CREATE-VIEW.html.md.erb b/reference/sql/CREATE-VIEW.html.md.erb
index 4a24bb7..e39d8d3 100644
--- a/reference/sql/CREATE-VIEW.html.md.erb
+++ b/reference/sql/CREATE-VIEW.html.md.erb
@@ -77,7 +77,7 @@ names, rank WHERE rank < '11' AND names.id=rank.id;
The SQL standard specifies some additional capabilities for the `CREATE VIEW` statement that are not in HAWQ. The optional clauses for the full SQL command in the standard are:
-- **CHECK OPTION** \u2014 This option has to do with updatable views. All `INSERT` and `UPDATE` commands on the view will be checked to ensure data satisfy the view-defining condition (that is, the new data would be visible through the view). If they do not, the update will be rejected.
+- **CHECK OPTION** \u2014 This option has to do with updatable views. All `INSERT` commands on the view will be checked to ensure data satisfy the view-defining condition (that is, the new data would be visible through the view). If they do not, the insert will be rejected.
- **LOCAL** \u2014 Check for integrity on this view.
- **CASCADED** \u2014 Check for integrity on this view and on any dependent view. `CASCADED` is assumed if neither `CASCADED` nor `LOCAL` is specified.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/ffda5ae3/reference/sql/DROP-TABLE.html.md.erb
----------------------------------------------------------------------
diff --git a/reference/sql/DROP-TABLE.html.md.erb b/reference/sql/DROP-TABLE.html.md.erb
index 98022ce..b277273 100644
--- a/reference/sql/DROP-TABLE.html.md.erb
+++ b/reference/sql/DROP-TABLE.html.md.erb
@@ -12,7 +12,7 @@ DROP TABLE [IF EXISTS] <name> [, ...] [CASCADE | RESTRICT]
## <a id="topic1__section3"></a>Description
-`DROP TABLE` removes tables from the database. Only its owner may drop a table. To empty a table of rows without removing the table definition, use `DELETE` or `TRUNCATE`.
+`DROP TABLE` removes tables from the database. Only its owner may drop a table. To empty a table of rows without removing the table definition, use `TRUNCATE`.
`DROP TABLE` always removes any indexes, rules, and constraints that exist for the target table. However, to drop a table that is referenced by a view, `CASCADE` must be specified. `CASCADE` will remove a dependent view entirely.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/ffda5ae3/reference/sql/PREPARE.html.md.erb
----------------------------------------------------------------------
diff --git a/reference/sql/PREPARE.html.md.erb b/reference/sql/PREPARE.html.md.erb
index 6f6836c..c633f14 100644
--- a/reference/sql/PREPARE.html.md.erb
+++ b/reference/sql/PREPARE.html.md.erb
@@ -30,7 +30,7 @@ Prepared statements have the largest performance advantage when a single session
<dd>The data type of a parameter to the prepared statement. If the data type of a particular parameter is unspecified or is specified as unknown, it will be inferred from the context in which the parameter is used. To refer to the parameters in the prepared statement itself, use `$1`, `$2`, etc.</dd>
<dt> \<statement\> </dt>
-<dd>Any `SELECT`, `INSERT`, `UPDATE`, `DELETE`, or `VALUES` statement.</dd>
+<dd>Any `SELECT`, `INSERT`, or `VALUES` statement.</dd>
## <a id="topic1__section5"></a>Notes
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/ffda5ae3/reference/sql/VACUUM.html.md.erb
----------------------------------------------------------------------
diff --git a/reference/sql/VACUUM.html.md.erb b/reference/sql/VACUUM.html.md.erb
index 2db5757..9b39a32 100644
--- a/reference/sql/VACUUM.html.md.erb
+++ b/reference/sql/VACUUM.html.md.erb
@@ -2,7 +2,9 @@
title: VACUUM
---
-Garbage-collects and optionally analyzes a database.
+Garbage-collects and optionally analyzes a database.
+
+**Note**: HAWQ `VACUUM` support is provided only for system catalog tables. `VACUUM`ing a HAWQ user table has no effect.
## <a id="topic1__section2"></a>Synopsis
@@ -14,11 +16,13 @@ VACUUM [FULL] [FREEZE] [VERBOSE] ANALYZE
## <a id="topic1__section3"></a>Description
-`VACUUM` reclaims storage occupied by deleted tuples. In normal HAWQ operation, tuples that are deleted or obsoleted by an update are not physically removed from their table; they remain present on disk until a `VACUUM` is done. Therefore it is necessary to do `VACUUM` periodically, especially on frequently-updated catalog tables. `VACUUM` has no effect on a normal HAWQ table, since the delete or update operations are not supported on normal HAWQ table.
+`VACUUM` reclaims storage occupied by deleted tuples. In normal HAWQ operation, tuples that are deleted or obsoleted by an update are not physically removed from their table; they remain present on disk until a `VACUUM` is done. Therefore it is necessary to do `VACUUM` periodically, especially on frequently-updated catalog tables. (`VACUUM` has no effect on a normal HAWQ table, since the delete or update operations are not supported on normal HAWQ table.)
With no parameter, `VACUUM` processes every table in the current database. With a parameter, `VACUUM` processes only that table. `VACUUM ANALYZE` performs a `VACUUM` and then an `ANALYZE` for each selected table. This is a handy combination form for routine maintenance scripts. See [ANALYZE](ANALYZE.html) for more details about its processing.
-Plain `VACUUM` (without `FULL`) simply reclaims space and makes it available for re-use. This form of the command can operate in parallel with normal reading and writing of the table, as an exclusive lock is not obtained. `VACUUM FULL` does more extensive processing, including moving of tuples across blocks to try to compact the table to the minimum number of disk blocks. This form is much slower and requires an exclusive lock on each table while it is being processed.
+Plain `VACUUM` (without `FULL`) simply reclaims space and makes it available for re-use. This form of the command can operate in parallel with normal reading and writing of the table, as an exclusive lock is not obtained. `VACUUM FULL` does more extensive processing, including moving of tuples across blocks to try to compact the table to the minimum number of disk blocks. This form is much slower and requires an exclusive lock on each table while it is being processed.
+
+**Note:** `VACUUM FULL` is not recommended in HAWQ.
**Outputs**
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/ffda5ae3/reference/toolkit/hawq_toolkit.html.md.erb
----------------------------------------------------------------------
diff --git a/reference/toolkit/hawq_toolkit.html.md.erb b/reference/toolkit/hawq_toolkit.html.md.erb
index f76963c..45a8a56 100644
--- a/reference/toolkit/hawq_toolkit.html.md.erb
+++ b/reference/toolkit/hawq_toolkit.html.md.erb
@@ -16,7 +16,7 @@ The following views can help identify tables that need routine table maintenance
- [hawq\_stats\_missing](#topic4)
-The `VACUUM` command reclaims disk space occupied by deleted or obsolete rows. Because of the MVCC transaction concurrency model used in HAWQ, data rows that are deleted or updated still occupy physical space on disk even though they are not visible to any new transactions. Expired rows increase table size on disk and eventually slow down scans of the table.
+The `VACUUM` command is applicable only to system catalog tables. The `VACUUM` command reclaims disk space occupied by deleted or obsolete rows. Because of the MVCC transaction concurrency model used in HAWQ, data rows that are deleted or updated still occupy physical space on disk even though they are not visible to any new transactions. Expired rows increase table size on disk and eventually slow down scans of the table.
**Note:** VACUUM FULL is not recommended in HAWQ. See [VACUUM](../sql/VACUUM.html#topic1).
[22/50] [abbrv] incubator-hawq-docs git commit: misc doc
clarifications
Posted by yo...@apache.org.
misc doc clarifications
Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/commit/cd4a3db2
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/tree/cd4a3db2
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/diff/cd4a3db2
Branch: refs/heads/master
Commit: cd4a3db248a2b882683dc496c9f07872b0cb2783
Parents: 1661c62
Author: Lisa Owen <lo...@pivotal.io>
Authored: Mon Sep 19 10:44:24 2016 -0700
Committer: Lisa Owen <lo...@pivotal.io>
Committed: Mon Sep 19 10:44:24 2016 -0700
----------------------------------------------------------------------
install/aws-config.html.md.erb | 18 ++----------------
pxf/HawqExtensionFrameworkPXF.html.md.erb | 4 ++++
reference/HAWQDataTypes.html.md.erb | 4 +++-
requirements/system-requirements.html.md.erb | 3 +++
4 files changed, 12 insertions(+), 17 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/cd4a3db2/install/aws-config.html.md.erb
----------------------------------------------------------------------
diff --git a/install/aws-config.html.md.erb b/install/aws-config.html.md.erb
index 395970b..fb2a623 100644
--- a/install/aws-config.html.md.erb
+++ b/install/aws-config.html.md.erb
@@ -29,9 +29,7 @@ Instance store-backed storage is generally better performing than EBS and recomm
**Note** EC2 *instance store* provides temporary block-level storage. This storage is located on disks that are physically attached to the host computer. While instance store provides high performance, powering off the instance causes data loss. Soft reboots preserve instance store data.
Virtual devices for instance store volumes for HAWQ EC2 instance store instances are named `ephemeralN` (where *N* varies based on instance type). CentOS instance store block device are named `/dev/xvdletter` (where *letter* is a lower case letter of the alphabet).
-
-?? DO WE RECOMMEND EBS-backed VOLUMES FOR ANYTHING? I.E. BACKUPS, ...??)
-
+
### <a id="topic_cfgplacegrp"></a>Configure Placement Group
@@ -75,24 +73,14 @@ Ensure your Ambari and HAWQ master instances are each assigned a public IP addre
A security group is a set of rules that control network traffic to and from your HAWQ instance. One or more rules may be associated with a security group, and one or more security groups may be associated with an instance.
-To configure HAWQ communication between nodes in the HAWQ cluster, include and open the following ports in the appropriate security group for the HAWQ master and segment nodes:
-
-?? PROVIDE SPECIFIC type/protocol/portrange/source settings from EC2 console ??
+To configure HAWQ communication between nodes in the HAWQ cluster, include and open the following ports in the appropriate security group for the HAWQ master and segment nodes.
| Port | Application |
|-------|-------------------------------------|
| 22 | ssh - secure connect to other hosts |
-| XX | XX |
-
-UDP access within the VPC should be enabled by default.
-
-
-?? DIFFENT SECURITY GROUP FOR AMBARI NODE ??
To allow access to/from a source external to the Ambari management node, include and open the following ports in an appropriate security group for your Ambari node:
-?? PROVIDE SPECIFIC type/protocol/portrange/source settings from EC2 console ??
-
| Port | Application |
|-------|-------------------------------------|
| 22 | ssh - secure connect to other hosts |
@@ -116,8 +104,6 @@ After launching your HAWQ instance, you will connect to and configure the instan
Before installing HAWQ, set up the EC2 instances as you would local host server machines. Configure the host operating system, configure host network information (for example, update the `/etc/hosts` file), set operating system parameters, and install operating system packages. For information about how to prepare your operating system environment for HAWQ, see [Select HAWQ Host Machines](../install/select-hosts.html).
-?? PACKAGES, KERNEL VERSION, KERNEL SETTINGS, BEST PRACTICES HERE OR WILL THEY BE THE SAME AS FOR PHYSICAL DEVICE??
-
##References<a id="topic_hgz_zwy_bv"></a>
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/cd4a3db2/pxf/HawqExtensionFrameworkPXF.html.md.erb
----------------------------------------------------------------------
diff --git a/pxf/HawqExtensionFrameworkPXF.html.md.erb b/pxf/HawqExtensionFrameworkPXF.html.md.erb
index b705984..c0d7c0f 100644
--- a/pxf/HawqExtensionFrameworkPXF.html.md.erb
+++ b/pxf/HawqExtensionFrameworkPXF.html.md.erb
@@ -28,6 +28,10 @@ PXF allows users to create custom connectors to access other parallel data store
This topic describes how to access HBase data using PXF.
+- **[Accessing JSON Data](../pxf/JsonPXF.html)**
+
+ This topic describes how to access JSON data using PXF.
+
- **[Using Profiles to Read and Write Data](../pxf/ReadWritePXF.html)**
PXF profiles are collections of common metadata attributes that can be used to simplify the reading and writing of data. You can use any of the built-in profiles that come with PXF or you can create your own.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/cd4a3db2/reference/HAWQDataTypes.html.md.erb
----------------------------------------------------------------------
diff --git a/reference/HAWQDataTypes.html.md.erb b/reference/HAWQDataTypes.html.md.erb
index d894b38..fe5cff7 100644
--- a/reference/HAWQDataTypes.html.md.erb
+++ b/reference/HAWQDataTypes.html.md.erb
@@ -27,7 +27,7 @@ The following data types are specified by SQL:
- timestamp�(with or without time zone)
- varchar
-**Note**: HAWQ supports the array data type for append-only tables; parquet table storage does *not* support the array type.
+**Note**(\*): HAWQ supports the array data type for append-only tables; parquet table storage does *not* support the array type.
Each data type has an external representation determined by its input and output functions. Many of the built-in types have obvious external formats. However, several types are unique to HAWQ, such as geometric paths, or have several possibilities for formats, such as the date and time types. Some of the input and output functions are not invertible. That is, the result of an output function may lose accuracy when compared to the original input.
@@ -75,6 +75,8 @@ Each data type has an external representation determined by its input and output
For variable length data types (such as char, varchar, text, xml, etc.) if the data is greater than or equal to 127 bytes, the storage overhead is 4 bytes instead of 1.
+**Note**: Use these documented built-in types when creating user tables. Any other data types that might be visible in the source code are for internal use only.
+
## <a id="timezones"></a>Time Zones
Time zones, and time-zone conventions, are influenced by political decisions, not just earth geometry. Time zones around the world became somewhat standardized during the 1900's, but continue to be prone to arbitrary changes, particularly with respect to daylight-savings rules.�HAWQ�uses the widely-used�zoneinfo�time zone database for information about historical time zone rules. For times in the future, the assumption is that the latest known rules for a given time zone will continue to be observed indefinitely far into the future.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/cd4a3db2/requirements/system-requirements.html.md.erb
----------------------------------------------------------------------
diff --git a/requirements/system-requirements.html.md.erb b/requirements/system-requirements.html.md.erb
index bc08ef0..35a2ed9 100644
--- a/requirements/system-requirements.html.md.erb
+++ b/requirements/system-requirements.html.md.erb
@@ -187,6 +187,9 @@ If this system uses the default HAWQ resource manager, you would set `hawq_rm_me
- Dedicated, non-blocking switch.
- Systems with multiple NICs require NIC bonding to utilize all available network bandwidth.
+## <a id="topic_rdb_jhv_2v"></a>Networking Requirements
+
+Communications between the HAWQ master and segments require reverse DNS lookup be configured in your cluster network.
## <a id="port-req"></a>Port Requirements
Individual PXF plug-ins, which you install after adding the HAWQ and PXF services, require that you Tomcat on the host machine. Tomcat reserves ports 8005, 8080, and 8009.
[34/50] [abbrv] incubator-hawq-docs git commit: even more updates -
clean up update, delete, vacuum references
Posted by yo...@apache.org.
even more updates - clean up update, delete, vacuum references
Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/commit/b6dd1372
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/tree/b6dd1372
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/diff/b6dd1372
Branch: refs/heads/master
Commit: b6dd13722077d4c2c9acada68c5eff07cc56359e
Parents: 94eac9f
Author: Lisa Owen <lo...@pivotal.io>
Authored: Fri Sep 23 10:27:29 2016 -0700
Committer: Lisa Owen <lo...@pivotal.io>
Committed: Fri Sep 23 10:27:29 2016 -0700
----------------------------------------------------------------------
datamgmt/ConcurrencyControl.html.md.erb | 7 -------
datamgmt/Transactions.html.md.erb | 2 --
reference/sql/GRANT.html.md.erb | 2 +-
3 files changed, 1 insertion(+), 10 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/b6dd1372/datamgmt/ConcurrencyControl.html.md.erb
----------------------------------------------------------------------
diff --git a/datamgmt/ConcurrencyControl.html.md.erb b/datamgmt/ConcurrencyControl.html.md.erb
index 0291ddb..2ced135 100644
--- a/datamgmt/ConcurrencyControl.html.md.erb
+++ b/datamgmt/ConcurrencyControl.html.md.erb
@@ -17,15 +17,8 @@ HAWQ provides multiple lock modes to control concurrent access to data in tables
| Lock Mode | Associated SQL Commands | Conflicts With |
|------------------------|-------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------|
| ACCESS SHARE | `SELECT` | ACCESS EXCLUSIVE |
-| ROW SHARE | `SELECT FOR UPDATE`, `SELECT FOR SHARE` | EXCLUSIVE, ACCESS EXCLUSIVE |
| ROW EXCLUSIVE | `INSERT`, `COPY` | SHARE, SHARE ROW EXCLUSIVE, EXCLUSIVE, ACCESS EXCLUSIVE |
| SHARE UPDATE EXCLUSIVE | `VACUUM` (without `FULL`), `ANALYZE` | SHARE UPDATE EXCLUSIVE, SHARE, SHARE ROW EXCLUSIVE, EXCLUSIVE, ACCESS EXCLUSIVE |
| SHARE | `CREATE INDEX` | ROW EXCLUSIVE, SHARE UPDATE EXCLUSIVE, SHARE ROW EXCLUSIVE, EXCLUSIVE, ACCESS EXCLUSIVE |
| SHARE ROW EXCLUSIVE | � | ROW EXCLUSIVE, SHARE UPDATE EXCLUSIVE, SHARE, SHARE ROW EXCLUSIVE, EXCLUSIVE, ACCESS EXCLUSIVE |
-| EXCLUSIVE | `DELETE`, `UPDATE` See [Note](#topic_f5l_qnh_kr__lock_note) | ROW SHARE, ROW EXCLUSIVE, SHARE UPDATE EXCLUSIVE, SHARE, SHARE ROW EXCLUSIVE, EXCLUSIVE, ACCESS EXCLUSIVE |
| ACCESS EXCLUSIVE | `ALTER TABLE`, `DROP TABLE`, `TRUNCATE`, `REINDEX`, `CLUSTER`, `VACUUM FULL` | ACCESS SHARE, ROW SHARE, ROW EXCLUSIVE, SHARE UPDATE EXCLUSIVE, SHARE, SHARE ROW EXCLUSIVE, EXCLUSIVE, ACCESS EXCLUSIVE |
-
-
-**Note:** In HAWQ, `UPDATE` and `DELETE` acquire the more restrictive lock EXCLUSIVE rather than ROW EXCLUSIVE.
-
-
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/b6dd1372/datamgmt/Transactions.html.md.erb
----------------------------------------------------------------------
diff --git a/datamgmt/Transactions.html.md.erb b/datamgmt/Transactions.html.md.erb
index a057121..dfc9a5e 100644
--- a/datamgmt/Transactions.html.md.erb
+++ b/datamgmt/Transactions.html.md.erb
@@ -49,8 +49,6 @@ A `SELECT` query:
Successive `SELECT` commands within a single transaction always see the same data.
- `SELECT FOR UPDATE` and `SELECT FOR SHARE` commands find only rows committed before the command started. If a concurrent transaction has already locked a target row when the row is found, the serializable or repeatable read transaction waits for the concurrent transaction to update the row, delete the row, or roll back.
-
The default transaction isolation level in HAWQ is *read committed*. To change the isolation level for a transaction, declare the isolation level when you `BEGIN` the transaction or use the `SET TRANSACTION` command after the transaction starts.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/b6dd1372/reference/sql/GRANT.html.md.erb
----------------------------------------------------------------------
diff --git a/reference/sql/GRANT.html.md.erb b/reference/sql/GRANT.html.md.erb
index 4baed38..1673df5 100644
--- a/reference/sql/GRANT.html.md.erb
+++ b/reference/sql/GRANT.html.md.erb
@@ -167,7 +167,7 @@ GRANT admins TO joe;
The `PRIVILEGES` key word in is required in the SQL standard, but optional in HAWQ. The SQL standard does not support setting the privileges on more than one object per command.
-HAWQ allows an object owner to revoke his own ordinary privileges: for example, a table owner can make the table read-only to himself by revoking his own `INSERT`, `UPDATE`, and `DELETE` privileges. This is not possible according to the SQL standard. HAWQ treats the owner's privileges as having been granted by the owner to himself; therefore he can revoke them too. In the SQL standard, the owner's privileges are granted by an assumed *system* entity.
+HAWQ allows an object owner to revoke his own ordinary privileges: for example, a table owner can make the table read-only to himself by revoking his own `INSERT` privileges. This is not possible according to the SQL standard. HAWQ treats the owner's privileges as having been granted by the owner to himself; therefore he can revoke them too. In the SQL standard, the owner's privileges are granted by an assumed *system* entity.
The SQL standard allows setting privileges for individual columns within a table.
[42/50] [abbrv] incubator-hawq-docs git commit: instruct user to turn
off pwd based auth after install if desired
Posted by yo...@apache.org.
instruct user to turn off pwd based auth after install if desired
Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/commit/268ed136
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/tree/268ed136
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/diff/268ed136
Branch: refs/heads/master
Commit: 268ed1368a81a2dc6c5756ba2008e35bd7328207
Parents: dd25c6f
Author: Lisa Owen <lo...@pivotal.io>
Authored: Tue Sep 27 13:52:59 2016 -0700
Committer: Lisa Owen <lo...@pivotal.io>
Committed: Wed Sep 28 09:10:24 2016 -0700
----------------------------------------------------------------------
install/aws-config.html.md.erb | 17 ++---------------
1 file changed, 2 insertions(+), 15 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/268ed136/install/aws-config.html.md.erb
----------------------------------------------------------------------
diff --git a/install/aws-config.html.md.erb b/install/aws-config.html.md.erb
index 98aa35a..881f742 100644
--- a/install/aws-config.html.md.erb
+++ b/install/aws-config.html.md.erb
@@ -30,9 +30,6 @@ Instance store-backed storage is generally better performing than EBS and recomm
Virtual devices for instance store volumes for HAWQ EC2 instance store instances are named `ephemeralN` (where *N* varies based on instance type). CentOS instance store block device are named `/dev/xvdletter` (where *letter* is a lower case letter of the alphabet).
-?? DO WE RECOMMEND EBS-backed VOLUMES FOR ANYTHING? I.E. BACKUPS, ...??)
-
-
### <a id="topic_cfgplacegrp"></a>Configure Placement Group
A placement group is a logical grouping of instances within a single availability zone that together participate in a low-latency, 10 Gbps network. Your HAWQ master and segment cluster instances should support enhanced networking and reside in a single placement group (and subnet) for optimal network performance.
@@ -77,22 +74,12 @@ A security group is a set of rules that control network traffic to and from your
To configure HAWQ communication between nodes in the HAWQ cluster, include and open the following ports in the appropriate security group for the HAWQ master and segment nodes:
-?? PROVIDE SPECIFIC type/protocol/portrange/source settings from EC2 console ??
-
| Port | Application |
|-------|-------------------------------------|
-| 22 | ssh - secure connect to other hosts |
-| XX | XX |
-
-UDP access within the VPC should be enabled by default.
-
-
-?? DIFFENT SECURITY GROUP FOR AMBARI NODE ??
+| 22 | ssh - secure connect to other hosts |
To allow access to/from a source external to the Ambari management node, include and open the following ports in an appropriate security group for your Ambari node:
-?? PROVIDE SPECIFIC type/protocol/portrange/source settings from EC2 console ??
-
| Port | Application |
|-------|-------------------------------------|
| 22 | ssh - secure connect to other hosts |
@@ -118,7 +105,7 @@ Before installing HAWQ, set up the EC2 instances as you would local host server
###Passwordless SSH Configuration<a id="topic_pwdlessssh_cc"></a>
-HAWQ hosts will be configured during the installation process to use passwordless SSH for intra-cluster communications. Temporary password-based authentication must be enabled on each HAWQ host in preparation for this configuration. Password authentication is typically disabled by default in cloud images. Update the cloud configuration in `/etc/cloud/cloud.cfg` to enable password authentication in your AMI(s). Set `ssh_pwauth: True` in this file.
+HAWQ hosts will be configured during the installation process to use passwordless SSH for intra-cluster communications. Temporary password-based authentication must be enabled on each HAWQ host in preparation for this configuration. Password authentication is typically disabled by default in cloud images. Update the cloud configuration in `/etc/cloud/cloud.cfg` to enable password authentication in your AMI(s). Set `ssh_pwauth: True` in this file. If desired, disable password authentication after HAWQ installation by setting the property back to `False`.
##References<a id="topic_hgz_zwy_bv"></a>
[36/50] [abbrv] incubator-hawq-docs git commit: clean up update,
delete, vacuum references
Posted by yo...@apache.org.
clean up update, delete, vacuum references
Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/commit/85e8a5da
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/tree/85e8a5da
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/diff/85e8a5da
Branch: refs/heads/master
Commit: 85e8a5da28b810649921f482c56d7530d1e5ac2c
Parents: 37857ea
Author: Lisa Owen <lo...@pivotal.io>
Authored: Thu Sep 22 14:37:13 2016 -0700
Committer: Lisa Owen <lo...@pivotal.io>
Committed: Fri Sep 23 12:01:04 2016 -0700
----------------------------------------------------------------------
admin/RecommendedMonitoringTasks.html.md.erb | 6 ++----
admin/monitor.html.md.erb | 2 +-
clientaccess/roles_privs.html.md.erb | 2 +-
datamgmt/BasicDataOperations.html.md.erb | 12 +++++++-----
datamgmt/ConcurrencyControl.html.md.erb | 7 -------
datamgmt/Transactions.html.md.erb | 10 +++-------
datamgmt/about_statistics.html.md.erb | 8 +++-----
...timizing-data-load-and-query-performance.html.md.erb | 6 +++---
ddl/ddl-partition.html.md.erb | 4 ++--
ddl/ddl-storage.html.md.erb | 9 +++------
reference/catalog/pg_class.html.md.erb | 2 +-
reference/catalog/pg_index.html.md.erb | 2 +-
reference/cli/admin_utilities/hawqload.html.md.erb | 2 +-
reference/cli/client_utilities/vacuumdb.html.md.erb | 2 ++
reference/guc/guc_category-list.html.md.erb | 2 +-
reference/guc/parameter_definitions.html.md.erb | 6 +-----
reference/sql/ALTER-TABLE.html.md.erb | 2 +-
reference/sql/BEGIN.html.md.erb | 2 +-
reference/sql/COPY.html.md.erb | 2 +-
reference/sql/CREATE-EXTERNAL-TABLE.html.md.erb | 2 +-
reference/sql/CREATE-TABLE.html.md.erb | 4 ++--
reference/sql/CREATE-VIEW.html.md.erb | 2 +-
reference/sql/DROP-TABLE.html.md.erb | 2 +-
reference/sql/GRANT.html.md.erb | 2 +-
reference/sql/PREPARE.html.md.erb | 2 +-
reference/sql/VACUUM.html.md.erb | 10 +++++++---
reference/toolkit/hawq_toolkit.html.md.erb | 2 +-
27 files changed, 50 insertions(+), 64 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/85e8a5da/admin/RecommendedMonitoringTasks.html.md.erb
----------------------------------------------------------------------
diff --git a/admin/RecommendedMonitoringTasks.html.md.erb b/admin/RecommendedMonitoringTasks.html.md.erb
index a01ea55..3007aee 100644
--- a/admin/RecommendedMonitoringTasks.html.md.erb
+++ b/admin/RecommendedMonitoringTasks.html.md.erb
@@ -193,7 +193,7 @@ GROUP BY 1;
<p>Severity: CRITICAL</p>
</td>
<td>
- <p><p>Vacuum an individual table:</p>
+ <p><p>Vacuum an individual system catalog table:</p>
<pre><code>VACUUM <<i>table</i>>;</code></pre>
</td>
<td>After the <a href="/20/reference/guc/parameter_definitions.html">vacuum_freeze_min_age</a> value is reached, VACUUM will no longer replace transaction IDs with <code>FrozenXID</code> while scanning a table. Perform vacuum on these tables before the limit is reached.</td>
@@ -224,9 +224,7 @@ GROUP BY 1;
<p>Recommended frequency: weekly, or more often if database objects are created and dropped frequently</p>
</td>
<td>
- <ol>
- <li><code>VACUUM</code> the system tables in each database.</li>
- </ol>
+ <p><code>VACUUM</code> the system tables in each database.</p>
</td>
<td>The optimizer retrieves information from the system tables to create query plans. If system tables and indexes are allowed to become bloated over time, scanning the system tables increases query execution time.</td>
</tr>
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/85e8a5da/admin/monitor.html.md.erb
----------------------------------------------------------------------
diff --git a/admin/monitor.html.md.erb b/admin/monitor.html.md.erb
index 1e464e2..d1fbf31 100644
--- a/admin/monitor.html.md.erb
+++ b/admin/monitor.html.md.erb
@@ -78,7 +78,7 @@ HAWQ tracks various metadata information in its system catalogs about the object
#### <a id="topic25"></a>Viewing the Last Operation Performed
-You can use the system views *pg\_stat\_operations* and *pg\_stat\_partition\_operations* to look up actions performed on an object, such as a table. For example, to see the actions performed on a table, such as when it was created and when it was last vacuumed and analyzed:
+You can use the system views *pg\_stat\_operations* and *pg\_stat\_partition\_operations* to look up actions performed on an object, such as a table. For example, to see the actions performed on a table, such as when it was created and when it was last analyzed:
```sql
=> SELECT schemaname as schema, objname as table,
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/85e8a5da/clientaccess/roles_privs.html.md.erb
----------------------------------------------------------------------
diff --git a/clientaccess/roles_privs.html.md.erb b/clientaccess/roles_privs.html.md.erb
index bd2de0a..4503951 100644
--- a/clientaccess/roles_privs.html.md.erb
+++ b/clientaccess/roles_privs.html.md.erb
@@ -16,7 +16,7 @@ In order to bootstrap the HAWQ system, a freshly initialized system always conta
## <a id="topic2"></a>Security Best Practices for Roles and Privileges
-- **Secure the gpadmin system user.** HAWQ requires a UNIX user id to install and initialize the HAWQ system. This system user is referred to as `gpadmin` in the HAWQ documentation. This `gpadmin` user is the default database superuser in HAWQ, as well as the file system owner of the HAWQ installation and its underlying data files. This default administrator account is fundamental to the design of HAWQ. The system cannot run without it, and there is no way to limit the access of this gpadmin user id. Use roles to manage who has access to the database for specific purposes. You should only use the `gpadmin` account for system maintenance tasks such as expansion and upgrade. Anyone who logs on to a HAWQ host as this user id can read, alter or delete any data; including system catalog data and database access rights. Therefore, it is very important to secure the gpadmin user id and only provide access to essential system administrators. Administrators should only log in to HAWQ as `g
padmin` when performing certain system maintenance tasks \(such as upgrade or expansion\). Database users should never log on as `gpadmin`, and ETL or production workloads should never run as `gpadmin`.
+- **Secure the gpadmin system user.** HAWQ requires a UNIX user id to install and initialize the HAWQ system. This system user is referred to as `gpadmin` in the HAWQ documentation. This `gpadmin` user is the default database superuser in HAWQ, as well as the file system owner of the HAWQ installation and its underlying data files. This default administrator account is fundamental to the design of HAWQ. The system cannot run without it, and there is no way to limit the access of this gpadmin user id. Use roles to manage who has access to the database for specific purposes. You should only use the `gpadmin` account for system maintenance tasks such as expansion and upgrade. Anyone who logs on to a HAWQ host as this user id can read, alter or delete any data; specifically system catalog data and database access rights. Therefore, it is very important to secure the gpadmin user id and only provide access to essential system administrators. Administrators should only log in to HAWQ as
`gpadmin` when performing certain system maintenance tasks \(such as upgrade or expansion\). Database users should never log on as `gpadmin`, and ETL or production workloads should never run as `gpadmin`.
- **Assign a distinct role to each user that logs in.** For logging and auditing purposes, each user that is allowed to log in to HAWQ should be given their own database role. For applications or web services, consider creating a distinct role for each application or service. See [Creating New Roles \(Users\)](#topic3).
- **Use groups to manage access privileges.** See [Role Membership](#topic5).
- **Limit users who have the SUPERUSER role attribute.** Roles that are superusers bypass all access privilege checks in HAWQ, as well as resource queuing. Only system administrators should be given superuser rights. See [Altering Role Attributes](#topic4).
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/85e8a5da/datamgmt/BasicDataOperations.html.md.erb
----------------------------------------------------------------------
diff --git a/datamgmt/BasicDataOperations.html.md.erb b/datamgmt/BasicDataOperations.html.md.erb
index cd2cf16..6c3323c 100644
--- a/datamgmt/BasicDataOperations.html.md.erb
+++ b/datamgmt/BasicDataOperations.html.md.erb
@@ -40,16 +40,18 @@ To insert data into a partitioned table, you specify the root partitioned table,
To insert large amounts of data, use external tables or the `COPY` command. These load mechanisms are more efficient than `INSERT` for inserting large quantities of rows. See [Loading and Unloading Data](load/g-loading-and-unloading-data.html#topic1) for more information about bulk data loading.
-## <a id="topic9"></a>Vacuuming the Database
+## <a id="topic9"></a>Vacuuming the System Catalog Tables
-Deleted or updated data rows occupy physical space on disk even though new transactions cannot see them. Periodically running the `VACUUM` command removes these expired rows. For example:
+Only HAWQ system catalog tables use multiple version concurrency control. Deleted or updated data rows in the catalog tables occupy physical space on disk even though new transactions cannot see them. Periodically running the `VACUUM` command removes these expired rows.
+
+Periodically running the `VACUUM` command on system catalog tables removes these expired rows. The `VACUUM` command also collects table-level statistics such as the number of rows and pages.
+
+For example:
``` sql
-VACUUM mytable;
+VACUUM pg_class;
```
-The `VACUUM` command collects table-level statistics such as the number of rows and pages. Vacuum all tables after loading data.
-
### <a id="topic10"></a>Configuring the Free Space Map
Expired rows are held in the *free space map*. The free space map must be sized large enough to hold all expired rows in your database. If not, a regular `VACUUM` command cannot reclaim space occupied by expired rows that overflow the free space map.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/85e8a5da/datamgmt/ConcurrencyControl.html.md.erb
----------------------------------------------------------------------
diff --git a/datamgmt/ConcurrencyControl.html.md.erb b/datamgmt/ConcurrencyControl.html.md.erb
index 0291ddb..2ced135 100644
--- a/datamgmt/ConcurrencyControl.html.md.erb
+++ b/datamgmt/ConcurrencyControl.html.md.erb
@@ -17,15 +17,8 @@ HAWQ provides multiple lock modes to control concurrent access to data in tables
| Lock Mode | Associated SQL Commands | Conflicts With |
|------------------------|-------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------|
| ACCESS SHARE | `SELECT` | ACCESS EXCLUSIVE |
-| ROW SHARE | `SELECT FOR UPDATE`, `SELECT FOR SHARE` | EXCLUSIVE, ACCESS EXCLUSIVE |
| ROW EXCLUSIVE | `INSERT`, `COPY` | SHARE, SHARE ROW EXCLUSIVE, EXCLUSIVE, ACCESS EXCLUSIVE |
| SHARE UPDATE EXCLUSIVE | `VACUUM` (without `FULL`), `ANALYZE` | SHARE UPDATE EXCLUSIVE, SHARE, SHARE ROW EXCLUSIVE, EXCLUSIVE, ACCESS EXCLUSIVE |
| SHARE | `CREATE INDEX` | ROW EXCLUSIVE, SHARE UPDATE EXCLUSIVE, SHARE ROW EXCLUSIVE, EXCLUSIVE, ACCESS EXCLUSIVE |
| SHARE ROW EXCLUSIVE | � | ROW EXCLUSIVE, SHARE UPDATE EXCLUSIVE, SHARE, SHARE ROW EXCLUSIVE, EXCLUSIVE, ACCESS EXCLUSIVE |
-| EXCLUSIVE | `DELETE`, `UPDATE` See [Note](#topic_f5l_qnh_kr__lock_note) | ROW SHARE, ROW EXCLUSIVE, SHARE UPDATE EXCLUSIVE, SHARE, SHARE ROW EXCLUSIVE, EXCLUSIVE, ACCESS EXCLUSIVE |
| ACCESS EXCLUSIVE | `ALTER TABLE`, `DROP TABLE`, `TRUNCATE`, `REINDEX`, `CLUSTER`, `VACUUM FULL` | ACCESS SHARE, ROW SHARE, ROW EXCLUSIVE, SHARE UPDATE EXCLUSIVE, SHARE, SHARE ROW EXCLUSIVE, EXCLUSIVE, ACCESS EXCLUSIVE |
-
-
-**Note:** In HAWQ, `UPDATE` and `DELETE` acquire the more restrictive lock EXCLUSIVE rather than ROW EXCLUSIVE.
-
-
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/85e8a5da/datamgmt/Transactions.html.md.erb
----------------------------------------------------------------------
diff --git a/datamgmt/Transactions.html.md.erb b/datamgmt/Transactions.html.md.erb
index b5d7e97..dfc9a5e 100644
--- a/datamgmt/Transactions.html.md.erb
+++ b/datamgmt/Transactions.html.md.erb
@@ -24,7 +24,7 @@ HAWQ accepts the standard SQL transaction levels as follows:
The following information describes the behavior of the HAWQ transaction levels:
-- **read committed/read uncommitted** \u2014 Provides fast, simple, partial transaction isolation. With read committed and read uncommitted transaction isolation, `SELECT`, `UPDATE`, and `DELETE` transactions operate on a snapshot of the database taken when the query started.
+- **read committed/read uncommitted** \u2014 Provides fast, simple, partial transaction isolation. With read committed and read uncommitted transaction isolation, `SELECT` transactions operate on a snapshot of the database taken when the query started.
A `SELECT` query:
@@ -33,9 +33,9 @@ A `SELECT` query:
- Does not see uncommitted data outside the transaction.
- Can possibly see changes that concurrent transactions made if the concurrent transaction is committed after the initial read in its own transaction.
-Successive `SELECT` queries in the same transaction can see different data if other concurrent transactions commit changes before the queries start. `UPDATE` and `DELETE` commands find only rows committed before the commands started.
+Successive `SELECT` queries in the same transaction can see different data if other concurrent transactions commit changes before the queries start.
-Read committed or read uncommitted transaction isolation allows concurrent transactions to modify or lock a row before `UPDATE` or `DELETE` finds the row. Read committed or read uncommitted transaction isolation may be inadequate for applications that perform complex queries and updates and require a consistent view of the database.
+Read committed or read uncommitted transaction isolation may be inadequate for applications that perform complex queries and require a consistent view of the database.
- **serializable/repeatable read** \u2014 Provides strict transaction isolation in which transactions execute as if they run one after another rather than concurrently. Applications on the serializable or repeatable read level must be designed to retry transactions in case of serialization failures.
@@ -49,10 +49,6 @@ A `SELECT` query:
Successive `SELECT` commands within a single transaction always see the same data.
- `UPDATE`, `DELETE, SELECT FOR UPDATE,` and `SELECT FOR SHARE` commands find only rows committed before the command started. If a concurrent transaction has already updated, deleted, or locked a target row when the row is found, the serializable or repeatable read transaction waits for the concurrent transaction to update the row, delete the row, or roll back.
-
- If the concurrent transaction updates or deletes the row, the serializable or repeatable read transaction rolls back. If the concurrent transaction rolls back, then the serializable or repeatable read transaction updates or deletes the row.
-
The default transaction isolation level in HAWQ is *read committed*. To change the isolation level for a transaction, declare the isolation level when you `BEGIN` the transaction or use the `SET TRANSACTION` command after the transaction starts.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/85e8a5da/datamgmt/about_statistics.html.md.erb
----------------------------------------------------------------------
diff --git a/datamgmt/about_statistics.html.md.erb b/datamgmt/about_statistics.html.md.erb
index d4b7665..25b3b24 100644
--- a/datamgmt/about_statistics.html.md.erb
+++ b/datamgmt/about_statistics.html.md.erb
@@ -20,7 +20,7 @@ Calculating statistics consumes time and resources, so HAWQ produces estimates b
### <a id="tablesize"></a>Table Size
-The query planner seeks to minimize the disk I/O and network traffic required to execute a query, using estimates of the number of rows that must be processed and the number of disk pages the query must access. The data from which these estimates are derived are the `pg_class` system table columns `reltuples` and `relpages`, which contain the number of rows and pages at the time a `VACUUM` or `ANALYZE` command was last run. As rows are added or deleted, the numbers become less accurate. However, an accurate count of disk pages is always available from the operating system, so as long as the ratio of `reltuples` to `relpages` does not change significantly, the optimizer can produce an estimate of the number of rows that is sufficiently accurate to choose the correct query execution plan.
+The query planner seeks to minimize the disk I/O and network traffic required to execute a query, using estimates of the number of rows that must be processed and the number of disk pages the query must access. The data from which these estimates are derived are the `pg_class` system table columns `reltuples` and `relpages`, which contain the number of rows and pages at the time a `VACUUM` or `ANALYZE` command was last run. As rows are added, the numbers become less accurate. However, an accurate count of disk pages is always available from the operating system, so as long as the ratio of `reltuples` to `relpages` does not change significantly, the optimizer can produce an estimate of the number of rows that is sufficiently accurate to choose the correct query execution plan.
In append-optimized tables, the number of tuples is kept up-to-date in the system catalogs, so the `reltuples` statistic is not an estimate. Non-visible tuples in the table are subtracted from the total. The `relpages` value is estimated from the append-optimized block sizes.
@@ -122,8 +122,6 @@ If a sample table is created, the number of rows in the sample is calculated to
Running `ANALYZE` with no arguments updates statistics for all tables in the database. This could take a very long time, so it is better to analyze tables selectively after data has changed. You can also analyze a subset of the columns in a table, for example columns used in joins, `WHERE` clauses, `SORT` clauses, `GROUP BY` clauses, or `HAVING` clauses.
-Analyzing a severely bloated table can generate poor statistics if the sample contains empty pages, so it is good practice to vacuum a bloated table before analyzing it.
-
See the SQL Command Reference for details of running the `ANALYZE` command.
Refer to the Management Utility Reference for details of running the `analyzedb` command.
@@ -150,7 +148,7 @@ Set the system default statistics target to a different value by setting the `de
$ hawq config -c default_statistics_target -v 50
```
-The statististics target for individual columns can be set with the `ALTER TABLE` command. For example, some queries can be improved by increasing the target for certain columns, especially columns that have irregular distributions. You can set the target to zero for columns that never contribute to query otpimization. When the target is 0, `ANALYZE` ignores the column. For example, the following `ALTER TABLE` command sets the statistics target for the `notes` column in the `emp` table to zero:
+The statististics target for individual columns can be set with the `ALTER TABLE` command. For example, some queries can be improved by increasing the target for certain columns, especially columns that have irregular distributions. You can set the target to zero for columns that never contribute to query optimization. When the target is 0, `ANALYZE` ignores the column. For example, the following `ALTER TABLE` command sets the statistics target for the `notes` column in the `emp` table to zero:
``` sql
ALTER TABLE emp ALTER COLUMN notes SET STATISTICS 0;
@@ -168,7 +166,7 @@ Automatic statistics collection has three modes:
- `none` disables automatic statistics collection.
- `on_no_stats` triggers an analyze operation for a table with no existing statistics when any of the commands `CREATE TABLE AS SELECT`, `INSERT`, or `COPY` are executed on the table.
-- `on_change` triggers an analyze operation when any of the commands `CREATE TABLE AS SELECT`, `UPDATE`, `DELETE`, `INSERT`, or `COPY` are executed on the table and the number of rows affected exceeds the threshold defined by the `gp_autostats_on_change_threshold` configuration parameter.
+- `on_change` triggers an analyze operation when any of the commands `CREATE TABLE AS SELECT`, `INSERT`, or `COPY` are executed on the table and the number of rows affected exceeds the threshold defined by the `gp_autostats_on_change_threshold` configuration parameter.
The automatic statistics collection mode is set separately for commands that occur within a procedural language function and commands that execute outside of a function:
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/85e8a5da/datamgmt/load/g-optimizing-data-load-and-query-performance.html.md.erb
----------------------------------------------------------------------
diff --git a/datamgmt/load/g-optimizing-data-load-and-query-performance.html.md.erb b/datamgmt/load/g-optimizing-data-load-and-query-performance.html.md.erb
index 203efc5..ff1c230 100644
--- a/datamgmt/load/g-optimizing-data-load-and-query-performance.html.md.erb
+++ b/datamgmt/load/g-optimizing-data-load-and-query-performance.html.md.erb
@@ -2,9 +2,9 @@
title: Optimizing Data Load and Query Performance
---
-Use the following tips to help optimize your data load and subsequent query performance.
+Use the following tip to help optimize your data load and subsequent query performance.
+
+- Run `ANALYZE` after loading data. If you significantly altered the data in a table, run `ANALYZE` or `VACUUM ANALYZE` (system catalog tables only) to update table statistics for the query optimizer. Current statistics ensure that the optimizer makes the best decisions during query planning and avoids poor performance due to inaccurate or nonexistent statistics.
-- Run `ANALYZE` after loading data. If you significantly altered the data in a table, run `ANALYZE` or `VACUUM ANALYZE` to update table statistics for the query optimizer. Current statistics ensure that the optimizer makes the best decisions during query planning and avoids poor performance due to inaccurate or nonexistent statistics.
-- Run `VACUUM` after load errors. If the load operation does not run in single row error isolation mode, the operation stops at the first error. The target table contains the rows loaded before the error occurred. You cannot access these rows, but they occupy disk space. Use the `VACUUM` command to recover the wasted space.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/85e8a5da/ddl/ddl-partition.html.md.erb
----------------------------------------------------------------------
diff --git a/ddl/ddl-partition.html.md.erb b/ddl/ddl-partition.html.md.erb
index 42b7ebe..39fd43c 100644
--- a/ddl/ddl-partition.html.md.erb
+++ b/ddl/ddl-partition.html.md.erb
@@ -23,7 +23,7 @@ HAWQ uses the partition criteria defined during table creation to create each pa
The HAWQ system catalog stores partition hierarchy information so that rows inserted into the top-level parent table propagate correctly to the child table partitions. To change the partition design or table structure, alter the parent table using `ALTER TABLE` with the `PARTITION` clause.
-To insert data into a partitioned table, you specify the root partitioned table, the table created with the `CREATE TABLE` command. You also can specify a leaf child table of the partitioned table in an `INSERT` command. An error is returned if the data is not valid for the specified leaf child table. Specifying a child table that is not a leaf child table in the `INSERT` command is not supported. Execution of other DML commands such as `UPDATE` and `DELETE` on any child table of a partitioned table is not supported. These commands must be executed on the root partitioned table, the table created with the `CREATE TABLE` command.
+To insert data into a partitioned table, you specify the root partitioned table, the table created with the `CREATE TABLE` command. You also can specify a leaf child table of the partitioned table in an `INSERT` command. An error is returned if the data is not valid for the specified leaf child table. Specifying a child table that is not a leaf child table in the `INSERT` command is not supported.
## <a id="topic65"></a>Deciding on a Table Partitioning Strategy
@@ -258,7 +258,7 @@ WHERE tablename='sales';
The following table and views show information about partitioned tables.
-- *pg\_partition*- Tracks partitioned tables and their inheritance level relationships.
+- *pg\_partition* - Tracks partitioned tables and their inheritance level relationships.
- *pg\_partition\_templates* - Shows the subpartitions created using a subpartition template.
- *pg\_partition\_columns* - Shows the partition key columns used in a partition design.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/85e8a5da/ddl/ddl-storage.html.md.erb
----------------------------------------------------------------------
diff --git a/ddl/ddl-storage.html.md.erb b/ddl/ddl-storage.html.md.erb
index 4711d00..264e552 100644
--- a/ddl/ddl-storage.html.md.erb
+++ b/ddl/ddl-storage.html.md.erb
@@ -16,7 +16,6 @@ HAWQ provides storage orientation models of either row-oriented or Parquet table
Row-oriented storage provides the best options for the following situations:
-- **Updates of table data.** Where you load and update the table data frequently.
- **Frequent INSERTs.** Where rows are frequently inserted into the table
- **Number of columns requested in queries.** Where you typically request all or the majority of columns in the `SELECT` list or `WHERE` clause of your queries, choose a row-oriented model.
- **Number of columns in the table.** Row-oriented storage is most efficient when many columns are required at the same time, or when the row-size of a table is relatively small.
@@ -63,12 +62,10 @@ The`DROP TABLE`command removes tables from the database. For example:
DROP TABLE mytable;
```
-To empty a table of rows without removing the table definition, use `DELETE` or `TRUNCATE`. For example:
+`DROP TABLE` always removes any indexes, rules, triggers, and constraints that exist for the target table. Specify `CASCADE`to drop a table that is referenced by a view. `CASCADE` removes dependent views.
-``` sql
-DELETE FROM mytable;
+To empty a table of rows without removing the table definition, use `TRUNCATE`. For example:
+``` sql
TRUNCATE mytable;
```
-
-`DROP TABLE`always removes any indexes, rules, triggers, and constraints that exist for the target table. Specify `CASCADE`to drop a table that is referenced by a view. `CASCADE` removes dependent views.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/85e8a5da/reference/catalog/pg_class.html.md.erb
----------------------------------------------------------------------
diff --git a/reference/catalog/pg_class.html.md.erb b/reference/catalog/pg_class.html.md.erb
index 5dff1f2..c0244b1 100644
--- a/reference/catalog/pg_class.html.md.erb
+++ b/reference/catalog/pg_class.html.md.erb
@@ -69,7 +69,7 @@ The system catalog table `pg_class` catalogs tables and most everything else tha
<td><code class="ph codeph">relpages</code></td>
<td>integer</td>
<td>�</td>
-<td>Size of the on-disk representation of this table in pages (of 32K each). This is only an estimate used by the planner. It is updated by <code class="ph codeph">VACUUM</code>, <code class="ph codeph">ANALYZE</code>, and a few DDL commands.</td>
+<td>Size of the on-disk representation of this table in pages (of 32K each). This is only an estimate used by the planner. It is updated by <code class="ph codeph">ANALYZE</code>, and a few DDL commands.</td>
</tr>
<tr class="odd">
<td><code class="ph codeph">reltuples</code></td>
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/85e8a5da/reference/catalog/pg_index.html.md.erb
----------------------------------------------------------------------
diff --git a/reference/catalog/pg_index.html.md.erb b/reference/catalog/pg_index.html.md.erb
index cd1a054..7abe921 100644
--- a/reference/catalog/pg_index.html.md.erb
+++ b/reference/catalog/pg_index.html.md.erb
@@ -14,7 +14,7 @@ The `pg_index` system catalog table contains part of the information about index
| `indnatts` | smallint | � | The number of columns in the index (duplicates pg\_class.relnatts). |
| `indisunique` | boolean | � | If true, this is a unique index. |
| `indisclustered` | boolean | � | If true, the table was last clustered on this index via the `CLUSTER` command. |
-| `indisvalid` | boolean | � | If true, the index is currently valid for queries. False means the index is possibly incomplete: it must still be modified by `INSERT`/`UPDATE` operations, but it cannot safely be used for queries. |
+| `indisvalid` | boolean | � | If true, the index is currently valid for queries. False means the index is possibly incomplete: it must still be modified by `INSERT` operations, but it cannot safely be used for queries. |
| `indkey` | int2vector | pg\_attribute.attnum | This is an array of indnatts values that indicate which table columns this index indexes. For example a value of 1 3 would mean that the first and the third table columns make up the index key. A zero in this array indicates that the corresponding index attribute is an expression over the table columns, rather than a simple column reference. |
| `indclass` | oidvector | pg\_opclass.oid | For each column in the index key this contains the OID of the operator class to use. |
| `indexprs` | text | � | Expression trees (in `nodeToString()` representation) for index attributes that are not simple column references. This is a list with one element for each zero entry in indkey. NULL if all index attributes are simple references. |
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/85e8a5da/reference/cli/admin_utilities/hawqload.html.md.erb
----------------------------------------------------------------------
diff --git a/reference/cli/admin_utilities/hawqload.html.md.erb b/reference/cli/admin_utilities/hawqload.html.md.erb
index 8d7d984..b9fe441 100644
--- a/reference/cli/admin_utilities/hawqload.html.md.erb
+++ b/reference/cli/admin_utilities/hawqload.html.md.erb
@@ -42,7 +42,7 @@ The client machine where `hawq load` is executed must have the following:
## <a id="topic1__section4"></a>Description
-`hawq load` is a data loading utility that acts as an interface to HAWQ's external table parallel loading feature. Using a load specification defined in a YAML formatted control file, `hawq load` executes a load by invoking the HAWQ parallel file server ([gpfdist](gpfdist.html#topic1)), creating an external table definition based on the source data defined, and executing an `INSERT`, `UPDATE` or `MERGE` operation to load the source data into the target table in the database.
+`hawq load` is a data loading utility that acts as an interface to HAWQ's external table parallel loading feature. Using a load specification defined in a YAML formatted control file, `hawq load` executes a load by invoking the HAWQ parallel file server ([gpfdist](gpfdist.html#topic1)), creating an external table definition based on the source data defined, and executing an `INSERT` operation to load the source data into the target table in the database.
The operation, including any SQL commands specified in the `SQL` collection of the YAML control file (see [Control File Format](#topic1__section7)), are performed as a single transaction to prevent inconsistent data when performing multiple, simultaneous load operations on a target table.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/85e8a5da/reference/cli/client_utilities/vacuumdb.html.md.erb
----------------------------------------------------------------------
diff --git a/reference/cli/client_utilities/vacuumdb.html.md.erb b/reference/cli/client_utilities/vacuumdb.html.md.erb
index 30617df..cbc37f3 100644
--- a/reference/cli/client_utilities/vacuumdb.html.md.erb
+++ b/reference/cli/client_utilities/vacuumdb.html.md.erb
@@ -4,6 +4,8 @@ title: vacuumdb
Garbage-collects and analyzes a database.
+`vacuumdb` is typically run on system catalog tables. It has no effect when run on HAWQ user tables.
+
## <a id="topic1__section2"></a>Synopsis
``` pre
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/85e8a5da/reference/guc/guc_category-list.html.md.erb
----------------------------------------------------------------------
diff --git a/reference/guc/guc_category-list.html.md.erb b/reference/guc/guc_category-list.html.md.erb
index 715583b..e77f925 100644
--- a/reference/guc/guc_category-list.html.md.erb
+++ b/reference/guc/guc_category-list.html.md.erb
@@ -344,7 +344,7 @@ These parameters adjust the amount of data sampled by an `ANALYZE` operation. Ad
### <a id="topic_qvz_nz3_yv"></a>Automatic Statistics Collection
-When automatic statistics collection is enabled, you can run `ANALYZE` automatically in the same transaction as an `INSERT`, `UPDATE`, `DELETE`, `COPY` or `CREATE TABLE...AS SELECT` statement when a certain threshold of rows is affected (`on_change`), or when a newly generated table has no statistics (`on_no_stats`). To enable this feature, set the following server configuration parameters in your HAWQ `hawq-site.xml` file by using the `hawq config` utility and restart HAWQ:
+When automatic statistics collection is enabled, you can run `ANALYZE` automatically in the same transaction as an `INSERT`, `COPY` or `CREATE TABLE...AS SELECT` statement when a certain threshold of rows is affected (`on_change`), or when a newly generated table has no statistics (`on_no_stats`). To enable this feature, set the following server configuration parameters in your HAWQ `hawq-site.xml` file by using the `hawq config` utility and restart HAWQ:
- [gp\_autostats\_mode](parameter_definitions.html#gp_autostats_mode)
- [log\_autostats](parameter_definitions.html#log_autostats)
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/85e8a5da/reference/guc/parameter_definitions.html.md.erb
----------------------------------------------------------------------
diff --git a/reference/guc/parameter_definitions.html.md.erb b/reference/guc/parameter_definitions.html.md.erb
index c792d1b..645335b 100644
--- a/reference/guc/parameter_definitions.html.md.erb
+++ b/reference/guc/parameter_definitions.html.md.erb
@@ -1076,10 +1076,6 @@ The `on_change` option triggers statistics collection only when the number of ro
`CREATE TABLE AS SELECT`
-`UPDATE`
-
-`DELETE`
-
`INSERT`
`COPY`
@@ -2391,7 +2387,7 @@ The maximum value is INT\_MAX/1024. If an invalid value is specified, the defaul
## <a name="log_statement"></a>log\_statement
-Controls which SQL statements are logged. DDL logs all data definition commands like CREATE, ALTER, and DROP commands. MOD logs all DDL statements, plus INSERT, UPDATE, DELETE, TRUNCATE, and COPY FROM. PREPARE and EXPLAIN ANALYZE statements are also logged if their contained command is of an appropriate type.
+Controls which SQL statements are logged. DDL logs all data definition commands like CREATE, ALTER, and DROP commands. MOD logs all DDL statements, plus INSERT, TRUNCATE, and COPY FROM. PREPARE and EXPLAIN ANALYZE statements are also logged if their contained command is of an appropriate type.
<table>
<colgroup>
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/85e8a5da/reference/sql/ALTER-TABLE.html.md.erb
----------------------------------------------------------------------
diff --git a/reference/sql/ALTER-TABLE.html.md.erb b/reference/sql/ALTER-TABLE.html.md.erb
index 7b1d74d..4303f0c 100644
--- a/reference/sql/ALTER-TABLE.html.md.erb
+++ b/reference/sql/ALTER-TABLE.html.md.erb
@@ -314,7 +314,7 @@ When a column is added with `ADD COLUMN`, all existing rows in the table are ini
You can specify multiple changes in a single `ALTER TABLE` command, which will be done in a single pass over the table.
-The `DROP COLUMN` form does not physically remove the column, but simply makes it invisible to SQL operations. Subsequent insert and update operations in the table will store a null value for the column. Thus, dropping a column is quick but it will not immediately reduce the on-disk size of your table, as the space occupied by the dropped column is not reclaimed. The space will be reclaimed over time as existing rows are updated.
+The `DROP COLUMN` form does not physically remove the column, but simply makes it invisible to SQL operations. Subsequent insert operations in the table will store a null value for the column. Thus, dropping a column is quick but it will not immediately reduce the on-disk size of your table, as the space occupied by the dropped column is not reclaimed.
The fact that `ALTER TYPE` requires rewriting the whole table is sometimes an advantage, because the rewriting process eliminates any dead space in the table. For example, to reclaim the space occupied by a dropped column immediately, the fastest way is: `ALTER TABLE <table> ALTER COLUMN <anycol> TYPE <sametype>;` Where�\<anycol\>�is any remaining table column and�\<sametype\>�is the same type that column already has. This results in no semantically-visible change in the table, but the command forces rewriting, which gets rid of no-longer-useful data.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/85e8a5da/reference/sql/BEGIN.html.md.erb
----------------------------------------------------------------------
diff --git a/reference/sql/BEGIN.html.md.erb b/reference/sql/BEGIN.html.md.erb
index 5c2a9bb..265e66e 100644
--- a/reference/sql/BEGIN.html.md.erb
+++ b/reference/sql/BEGIN.html.md.erb
@@ -31,7 +31,7 @@ READ UNCOMMITTED </dt>
<dt>READ WRITE
READ ONLY </dt>
-<dd>Determines whether the transaction is read/write or read-only. Read/write is the default. When a transaction is read-only, the following SQL commands are disallowed: `INSERT`, `UPDATE`, `DELETE`, and `COPY FROM` if the table they would write to is not a temporary table; all `CREATE`, `ALTER`, and `DROP` commands; `GRANT`, `REVOKE`, `TRUNCATE`; and `EXPLAIN ANALYZE` and `EXECUTE` if the command they would execute is among those listed.</dd>
+<dd>Determines whether the transaction is read/write or read-only. Read/write is the default. When a transaction is read-only, the following SQL commands are disallowed: `INSERT` and `COPY FROM` if the table they would write to is not a temporary table; all `CREATE`, `ALTER`, and `DROP` commands; `GRANT`, `REVOKE`, `TRUNCATE`; and `EXPLAIN ANALYZE` and `EXECUTE` if the command they would execute is among those listed.</dd>
## <a id="topic1__section5"></a>Notes
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/85e8a5da/reference/sql/COPY.html.md.erb
----------------------------------------------------------------------
diff --git a/reference/sql/COPY.html.md.erb b/reference/sql/COPY.html.md.erb
index 6069aa5..aaa2270 100644
--- a/reference/sql/COPY.html.md.erb
+++ b/reference/sql/COPY.html.md.erb
@@ -139,7 +139,7 @@ Files named in a `COPY` command are read or written directly by the database ser
`COPY` input and output is affected by `DateStyle`. To ensure portability to other HAWQ installations that might use non-default `DateStyle` settings, `DateStyle` should be set to ISO before using `COPY TO`.
-By default, `COPY` stops operation at the first error. This should not lead to problems in the event of a `COPY TO`, but the target table will already have received earlier rows in a `COPY FROM`. These rows will not be visible or accessible, but they still occupy disk space. This may amount to a considerable amount of wasted disk space if the failure happened well into a large `COPY FROM` operation. You may wish to invoke `VACUUM` to recover the wasted space. Another option would be to use single row error isolation mode to filter out error rows while still loading good rows.
+By default, `COPY` stops operation at the first error. This should not lead to problems in the event of a `COPY TO`, but the target table will already have received earlier rows in a `COPY FROM`. These rows will not be visible or accessible, but they still occupy disk space. This may amount to a considerable amount of wasted disk space if the failure happened well into a large `COPY FROM` operation. You may wish to use single row error isolation mode to filter out error rows while still loading good rows.
COPY supports creating readable foreign tables with error tables. The default for concurrently inserting into the error table is 127.�You can use�error tables with foreign tables under the following circumstances:
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/85e8a5da/reference/sql/CREATE-EXTERNAL-TABLE.html.md.erb
----------------------------------------------------------------------
diff --git a/reference/sql/CREATE-EXTERNAL-TABLE.html.md.erb b/reference/sql/CREATE-EXTERNAL-TABLE.html.md.erb
index 2b164dc..2f19eab 100644
--- a/reference/sql/CREATE-EXTERNAL-TABLE.html.md.erb
+++ b/reference/sql/CREATE-EXTERNAL-TABLE.html.md.erb
@@ -109,7 +109,7 @@ where \<pxf parameters\> is:
## <a id="topic1__section3"></a>Description
-`CREATE EXTERNAL TABLE` or `CREATE EXTERNAL WEB TABLE` creates a new readable external table definition in HAWQ. Readable external tables are typically used for fast, parallel data loading. Once an external table is defined, you can query its data directly (and in parallel) using SQL commands. For example, you can select, join, or sort external table data. You can also create views for external tables. DML operations (`UPDATE`, `INSERT`, `DELETE`, or` TRUNCATE`) are not allowed on readable external tables.
+`CREATE EXTERNAL TABLE` or `CREATE EXTERNAL WEB TABLE` creates a new readable external table definition in HAWQ. Readable external tables are typically used for fast, parallel data loading. Once an external table is defined, you can query its data directly (and in parallel) using SQL commands. For example, you can select, join, or sort external table data. You can also create views for external tables. DML operations (`UPDATE`, `INSERT`, `DELETE`, or `TRUNCATE`) are not permitted on readable external tables.
`CREATE WRITABLE EXTERNAL TABLE` or `CREATE WRITABLE EXTERNAL WEB TABLE` creates a new writable external table definition in HAWQ. Writable external tables are typically used for unloading data from the database into a set of files or named pipes.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/85e8a5da/reference/sql/CREATE-TABLE.html.md.erb
----------------------------------------------------------------------
diff --git a/reference/sql/CREATE-TABLE.html.md.erb b/reference/sql/CREATE-TABLE.html.md.erb
index 99ff35e..162a438 100644
--- a/reference/sql/CREATE-TABLE.html.md.erb
+++ b/reference/sql/CREATE-TABLE.html.md.erb
@@ -161,7 +161,7 @@ where \<storage\_parameter\> for a partition is:
`CREATE TABLE` creates a new, initially empty table in the current database. The table is owned by the user issuing the command. If a schema name is given then the table is created in the specified schema. Otherwise it is created in the current schema. Temporary tables exist in a special schema, so a schema name may not be given when creating a temporary table. The name of the table must be distinct from the name of any other table, external table, sequence, or view in the same schema.
-The optional constraint clauses specify conditions that new or updated rows must satisfy for an insert or update operation to succeed. A constraint is an SQL object that helps define the set of valid values in the table in various ways. Constraints apply to tables, not to partitions. You cannot add a constraint to a partition or subpartition.
+The optional constraint clauses specify conditions that new rows must satisfy for an insert operation to succeed. A constraint is an SQL object that helps define the set of valid values in the table in various ways. Constraints apply to tables, not to partitions. You cannot add a constraint to a partition or subpartition.
There are two ways to define constraints: table constraints and column constraints. A column constraint is defined as part of a column definition. A table constraint definition is not tied to a particular column, and it can encompass more than one column. Every column constraint can also be written as a table constraint; a column constraint is only a notational convenience for use when the constraint only affects one column.
@@ -213,7 +213,7 @@ Note also that unlike `INHERITS`, copied columns and constraints are not merged
<dd>Specifies if the column is or is not allowed to contain null values. `NULL` is the default.</dd>
<dt>CHECK ( \<expression\> ) </dt>
-<dd>The `CHECK` clause specifies an expression producing a Boolean result which new or updated rows must satisfy for an insert or update operation to succeed. Expressions evaluating to `TRUE` or `UNKNOWN` succeed. Should any row of an insert or update operation produce a `FALSE` result an error exception is raised and the insert or update does not alter the database. A check constraint specified as a column constraint should reference that column's value only, while an expression appearing in a table constraint may reference multiple columns. `CHECK` expressions cannot contain subqueries nor refer to variables other than columns of the current row.</dd>
+<dd>The `CHECK` clause specifies an expression producing a Boolean result which new rows must satisfy for an insert operation to succeed. Expressions evaluating to `TRUE` or `UNKNOWN` succeed. Should any row of an insert operation produce a `FALSE` result an error exception is raised and the insert does not alter the database. A check constraint specified as a column constraint should reference that column's value only, while an expression appearing in a table constraint may reference multiple columns. `CHECK` expressions cannot contain subqueries nor refer to variables other than columns of the current row.</dd>
<dt>WITH ( \<storage\_option\>=\<value\> ) </dt>
<dd>The `WITH` clause can be used to set storage options for the table or its indexes. Note that you can also set storage parameters on a particular partition or subpartition by declaring the `WITH` clause in the partition specification.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/85e8a5da/reference/sql/CREATE-VIEW.html.md.erb
----------------------------------------------------------------------
diff --git a/reference/sql/CREATE-VIEW.html.md.erb b/reference/sql/CREATE-VIEW.html.md.erb
index 4a24bb7..e39d8d3 100644
--- a/reference/sql/CREATE-VIEW.html.md.erb
+++ b/reference/sql/CREATE-VIEW.html.md.erb
@@ -77,7 +77,7 @@ names, rank WHERE rank < '11' AND names.id=rank.id;
The SQL standard specifies some additional capabilities for the `CREATE VIEW` statement that are not in HAWQ. The optional clauses for the full SQL command in the standard are:
-- **CHECK OPTION** \u2014 This option has to do with updatable views. All `INSERT` and `UPDATE` commands on the view will be checked to ensure data satisfy the view-defining condition (that is, the new data would be visible through the view). If they do not, the update will be rejected.
+- **CHECK OPTION** \u2014 This option has to do with updatable views. All `INSERT` commands on the view will be checked to ensure data satisfy the view-defining condition (that is, the new data would be visible through the view). If they do not, the insert will be rejected.
- **LOCAL** \u2014 Check for integrity on this view.
- **CASCADED** \u2014 Check for integrity on this view and on any dependent view. `CASCADED` is assumed if neither `CASCADED` nor `LOCAL` is specified.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/85e8a5da/reference/sql/DROP-TABLE.html.md.erb
----------------------------------------------------------------------
diff --git a/reference/sql/DROP-TABLE.html.md.erb b/reference/sql/DROP-TABLE.html.md.erb
index 98022ce..b277273 100644
--- a/reference/sql/DROP-TABLE.html.md.erb
+++ b/reference/sql/DROP-TABLE.html.md.erb
@@ -12,7 +12,7 @@ DROP TABLE [IF EXISTS] <name> [, ...] [CASCADE | RESTRICT]
## <a id="topic1__section3"></a>Description
-`DROP TABLE` removes tables from the database. Only its owner may drop a table. To empty a table of rows without removing the table definition, use `DELETE` or `TRUNCATE`.
+`DROP TABLE` removes tables from the database. Only its owner may drop a table. To empty a table of rows without removing the table definition, use `TRUNCATE`.
`DROP TABLE` always removes any indexes, rules, and constraints that exist for the target table. However, to drop a table that is referenced by a view, `CASCADE` must be specified. `CASCADE` will remove a dependent view entirely.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/85e8a5da/reference/sql/GRANT.html.md.erb
----------------------------------------------------------------------
diff --git a/reference/sql/GRANT.html.md.erb b/reference/sql/GRANT.html.md.erb
index 4baed38..1673df5 100644
--- a/reference/sql/GRANT.html.md.erb
+++ b/reference/sql/GRANT.html.md.erb
@@ -167,7 +167,7 @@ GRANT admins TO joe;
The `PRIVILEGES` key word in is required in the SQL standard, but optional in HAWQ. The SQL standard does not support setting the privileges on more than one object per command.
-HAWQ allows an object owner to revoke his own ordinary privileges: for example, a table owner can make the table read-only to himself by revoking his own `INSERT`, `UPDATE`, and `DELETE` privileges. This is not possible according to the SQL standard. HAWQ treats the owner's privileges as having been granted by the owner to himself; therefore he can revoke them too. In the SQL standard, the owner's privileges are granted by an assumed *system* entity.
+HAWQ allows an object owner to revoke his own ordinary privileges: for example, a table owner can make the table read-only to himself by revoking his own `INSERT` privileges. This is not possible according to the SQL standard. HAWQ treats the owner's privileges as having been granted by the owner to himself; therefore he can revoke them too. In the SQL standard, the owner's privileges are granted by an assumed *system* entity.
The SQL standard allows setting privileges for individual columns within a table.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/85e8a5da/reference/sql/PREPARE.html.md.erb
----------------------------------------------------------------------
diff --git a/reference/sql/PREPARE.html.md.erb b/reference/sql/PREPARE.html.md.erb
index 6f6836c..c633f14 100644
--- a/reference/sql/PREPARE.html.md.erb
+++ b/reference/sql/PREPARE.html.md.erb
@@ -30,7 +30,7 @@ Prepared statements have the largest performance advantage when a single session
<dd>The data type of a parameter to the prepared statement. If the data type of a particular parameter is unspecified or is specified as unknown, it will be inferred from the context in which the parameter is used. To refer to the parameters in the prepared statement itself, use `$1`, `$2`, etc.</dd>
<dt> \<statement\> </dt>
-<dd>Any `SELECT`, `INSERT`, `UPDATE`, `DELETE`, or `VALUES` statement.</dd>
+<dd>Any `SELECT`, `INSERT`, or `VALUES` statement.</dd>
## <a id="topic1__section5"></a>Notes
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/85e8a5da/reference/sql/VACUUM.html.md.erb
----------------------------------------------------------------------
diff --git a/reference/sql/VACUUM.html.md.erb b/reference/sql/VACUUM.html.md.erb
index 2db5757..9b39a32 100644
--- a/reference/sql/VACUUM.html.md.erb
+++ b/reference/sql/VACUUM.html.md.erb
@@ -2,7 +2,9 @@
title: VACUUM
---
-Garbage-collects and optionally analyzes a database.
+Garbage-collects and optionally analyzes a database.
+
+**Note**: HAWQ `VACUUM` support is provided only for system catalog tables. `VACUUM`ing a HAWQ user table has no effect.
## <a id="topic1__section2"></a>Synopsis
@@ -14,11 +16,13 @@ VACUUM [FULL] [FREEZE] [VERBOSE] ANALYZE
## <a id="topic1__section3"></a>Description
-`VACUUM` reclaims storage occupied by deleted tuples. In normal HAWQ operation, tuples that are deleted or obsoleted by an update are not physically removed from their table; they remain present on disk until a `VACUUM` is done. Therefore it is necessary to do `VACUUM` periodically, especially on frequently-updated catalog tables. `VACUUM` has no effect on a normal HAWQ table, since the delete or update operations are not supported on normal HAWQ table.
+`VACUUM` reclaims storage occupied by deleted tuples. In normal HAWQ operation, tuples that are deleted or obsoleted by an update are not physically removed from their table; they remain present on disk until a `VACUUM` is done. Therefore it is necessary to do `VACUUM` periodically, especially on frequently-updated catalog tables. (`VACUUM` has no effect on a normal HAWQ table, since the delete or update operations are not supported on normal HAWQ table.)
With no parameter, `VACUUM` processes every table in the current database. With a parameter, `VACUUM` processes only that table. `VACUUM ANALYZE` performs a `VACUUM` and then an `ANALYZE` for each selected table. This is a handy combination form for routine maintenance scripts. See [ANALYZE](ANALYZE.html) for more details about its processing.
-Plain `VACUUM` (without `FULL`) simply reclaims space and makes it available for re-use. This form of the command can operate in parallel with normal reading and writing of the table, as an exclusive lock is not obtained. `VACUUM FULL` does more extensive processing, including moving of tuples across blocks to try to compact the table to the minimum number of disk blocks. This form is much slower and requires an exclusive lock on each table while it is being processed.
+Plain `VACUUM` (without `FULL`) simply reclaims space and makes it available for re-use. This form of the command can operate in parallel with normal reading and writing of the table, as an exclusive lock is not obtained. `VACUUM FULL` does more extensive processing, including moving of tuples across blocks to try to compact the table to the minimum number of disk blocks. This form is much slower and requires an exclusive lock on each table while it is being processed.
+
+**Note:** `VACUUM FULL` is not recommended in HAWQ.
**Outputs**
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/85e8a5da/reference/toolkit/hawq_toolkit.html.md.erb
----------------------------------------------------------------------
diff --git a/reference/toolkit/hawq_toolkit.html.md.erb b/reference/toolkit/hawq_toolkit.html.md.erb
index f76963c..45a8a56 100644
--- a/reference/toolkit/hawq_toolkit.html.md.erb
+++ b/reference/toolkit/hawq_toolkit.html.md.erb
@@ -16,7 +16,7 @@ The following views can help identify tables that need routine table maintenance
- [hawq\_stats\_missing](#topic4)
-The `VACUUM` command reclaims disk space occupied by deleted or obsolete rows. Because of the MVCC transaction concurrency model used in HAWQ, data rows that are deleted or updated still occupy physical space on disk even though they are not visible to any new transactions. Expired rows increase table size on disk and eventually slow down scans of the table.
+The `VACUUM` command is applicable only to system catalog tables. The `VACUUM` command reclaims disk space occupied by deleted or obsolete rows. Because of the MVCC transaction concurrency model used in HAWQ, data rows that are deleted or updated still occupy physical space on disk even though they are not visible to any new transactions. Expired rows increase table size on disk and eventually slow down scans of the table.
**Note:** VACUUM FULL is not recommended in HAWQ. See [VACUUM](../sql/VACUUM.html#topic1).
[19/50] [abbrv] incubator-hawq-docs git commit: hawq check - add
h/host option, cleanup
Posted by yo...@apache.org.
hawq check - add h/host option, cleanup
Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/commit/01663016
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/tree/01663016
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/diff/01663016
Branch: refs/heads/master
Commit: 016630163015e782ef630338998ef1696f5f005e
Parents: 6704cc0
Author: Lisa Owen <lo...@pivotal.io>
Authored: Thu Sep 15 10:52:32 2016 -0700
Committer: Lisa Owen <lo...@pivotal.io>
Committed: Thu Sep 15 10:52:32 2016 -0700
----------------------------------------------------------------------
reference/cli/admin_utilities/hawqcheck.html.md.erb | 15 +++++++++------
1 file changed, 9 insertions(+), 6 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/01663016/reference/cli/admin_utilities/hawqcheck.html.md.erb
----------------------------------------------------------------------
diff --git a/reference/cli/admin_utilities/hawqcheck.html.md.erb b/reference/cli/admin_utilities/hawqcheck.html.md.erb
index 3b31d82..de67903 100644
--- a/reference/cli/admin_utilities/hawqcheck.html.md.erb
+++ b/reference/cli/admin_utilities/hawqcheck.html.md.erb
@@ -7,9 +7,9 @@ Verifies and validates HAWQ platform settings.
## <a id="topic1__section2"></a>Synopsis
``` pre
-hawq check -f <hostfile_hawq_check>
+hawq check -f <hostfile_hawq_check> | (-h <hostname> | --host <hostname>)
+ --hadoop <hadoop_home> | --hadoop-home <hadoop_home>
[--config <config_file>]
- [--hadoop <hadoop_home> | --hadoop-home <hadoop_home>]
[--stdout | --zipout]
[--kerberos]
[--hdfs-ha]
@@ -43,6 +43,9 @@ diskusage.monitor.mounts = /,/data1,/data2
<dt>-f \<hostfile\_hawq\_check\> </dt>
<dd>The name of a file that contains a list of hosts that `hawq check` uses to validate platform-specific settings. This file should contain a single host name for all hosts in your HAWQ system (master, standby master, and segments).</dd>
+<dt>-h, -\\\-host \<hostname\> </dt>
+<dd>Specifies a single host on which platform-specific settings will be validated.</dd>
+
<dt>-\\\-zipin \<hawq\_check\_zipfile\> </dt>
<dd>Use this option to decompress and check a .zip file created with the `--zipout` option. If you specify the `--zipin` option, `hawq check` performs validation tasks against the specified file.</dd>
@@ -85,19 +88,19 @@ diskusage.monitor.mounts = /,/data1,/data2
Verify and validate the HAWQ platform settings by entering a host file and specifying the full hadoop install path:
``` shell
-$ hawq check -f hostfile_hawq_check --hadoop /usr/hdp/version/hadoop
+$ hawq check -f hostfile_hawq_check --hadoop /usr/hdp/<version>/hadoop
```
Verify and validate the HAWQ platform settings with HDFS HA enabled, YARN HA enabled and Kerberos enabled:
``` shell
-$ hawq check -f hostfile_hawq_check --hadoop /usr/hdp/version/hadoop --hdfs-ha --yarn-ha --kerberos
+$ hawq check -f hostfile_hawq_check --hadoop /usr/local/hadoop-<version> --hdfs-ha --yarn-ha --kerberos
```
Verify and validate the HAWQ platform settings with HDFS HA enabled, and Kerberos enabled:
``` shell
-$ hawq check -f hostfile_hawq_check --hadoop /usr/hdp/version/hadoop --hdfs-ha --kerberos
+$ hawq check -f hostfile_hawq_check --hadoop /usr/hdp/<version>/hadoop --hdfs-ha --kerberos
```
Save HAWQ platform settings to a zip file, when the `$HADOOP_HOME` environment variable is set:
@@ -115,7 +118,7 @@ $ hawq check --zipin hawq_check_timestamp.tar.gz
View collected HAWQ platform settings:
``` shell
-$ hawq check -f hostfile_hawq_check --hadoop /usr/hdp/version/hadoop --stdout
+$ hawq check -f hostfile_hawq_check --hadoop /usr/local/hadoop-<version> --stdout
```
## <a id="topic1__section6"></a>See Also
[20/50] [abbrv] incubator-hawq-docs git commit: hawq check - hadoop
home is optional
Posted by yo...@apache.org.
hawq check - hadoop home is optional
Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/commit/4a617974
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/tree/4a617974
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/diff/4a617974
Branch: refs/heads/master
Commit: 4a617974cf04d1b1758bdfbc490116b60bdefb79
Parents: 0166301
Author: Lisa Owen <lo...@pivotal.io>
Authored: Thu Sep 15 11:38:36 2016 -0700
Committer: Lisa Owen <lo...@pivotal.io>
Committed: Thu Sep 15 11:38:36 2016 -0700
----------------------------------------------------------------------
reference/cli/admin_utilities/hawqcheck.html.md.erb | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/4a617974/reference/cli/admin_utilities/hawqcheck.html.md.erb
----------------------------------------------------------------------
diff --git a/reference/cli/admin_utilities/hawqcheck.html.md.erb b/reference/cli/admin_utilities/hawqcheck.html.md.erb
index de67903..23a496d 100644
--- a/reference/cli/admin_utilities/hawqcheck.html.md.erb
+++ b/reference/cli/admin_utilities/hawqcheck.html.md.erb
@@ -8,7 +8,7 @@ Verifies and validates HAWQ platform settings.
``` pre
hawq check -f <hostfile_hawq_check> | (-h <hostname> | --host <hostname>)
- --hadoop <hadoop_home> | --hadoop-home <hadoop_home>
+ [--hadoop <hadoop_home> | --hadoop-home <hadoop_home>]
[--config <config_file>]
[--stdout | --zipout]
[--kerberos]
[21/50] [abbrv] incubator-hawq-docs git commit: add debug logging
info to PXF troubleshooting page
Posted by yo...@apache.org.
add debug logging info to PXF troubleshooting page
Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/commit/a15e1862
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/tree/a15e1862
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/diff/a15e1862
Branch: refs/heads/master
Commit: a15e18625ce215873747fc3fb3ece97403dcfe1d
Parents: 1661c62
Author: Lisa Owen <lo...@pivotal.io>
Authored: Fri Sep 16 12:54:28 2016 -0700
Committer: Lisa Owen <lo...@pivotal.io>
Committed: Fri Sep 16 12:54:28 2016 -0700
----------------------------------------------------------------------
pxf/TroubleshootingPXF.html.md.erb | 71 ++++++++++++++++++++++++++++++++-
1 file changed, 70 insertions(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a15e1862/pxf/TroubleshootingPXF.html.md.erb
----------------------------------------------------------------------
diff --git a/pxf/TroubleshootingPXF.html.md.erb b/pxf/TroubleshootingPXF.html.md.erb
index 050e360..7b53065 100644
--- a/pxf/TroubleshootingPXF.html.md.erb
+++ b/pxf/TroubleshootingPXF.html.md.erb
@@ -2,7 +2,9 @@
title: Troubleshooting PXF
---
-The following table describes some common errors while using PXF:
+## <a id="pxerrortbl"></a>PXF Errors
+
+The following table lists some common errors encountered while using PXF:
<table>
<caption><span class="tablecap">Table 1. PXF Errors and Explanation</span></caption>
@@ -132,3 +134,70 @@ The following table describes some common errors while using PXF:
</table>
+## <a id="pxflogging"></a>PXF Logging
+Enabling more verbose logging may aid PXF troubleshooting efforts.
+
+PXF provides two categories of message logging - service-level and database-level.
+
+### <a id="pxfsvclogmsg"></a>Service-Level Logging
+
+PXF utilizes `log4j` for service-level logging. PXF-service-related log messages are captured in a log file specified by PXF's `log4j` properties file, `/etc/pxf/conf/pxf-log4j.properties`. The default PXF logging configuration will write `INFO` and more severe level logs to `/var/log/pxf/pxf-service.log`.
+
+PXF provides more detailed logging when the `DEBUG` level is enabled. To configure PXF `DEBUG` logging, uncomment the following line in `pxf-log4j.properties`:
+
+``` shell
+#log4j.logger.org.apache.hawq.pxf=DEBUG
+```
+
+and restart the PXF service:
+
+``` shell
+$ sudo service pxf-service restart
+```
+
+With `DEBUG` level logging now enabled, perform your PXF operations; for example, creating and querying an external table. (Make note of the time; this will direct you to the relevant log messages in `/var/log/pxf/pxf-service.log`.)
+
+``` shell
+$ psql
+```
+
+``` sql
+gpadmin=# CREATE EXTERNAL TABLE hivetest(id int, newid int)
+ LOCATION ('pxf://namenode:51200/pxf_hive1?PROFILE=Hive')
+ FORMAT 'custom' (formatter='pxfwritable_import');
+gpadmin=# select * from hivetest;
+<select output>
+```
+
+Examine/collect the log messages from `pxf-service.log`.
+
+**Note**: `DEBUG` logging is verbose and has a performance impact. Remember to turn off PXF service `DEBUG` logging after you have collected the desired information.
+
+
+### <a id="pxfdblogmsg"></a>Database-Level Logging
+
+Enable HAWQ and PXF debug message logging during operations on PXF external tables by setting the `client_min_messages` server configuration parameter to `DEBUG2` in your `psql` session.
+
+``` shell
+$ psql
+```
+
+``` sql
+gpadmin=# set client_min_messages=DEBUG2
+gpadmin=# select * from hivetest;
+...
+DEBUG2: churl http header: cell #19: X-GP-URL-HOST: localhost
+DEBUG2: churl http header: cell #20: X-GP-URL-PORT: 51200
+DEBUG2: churl http header: cell #21: X-GP-DATA-DIR: pxf_hive1
+DEBUG2: churl http header: cell #22: X-GP-profile: Hive
+DEBUG2: churl http header: cell #23: X-GP-URI: pxf://namenode:51200/pxf_hive1?profile=Hive
+...
+```
+
+Examine/collect the log messages from `stdout`.
+
+**Note**: `DEBUG2` database session logging has a performance impact. Remember to turn off `DEBUG2` logging after you have collected the desired information.
+
+``` sql
+gpadmin=# set client_min_messages=NOTICE
+```
\ No newline at end of file
[18/50] [abbrv] incubator-hawq-docs git commit: hawq check --hadoop
option - misc cleanup
Posted by yo...@apache.org.
hawq check --hadoop option - misc cleanup
Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/commit/6704cc0b
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/tree/6704cc0b
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/diff/6704cc0b
Branch: refs/heads/master
Commit: 6704cc0b7a358fafba08b3fd66a5a12b5bb97f85
Parents: 0f642f1
Author: Lisa Owen <lo...@pivotal.io>
Authored: Wed Sep 14 14:21:07 2016 -0700
Committer: Lisa Owen <lo...@pivotal.io>
Committed: Wed Sep 14 14:21:07 2016 -0700
----------------------------------------------------------------------
reference/cli/admin_utilities/hawqcheck.html.md.erb | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/6704cc0b/reference/cli/admin_utilities/hawqcheck.html.md.erb
----------------------------------------------------------------------
diff --git a/reference/cli/admin_utilities/hawqcheck.html.md.erb b/reference/cli/admin_utilities/hawqcheck.html.md.erb
index 517004d..3b31d82 100644
--- a/reference/cli/admin_utilities/hawqcheck.html.md.erb
+++ b/reference/cli/admin_utilities/hawqcheck.html.md.erb
@@ -25,7 +25,7 @@ hawq check -?
## <a id="topic1__section3"></a>Description
-The `hawq check` utility determines the platform on which you are running HAWQ and validates various platform-specific configuration settings as well as HAWQ and HDFS-specific configuration settings. In order to perform HAWQ configuration checks, make sure HAWQ has been already started and `hawq config` works. For HDFS checks, you should either set the HADOOP\_HOME environment variable or provide the hadoop installation location using `--hadoop` option.
+The `hawq check` utility determines the platform on which you are running HAWQ and validates various platform-specific configuration settings as well as HAWQ and HDFS-specific configuration settings. In order to perform HAWQ configuration checks, make sure HAWQ has been already started and `hawq config` works. For HDFS checks, you should either set the `$HADOOP_HOME` environment variable or provide the full path to the hadoop installation location using the `--hadoop` option.
The `hawq check` utility can use a host file or a file previously created with the `--zipout `option to validate platform settings. If `GPCHECK_ERROR` displays, one or more validation checks failed. You can also use `hawq check` to gather and view platform settings on hosts without running validation checks. When running checks, `hawq check` compares your actual configuration setting with an expected value listed in a config file (`$GPHOME/etc/hawq_check.cnf` by default). You must modify your configuration values for "mount.points" and "diskusage.monitor.mounts" to reflect the actual mount points you want to check, as a comma-separated list. Otherwise, the utility only checks the root directory, which may not be helpful.
@@ -54,7 +54,7 @@ diskusage.monitor.mounts = /,/data1,/data2
<dd>The name of a configuration file to use instead of the default file `$GPHOME/etc/hawq_check.cnf`.</dd>
<dt>-\\\-hadoop, -\\\-hadoop-home \<hadoop\_home\> </dt>
-<dd>Use this option to specify the full path to your hadoop installation location so that `hawq check` can validate HDFS settings. This option is not needed if the `HADOOP_HOME` environment variable is set.</dd>
+<dd>Use this option to specify the full path to your hadoop installation location so that `hawq check` can validate HDFS settings. This option is not needed if the `$HADOOP_HOME` environment variable is set.</dd>
<dt>-\\\-stdout </dt>
<dd>Send collected host information from `hawq check` to standard output. No checks or validations are performed.</dd>
@@ -100,7 +100,7 @@ Verify and validate the HAWQ platform settings with HDFS HA enabled, and Kerbero
$ hawq check -f hostfile_hawq_check --hadoop /usr/hdp/version/hadoop --hdfs-ha --kerberos
```
-Save HAWQ platform settings to a zip file, when the $HADOOP\_HOME environment variable is set:
+Save HAWQ platform settings to a zip file, when the `$HADOOP_HOME` environment variable is set:
``` shell
$ hawq check -f hostfile_hawq_check --zipout
[27/50] [abbrv] incubator-hawq-docs git commit: include links to
gpfdist, hawq load, gpload
Posted by yo...@apache.org.
include links to gpfdist, hawq load, gpload
Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/commit/19b9c98d
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/tree/19b9c98d
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/diff/19b9c98d
Branch: refs/heads/master
Commit: 19b9c98d5ebe5bd2f8cceaddedac795b97f0b964
Parents: 23759ec
Author: Lisa Owen <lo...@pivotal.io>
Authored: Mon Sep 19 15:45:18 2016 -0700
Committer: Lisa Owen <lo...@pivotal.io>
Committed: Mon Sep 19 15:45:18 2016 -0700
----------------------------------------------------------------------
datamgmt/load/client-loadtools.html.md.erb | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/19b9c98d/datamgmt/load/client-loadtools.html.md.erb
----------------------------------------------------------------------
diff --git a/datamgmt/load/client-loadtools.html.md.erb b/datamgmt/load/client-loadtools.html.md.erb
index 7555621..af5d4c2 100644
--- a/datamgmt/load/client-loadtools.html.md.erb
+++ b/datamgmt/load/client-loadtools.html.md.erb
@@ -20,7 +20,7 @@ The RHEL Load Tools are provided in the HAWQ installation package.
The files/directories of interest in a HAWQ RHEL Load Tools installation include:
-`bin/` \u2014 data loading command-line tools (`gpfdist` and `hawq load`)
+`bin/` \u2014 data loading command-line tools ([gpfdist](../../reference/cli/admin_utilities/gpfdist.html) and [hawq load](../../reference/cli/admin_utilities/hawqload.html))
`greenplum_path.sh` \u2014 environment set up file
### <a id="installloadrhelcfgenv"></a>Configuring the RHEL Load Environment
@@ -64,7 +64,7 @@ The HAWQ Load Tools for Windows requires that the 32-bit version of Python 2.5 b
### <a id="installloadabout"></a>About the Windows Loader Installation
Your HAWQ Windows Load Tools installation includes the following files and directories:
-`bin/` \u2014 data loading command-line tools (`gpfdist` and `gpload`)
+`bin/` \u2014 data loading command-line tools ([gpfdist](http://gpdb.docs.pivotal.io/4380/client_tool_guides/load/unix/gpfdist.html) and [gpload](http://gpdb.docs.pivotal.io/4380/client_tool_guides/load/unix/gpload.html))
`lib/` \u2014 data loading library files
`greenplum_loaders_path.bat` \u2014 environment set up file
[49/50] [abbrv] incubator-hawq-docs git commit: Adding note to say
manual Kerberos config isn't required for Ambari installs.
Posted by yo...@apache.org.
Adding note to say manual Kerberos config isn't required for Ambari installs.
Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/commit/10cde80c
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/tree/10cde80c
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/diff/10cde80c
Branch: refs/heads/master
Commit: 10cde80c922ea6fe65252d32e5d1d60df326cf91
Parents: 459e3bc
Author: David Yozie <yo...@apache.org>
Authored: Thu Sep 29 08:27:43 2016 -0700
Committer: David Yozie <yo...@apache.org>
Committed: Thu Sep 29 08:27:43 2016 -0700
----------------------------------------------------------------------
clientaccess/kerberos.html.md.erb | 2 ++
1 file changed, 2 insertions(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/10cde80c/clientaccess/kerberos.html.md.erb
----------------------------------------------------------------------
diff --git a/clientaccess/kerberos.html.md.erb b/clientaccess/kerberos.html.md.erb
index 77d6998..a609471 100644
--- a/clientaccess/kerberos.html.md.erb
+++ b/clientaccess/kerberos.html.md.erb
@@ -2,6 +2,8 @@
title: Using Kerberos Authentication
---
+**Note:** The following steps for enabling Kerberos *are not required* if you install HAWQ using Ambari.
+
You can control access to HAWQ with a Kerberos authentication server.
HAWQ supports the Generic Security Service Application Program Interface \(GSSAPI\) with Kerberos authentication. GSSAPI provides automatic authentication \(single sign-on\) for systems that support it. You specify the HAWQ users \(roles\) that require Kerberos authentication in the HAWQ configuration file `pg_hba.conf`. The login fails if Kerberos authentication is not available when a role attempts to log in to HAWQ.
[44/50] [abbrv] incubator-hawq-docs git commit: passwdless ssh -
restart cmd change, remove some props
Posted by yo...@apache.org.
passwdless ssh - restart cmd change, remove some props
Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/commit/e8d612e5
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/tree/e8d612e5
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/diff/e8d612e5
Branch: refs/heads/master
Commit: e8d612e57246fc457dc0374c22ff9edcb9f4e19b
Parents: 5159281
Author: Lisa Owen <lo...@pivotal.io>
Authored: Tue Sep 27 08:35:02 2016 -0700
Committer: Lisa Owen <lo...@pivotal.io>
Committed: Wed Sep 28 09:34:02 2016 -0700
----------------------------------------------------------------------
requirements/system-requirements.html.md.erb | 19 ++++++++-----------
1 file changed, 8 insertions(+), 11 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/e8d612e5/requirements/system-requirements.html.md.erb
----------------------------------------------------------------------
diff --git a/requirements/system-requirements.html.md.erb b/requirements/system-requirements.html.md.erb
index c61ba3d..7f117dc 100644
--- a/requirements/system-requirements.html.md.erb
+++ b/requirements/system-requirements.html.md.erb
@@ -178,43 +178,40 @@ If this system uses the default HAWQ resource manager, you would set `hawq_rm_me
HAWQ hosts will be configured to use passwordless SSH for intra-cluster communications during the installation process. Temporary password-based authentication must be enabled on each HAWQ host in preparation for this configuration.
1. Install the SSH server if not already configured on the HAWQ system:
-
+
``` shell
$ yum list installed | grep openssh-server
$ yum -y install openssh-server
```
-
+
2. Update the host's SSH configuration to allow password-based authentication. Edit the SSH config file and change the `PasswordAuthentication` configuration value from `no` to `yes`:
-
+
``` shell
$ sudo vi /etc/ssh/sshd_config
```
-
+
```
PasswordAuthentication yes
```
3. Restart SSH:
-
+
``` shell
- $ sudo /etc/init.d/sshd restart
+ $ sudo service sshd restart
```
*After installation is complete*, you may choose to turn off the temporary password-based authentication configured in the previous steps:
-1. Open the SSH `/etc/ssh/sshd_config` file in a text editor and update/uncomment the following configuration options.
+1. Open the SSH `/etc/ssh/sshd_config` file in a text editor and update the configuration option you enabled in step 2 above:
```
- RSAAuthentication yes
PasswordAuthentication no
- PubkeyAuthentication yes
- AuthorizedKeyFile .ssh/authorized_keys
```
2. Restart SSH:
``` shell
- $ sudo /etc/init.d/sshd restart
+ $ sudo service sshd restart
```
�
[06/50] [abbrv] incubator-hawq-docs git commit: remove
MASTER_DATA_DIRECTORY from gplogfilter HAWQ-1031
Posted by yo...@apache.org.
remove MASTER_DATA_DIRECTORY from gplogfilter HAWQ-1031
Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/commit/1ba8a23c
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/tree/1ba8a23c
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/diff/1ba8a23c
Branch: refs/heads/master
Commit: 1ba8a23c7d743d963f6fdf257b02dfd449875cc6
Parents: 1a7efef
Author: Lisa Owen <lo...@pivotal.io>
Authored: Thu Sep 1 13:53:54 2016 -0700
Committer: Lisa Owen <lo...@pivotal.io>
Committed: Thu Sep 1 13:53:54 2016 -0700
----------------------------------------------------------------------
.../cli/admin_utilities/gplogfilter.html.md.erb | 16 ++++++++--------
1 file changed, 8 insertions(+), 8 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/1ba8a23c/reference/cli/admin_utilities/gplogfilter.html.md.erb
----------------------------------------------------------------------
diff --git a/reference/cli/admin_utilities/gplogfilter.html.md.erb b/reference/cli/admin_utilities/gplogfilter.html.md.erb
index da37147..44e73c5 100644
--- a/reference/cli/admin_utilities/gplogfilter.html.md.erb
+++ b/reference/cli/admin_utilities/gplogfilter.html.md.erb
@@ -45,7 +45,7 @@ where:
## <a id="topic1__section3"></a>Description
-The `gplogfilter` utility can be used to search through a HAWQ log file for entries matching the specified criteria. If an input file is not supplied, then `gplogfilter` will use the `$MASTER_DATA_DIRECTORY` environment variable to locate the HAWQ master log file in the standard logging location. To read from standard input, use a dash (`-`) as the input file name. Input files may be compressed using `gzip`. In an input file, a log entry is identified by its timestamp in `YYYY-MM-DD [hh:mm[:ss]]` format.
+The `gplogfilter` utility can be used to search through a HAWQ log file for entries matching the specified criteria. To read from standard input, use a dash (`-`) as the input file name. Input files may be compressed using `gzip`. In an input file, a log entry is identified by its timestamp in `YYYY-MM-DD [hh:mm[:ss]]` format.
You can also use `gplogfilter` to search through all segment log files at once by running it through the [hawq ssh](hawqssh.html#topic1) utility. For example, to display the last three lines of each segment log file:
@@ -63,7 +63,7 @@ By default, the output of `gplogfilter` is sent to standard output. Use the `-o`
**\<input_options\>**
<dt>\<input\_file\></dt>
-<dd>The name of the input log file(s) to search through. If an input file is not supplied, `gplogfilter` will use the `$MASTER_DATA_DIRECTORY` environment variable to locate the HAWQ master log file. To read from standard input, use a dash (`-`) as the input file name.</dd>
+<dd>The name of the input log file(s) to search through. To read from standard input, use a dash (`-`) as the input file name.</dd>
<dt>-u, -\\\-unzip </dt>
<dd>Uncompress the input file using `gunzip`. If the input file name ends in `.gz`, it will be uncompressed by default.</dd>
@@ -144,10 +144,10 @@ $ gplogfilter -e '2016-02-13 14:23'
## <a id="topic1__section9"></a>Examples
-Display the last three error messages in the master log file:
+Display the last three error messages in the identified log file:
``` shell
-$ gplogfilter -t -n 3
+$ gplogfilter -t -n 3 "/data/hawq/master/pg_log/hawq-2016-09-01_134934.csv"
```
Display the last five error messages in a date-specified log file:
@@ -156,16 +156,16 @@ Display the last five error messages in a date-specified log file:
$ gplogfilter -t -n 5 "/data/hawq-file-path/hawq-yyyy-mm-dd*.csv"
```
-Display all log messages in the master log file timestamped in the last 10 minutes:
+Display all log messages in the date-specified log file timestamped in the last 10 minutes:
``` shell
-$ gplogfilter -d :10
+$ gplogfilter -d :10 "/data/hawq-file-path/hawq-yyyy-mm-dd*.csv"
```
-Display log messages in the master log file containing the string `|con6 cmd11|`:
+Display log messages in the identified log file containing the string `|con6 cmd11|`:
``` shell
-$ gplogfilter -f '|con6 cmd11|'
+$ gplogfilter -f '|con6 cmd11|' "/data/hawq/master/pg_log/hawq-2016-09-01_134934.csv"
```
Using [hawq ssh](hawqssh.html#topic1), run `gplogfilter` on the segment hosts and search for log messages in the segment log files containing the string `con6` and save output to a file.
[25/50] [abbrv] incubator-hawq-docs git commit: clarify client load
tools windows/rhel discussion
Posted by yo...@apache.org.
clarify client load tools windows/rhel discussion
- install HAWQ to get RHEL tools
- clarify windows loader download location
Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/commit/1d78599e
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/tree/1d78599e
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/diff/1d78599e
Branch: refs/heads/master
Commit: 1d78599eafd1f68dc5245354e855364c2fec403f
Parents: 1661c62
Author: Lisa Owen <lo...@pivotal.io>
Authored: Mon Sep 19 14:43:33 2016 -0700
Committer: Lisa Owen <lo...@pivotal.io>
Committed: Mon Sep 19 14:43:33 2016 -0700
----------------------------------------------------------------------
datamgmt/load/client-loadtools.html.md.erb | 58 +++++++++++---------
.../cli/admin_utilities/gpfdist.html.md.erb | 2 +-
2 files changed, 34 insertions(+), 26 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/1d78599e/datamgmt/load/client-loadtools.html.md.erb
----------------------------------------------------------------------
diff --git a/datamgmt/load/client-loadtools.html.md.erb b/datamgmt/load/client-loadtools.html.md.erb
index 6b8deff..7ad9ee2 100644
--- a/datamgmt/load/client-loadtools.html.md.erb
+++ b/datamgmt/load/client-loadtools.html.md.erb
@@ -6,26 +6,38 @@ HAWQ supports data loading from Red Hat Enterprise Linux 5, 6, and 7 and Windows
This topic presents the instructions to install the HAWQ Load Tools on your client machine. It also includes the information necessary to configure HAWQ databases to accept remote client connections.
## <a id="installloadrunrhel"></a>RHEL Load Tools
-### <a id="installloadrunux"></a>Running the RHEL Installer
-1. Download the `greenplum-loaders-4.3.x.x-build-n-RHEL5.zip` installer package from [Pivotal Network](https://network.pivotal.io/products/pivotal-gpdb). Make note of the directory to which the file was downloaded.
+The RHEL Load Tools are provided in the HAWQ installation package.
+
+
+### <a id="installloadrunux"></a>Installing the RHEL Loader
+
+1. Download the `hdb-2.0.1.0-nnnnn.tar.gz` installer package from [Pivotal Network](https://network.pivotal.io/products/pivotal-hdb). Make note of the directory to which the file was downloaded.
-2. Unzip the run the installer. `sudo` privileges are required if you plan to accept the default install location of `/usr/local/greenplum-loaders-4.3.x.x-build-n`.
+2. Refer to the HAWQ command line install instructions to set up your HDB repository and install the HAWQ binary.
- ``` shell
- $ unzip greenplum-loaders-4.3.x.x-build-n-RHEL5-x86_64.zip
- $ /bin/bash greenplum-loaders-4.3.x.x-build-n-RHEL5-x86_64.bin
- ```
-
- The installer will prompt you to accept the license agreement and to provide an installation path. Enter an absolute path if you choose not to accept the default install location.
+### <a id="installrhelloadabout"></a>About the RHEL Loader Installation
+
+The files/directories of interest in a HAWQ RHEL Load Tools installation include:
+
+`bin/` \u2014 data loading command-line tools (`gpfdist` and `hawq load`)
+`greenplum_path.sh0` \u2014 environment set up file
+
+### <a id="installloadrhelcfgenv"></a>Configuring the RHEL Load Environment
+
+A `greenplum_path.sh` file is located in the HAWQ base install directory following installation. Source `greenplum_path.sh` before running the HAWQ RHEL Load Tools to set up your HAWQ environment:
+
+``` shell
+$ . /usr/local/hawq/greenplum_path.sh
+```
## <a id="installloadrunwin"></a>Windows Load Tools
### <a id="installpythonwin"></a>Installing Python 2.5
-The HAWQ Load Tools for Windows requires that the 32-bit version of Python 2.5 be installed on your machine.
+The HAWQ Load Tools for Windows requires that the 32-bit version of Python 2.5 be installed on your system.
-**Note**: The 64-bit version of Python is not compatible with the HAWQ Load Tools for Windows.
+**Note**: The 64-bit version of Python is **not** compatible with the HAWQ Load Tools for Windows.
1. Download the [Python 2.5 installer for Windows](https://www.python.org/downloads/). Make note of the directory to which it was downloaded.
@@ -38,7 +50,7 @@ The HAWQ Load Tools for Windows requires that the 32-bit version of Python 2.5 b
### <a id="installloadrunwin"></a>Running the Windows Installer
-1. Download the `greenplum-loaders-4.3.x.x-build-n-WinXP-x86_32.msi` installer package from [Pivotal Network](https://network.pivotal.io/products/pivotal-gpdb). Make note of the directory to which it was downloaded.
+1. Download the `greenplum-loaders-4.3.x.x-build-n-WinXP-x86_32.msi` installer package from [Pivotal Network](https://network.pivotal.io/products/pivotal-hdb). Make note of the directory to which it was downloaded.
2. Double-click the `greenplum-loaders-4.3.x.x-build-n-WinXP-x86_32.msi` file to launch the installer.
3. Click **Next** on the **Welcome** screen.
@@ -49,27 +61,23 @@ The HAWQ Load Tools for Windows requires that the 32-bit version of Python 2.5 b
8. Click **Finish** to exit the installer.
-## <a id="installloadabout"></a>About the Loader Installation
-Your HAWQ Load Tools installation includes the following files and directories:
-
-`bin` \u2014 data loading command-line tools (`gpfdist` and `gpload`)
-`ext` \u2014 external dependent components (python)
-`lib` - data loading library files
-`greenplum_loaders_path.[sh | bat]` \u2014 environment set up file
-
+### <a id="installloadabout"></a>About the Windows Loader Installation
+Your HAWQ Windows Load Tools installation includes the following files and directories:
+`bin/` \u2014 data loading command-line tools (`gpfdist` and `gpload`)
+`lib/` \u2014 data loading library files
+`greenplum_loaders_path.bat` \u2014 environment set up file
-## <a id="installloadcfgenv"></a>Configuring the Load Environment
-?? NEED TO VERIFY RHEL/WIN HAVE SAME env varb NAMES ??
+### <a id="installloadcfgenv"></a>Configuring the Windows Load Environment
-A `greenplum_loaders_path.[sh | bat]` file is provided in your load tools base install directory following installation. This file sets the following environment variables:
+A `greenplum_loaders_path.bat` file is provided in your load tools base install directory following installation. This file sets the following environment variables:
- `GPHOME_LOADERS` - base directory of loader installation
- `PATH` - adds the loader and component program directories
-- *`LIBRARY_PATH`* (OS-specific name) - adds the loader and component library directories
+- `PYTHONPATH` - adds component library directories
-Source `greenplum_loaders.sh` or execute `greenplum_loaders.bat` to set up your HAWQ environment before running the HAWQ Load Tools.
+Execute `greenplum_loaders_path.bat` to set up your HAWQ environment before running the HAWQ Windows Load Tools.
## <a id="installloadenableclientconn"></a>Enabling Remote Client Connections
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/1d78599e/reference/cli/admin_utilities/gpfdist.html.md.erb
----------------------------------------------------------------------
diff --git a/reference/cli/admin_utilities/gpfdist.html.md.erb b/reference/cli/admin_utilities/gpfdist.html.md.erb
index b6f7a35..1683ddb 100644
--- a/reference/cli/admin_utilities/gpfdist.html.md.erb
+++ b/reference/cli/admin_utilities/gpfdist.html.md.erb
@@ -31,7 +31,7 @@ For readable external tables, if load files are compressed using `gzip` or `bzip
**Note:** Currently, readable external tables do not support compression on Windows platforms, and writable external tables do not support compression on any platforms.
-Most likely, you will want to run `gpfdist` on your ETL machines rather than the hosts where HAWQ is installed. To install `gpfdist` on another host, simply copy the utility over to that host and add `gpfdist` to your `$PATH`.
+To run `gpfdist` on your ETL machines, refer to [Client-Based HAWQ Load Tools](../../../datamgmt/load/client-loadtools.html) for more information.
**Note:** When using IPv6, always enclose the numeric IP address in brackets.
[47/50] [abbrv] incubator-hawq-docs git commit: fix merge conflict
Posted by yo...@apache.org.
fix merge conflict
Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/commit/97317c4d
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/tree/97317c4d
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/diff/97317c4d
Branch: refs/heads/master
Commit: 97317c4df540a58d07756751f73dfd3ff507c36d
Parents: 6e10024 268ed13
Author: Lisa Owen <lo...@pivotal.io>
Authored: Wed Sep 28 09:39:18 2016 -0700
Committer: Lisa Owen <lo...@pivotal.io>
Committed: Wed Sep 28 09:39:18 2016 -0700
----------------------------------------------------------------------
----------------------------------------------------------------------
[46/50] [abbrv] incubator-hawq-docs git commit: fix merge conflict
Posted by yo...@apache.org.
fix merge conflict
Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/commit/6e100245
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/tree/6e100245
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/diff/6e100245
Branch: refs/heads/master
Commit: 6e100245f8cb1554bf7c2f78101f605381699630
Parents: e8d612e
Author: Lisa Owen <lo...@pivotal.io>
Authored: Wed Sep 28 09:35:34 2016 -0700
Committer: Lisa Owen <lo...@pivotal.io>
Committed: Wed Sep 28 09:35:34 2016 -0700
----------------------------------------------------------------------
install/aws-config.html.md.erb | 1 -
1 file changed, 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/6e100245/install/aws-config.html.md.erb
----------------------------------------------------------------------
diff --git a/install/aws-config.html.md.erb b/install/aws-config.html.md.erb
index 8bbb9b4..e4106b1 100644
--- a/install/aws-config.html.md.erb
+++ b/install/aws-config.html.md.erb
@@ -29,7 +29,6 @@ Instance store-backed storage is generally better performing than EBS and recomm
**Note** EC2 *instance store* provides temporary block-level storage. This storage is located on disks that are physically attached to the host computer. While instance store provides high performance, powering off the instance causes data loss. Soft reboots preserve instance store data.
Virtual devices for instance store volumes for HAWQ EC2 instance store instances are named `ephemeralN` (where *N* varies based on instance type). CentOS instance store block device are named `/dev/xvdletter` (where *letter* is a lower case letter of the alphabet).
-
### <a id="topic_cfgplacegrp"></a>Configure Placement Group
[45/50] [abbrv] incubator-hawq-docs git commit: enhance passwordless
ssh instructions
Posted by yo...@apache.org.
enhance passwordless ssh instructions
- include passwdless ssh instructions in system requirement section
- add appropriate xrefs to these instructions where appropriate
(expanding cluster, ambari mgmt)
- include cloud.cfg config info in AWS doc
Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/commit/433de8e0
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/tree/433de8e0
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/diff/433de8e0
Branch: refs/heads/master
Commit: 433de8e0835dbe52c7bb86e75a94a0aeb1ba337b
Parents: 29a7f42
Author: Lisa Owen <lo...@pivotal.io>
Authored: Mon Sep 12 11:44:34 2016 -0700
Committer: Lisa Owen <lo...@pivotal.io>
Committed: Wed Sep 28 09:34:02 2016 -0700
----------------------------------------------------------------------
admin/ClusterExpansion.html.md.erb | 4 +-
admin/ambari-admin.html.md.erb | 8 +++-
install/aws-config.html.md.erb | 5 ++-
requirements/system-requirements.html.md.erb | 45 +++++++++++++++++++++++
4 files changed, 58 insertions(+), 4 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/433de8e0/admin/ClusterExpansion.html.md.erb
----------------------------------------------------------------------
diff --git a/admin/ClusterExpansion.html.md.erb b/admin/ClusterExpansion.html.md.erb
index 87b811c..f7d3856 100644
--- a/admin/ClusterExpansion.html.md.erb
+++ b/admin/ClusterExpansion.html.md.erb
@@ -20,7 +20,7 @@ There are several recommendations to keep in mind when modifying the size of you
## <a id="task_hawq_expand"></a>Adding a New Node to an Existing HAWQ Cluster
-The following procedure describes the steps required to add a node to an existing HAWQ cluster.
+The following procedure describes the steps required to add a node to an existing HAWQ cluster. First ensure that the new node has been configured per the instructions in [Apache HAWQ System Requirements](/20/requirements/system-requirements.html) and [Select HAWQ Host Machines](/20/install/select-hosts.html).
For example purposes in this procedure, we are adding a new node named `sdw4`.
@@ -71,6 +71,8 @@ For example purposes in this procedure, we are adding a new node named `sdw4`.
$ hawq ssh-exkeys -e hawq_hosts -x new_hosts
```
+ 8. (Optional) Turn off temporary password-based authentication as described in [Apache HAWQ System Requirements](/20/requirements/system-requirements.html#topic_pwdlessssh).
+
8. After setting up passwordless ssh, you can execute the following hawq command to check the target machine's configuration.
```shell
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/433de8e0/admin/ambari-admin.html.md.erb
----------------------------------------------------------------------
diff --git a/admin/ambari-admin.html.md.erb b/admin/ambari-admin.html.md.erb
index ddd986a..5c98dc0 100644
--- a/admin/ambari-admin.html.md.erb
+++ b/admin/ambari-admin.html.md.erb
@@ -159,7 +159,9 @@ There are several recommendations to keep in mind when modifying the size of you
- Note that for hash distributed tables, expanding the cluster will not immediately improve performance since hash distributed tables use a fixed number of virtual segments. In order to obtain better performance with hash distributed tables, you must redistribute the table to the updated cluster by either the [ALTER TABLE](/20/reference/sql/ALTER-TABLE.html) or [CREATE TABLE AS](/20/reference/sql/CREATE-TABLE-AS.html) command.
- If you are using hash tables, consider updating the `default_hash_table_bucket_number` server configuration parameter to a larger value after expanding the cluster but before redistributing the hash tables.
-### Procedure
+### Procedure
+First ensure that the new node(s) has been configured per the instructions in [Apache HAWQ System Requirements](/20/requirements/system-requirements.html) and [Select HAWQ Host Machines](/20/install/select-hosts.html).
+
1. If you have any user-defined function (UDF) libraries installed in your existing HAWQ cluster, install them on the new node(s) that you want to add to the HAWQ cluster.
2. Access the Ambari web console at http://ambari.server.hostname:8080, and login as the "admin" user. \(The default password is also "admin".\)
3. Click **HAWQ** in the list of installed services.
@@ -200,7 +202,9 @@ There are several recommendations to keep in mind when modifying the size of you
21. If you are using hash distributed tables and wish to take advantage of the performance benefits of using a larger cluster, redistribute the data in all hash-distributed tables by using either the [ALTER TABLE](/20/reference/sql/ALTER-TABLE.html) or [CREATE TABLE AS](/20/reference/sql/CREATE-TABLE-AS.html) command. You should redistribute the table data if you modified the `default_hash_table_bucket_number` configuration parameter.
**Note:** The redistribution of table data can take a significant amount of time.
-22. (Optional.) If you changed the **Exchange SSH Keys** property value before adding the host(s), change the value back to `false` after Ambari exchanges keys with the new hosts. This prevents Ambari from exchanging keys with all hosts every time the HAWQ master is started or restarted.
+22. (Optional.) If you changed the **Exchange SSH Keys** property value before adding the host(s), change the value back to `false` after Ambari exchanges keys with the new hosts. This prevents Ambari from exchanging keys with all hosts every time the HAWQ master is started or restarted.
+
+23. (Optional.) Turn off temporary password-based authentication as described in [Apache HAWQ System Requirements](/20/requirements/system-requirements.html#topic_pwdlessssh).
#### <a id="manual-config-steps"></a>Manually Updating the HAWQ Configuration
If you need to expand your HAWQ cluster without restarting the HAWQ service, follow these steps to manually apply the new HAWQ configuration. (Use these steps *instead* of following Step 7 in the above procedure.):
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/433de8e0/install/aws-config.html.md.erb
----------------------------------------------------------------------
diff --git a/install/aws-config.html.md.erb b/install/aws-config.html.md.erb
index 32675fe..8bbb9b4 100644
--- a/install/aws-config.html.md.erb
+++ b/install/aws-config.html.md.erb
@@ -102,8 +102,11 @@ $ ssh -i my-test.pem user1@192.0.2.0
After launching your HAWQ instance, you will connect to and configure the instance. The *Instances* page of the EC2 Console lists the running instances and their associated network access information.
-Before installing HAWQ, set up the EC2 instances as you would local host server machines. Configure the host operating system, configure host network information (for example, update the `/etc/hosts` file), set operating system parameters, and install operating system packages. For information about how to prepare your operating system environment for HAWQ, see [Select HAWQ Host Machines](../install/select-hosts.html).
+Before installing HAWQ, set up the EC2 instances as you would local host server machines. Configure the host operating system, configure host network information (for example, update the `/etc/hosts` file), set operating system parameters, and install operating system packages. For information about how to prepare your operating system environment for HAWQ, see [Apache HAWQ System Requirements](../requirements/system-requirements.html) and [Select HAWQ Host Machines](../install/select-hosts.html).
+###Passwordless SSH Configuration<a id="topic_pwdlessssh_cc"></a>
+
+HAWQ hosts will be configured during the installation process to use passwordless SSH for intra-cluster communications. Temporary password-based authentication must be enabled on each HAWQ host in preparation for this configuration. Password authentication is typically disabled by default in cloud images. Update the cloud configuration in `/etc/cloud/cloud.cfg` to enable password authentication in your AMI(s). Set `ssh_pwauth: True` in this file. If desired, disable password authentication after HAWQ installation by setting the property back to `False`.
##References<a id="topic_hgz_zwy_bv"></a>
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/433de8e0/requirements/system-requirements.html.md.erb
----------------------------------------------------------------------
diff --git a/requirements/system-requirements.html.md.erb b/requirements/system-requirements.html.md.erb
index cf9214d..c61ba3d 100644
--- a/requirements/system-requirements.html.md.erb
+++ b/requirements/system-requirements.html.md.erb
@@ -173,6 +173,51 @@ If this system uses YARN for resource management, you would set `yarn.nodemanage
If this system uses the default HAWQ resource manager, you would set `hawq_rm_memory_limit_perseg` = `RAM - NON_HAWQ_MEMORY` = 8 GB - 7GB = 1.
+## <a id="topic_pwdlessssh"></a>Passwordless SSH Configuration
+
+HAWQ hosts will be configured to use passwordless SSH for intra-cluster communications during the installation process. Temporary password-based authentication must be enabled on each HAWQ host in preparation for this configuration.
+
+1. Install the SSH server if not already configured on the HAWQ system:
+
+ ``` shell
+ $ yum list installed | grep openssh-server
+ $ yum -y install openssh-server
+ ```
+
+2. Update the host's SSH configuration to allow password-based authentication. Edit the SSH config file and change the `PasswordAuthentication` configuration value from `no` to `yes`:
+
+ ``` shell
+ $ sudo vi /etc/ssh/sshd_config
+ ```
+
+ ```
+ PasswordAuthentication yes
+ ```
+
+3. Restart SSH:
+
+ ``` shell
+ $ sudo /etc/init.d/sshd restart
+ ```
+
+*After installation is complete*, you may choose to turn off the temporary password-based authentication configured in the previous steps:
+
+1. Open the SSH `/etc/ssh/sshd_config` file in a text editor and update/uncomment the following configuration options.
+
+ ```
+ RSAAuthentication yes
+ PasswordAuthentication no
+ PubkeyAuthentication yes
+ AuthorizedKeyFile .ssh/authorized_keys
+ ```
+
+2. Restart SSH:
+
+ ``` shell
+ $ sudo /etc/init.d/sshd restart
+ ```
+�
+
## <a id="topic_bsm_hhv_2v"></a>Disk Requirements
- 2GB per host for HAWQ installation.�
[16/50] [abbrv] incubator-hawq-docs git commit: Merge branch
'feature/GUC-gp_vmem_idle_resource_timeout' of
https://github.com/lisakowen/incubator-hawq-docs into develop
Posted by yo...@apache.org.
Merge branch 'feature/GUC-gp_vmem_idle_resource_timeout' of https://github.com/lisakowen/incubator-hawq-docs into develop
Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/commit/1661c625
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/tree/1661c625
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/diff/1661c625
Branch: refs/heads/master
Commit: 1661c625db3a904168755707d8858995ede0ffdb
Parents: 32ccc73 eb9f915
Author: David Yozie <yo...@apache.org>
Authored: Wed Sep 14 09:27:54 2016 -0700
Committer: David Yozie <yo...@apache.org>
Committed: Wed Sep 14 09:27:54 2016 -0700
----------------------------------------------------------------------
reference/guc/guc_category-list.html.md.erb | 1 +
reference/guc/parameter_definitions.html.md.erb | 11 +++++++++++
2 files changed, 12 insertions(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/1661c625/reference/guc/parameter_definitions.html.md.erb
----------------------------------------------------------------------
[05/50] [abbrv] incubator-hawq-docs git commit: MASTER_DATA_DIRECTORY
clarifications - HAWQ-1031
Posted by yo...@apache.org.
MASTER_DATA_DIRECTORY clarifications - HAWQ-1031
Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/commit/1a7efeff
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/tree/1a7efeff
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/diff/1a7efeff
Branch: refs/heads/master
Commit: 1a7efeffc1e1a5e010f7aa4cba4622f9da1357bb
Parents: bec5e3e
Author: Lisa Owen <lo...@pivotal.io>
Authored: Mon Aug 29 13:23:01 2016 -0700
Committer: Lisa Owen <lo...@pivotal.io>
Committed: Mon Aug 29 13:23:01 2016 -0700
----------------------------------------------------------------------
...acesandHighAvailabilityEnabledHDFS.html.md.erb | 18 ++++++++----------
clientaccess/client_auth.html.md.erb | 14 +++++++-------
clientaccess/disable-kerberos.html.md.erb | 11 +++--------
clientaccess/kerberos.html.md.erb | 18 +++++++++---------
reference/HAWQEnvironmentVariables.html.md.erb | 8 --------
5 files changed, 27 insertions(+), 42 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/1a7efeff/admin/HAWQFilespacesandHighAvailabilityEnabledHDFS.html.md.erb
----------------------------------------------------------------------
diff --git a/admin/HAWQFilespacesandHighAvailabilityEnabledHDFS.html.md.erb b/admin/HAWQFilespacesandHighAvailabilityEnabledHDFS.html.md.erb
index 54cec32..3147033 100644
--- a/admin/HAWQFilespacesandHighAvailabilityEnabledHDFS.html.md.erb
+++ b/admin/HAWQFilespacesandHighAvailabilityEnabledHDFS.html.md.erb
@@ -12,7 +12,7 @@ To enable the HDFS NameNode HA feature for use with HAWQ, you need to perform th
1. Collect information about the target filespace.
1. Stop the HAWQ cluster and backup the catalog (**Note:** Ambari users must perform this manual step.)
1. Move the filespace location using the command line tool (**Note:** Ambari users must perform this manual step.)
-1. Reconfigure $\{GPHOME\}/etc/hdfs-client.xml and $\{GPHOME\}/etc/hawq-site.xml files. Then, synchronize updated configuration files to all HAWQ nodes.
+1. Reconfigure `${GPHOME}/etc/hdfs-client.xml` and `${GPHOME}/etc/hawq-site.xml` files. Then, synchronize updated configuration files to all HAWQ nodes.
1. Start the HAWQ cluster and resynchronize the standby master after moving the filespace.
@@ -76,26 +76,24 @@ To move the filespace location to a HA-enabled HDFS location, you must move the
When you enable HA HDFS, you are�changing the HAWQ catalog and persistent tables. You cannot perform transactions while�persistent tables are being updated. Therefore, before you move the filespace location, back up the catalog. This is to ensure that you do not lose data due to a�hardware failure or during an operation \(such as killing the HAWQ process\).�
-1. If you defined a custom port for HAWQ master, export the PGPORT environment variable. For example:
+1. If you defined a custom port for HAWQ master, export the `PGPORT` environment variable. For example:
```shell
export PGPORT=9000
```
-1. If you have not configured it already, export the MASTER\_DATA\_DIRECTORY environment variable.
+1. Save the HAWQ master data directory, found in the `hawq_master_directory` property value from `hawq-site.xml` to an environment variable.
```bash
- export MASTER_DATA_DIRECTORY=/path/to/master/catalog
+ export MDATA_DIR=/path/to/hawq_master_directory
```
- See [Environment Variables](/20/reference/HAWQEnvironmentVariables.html) for more information on environment variables.
-
1. Disconnect all workload connections. Check the active connection with:
```shell
$ psql -p ${PGPORT} -c "select * from pg_catalog.pg_stat_activity" -d template1
```
- where $\{PGPORT\} corresponds to the port number you optionally customized for HAWQ master.
+ where `${PGPORT}` corresponds to the port number you optionally customized for HAWQ master.
2. Issue a checkpoint:�
@@ -113,7 +111,7 @@ When you enable HA HDFS, you are�changing the HAWQ catalog and persistent table
4. Copy the master data directory to a backup location:
```shell
- $ cp -r ${MASTER_DATA_DIRECTORY} /catalog/backup/location
+ $ cp -r ${MDATA_DIR} /catalog/backup/location
```
The master data directory contains the catalog. Fatal errors can occur due to hardware failure or if you fail to kill a HAWQ process before attempting a filespace location change. Make sure you back this directory up.
@@ -123,7 +121,7 @@ When you enable HA HDFS, you are�changing the HAWQ catalog and persistent table
HAWQ provides the command line tool, `hawq filespace`, to move the location of the filespace.
-1. If you defined a custom port for HAWQ master, export the PGPORT environment variable. For example:
+1. If you defined a custom port for HAWQ master, export the `PGPORT` environment variable. For example:
```shell
export PGPORT=9000
@@ -139,7 +137,7 @@ HAWQ provides the command line tool, `hawq filespace`, to move the location of t
Non-fatal error can occur if you provide invalid input or if you have not stopped HAWQ before attempting a filespace location change. Check that you have followed the instructions from the beginning, or correct the input error before you re-run `hawq filespace`.
-Fatal errors can occur due to hardware failure or if you fail to kill a HAWQ process before attempting a filespace location change. When a fatal error occurs, you will see the message, "PLEASE RESTORE MASTER DATA DIRECTORY" in the output. If this occurs, shut down the database and restore the `${MASTER_DATA_DIRECTORY}` that you backed up in Step 4.
+Fatal errors can occur due to hardware failure or if you fail to kill a HAWQ process before attempting a filespace location change. When a fatal error occurs, you will see the message, "PLEASE RESTORE MASTER DATA DIRECTORY" in the output. If this occurs, shut down the database and restore the `${MDATA_DIR}` that you backed up in Step 4.
### <a id="configuregphomeetchdfsclientxml"></a>Step 5: Update HAWQ to Use NameNode HA by Reconfiguring hdfs-client.xml and hawq-site.xml
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/1a7efeff/clientaccess/client_auth.html.md.erb
----------------------------------------------------------------------
diff --git a/clientaccess/client_auth.html.md.erb b/clientaccess/client_auth.html.md.erb
index 1c06d42..9173aed 100644
--- a/clientaccess/client_auth.html.md.erb
+++ b/clientaccess/client_auth.html.md.erb
@@ -6,11 +6,11 @@ When a HAWQ system is first initialized, the system contains one predefined *sup
## <a id="topic2"></a>Allowing Connections to HAWQ
-Client access and authentication is controlled by the standard PostgreSQL host-based authentication file, pg\_hba.conf. In HAWQ, the pg\_hba.conf file of the master instance controls client access and authentication to your HAWQ system. HAWQ segments have pg\_hba.conf files that are configured to allow only client connections from the master host and never accept client connections. Do not alter the pg\_hba.conf file on your segments.
+Client access and authentication is controlled by the standard PostgreSQL host-based authentication file, `pg_hba.conf`. In HAWQ, the `pg_hba.conf` file of the master instance controls client access and authentication to your HAWQ system. HAWQ segments have `pg_hba.conf` files that are configured to allow only client connections from the master host and never accept client connections. Do not alter the `pg_hba.conf` file on your segments.
See [The pg\_hba.conf File](http://www.postgresql.org/docs/9.0/interactive/auth-pg-hba-conf.html) in the PostgreSQL documentation for more information.
-The general format of the pg\_hba.conf file is a set of records, one per line. HAWQ ignores blank lines and any text after the `#` comment character. A record consists of a number of fields that are separated by spaces and/or tabs. Fields can contain white space if the field value is quoted. Records cannot be continued across lines. Each remote client access record has the following format:
+The general format of the `pg_hba.conf` file is a set of records, one per line. HAWQ ignores blank lines and any text after the `#` comment character. A record consists of a number of fields that are separated by spaces and/or tabs. Fields can contain white space if the field value is quoted. Records cannot be continued across lines. Each remote client access record has the following format:
```
*host*���*database*���*role*���*CIDR-address*���*authentication-method*
@@ -38,13 +38,13 @@ The following table describes meaning of each field.
### <a id="topic3"></a>Editing the pg\_hba.conf File
-This example shows how to edit the pg\_hba.conf file of the master to allow remote client access to all databases from all roles using encrypted password authentication.
+This example shows how to edit the `pg_hba.conf` file of the master to allow remote client access to all databases from all roles using encrypted password authentication.
-**Note:** For a more secure system, consider removing all connections that use trust authentication from your master pg\_hba.conf. Trust authentication means the role is granted access without any authentication, therefore bypassing all security. Replace trust entries with ident authentication if your system has an ident service available.
+**Note:** For a more secure system, consider removing all connections that use trust authentication from your master `pg_hba.conf`. Trust authentication means the role is granted access without any authentication, therefore bypassing all security. Replace trust entries with ident authentication if your system has an ident service available.
#### <a id="ip144328"></a>Editing pg\_hba.conf
-1. Open the file $MASTER\_DATA\_DIRECTORY/pg\_hba.conf in a text editor.
+1. Obtain the master data directory from the `hawq_master_directory` property value in `hawq-site.xml` and use a text editor to open the `pg_hba.conf` file in this directory.
2. Add a line to the file for each type of connection you want to allow. Records are read sequentially, so the order of the records is significant. Typically, earlier records will have tight connection match parameters and weaker authentication methods, while later records will have looser match parameters and stronger authentication methods. For example:
```
@@ -68,7 +68,7 @@ This example shows how to edit the pg\_hba.conf file of the master to allow remo
```
3. Save and close the file.
-4. Reload the pg\_hba.conf configuration file for your changes to take effect:
+4. Reload the `pg_hba.conf `configuration file for your changes to take effect:
``` bash
$ hawq stop -u
@@ -104,7 +104,7 @@ The following steps set the parameter values with the HAWQ utility `hawq config`
### <a id="ip142411"></a>To change the number of allowed connections
-1. Log into the HAWQ master host as the HAWQ administrator and source the file `$GPHOME/greenplum_path.sh`.
+1. Log into the HAWQ master host as the HAWQ administrator and source the file `/usr/local/hawq/greenplum_path.sh`.
2. Set the value of the `max_connections` parameter. This `hawq config` command sets the value to 100 on all HAWQ instances.
``` bash
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/1a7efeff/clientaccess/disable-kerberos.html.md.erb
----------------------------------------------------------------------
diff --git a/clientaccess/disable-kerberos.html.md.erb b/clientaccess/disable-kerberos.html.md.erb
index 8986b25..b5d7eeb 100644
--- a/clientaccess/disable-kerberos.html.md.erb
+++ b/clientaccess/disable-kerberos.html.md.erb
@@ -14,15 +14,12 @@ Follow these steps to disable Kerberos security for HAWQ and PXF for manual inst
$ ssh hawq_master_fqdn
```
- 2. Run the following commands to set environment variables:
+ 2. Run the following command to set up HAWQ environment variables:
``` bash
$ source /usr/local/hawq/greenplum_path.sh
- $ export MASTER_DATA_DIRECTORY = /gpsql
```
- **Note:** Substitute the correct value of MASTER\_DATA\_DIRECTORY for your configuration.
-
3. Start HAWQ if necessary:
``` bash
@@ -35,15 +32,13 @@ Follow these steps to disable Kerberos security for HAWQ and PXF for manual inst
$ hawq config --masteronly -c enable_secure_filesystem -v \u201coff\u201d
```
- **Note:** Substitute the correct value of MASTER\_DATA\_DIRECTORY for your configuration.
-
5. Change the permission of the HAWQ HDFS data directory:
``` bash
$ sudo -u hdfs hdfs dfs -chown -R gpadmin:gpadmin /hawq_data
```
- 6. On the HAWQ master node and on all segment server nodes, edit the /usr/local/hawq/etc/hdfs-client.xml file to disable kerberos security. Comment or remove the following properties in each file:
+ 6. On the HAWQ master node and on all segment server nodes, edit the `/usr/local/hawq/etc/hdfs-client.xml` file to disable kerberos security. Comment or remove the following properties in each file:
``` xml
<!--
@@ -66,7 +61,7 @@ Follow these steps to disable Kerberos security for HAWQ and PXF for manual inst
```
3. Disable security for PXF:
- 1. On each PXF node, edit the /etc/gphd/pxf/conf/pxf-site.xml to comment or remove the properties:
+ 1. On each PXF node, edit the `/etc/gphd/pxf/conf/pxf-site.xml` to comment or remove the properties:
``` xml
<!--
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/1a7efeff/clientaccess/kerberos.html.md.erb
----------------------------------------------------------------------
diff --git a/clientaccess/kerberos.html.md.erb b/clientaccess/kerberos.html.md.erb
index 5db0f3b..77d6998 100644
--- a/clientaccess/kerberos.html.md.erb
+++ b/clientaccess/kerberos.html.md.erb
@@ -4,9 +4,9 @@ title: Using Kerberos Authentication
You can control access to HAWQ with a Kerberos authentication server.
-HAWQ supports the Generic Security Service Application Program Interface \(GSSAPI\) with Kerberos authentication. GSSAPI provides automatic authentication \(single sign-on\) for systems that support it. You specify the HAWQ users \(roles\) that require Kerberos authentication in the HAWQ configuration file pg\_hba.conf. The login fails if Kerberos authentication is not available when a role attempts to log in to HAWQ.
+HAWQ supports the Generic Security Service Application Program Interface \(GSSAPI\) with Kerberos authentication. GSSAPI provides automatic authentication \(single sign-on\) for systems that support it. You specify the HAWQ users \(roles\) that require Kerberos authentication in the HAWQ configuration file `pg_hba.conf`. The login fails if Kerberos authentication is not available when a role attempts to log in to HAWQ.
-Kerberos provides a secure, encrypted authentication service. It does not encrypt data exchanged between the client and database and provides no authorization services. To encrypt data exchanged over the network, you must use an SSL connection. To manage authorization for access to HAWQ databases and objects such as schemas and tables, you use settings in the pg\_hba.conf file and privileges given to HAWQ users and roles within the database. For information about managing authorization privileges, see [Managing Roles and Privileges](roles_privs.html).
+Kerberos provides a secure, encrypted authentication service. It does not encrypt data exchanged between the client and database and provides no authorization services. To encrypt data exchanged over the network, you must use an SSL connection. To manage authorization for access to HAWQ databases and objects such as schemas and tables, you use settings in the `pg_hba.conf` file and privileges given to HAWQ users and roles within the database. For information about managing authorization privileges, see [Managing Roles and Privileges](roles_privs.html).
For more information about Kerberos, see [http://web.mit.edu/kerberos/](http://web.mit.edu/kerberos/).
@@ -54,7 +54,7 @@ Follow these steps to install and configure a Kerberos Key Distribution Center \
sudo yum install krb5-libs krb5-server krb5-workstation
```
-2. Edit the /etc/krb5.conf configuration file. The following example shows a Kerberos server with a default `KRB.EXAMPLE.COM` realm.
+2. Edit the `/etc/krb5.conf` configuration file. The following example shows a Kerberos server with a default `KRB.EXAMPLE.COM` realm.
```
[logging]
@@ -104,7 +104,7 @@ Follow these steps to install and configure a Kerberos Key Distribution Center \
kdb5_util create -s
```
- The `kdb5_util`create option creates the database to store keys for the Kerberos realms that are managed by this KDC server. The -s option creates a stash file. Without the stash file, every time the KDC server starts it requests a password.
+ The `kdb5_util`create option creates the database to store keys for the Kerberos realms that are managed by this KDC server. The `-s` option creates a stash file. Without the stash file, every time the KDC server starts it requests a password.
4. Add an administrative user to the KDC database with the `kadmin.local` utility. Because it does not itself depend on Kerberos authentication, the `kadmin.local` utility allows you to add an initial administrative user to the local Kerberos server. To add the user `gpadmin` as an administrative user to the KDC database, run the following command:
@@ -114,7 +114,7 @@ Follow these steps to install and configure a Kerberos Key Distribution Center \
Most users do not need administrative access to the Kerberos server. They can use `kadmin` to manage their own principals \(for example, to change their own password\). For information about `kadmin`, see the [Kerberos documentation](http://web.mit.edu/kerberos/krb5-latest/doc/).
-5. If needed, edit the /var/kerberos/krb5kdc/kadm5.acl file to grant the appropriate permissions to `gpadmin`.
+5. If needed, edit the `/var/kerberos/krb5kdc/kadm5.acl` file to grant the appropriate permissions to `gpadmin`.
6. Start the Kerberos daemons:
```
@@ -183,7 +183,7 @@ Install the Kerberos client libraries on the HAWQ master and configure the Kerbe
sudo kdestroy
```
-5. Use the Kerberos utility `kinit` to request a ticket using the keytab file on the HAWQ master for `gpadmin/kerberos-gpdb@KRB.EXAMPLE.COM`. The -t option specifies the keytab file on the HAWQ master.
+5. Use the Kerberos utility `kinit` to request a ticket using the keytab file on the HAWQ master for `gpadmin/kerberos-gpdb@KRB.EXAMPLE.COM`. The `-t` option specifies the keytab file on the HAWQ master.
```
# kinit -k -t gpdb-kerberos.keytab gpadmin/kerberos-gpdb@KRB.EXAMPLE.COM
@@ -246,13 +246,13 @@ After you have set up Kerberos on the HAWQ master, you can configure HAWQ to use
$ psql -U "adminuser/mdw.proddb" -h mdw.proddb
```
- If the default user is `adminuser`, the pg\_ident.conf file and the pg\_hba.conf file can be configured so that the `adminuser` can log in to the database as the Kerberos principal `adminuser/mdw.proddb` without specifying the `-U` option:
+ If the default user is `adminuser`, the `pg_ident.conf` file and the `pg_hba.conf` file can be configured so that the `adminuser` can log in to the database as the Kerberos principal `adminuser/mdw.proddb` without specifying the `-U` option:
``` bash
$ psql -h mdw.proddb
```
- The following username map is defined in the HAWQ file `$MASTER_DATA_DIRECTORY/pg_ident.conf`:
+ The `pg_ident.conf` file defines the username map. This file is located in the HAWQ master data directory (identified by the `hawq_master_directory` property value in `hawq-site.xml`):
```
# MAPNAME ��SYSTEM-USERNAME �������GP-USERNAME
@@ -283,7 +283,7 @@ Enable Kerberos-authenticated JDBC access to HAWQ.
You can configure HAWQ to use Kerberos to run user-defined Java functions.
1. Ensure that Kerberos is installed and configured on the HAWQ master. See [Install and Configure the Kerberos Client](#topic6).
-2. Create the file .java.login.config in the folder /home/gpadmin and add the following text to the file:
+2. Create the file `.java.login.config` in the folder `/home/gpadmin` and add the following text to the file:
```
pgjdbc {
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/1a7efeff/reference/HAWQEnvironmentVariables.html.md.erb
----------------------------------------------------------------------
diff --git a/reference/HAWQEnvironmentVariables.html.md.erb b/reference/HAWQEnvironmentVariables.html.md.erb
index beb28d3..8061781 100644
--- a/reference/HAWQEnvironmentVariables.html.md.erb
+++ b/reference/HAWQEnvironmentVariables.html.md.erb
@@ -41,14 +41,6 @@ export LD_LIBRARY_PATH
The following are HAWQ environment variables. You may want to add the connection-related environment variables to your profile, for convenience. That way, you do not have to type so many options on the command line for client connections. Note that these environment variables should be set on the HAWQ master host only.
-### <a id="master_data_directory"></a>MASTER\_DATA\_DIRECTORY
-
-This variable is only needed for legacy compatibility. The master data directory is now set in hawq-site.xml, by using the `hawq config` command. If used, this variable should point to the directory created by the `hawq init` utility in the master data directory location. For example:
-
-``` pre
-MASTER_DATA_DIRECTORY=/data/master
-export MASTER_DATA_DIRECTORY
-```
### <a id="pgappname"></a>PGAPPNAME
[33/50] [abbrv] incubator-hawq-docs git commit: more updates -
cleaning up update, delete, vacuum references
Posted by yo...@apache.org.
more updates - cleaning up update, delete, vacuum references
Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/commit/94eac9f9
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/tree/94eac9f9
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/diff/94eac9f9
Branch: refs/heads/master
Commit: 94eac9f94147850c092fa04fcf4ca828d7266b28
Parents: ffda5ae
Author: Lisa Owen <lo...@pivotal.io>
Authored: Thu Sep 22 15:36:43 2016 -0700
Committer: Lisa Owen <lo...@pivotal.io>
Committed: Thu Sep 22 15:36:43 2016 -0700
----------------------------------------------------------------------
datamgmt/BasicDataOperations.html.md.erb | 2 +-
reference/sql/CREATE-EXTERNAL-TABLE.html.md.erb | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/94eac9f9/datamgmt/BasicDataOperations.html.md.erb
----------------------------------------------------------------------
diff --git a/datamgmt/BasicDataOperations.html.md.erb b/datamgmt/BasicDataOperations.html.md.erb
index e50060c..6c3323c 100644
--- a/datamgmt/BasicDataOperations.html.md.erb
+++ b/datamgmt/BasicDataOperations.html.md.erb
@@ -49,7 +49,7 @@ Periodically running the `VACUUM` command on system catalog tables removes these
For example:
``` sql
-VACUUM mytable;
+VACUUM pg_class;
```
### <a id="topic10"></a>Configuring the Free Space Map
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/94eac9f9/reference/sql/CREATE-EXTERNAL-TABLE.html.md.erb
----------------------------------------------------------------------
diff --git a/reference/sql/CREATE-EXTERNAL-TABLE.html.md.erb b/reference/sql/CREATE-EXTERNAL-TABLE.html.md.erb
index 4eec64b..2f19eab 100644
--- a/reference/sql/CREATE-EXTERNAL-TABLE.html.md.erb
+++ b/reference/sql/CREATE-EXTERNAL-TABLE.html.md.erb
@@ -109,7 +109,7 @@ where \<pxf parameters\> is:
## <a id="topic1__section3"></a>Description
-`CREATE EXTERNAL TABLE` or `CREATE EXTERNAL WEB TABLE` creates a new readable external table definition in HAWQ. Readable external tables are typically used for fast, parallel data loading. Once an external table is defined, you can query its data directly (and in parallel) using SQL commands. For example, you can select, join, or sort external table data. You can also create views for external tables. DML operations (`INSERT` or `TRUNCATE`) are not allowed on readable external tables.
+`CREATE EXTERNAL TABLE` or `CREATE EXTERNAL WEB TABLE` creates a new readable external table definition in HAWQ. Readable external tables are typically used for fast, parallel data loading. Once an external table is defined, you can query its data directly (and in parallel) using SQL commands. For example, you can select, join, or sort external table data. You can also create views for external tables. DML operations (`UPDATE`, `INSERT`, `DELETE`, or `TRUNCATE`) are not permitted on readable external tables.
`CREATE WRITABLE EXTERNAL TABLE` or `CREATE WRITABLE EXTERNAL WEB TABLE` creates a new writable external table definition in HAWQ. Writable external tables are typically used for unloading data from the database into a set of files or named pipes.
[14/50] [abbrv] incubator-hawq-docs git commit: update
gp_vmem_idle_resource_timeout guc set classifications
Posted by yo...@apache.org.
update gp_vmem_idle_resource_timeout guc set classifications
Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/commit/eb9f915a
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/tree/eb9f915a
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/diff/eb9f915a
Branch: refs/heads/master
Commit: eb9f915a79e10ec5f7c81dee411fe53910c900db
Parents: 1212af9
Author: Lisa Owen <lo...@pivotal.io>
Authored: Wed Sep 14 08:32:16 2016 -0700
Committer: Lisa Owen <lo...@pivotal.io>
Committed: Wed Sep 14 08:32:16 2016 -0700
----------------------------------------------------------------------
reference/guc/parameter_definitions.html.md.erb | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/eb9f915a/reference/guc/parameter_definitions.html.md.erb
----------------------------------------------------------------------
diff --git a/reference/guc/parameter_definitions.html.md.erb b/reference/guc/parameter_definitions.html.md.erb
index af87b75..ac05fa2 100644
--- a/reference/guc/parameter_definitions.html.md.erb
+++ b/reference/guc/parameter_definitions.html.md.erb
@@ -1667,7 +1667,7 @@ Sets the time in milliseconds a session can be idle before gangs on the segment
| Value Range | Default | Set Classifications |
|----------------|---------|-------------------------|
-| number of milliseconds | 18000 | master, session, reload |
+| number of milliseconds | 18000 | master, system, restart |
## <a name="gp_vmem_protect_segworker_cache_limit"></a>gp\_vmem\_protect\_segworker\_cache\_limit
[31/50] [abbrv] incubator-hawq-docs git commit: Merge branch
'feature/troubleshoot-pxf' of
https://github.com/lisakowen/incubator-hawq-docs into develop
Posted by yo...@apache.org.
Merge branch 'feature/troubleshoot-pxf' of https://github.com/lisakowen/incubator-hawq-docs into develop
Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/commit/37857eae
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/tree/37857eae
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/diff/37857eae
Branch: refs/heads/master
Commit: 37857eaedce5b9bd78cbb84569a4faac4980c16a
Parents: ecb0097 a15e186
Author: David Yozie <yo...@apache.org>
Authored: Tue Sep 20 15:48:10 2016 -0700
Committer: David Yozie <yo...@apache.org>
Committed: Tue Sep 20 15:48:10 2016 -0700
----------------------------------------------------------------------
pxf/TroubleshootingPXF.html.md.erb | 71 ++++++++++++++++++++++++++++++++-
1 file changed, 70 insertions(+), 1 deletion(-)
----------------------------------------------------------------------