You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hawq.apache.org by yo...@apache.org on 2016/09/28 20:32:39 UTC
[1/9] incubator-hawq-docs git commit: enhance passwordless ssh
instructions
Repository: incubator-hawq-docs
Updated Branches:
refs/heads/develop 29a7f42b2 -> 97317c4df
enhance passwordless ssh instructions
- include passwdless ssh instructions in system requirement section
- add appropriate xrefs to these instructions where appropriate
(expanding cluster, ambari mgmt)
- include cloud.cfg config info in AWS doc
Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/commit/87930225
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/tree/87930225
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/diff/87930225
Branch: refs/heads/develop
Commit: 8793022527080ca80198ba43e2f75550c3f157f1
Parents: fa5ed53
Author: Lisa Owen <lo...@pivotal.io>
Authored: Mon Sep 12 11:44:34 2016 -0700
Committer: Lisa Owen <lo...@pivotal.io>
Committed: Mon Sep 12 11:44:34 2016 -0700
----------------------------------------------------------------------
admin/ClusterExpansion.html.md.erb | 4 +-
admin/ambari-admin.html.md.erb | 8 +++-
install/aws-config.html.md.erb | 5 ++-
requirements/system-requirements.html.md.erb | 45 +++++++++++++++++++++++
4 files changed, 57 insertions(+), 5 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/87930225/admin/ClusterExpansion.html.md.erb
----------------------------------------------------------------------
diff --git a/admin/ClusterExpansion.html.md.erb b/admin/ClusterExpansion.html.md.erb
index dcd96df..e72ad14 100644
--- a/admin/ClusterExpansion.html.md.erb
+++ b/admin/ClusterExpansion.html.md.erb
@@ -20,7 +20,7 @@ There are several recommendations to keep in mind when modifying the size of you
## <a id="task_hawq_expand"></a>Adding a New Node to an Existing HAWQ Cluster
-The following procedure describes the steps required to add a node to an existing HAWQ cluster.
+The following procedure describes the steps required to add a node to an existing HAWQ cluster. First ensure that the new node has been configured per the instructions in [Apache HAWQ System Requirements](/20/requirements/system-requirements.html) and [Select HAWQ Host Machines](/20/install/select-hosts.html).
For example purposes in this procedure, we are adding a new node named `sdw4`.
@@ -71,6 +71,8 @@ For example purposes in this procedure, we are adding a new node named `sdw4`.
$ hawq ssh-exkeys -e hawq_hosts -x new_hosts
```
+ 8. (Optional) Turn off temporary password-based authentication as described in [Apache HAWQ System Requirements](/20/requirements/system-requirements.html#topic_pwdlessssh).
+
8. After setting up passwordless ssh, you can execute the following hawq command to check the target machine's configuration.
```shell
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/87930225/admin/ambari-admin.html.md.erb
----------------------------------------------------------------------
diff --git a/admin/ambari-admin.html.md.erb b/admin/ambari-admin.html.md.erb
index 16513d4..54ac91c 100644
--- a/admin/ambari-admin.html.md.erb
+++ b/admin/ambari-admin.html.md.erb
@@ -159,7 +159,9 @@ There are several recommendations to keep in mind when modifying the size of you
- Note that for hash distributed tables, expanding the cluster will not immediately improve performance since hash distributed tables use a fixed number of virtual segments. In order to obtain better performance with hash distributed tables, you must redistribute the table to the updated cluster by either the [ALTER TABLE](/20/reference/sql/ALTER-TABLE.html) or [CREATE TABLE AS](/20/reference/sql/CREATE-TABLE-AS.html) command.
- If you are using hash tables, consider updating the `default_hash_table_bucket_number` server configuration parameter to a larger value after expanding the cluster but before redistributing the hash tables.
-### Procedure
+### Procedure
+First ensure that the new node(s) has been configured per the instructions in [Apache HAWQ System Requirements](/20/requirements/system-requirements.html) and [Select HAWQ Host Machines](/20/install/select-hosts.html).
+
1. If you have any user-defined function (UDF) libraries installed in your existing HAWQ cluster, install them on the new node(s) that you want to add to the HAWQ cluster.
2. Access the Ambari web console at http://ambari.server.hostname:8080, and login as the "admin" user. \(The default password is also "admin".\)
3. Click **HAWQ** in the list of installed services.
@@ -200,7 +202,9 @@ There are several recommendations to keep in mind when modifying the size of you
21. If you are using hash distributed tables and wish to take advantage of the performance benefits of using a larger cluster, redistribute the data in all hash-distributed tables by using either the [ALTER TABLE](/20/reference/sql/ALTER-TABLE.html) or [CREATE TABLE AS](/20/reference/sql/CREATE-TABLE-AS.html) command. You should redistribute the table data if you modified the `default_hash_table_bucket_number` configuration parameter.
**Note:** The redistribution of table data can take a significant amount of time.
-22. (Optional.) If you changed the **Exchange SSH Keys** property value before adding the host(s), change the value back to `false` after Ambari exchanges keys with the new hosts. This prevents Ambari from exchanging keys with all hosts every time the HAWQ master is started or restarted.
+22. (Optional.) If you changed the **Exchange SSH Keys** property value before adding the host(s), change the value back to `false` after Ambari exchanges keys with the new hosts. This prevents Ambari from exchanging keys with all hosts every time the HAWQ master is started or restarted.
+
+23. (Optional.) Turn off temporary password-based authentication as described in [Apache HAWQ System Requirements](/20/requirements/system-requirements.html#topic_pwdlessssh).
#### <a id="manual-config-steps"></a>Manually Updating the HAWQ Configuration
If you need to expand your HAWQ cluster without restarting the HAWQ service, follow these steps to manually apply the new HAWQ configuration. (Use these steps *instead* of following Step 7 in the above procedure.):
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/87930225/install/aws-config.html.md.erb
----------------------------------------------------------------------
diff --git a/install/aws-config.html.md.erb b/install/aws-config.html.md.erb
index 395970b..98aa35a 100644
--- a/install/aws-config.html.md.erb
+++ b/install/aws-config.html.md.erb
@@ -114,10 +114,11 @@ $ ssh -i my-test.pem user1@192.0.2.0
After launching your HAWQ instance, you will connect to and configure the instance. The *Instances* page of the EC2 Console lists the running instances and their associated network access information.
-Before installing HAWQ, set up the EC2 instances as you would local host server machines. Configure the host operating system, configure host network information (for example, update the `/etc/hosts` file), set operating system parameters, and install operating system packages. For information about how to prepare your operating system environment for HAWQ, see [Select HAWQ Host Machines](../install/select-hosts.html).
+Before installing HAWQ, set up the EC2 instances as you would local host server machines. Configure the host operating system, configure host network information (for example, update the `/etc/hosts` file), set operating system parameters, and install operating system packages. For information about how to prepare your operating system environment for HAWQ, see [Apache HAWQ System Requirements](../requirements/system-requirements.html) and [Select HAWQ Host Machines](../install/select-hosts.html).
-?? PACKAGES, KERNEL VERSION, KERNEL SETTINGS, BEST PRACTICES HERE OR WILL THEY BE THE SAME AS FOR PHYSICAL DEVICE??
+###Passwordless SSH Configuration<a id="topic_pwdlessssh_cc"></a>
+HAWQ hosts will be configured during the installation process to use passwordless SSH for intra-cluster communications. Temporary password-based authentication must be enabled on each HAWQ host in preparation for this configuration. Password authentication is typically disabled by default in cloud images. Update the cloud configuration in `/etc/cloud/cloud.cfg` to enable password authentication in your AMI(s). Set `ssh_pwauth: True` in this file.
##References<a id="topic_hgz_zwy_bv"></a>
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/87930225/requirements/system-requirements.html.md.erb
----------------------------------------------------------------------
diff --git a/requirements/system-requirements.html.md.erb b/requirements/system-requirements.html.md.erb
index bc08ef0..3f47092 100644
--- a/requirements/system-requirements.html.md.erb
+++ b/requirements/system-requirements.html.md.erb
@@ -173,6 +173,51 @@ If this system uses YARN for resource management, you would set `yarn.nodemanage
If this system uses the default HAWQ resource manager, you would set `hawq_rm_memory_limit_perseg` = `RAM - NON_HAWQ_MEMORY` = 8 GB - 7GB = 1.
+## <a id="topic_pwdlessssh"></a>Passwordless SSH Configuration
+
+HAWQ hosts will be configured to use passwordless SSH for intra-cluster communications during the installation process. Temporary password-based authentication must be enabled on each HAWQ host in preparation for this configuration.
+
+1. Install the SSH server if not already configured on the HAWQ system:
+
+ ``` shell
+ $ yum list installed | grep openssh-server
+ $ yum -y install openssh-server
+ ```
+
+2. Update the host's SSH configuration to allow password-based authentication. Edit the SSH config file and change the `PasswordAuthentication` configuration value from `no` to `yes`:
+
+ ``` shell
+ $ sudo vi /etc/ssh/sshd_config
+ ```
+
+ ```
+ PasswordAuthentication yes
+ ```
+
+3. Restart SSH:
+
+ ``` shell
+ $ sudo /etc/init.d/sshd restart
+ ```
+
+*After installation is complete*, you may choose to turn off the temporary password-based authentication configured in the previous steps:
+
+1. Open the SSH `/etc/ssh/sshd_config` file in a text editor and update/uncomment the following configuration options.
+
+ ```
+ RSAAuthentication yes
+ PasswordAuthentication no
+ PubkeyAuthentication yes
+ AuthorizedKeyFile .ssh/authorized_keys
+ ```
+
+2. Restart SSH:
+
+ ``` shell
+ $ sudo /etc/init.d/sshd restart
+ ```
+�
+
## <a id="topic_bsm_hhv_2v"></a>Disk Requirements
- 2GB per host for HAWQ installation.�
[2/9] incubator-hawq-docs git commit: clarify wording for disabling
passwd-based auth
Posted by yo...@apache.org.
clarify wording for disabling passwd-based auth
Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/commit/831a83e7
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/tree/831a83e7
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/diff/831a83e7
Branch: refs/heads/develop
Commit: 831a83e7ca4364d73a901b4288c5fcbc5e2e949f
Parents: 8793022
Author: Lisa Owen <lo...@pivotal.io>
Authored: Mon Sep 12 12:07:28 2016 -0700
Committer: Lisa Owen <lo...@pivotal.io>
Committed: Mon Sep 12 12:07:28 2016 -0700
----------------------------------------------------------------------
admin/ClusterExpansion.html.md.erb | 4 ++--
admin/ambari-admin.html.md.erb | 4 ++--
2 files changed, 4 insertions(+), 4 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/831a83e7/admin/ClusterExpansion.html.md.erb
----------------------------------------------------------------------
diff --git a/admin/ClusterExpansion.html.md.erb b/admin/ClusterExpansion.html.md.erb
index e72ad14..e335bd5 100644
--- a/admin/ClusterExpansion.html.md.erb
+++ b/admin/ClusterExpansion.html.md.erb
@@ -20,7 +20,7 @@ There are several recommendations to keep in mind when modifying the size of you
## <a id="task_hawq_expand"></a>Adding a New Node to an Existing HAWQ Cluster
-The following procedure describes the steps required to add a node to an existing HAWQ cluster. First ensure that the new node has been configured per the instructions in [Apache HAWQ System Requirements](/20/requirements/system-requirements.html) and [Select HAWQ Host Machines](/20/install/select-hosts.html).
+The following procedure describes the steps required to add a node to an existing HAWQ cluster. First ensure that the new node has been configured per the instructions found in [Apache HAWQ System Requirements](/20/requirements/system-requirements.html) and [Select HAWQ Host Machines](/20/install/select-hosts.html).
For example purposes in this procedure, we are adding a new node named `sdw4`.
@@ -71,7 +71,7 @@ For example purposes in this procedure, we are adding a new node named `sdw4`.
$ hawq ssh-exkeys -e hawq_hosts -x new_hosts
```
- 8. (Optional) Turn off temporary password-based authentication as described in [Apache HAWQ System Requirements](/20/requirements/system-requirements.html#topic_pwdlessssh).
+ 8. (Optional) If you enabled temporary password-based authentication while preparing/configuring your new HAWQ host system, turn off password-based authentication as described in [Apache HAWQ System Requirements](/20/requirements/system-requirements.html#topic_pwdlessssh).
8. After setting up passwordless ssh, you can execute the following hawq command to check the target machine's configuration.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/831a83e7/admin/ambari-admin.html.md.erb
----------------------------------------------------------------------
diff --git a/admin/ambari-admin.html.md.erb b/admin/ambari-admin.html.md.erb
index 54ac91c..96c292f 100644
--- a/admin/ambari-admin.html.md.erb
+++ b/admin/ambari-admin.html.md.erb
@@ -160,7 +160,7 @@ There are several recommendations to keep in mind when modifying the size of you
- If you are using hash tables, consider updating the `default_hash_table_bucket_number` server configuration parameter to a larger value after expanding the cluster but before redistributing the hash tables.
### Procedure
-First ensure that the new node(s) has been configured per the instructions in [Apache HAWQ System Requirements](/20/requirements/system-requirements.html) and [Select HAWQ Host Machines](/20/install/select-hosts.html).
+First ensure that the new node(s) has been configured per the instructions found in [Apache HAWQ System Requirements](/20/requirements/system-requirements.html) and [Select HAWQ Host Machines](/20/install/select-hosts.html).
1. If you have any user-defined function (UDF) libraries installed in your existing HAWQ cluster, install them on the new node(s) that you want to add to the HAWQ cluster.
2. Access the Ambari web console at http://ambari.server.hostname:8080, and login as the "admin" user. \(The default password is also "admin".\)
@@ -204,7 +204,7 @@ First ensure that the new node(s) has been configured per the instructions in [A
**Note:** The redistribution of table data can take a significant amount of time.
22. (Optional.) If you changed the **Exchange SSH Keys** property value before adding the host(s), change the value back to `false` after Ambari exchanges keys with the new hosts. This prevents Ambari from exchanging keys with all hosts every time the HAWQ master is started or restarted.
-23. (Optional.) Turn off temporary password-based authentication as described in [Apache HAWQ System Requirements](/20/requirements/system-requirements.html#topic_pwdlessssh).
+23. (Optional.) If you enabled temporary password-based authentication while preparing/configuring your HAWQ host systems, turn off password-based authentication as described in [Apache HAWQ System Requirements](/20/requirements/system-requirements.html#topic_pwdlessssh).
#### <a id="manual-config-steps"></a>Manually Updating the HAWQ Configuration
If you need to expand your HAWQ cluster without restarting the HAWQ service, follow these steps to manually apply the new HAWQ configuration. (Use these steps *instead* of following Step 7 in the above procedure.):
[4/9] incubator-hawq-docs git commit: instruct user to turn off pwd
based auth after install if desired
Posted by yo...@apache.org.
instruct user to turn off pwd based auth after install if desired
Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/commit/268ed136
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/tree/268ed136
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/diff/268ed136
Branch: refs/heads/develop
Commit: 268ed1368a81a2dc6c5756ba2008e35bd7328207
Parents: dd25c6f
Author: Lisa Owen <lo...@pivotal.io>
Authored: Tue Sep 27 13:52:59 2016 -0700
Committer: Lisa Owen <lo...@pivotal.io>
Committed: Wed Sep 28 09:10:24 2016 -0700
----------------------------------------------------------------------
install/aws-config.html.md.erb | 17 ++---------------
1 file changed, 2 insertions(+), 15 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/268ed136/install/aws-config.html.md.erb
----------------------------------------------------------------------
diff --git a/install/aws-config.html.md.erb b/install/aws-config.html.md.erb
index 98aa35a..881f742 100644
--- a/install/aws-config.html.md.erb
+++ b/install/aws-config.html.md.erb
@@ -30,9 +30,6 @@ Instance store-backed storage is generally better performing than EBS and recomm
Virtual devices for instance store volumes for HAWQ EC2 instance store instances are named `ephemeralN` (where *N* varies based on instance type). CentOS instance store block device are named `/dev/xvdletter` (where *letter* is a lower case letter of the alphabet).
-?? DO WE RECOMMEND EBS-backed VOLUMES FOR ANYTHING? I.E. BACKUPS, ...??)
-
-
### <a id="topic_cfgplacegrp"></a>Configure Placement Group
A placement group is a logical grouping of instances within a single availability zone that together participate in a low-latency, 10 Gbps network. Your HAWQ master and segment cluster instances should support enhanced networking and reside in a single placement group (and subnet) for optimal network performance.
@@ -77,22 +74,12 @@ A security group is a set of rules that control network traffic to and from your
To configure HAWQ communication between nodes in the HAWQ cluster, include and open the following ports in the appropriate security group for the HAWQ master and segment nodes:
-?? PROVIDE SPECIFIC type/protocol/portrange/source settings from EC2 console ??
-
| Port | Application |
|-------|-------------------------------------|
-| 22 | ssh - secure connect to other hosts |
-| XX | XX |
-
-UDP access within the VPC should be enabled by default.
-
-
-?? DIFFENT SECURITY GROUP FOR AMBARI NODE ??
+| 22 | ssh - secure connect to other hosts |
To allow access to/from a source external to the Ambari management node, include and open the following ports in an appropriate security group for your Ambari node:
-?? PROVIDE SPECIFIC type/protocol/portrange/source settings from EC2 console ??
-
| Port | Application |
|-------|-------------------------------------|
| 22 | ssh - secure connect to other hosts |
@@ -118,7 +105,7 @@ Before installing HAWQ, set up the EC2 instances as you would local host server
###Passwordless SSH Configuration<a id="topic_pwdlessssh_cc"></a>
-HAWQ hosts will be configured during the installation process to use passwordless SSH for intra-cluster communications. Temporary password-based authentication must be enabled on each HAWQ host in preparation for this configuration. Password authentication is typically disabled by default in cloud images. Update the cloud configuration in `/etc/cloud/cloud.cfg` to enable password authentication in your AMI(s). Set `ssh_pwauth: True` in this file.
+HAWQ hosts will be configured during the installation process to use passwordless SSH for intra-cluster communications. Temporary password-based authentication must be enabled on each HAWQ host in preparation for this configuration. Password authentication is typically disabled by default in cloud images. Update the cloud configuration in `/etc/cloud/cloud.cfg` to enable password authentication in your AMI(s). Set `ssh_pwauth: True` in this file. If desired, disable password authentication after HAWQ installation by setting the property back to `False`.
##References<a id="topic_hgz_zwy_bv"></a>
[6/9] incubator-hawq-docs git commit: enhance passwordless ssh
instructions
Posted by yo...@apache.org.
enhance passwordless ssh instructions
- include passwdless ssh instructions in system requirement section
- add appropriate xrefs to these instructions where appropriate
(expanding cluster, ambari mgmt)
- include cloud.cfg config info in AWS doc
Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/commit/433de8e0
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/tree/433de8e0
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/diff/433de8e0
Branch: refs/heads/develop
Commit: 433de8e0835dbe52c7bb86e75a94a0aeb1ba337b
Parents: 29a7f42
Author: Lisa Owen <lo...@pivotal.io>
Authored: Mon Sep 12 11:44:34 2016 -0700
Committer: Lisa Owen <lo...@pivotal.io>
Committed: Wed Sep 28 09:34:02 2016 -0700
----------------------------------------------------------------------
admin/ClusterExpansion.html.md.erb | 4 +-
admin/ambari-admin.html.md.erb | 8 +++-
install/aws-config.html.md.erb | 5 ++-
requirements/system-requirements.html.md.erb | 45 +++++++++++++++++++++++
4 files changed, 58 insertions(+), 4 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/433de8e0/admin/ClusterExpansion.html.md.erb
----------------------------------------------------------------------
diff --git a/admin/ClusterExpansion.html.md.erb b/admin/ClusterExpansion.html.md.erb
index 87b811c..f7d3856 100644
--- a/admin/ClusterExpansion.html.md.erb
+++ b/admin/ClusterExpansion.html.md.erb
@@ -20,7 +20,7 @@ There are several recommendations to keep in mind when modifying the size of you
## <a id="task_hawq_expand"></a>Adding a New Node to an Existing HAWQ Cluster
-The following procedure describes the steps required to add a node to an existing HAWQ cluster.
+The following procedure describes the steps required to add a node to an existing HAWQ cluster. First ensure that the new node has been configured per the instructions in [Apache HAWQ System Requirements](/20/requirements/system-requirements.html) and [Select HAWQ Host Machines](/20/install/select-hosts.html).
For example purposes in this procedure, we are adding a new node named `sdw4`.
@@ -71,6 +71,8 @@ For example purposes in this procedure, we are adding a new node named `sdw4`.
$ hawq ssh-exkeys -e hawq_hosts -x new_hosts
```
+ 8. (Optional) Turn off temporary password-based authentication as described in [Apache HAWQ System Requirements](/20/requirements/system-requirements.html#topic_pwdlessssh).
+
8. After setting up passwordless ssh, you can execute the following hawq command to check the target machine's configuration.
```shell
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/433de8e0/admin/ambari-admin.html.md.erb
----------------------------------------------------------------------
diff --git a/admin/ambari-admin.html.md.erb b/admin/ambari-admin.html.md.erb
index ddd986a..5c98dc0 100644
--- a/admin/ambari-admin.html.md.erb
+++ b/admin/ambari-admin.html.md.erb
@@ -159,7 +159,9 @@ There are several recommendations to keep in mind when modifying the size of you
- Note that for hash distributed tables, expanding the cluster will not immediately improve performance since hash distributed tables use a fixed number of virtual segments. In order to obtain better performance with hash distributed tables, you must redistribute the table to the updated cluster by either the [ALTER TABLE](/20/reference/sql/ALTER-TABLE.html) or [CREATE TABLE AS](/20/reference/sql/CREATE-TABLE-AS.html) command.
- If you are using hash tables, consider updating the `default_hash_table_bucket_number` server configuration parameter to a larger value after expanding the cluster but before redistributing the hash tables.
-### Procedure
+### Procedure
+First ensure that the new node(s) has been configured per the instructions in [Apache HAWQ System Requirements](/20/requirements/system-requirements.html) and [Select HAWQ Host Machines](/20/install/select-hosts.html).
+
1. If you have any user-defined function (UDF) libraries installed in your existing HAWQ cluster, install them on the new node(s) that you want to add to the HAWQ cluster.
2. Access the Ambari web console at http://ambari.server.hostname:8080, and login as the "admin" user. \(The default password is also "admin".\)
3. Click **HAWQ** in the list of installed services.
@@ -200,7 +202,9 @@ There are several recommendations to keep in mind when modifying the size of you
21. If you are using hash distributed tables and wish to take advantage of the performance benefits of using a larger cluster, redistribute the data in all hash-distributed tables by using either the [ALTER TABLE](/20/reference/sql/ALTER-TABLE.html) or [CREATE TABLE AS](/20/reference/sql/CREATE-TABLE-AS.html) command. You should redistribute the table data if you modified the `default_hash_table_bucket_number` configuration parameter.
**Note:** The redistribution of table data can take a significant amount of time.
-22. (Optional.) If you changed the **Exchange SSH Keys** property value before adding the host(s), change the value back to `false` after Ambari exchanges keys with the new hosts. This prevents Ambari from exchanging keys with all hosts every time the HAWQ master is started or restarted.
+22. (Optional.) If you changed the **Exchange SSH Keys** property value before adding the host(s), change the value back to `false` after Ambari exchanges keys with the new hosts. This prevents Ambari from exchanging keys with all hosts every time the HAWQ master is started or restarted.
+
+23. (Optional.) Turn off temporary password-based authentication as described in [Apache HAWQ System Requirements](/20/requirements/system-requirements.html#topic_pwdlessssh).
#### <a id="manual-config-steps"></a>Manually Updating the HAWQ Configuration
If you need to expand your HAWQ cluster without restarting the HAWQ service, follow these steps to manually apply the new HAWQ configuration. (Use these steps *instead* of following Step 7 in the above procedure.):
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/433de8e0/install/aws-config.html.md.erb
----------------------------------------------------------------------
diff --git a/install/aws-config.html.md.erb b/install/aws-config.html.md.erb
index 32675fe..8bbb9b4 100644
--- a/install/aws-config.html.md.erb
+++ b/install/aws-config.html.md.erb
@@ -102,8 +102,11 @@ $ ssh -i my-test.pem user1@192.0.2.0
After launching your HAWQ instance, you will connect to and configure the instance. The *Instances* page of the EC2 Console lists the running instances and their associated network access information.
-Before installing HAWQ, set up the EC2 instances as you would local host server machines. Configure the host operating system, configure host network information (for example, update the `/etc/hosts` file), set operating system parameters, and install operating system packages. For information about how to prepare your operating system environment for HAWQ, see [Select HAWQ Host Machines](../install/select-hosts.html).
+Before installing HAWQ, set up the EC2 instances as you would local host server machines. Configure the host operating system, configure host network information (for example, update the `/etc/hosts` file), set operating system parameters, and install operating system packages. For information about how to prepare your operating system environment for HAWQ, see [Apache HAWQ System Requirements](../requirements/system-requirements.html) and [Select HAWQ Host Machines](../install/select-hosts.html).
+###Passwordless SSH Configuration<a id="topic_pwdlessssh_cc"></a>
+
+HAWQ hosts will be configured during the installation process to use passwordless SSH for intra-cluster communications. Temporary password-based authentication must be enabled on each HAWQ host in preparation for this configuration. Password authentication is typically disabled by default in cloud images. Update the cloud configuration in `/etc/cloud/cloud.cfg` to enable password authentication in your AMI(s). Set `ssh_pwauth: True` in this file. If desired, disable password authentication after HAWQ installation by setting the property back to `False`.
##References<a id="topic_hgz_zwy_bv"></a>
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/433de8e0/requirements/system-requirements.html.md.erb
----------------------------------------------------------------------
diff --git a/requirements/system-requirements.html.md.erb b/requirements/system-requirements.html.md.erb
index cf9214d..c61ba3d 100644
--- a/requirements/system-requirements.html.md.erb
+++ b/requirements/system-requirements.html.md.erb
@@ -173,6 +173,51 @@ If this system uses YARN for resource management, you would set `yarn.nodemanage
If this system uses the default HAWQ resource manager, you would set `hawq_rm_memory_limit_perseg` = `RAM - NON_HAWQ_MEMORY` = 8 GB - 7GB = 1.
+## <a id="topic_pwdlessssh"></a>Passwordless SSH Configuration
+
+HAWQ hosts will be configured to use passwordless SSH for intra-cluster communications during the installation process. Temporary password-based authentication must be enabled on each HAWQ host in preparation for this configuration.
+
+1. Install the SSH server if not already configured on the HAWQ system:
+
+ ``` shell
+ $ yum list installed | grep openssh-server
+ $ yum -y install openssh-server
+ ```
+
+2. Update the host's SSH configuration to allow password-based authentication. Edit the SSH config file and change the `PasswordAuthentication` configuration value from `no` to `yes`:
+
+ ``` shell
+ $ sudo vi /etc/ssh/sshd_config
+ ```
+
+ ```
+ PasswordAuthentication yes
+ ```
+
+3. Restart SSH:
+
+ ``` shell
+ $ sudo /etc/init.d/sshd restart
+ ```
+
+*After installation is complete*, you may choose to turn off the temporary password-based authentication configured in the previous steps:
+
+1. Open the SSH `/etc/ssh/sshd_config` file in a text editor and update/uncomment the following configuration options.
+
+ ```
+ RSAAuthentication yes
+ PasswordAuthentication no
+ PubkeyAuthentication yes
+ AuthorizedKeyFile .ssh/authorized_keys
+ ```
+
+2. Restart SSH:
+
+ ``` shell
+ $ sudo /etc/init.d/sshd restart
+ ```
+�
+
## <a id="topic_bsm_hhv_2v"></a>Disk Requirements
- 2GB per host for HAWQ installation.�
[9/9] incubator-hawq-docs git commit: fix merge conflict
Posted by yo...@apache.org.
fix merge conflict
Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/commit/97317c4d
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/tree/97317c4d
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/diff/97317c4d
Branch: refs/heads/develop
Commit: 97317c4df540a58d07756751f73dfd3ff507c36d
Parents: 6e10024 268ed13
Author: Lisa Owen <lo...@pivotal.io>
Authored: Wed Sep 28 09:39:18 2016 -0700
Committer: Lisa Owen <lo...@pivotal.io>
Committed: Wed Sep 28 09:39:18 2016 -0700
----------------------------------------------------------------------
----------------------------------------------------------------------
[8/9] incubator-hawq-docs git commit: fix merge conflict
Posted by yo...@apache.org.
fix merge conflict
Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/commit/6e100245
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/tree/6e100245
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/diff/6e100245
Branch: refs/heads/develop
Commit: 6e100245f8cb1554bf7c2f78101f605381699630
Parents: e8d612e
Author: Lisa Owen <lo...@pivotal.io>
Authored: Wed Sep 28 09:35:34 2016 -0700
Committer: Lisa Owen <lo...@pivotal.io>
Committed: Wed Sep 28 09:35:34 2016 -0700
----------------------------------------------------------------------
install/aws-config.html.md.erb | 1 -
1 file changed, 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/6e100245/install/aws-config.html.md.erb
----------------------------------------------------------------------
diff --git a/install/aws-config.html.md.erb b/install/aws-config.html.md.erb
index 8bbb9b4..e4106b1 100644
--- a/install/aws-config.html.md.erb
+++ b/install/aws-config.html.md.erb
@@ -29,7 +29,6 @@ Instance store-backed storage is generally better performing than EBS and recomm
**Note** EC2 *instance store* provides temporary block-level storage. This storage is located on disks that are physically attached to the host computer. While instance store provides high performance, powering off the instance causes data loss. Soft reboots preserve instance store data.
Virtual devices for instance store volumes for HAWQ EC2 instance store instances are named `ephemeralN` (where *N* varies based on instance type). CentOS instance store block device are named `/dev/xvdletter` (where *letter* is a lower case letter of the alphabet).
-
### <a id="topic_cfgplacegrp"></a>Configure Placement Group
[3/9] incubator-hawq-docs git commit: passwdless ssh - restart cmd
change, remove some props
Posted by yo...@apache.org.
passwdless ssh - restart cmd change, remove some props
Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/commit/dd25c6fe
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/tree/dd25c6fe
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/diff/dd25c6fe
Branch: refs/heads/develop
Commit: dd25c6fe223ddc0baca9b55ad43dcc9b3820140e
Parents: 831a83e
Author: Lisa Owen <lo...@pivotal.io>
Authored: Tue Sep 27 08:35:02 2016 -0700
Committer: Lisa Owen <lo...@pivotal.io>
Committed: Tue Sep 27 08:35:02 2016 -0700
----------------------------------------------------------------------
requirements/system-requirements.html.md.erb | 19 ++++++++-----------
1 file changed, 8 insertions(+), 11 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/dd25c6fe/requirements/system-requirements.html.md.erb
----------------------------------------------------------------------
diff --git a/requirements/system-requirements.html.md.erb b/requirements/system-requirements.html.md.erb
index 3f47092..835d974 100644
--- a/requirements/system-requirements.html.md.erb
+++ b/requirements/system-requirements.html.md.erb
@@ -178,43 +178,40 @@ If this system uses the default HAWQ resource manager, you would set `hawq_rm_me
HAWQ hosts will be configured to use passwordless SSH for intra-cluster communications during the installation process. Temporary password-based authentication must be enabled on each HAWQ host in preparation for this configuration.
1. Install the SSH server if not already configured on the HAWQ system:
-
+
``` shell
$ yum list installed | grep openssh-server
$ yum -y install openssh-server
```
-
+
2. Update the host's SSH configuration to allow password-based authentication. Edit the SSH config file and change the `PasswordAuthentication` configuration value from `no` to `yes`:
-
+
``` shell
$ sudo vi /etc/ssh/sshd_config
```
-
+
```
PasswordAuthentication yes
```
3. Restart SSH:
-
+
``` shell
- $ sudo /etc/init.d/sshd restart
+ $ sudo service sshd restart
```
*After installation is complete*, you may choose to turn off the temporary password-based authentication configured in the previous steps:
-1. Open the SSH `/etc/ssh/sshd_config` file in a text editor and update/uncomment the following configuration options.
+1. Open the SSH `/etc/ssh/sshd_config` file in a text editor and update the configuration option you enabled in step 2 above:
```
- RSAAuthentication yes
PasswordAuthentication no
- PubkeyAuthentication yes
- AuthorizedKeyFile .ssh/authorized_keys
```
2. Restart SSH:
``` shell
- $ sudo /etc/init.d/sshd restart
+ $ sudo service sshd restart
```
�
[7/9] incubator-hawq-docs git commit: clarify wording for disabling
passwd-based auth
Posted by yo...@apache.org.
clarify wording for disabling passwd-based auth
Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/commit/51592812
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/tree/51592812
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/diff/51592812
Branch: refs/heads/develop
Commit: 51592812b88f5078d7092b9b4a00a9dc169e8a71
Parents: 433de8e
Author: Lisa Owen <lo...@pivotal.io>
Authored: Mon Sep 12 12:07:28 2016 -0700
Committer: Lisa Owen <lo...@pivotal.io>
Committed: Wed Sep 28 09:34:02 2016 -0700
----------------------------------------------------------------------
admin/ClusterExpansion.html.md.erb | 4 ++--
admin/ambari-admin.html.md.erb | 4 ++--
2 files changed, 4 insertions(+), 4 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/51592812/admin/ClusterExpansion.html.md.erb
----------------------------------------------------------------------
diff --git a/admin/ClusterExpansion.html.md.erb b/admin/ClusterExpansion.html.md.erb
index f7d3856..d74881e 100644
--- a/admin/ClusterExpansion.html.md.erb
+++ b/admin/ClusterExpansion.html.md.erb
@@ -20,7 +20,7 @@ There are several recommendations to keep in mind when modifying the size of you
## <a id="task_hawq_expand"></a>Adding a New Node to an Existing HAWQ Cluster
-The following procedure describes the steps required to add a node to an existing HAWQ cluster. First ensure that the new node has been configured per the instructions in [Apache HAWQ System Requirements](/20/requirements/system-requirements.html) and [Select HAWQ Host Machines](/20/install/select-hosts.html).
+The following procedure describes the steps required to add a node to an existing HAWQ cluster. First ensure that the new node has been configured per the instructions found in [Apache HAWQ System Requirements](/20/requirements/system-requirements.html) and [Select HAWQ Host Machines](/20/install/select-hosts.html).
For example purposes in this procedure, we are adding a new node named `sdw4`.
@@ -71,7 +71,7 @@ For example purposes in this procedure, we are adding a new node named `sdw4`.
$ hawq ssh-exkeys -e hawq_hosts -x new_hosts
```
- 8. (Optional) Turn off temporary password-based authentication as described in [Apache HAWQ System Requirements](/20/requirements/system-requirements.html#topic_pwdlessssh).
+ 8. (Optional) If you enabled temporary password-based authentication while preparing/configuring your new HAWQ host system, turn off password-based authentication as described in [Apache HAWQ System Requirements](/20/requirements/system-requirements.html#topic_pwdlessssh).
8. After setting up passwordless ssh, you can execute the following hawq command to check the target machine's configuration.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/51592812/admin/ambari-admin.html.md.erb
----------------------------------------------------------------------
diff --git a/admin/ambari-admin.html.md.erb b/admin/ambari-admin.html.md.erb
index 5c98dc0..6b5f1e1 100644
--- a/admin/ambari-admin.html.md.erb
+++ b/admin/ambari-admin.html.md.erb
@@ -160,7 +160,7 @@ There are several recommendations to keep in mind when modifying the size of you
- If you are using hash tables, consider updating the `default_hash_table_bucket_number` server configuration parameter to a larger value after expanding the cluster but before redistributing the hash tables.
### Procedure
-First ensure that the new node(s) has been configured per the instructions in [Apache HAWQ System Requirements](/20/requirements/system-requirements.html) and [Select HAWQ Host Machines](/20/install/select-hosts.html).
+First ensure that the new node(s) has been configured per the instructions found in [Apache HAWQ System Requirements](/20/requirements/system-requirements.html) and [Select HAWQ Host Machines](/20/install/select-hosts.html).
1. If you have any user-defined function (UDF) libraries installed in your existing HAWQ cluster, install them on the new node(s) that you want to add to the HAWQ cluster.
2. Access the Ambari web console at http://ambari.server.hostname:8080, and login as the "admin" user. \(The default password is also "admin".\)
@@ -204,7 +204,7 @@ First ensure that the new node(s) has been configured per the instructions in [A
**Note:** The redistribution of table data can take a significant amount of time.
22. (Optional.) If you changed the **Exchange SSH Keys** property value before adding the host(s), change the value back to `false` after Ambari exchanges keys with the new hosts. This prevents Ambari from exchanging keys with all hosts every time the HAWQ master is started or restarted.
-23. (Optional.) Turn off temporary password-based authentication as described in [Apache HAWQ System Requirements](/20/requirements/system-requirements.html#topic_pwdlessssh).
+23. (Optional.) If you enabled temporary password-based authentication while preparing/configuring your HAWQ host systems, turn off password-based authentication as described in [Apache HAWQ System Requirements](/20/requirements/system-requirements.html#topic_pwdlessssh).
#### <a id="manual-config-steps"></a>Manually Updating the HAWQ Configuration
If you need to expand your HAWQ cluster without restarting the HAWQ service, follow these steps to manually apply the new HAWQ configuration. (Use these steps *instead* of following Step 7 in the above procedure.):
[5/9] incubator-hawq-docs git commit: passwdless ssh - restart cmd
change, remove some props
Posted by yo...@apache.org.
passwdless ssh - restart cmd change, remove some props
Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/commit/e8d612e5
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/tree/e8d612e5
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/diff/e8d612e5
Branch: refs/heads/develop
Commit: e8d612e57246fc457dc0374c22ff9edcb9f4e19b
Parents: 5159281
Author: Lisa Owen <lo...@pivotal.io>
Authored: Tue Sep 27 08:35:02 2016 -0700
Committer: Lisa Owen <lo...@pivotal.io>
Committed: Wed Sep 28 09:34:02 2016 -0700
----------------------------------------------------------------------
requirements/system-requirements.html.md.erb | 19 ++++++++-----------
1 file changed, 8 insertions(+), 11 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/e8d612e5/requirements/system-requirements.html.md.erb
----------------------------------------------------------------------
diff --git a/requirements/system-requirements.html.md.erb b/requirements/system-requirements.html.md.erb
index c61ba3d..7f117dc 100644
--- a/requirements/system-requirements.html.md.erb
+++ b/requirements/system-requirements.html.md.erb
@@ -178,43 +178,40 @@ If this system uses the default HAWQ resource manager, you would set `hawq_rm_me
HAWQ hosts will be configured to use passwordless SSH for intra-cluster communications during the installation process. Temporary password-based authentication must be enabled on each HAWQ host in preparation for this configuration.
1. Install the SSH server if not already configured on the HAWQ system:
-
+
``` shell
$ yum list installed | grep openssh-server
$ yum -y install openssh-server
```
-
+
2. Update the host's SSH configuration to allow password-based authentication. Edit the SSH config file and change the `PasswordAuthentication` configuration value from `no` to `yes`:
-
+
``` shell
$ sudo vi /etc/ssh/sshd_config
```
-
+
```
PasswordAuthentication yes
```
3. Restart SSH:
-
+
``` shell
- $ sudo /etc/init.d/sshd restart
+ $ sudo service sshd restart
```
*After installation is complete*, you may choose to turn off the temporary password-based authentication configured in the previous steps:
-1. Open the SSH `/etc/ssh/sshd_config` file in a text editor and update/uncomment the following configuration options.
+1. Open the SSH `/etc/ssh/sshd_config` file in a text editor and update the configuration option you enabled in step 2 above:
```
- RSAAuthentication yes
PasswordAuthentication no
- PubkeyAuthentication yes
- AuthorizedKeyFile .ssh/authorized_keys
```
2. Restart SSH:
``` shell
- $ sudo /etc/init.d/sshd restart
+ $ sudo service sshd restart
```
�