You are viewing a plain text version of this content. The canonical link for it is here.
Posted to codereview@trafodion.apache.org by gtapper <gi...@git.apache.org> on 2016/02/17 09:42:30 UTC
[GitHub] incubator-trafodion pull request: Trafodion 1822
GitHub user gtapper opened a pull request:
https://github.com/apache/incubator-trafodion/pull/326
Trafodion 1822
Please publish these changes to the website. Need to build the 1.3.0 and 2.0.0 of the new Provisioning Guide.
Thanks!
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/gtapper/incubator-trafodion TRAFODION-1822
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-trafodion/pull/326.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #326
----
commit 4b71bcc89d3438f0d79c0ddfc4afa751ce306df1
Author: Gunnar Tapper <gt...@esgyn.local>
Date: 2016-02-12T03:04:03Z
Merge branch 'master' of git://git.apache.org/incubator-trafodion into TRAFODION-1822
commit b763c5fa6a0e152af0ae05b9b53c1f93577c5e9a
Author: Gunnar Tapper <gt...@esgyn.local>
Date: 2016-02-14T07:43:08Z
Backup checking for changes in the Provisioning Guide.
commit 72a907620b8dedbe9fc9a1db8b8cfccb7e49cd13
Author: Gunnar Tapper <gt...@esgyn.local>
Date: 2016-02-15T08:09:03Z
Backup checkin of changes to the Provisioning Guide.
commit 8ff14487fc139a3d45b3c2d95aacb99fb57fca22
Author: Gunnar Tapper <gt...@esgyn.local>
Date: 2016-02-15T19:26:09Z
More updates.
commit b19faadfdea985cec4dcd1fc2657efb9af18325b
Author: Gunnar Tapper <gt...@esgyn.local>
Date: 2016-02-17T08:34:00Z
[TRAFODION-1810] New provisioning guide.
[TRAFODION-1822] Use new Apache Logo: Added new logo.
[TRAFODION-1827] Broken link
Shrunk width of navigation panel for web books. Updated feedback text to
suggest contribution in all guides.
Please update the website. Build the 1.3.0 and 2.0.0 version of the
Provisioning Guide.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request: Trafodion 1822
Posted by gtapper <gi...@git.apache.org>.
Github user gtapper commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/326#discussion_r53246345
--- Diff: docs/provisioning_guide/src/asciidoc/_chapters/requirements.adoc ---
@@ -415,11 +456,34 @@ java.heapsize^1^ | 1073741824 (1GB) | Java Heap Size for the HDFS Secondary Name
1. Applies to Cloudera distributions only.
+[[requirements-recommended-configuration-changes]]
== Recommended Configuration Changes
The following configuration changes are recommended but not required.
NOTE: The Trafodion Installer does *not* make these changes.
+[[requirements-recommended-security-changes]]
+=== Recommended Security Changes
+
+The `trafodion` user ID should not be given other `sudo` privileges than what's specified in this manual. Also, we
+that this user ID is locked (`sudo passwd -l trafodion`) once the installation/upgrade activity has been completed.
--- End diff --
We recommend. :)
Fixed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request: Trafodion 1822
Posted by DaveBirdsall <gi...@git.apache.org>.
Github user DaveBirdsall commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/326#discussion_r53222922
--- Diff: docs/provisioning_guide/src/asciidoc/_chapters/introduction.adoc ---
@@ -0,0 +1,466 @@
+////
+/**
+* @@@ START COPYRIGHT @@@
+*
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing,
+* software distributed under the License is distributed on an
+* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+* KIND, either express or implied. See the License for the
+* specific language governing permissions and limitations
+* under the License.
+*
+* @@@ END COPYRIGHT @@@
+ */
+////
+
+[[introduction]]
+= Introduction
+
+Trafodion is a Hadoop add-on service that provides transactional SQL on top of HBase. Typically, you
+use Trafodion as the database for applications that require Online Transaction Processing (OLTP),
+Operational Data Store (ODS), and/or strong reporting capabilities. You access Trafodion using
+standard JDBC and ODBC APIs.
+
+You may choose whether to add Trafodion to an existing Hadoop environment or to create a standalone
+Hadoop environment specifically for Hadoop.
+
+This guide assumes that a Hadoop environment exists upon which your provisioning Trafodion. Refer to
+<<requirements-hadoop-software,Hadoop Software>> for information about what Hadoop software is required
+Trafodion.
+
+[[introduction-security-considerations]]
+== Security Considerations
+
+The following users need be considered for Trafodion:
+
+* *Provisioning User*: A Linux-level user that performs the Trafodion provisioning tasks. This user ID
+requires `sudo` access and passwordless ssh among the nodes where Trafodion is installed. In addition,
+this user ID requires access to Hadoop distribution, HDFS, and HBase administrative users to change
+respective environment's configuration settings per Trafodion requirements. Refer to
+<<requirements-trafodion-provisioning-user,Trafodion Provisioning User>> for more information
+about the requirements and usage associated with this user ID.
+
+* *Runtime User*: A linux-level user under which the Trafodion software runs. This user ID must be registered
--- End diff --
Should linux be capitalized? (It is elsewhere.)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request: Trafodion 1822
Posted by DaveBirdsall <gi...@git.apache.org>.
Github user DaveBirdsall commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/326#discussion_r53245371
--- Diff: docs/provisioning_guide/src/asciidoc/_chapters/introduction.adoc ---
@@ -0,0 +1,466 @@
+////
+/**
+* @@@ START COPYRIGHT @@@
+*
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing,
+* software distributed under the License is distributed on an
+* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+* KIND, either express or implied. See the License for the
+* specific language governing permissions and limitations
+* under the License.
+*
+* @@@ END COPYRIGHT @@@
+ */
+////
+
+[[introduction]]
+= Introduction
+
+Trafodion is a Hadoop add-on service that provides transactional SQL on top of HBase. Typically, you
+use Trafodion as the database for applications that require Online Transaction Processing (OLTP),
+Operational Data Store (ODS), and/or strong reporting capabilities. You access Trafodion using
+standard JDBC and ODBC APIs.
+
+You may choose whether to add Trafodion to an existing Hadoop environment or to create a standalone
+Hadoop environment specifically for Hadoop.
+
+This guide assumes that a Hadoop environment exists upon which your provisioning Trafodion. Refer to
+<<requirements-hadoop-software,Hadoop Software>> for information about what Hadoop software is required
+Trafodion.
+
+[[introduction-security-considerations]]
+== Security Considerations
+
+The following users need be considered for Trafodion:
+
+* *Provisioning User*: A Linux-level user that performs the Trafodion provisioning tasks. This user ID
+requires `sudo` access and passwordless ssh among the nodes where Trafodion is installed. In addition,
+this user ID requires access to Hadoop distribution, HDFS, and HBase administrative users to change
+respective environment's configuration settings per Trafodion requirements. Refer to
+<<requirements-trafodion-provisioning-user,Trafodion Provisioning User>> for more information
+about the requirements and usage associated with this user ID.
+
+* *Runtime User*: A linux-level user under which the Trafodion software runs. This user ID must be registered
+as a user in the Hadoop Distributed File System (HDFS) to store and access objects in HDFS, HBase, and Hive.
+In addition, this user ID requires passwordless access among the nodes where Trafodion is installed.
+Refer to <<requirements-trafodion-runtime-user,Trafodion Runtime User>> for more information about this user ID.
+
+* *Trafodion Database Users*: Trafodion users are managed by the Trafodion security features (grant, revoke, etc.),
+which can be integrated with LDAP if so desired. These users are referred to as *database users* and
+do not have direct access to the operating system. Refer to
+http://trafodion.apache.org/docs/sql_reference/index.html#register_user_statement[Register User],
+http://trafodion.apache.org/docs/sql_reference/index.html#grant_statement[Grant], and other SQL statements
+in the http://trafodion.apache.org/docs/sql_reference/index.html[Trafodion SQL Reference Manual] for
+more information about managing Trafodion Database Users.
+ +
+ +
+Optionally, you can enable Trafodion Security. If you do not enable security in Trafodion, then a client interface
+to Trafodion may request a user name and password, but Trafodion ignores the user name and password entered in the
+client interface, and the session runs as the database *root* user, `DB__ROOT`, without restrictions. If you want
+to restrict users, restrict access to certain users only, or restrict access to an object or operation, then you must
+enable security, which enforces authentication and authorization. Refer to
+<<enable-security,Enable Security>> for more information about this option.
+
+[[introduction-provisioning-options]]
+== Provisioning Options
+
+Trafodion ships with a set of scripts (the Trafodion Installer) that takes care of many of the installation and upgrade
--- End diff --
My comment is incorrect; "takes care" is correct. ("Takes care" refers to "set")
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request: Trafodion 1822
Posted by gtapper <gi...@git.apache.org>.
Github user gtapper commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/326#discussion_r53245244
--- Diff: docs/provisioning_guide/src/asciidoc/_chapters/about.adoc ---
@@ -24,19 +24,22 @@
////
= About This Document
-This guide describes how to provision Trafodion for end-user environments. This install allows you to store and query data using Trafodion, either via Trafodion clients
+This guide describes how to provision the end-user Trafodion binaries ontop of an existing Hadoop environment. This install allows you to store and query data using Trafodion, either via Trafodion clients
--- End diff --
Fixed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request: Trafodion 1822
Posted by gtapper <gi...@git.apache.org>.
Github user gtapper commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/326#discussion_r53244950
--- Diff: docs/provisioning_guide/src/asciidoc/_chapters/prepare.adoc ---
@@ -0,0 +1,379 @@
+////
+/**
+* @@@ START COPYRIGHT @@@
+*
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing,
+* software distributed under the License is distributed on an
+* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+* KIND, either express or implied. See the License for the
+* specific language governing permissions and limitations
+* under the License.
+*
+* @@@ END COPYRIGHT @@@
+ */
+////
+
+[[prepare]]
+= Prepare
+You need to prepare your Hadoop environment before installing Trafodion.
+
+1. <<prepare-install-optional-workstation-software,Install Optional Workstation Software>>
+2. <<configure-installation-user-id,Configure Installation User ID>>
+3. <<prepare-disable-requiretty,Disable requiretty>>
+4. <<prepare-verify-os-requirements-and-recommendations,Verify OS Requirements and Recommendations>>
+5. <<prepare-configure-ldap-identity-store,Configure LDAP Identity Store>>
+6. <<prepare-gather-configuration-information,Gather Configuration Information>>
+7. <<prepare-install-required-software-packages,Install Required Software Packages>>
+8. <<prepare-perform-recipe-based-provisioning-tasks,Perform Recipe-Based Provisioning Tasks>>
+
+[[prepare-install-optional-workstation-software]]
+== Install Optional Workstation Software
+
+If you are using a Windows workstation, then the following optional software helps installation process.
+We recommended that you pre-install the software before continuing with the Trafodion installation:
+
+* putty and puttygen (download from http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html[PuTTY web site])
+* VNC client (download from http://www.realvnc.com[RealVNC web site])
+* Firefox or Chrome browser
+* SFTP client to transfer files from your workstation to the Linux server: WinSCP or FileZilla
+
+[[configure-installation-user-id]]
+== Configure Installation User ID
+
+Trafodion installation requires a user ID with these attributes:
+
+* `sudo` access per the requirements documented in <<requirements-linux-installation-user,Linux Installation User>>.
+* passwordless ssh to all nodes on the cluster where Trafodion will be installed.
+
+NOTE: You may need to request permission from your cluster-management team to obtain this type of access.
+
+The following example shows how to set up your user ID to have "passwordless ssh" abilities.
+
+Do the following on the Provision Master Node:
+
+```
+echo -e 'y\n' | ssh-keygen -t rsa -N "" -f $HOME/.ssh/id_rsa
+cat $HOME/.ssh/id_rsa.pub >> $HOME/.ssh/authorized_keys
+echo localhost $(cat /etc/ssh/ssh_host_rsa_key.pub) >> $HOME/.ssh/known_hosts
+echo "NoHostAuthenticationForLocalhost=yes" >> $HOME/.ssh/config
+chmod 600 $HOME/.ssh/config
+```
+
+After running these commands, do the following:
+
+* Copy the content of the `$HOME/.ssh/id_rsa.pub` file (public key) and append it to the
+`$HOME/.ssh/authorized_keys` file on the other nodes in your cluster.
+* Copy the `$HOME/.ssh/id_rsa` file (private key) from the current node to the `$HOME/.ssh` directory on the other nodes in the cluster.
+* Secure `$HOME/.ssh/id_rsa` on all nodes as private to yourself (`chmod 700`).
+* `ssh` to the other nodes in the cluster. Answer `y` to the prompt asking you whether to continue the connection.
+This adds the node to the `$HOME/.ssh/known_hosts` file completing the passwordless ssh setup.
+
+
+[[prepare-disable-requiretty]]
+== Disable requiretty
+You need to disable `requiretty` in `/etc/sudoers` on all nodes in the cluster
+to ensure that `sudo` commands can be run from inside the installation scripts.
+
+Comment out the `Defaults requiretty` setting in the `/etc/sudoers` file to
+ensure that the `requiretty` option is NOT being used.
+
+[[prepare-verify-os-requirements-and-recommendations]]
+== Verify OS Requirements and Recommendations
+
+Please ensure that the <<requirements-os-requirements-and-recommendations,OS Requirements and Recommendations>>
+are met for each node in the cluster where you intend to install Trafodion.
+
+<<<
+[[prepare-configure-ldap-identity-store]]
+== Configure LDAP Identity Store
+
+If you plan to enable security in Trafodion, then you need to have an LDAP identity store available to perform authentication.
+The Trafodion Installer prompts you to set up an authentication configuration file that points to an LDAP server (or servers),
+which enables security (that is, authentication and authorization) in the Trafodion database.
+
+If you wish to manually set up the authentication configuration file and enable security, then refer to the section on
+<<enable-security,Enable Security>>.
+
+[[prepare-gather-configuration-information]]
+== Gather Configuration Information
+
+You need to gather/decide information about your environment to aid installation Trafodion, both for the Trafodion Installer
+and for recipe-based provisioning. (In alphabetical order to make it easier to find information later on in the provisioning process.)
+
+[cols="25%l,25%,15%l,35%",options="header"]
+|===
+| ID^1^ | Information | Default | Notes
+| ADMIN | Administrator user name for Apache Ambari or Cloudera Manager. | admin | A user that can change configuration and restart services via the
+distribution manager's REST API.
+| BACKUP_DCS_NODES | List of nodes where to start the backup DCS Master components. | None | Blank separated FQDN list. Not needed if $ENABLE_HA = N.
+| CLOUD_CONFIG | Whether you're installing Trafodion on a cloud environment. | N | N = bare-metal or VM installation.
+| CLOUD_TYPE | What type of cloud environment you're installing Trafodion on. | None | { AWS \| OpenStack \| Other }. Not applicable for bare-metal or VM installation.
+| CLUSTER_NAME | The name of the Hadoop Cluster. | None | From Apache Ambari or Cloudera Manager.
+| DCS_BUILD | Tar file containing the DCS component. | None | Not needed if using a Trafodion package installation tar file.
+| DCS_PRIMARY_MASTER_NODE | The node where the primary DCS should run. | None | The DCS Master handles JDBC and ODBC connection requests.
+| DCS_SERVER_PARM | Number of concurrent client sessions per node. | 16 | This number specifies the concurrent sessions per node to be supported. Each session could require up to 1GB of physical memory. The number can be changed post-installation. For more information,
+refer to the http://trafodion.apache.org/docs/client_install/index.html[Trafodion Client Installation Guide].
+| ENABLE_HA | Whether to run DCS in high-availability (HA) mode. | N | You need the floating IP address, the interface, and the backup nodes for DCS Master if enabling this feature.
+| EPEL_RPM | Location of EPEL RPM. | None | Specify if you don't have access to the Internet.
+Downloaded automatically by the Trafodion Installer.
+| FLOATING_IP | IP address if running DCS in HA mode. | None | Not needed if $ENABLE_HA = N. An FQDN name or IP address.
+| HADOOP_TYPE | The type of Hadoop distribution you're installing Trafodion on. | None | Lowercase. cloudera or hadoop.
+| HBASE_GROUP | Linux group name for the HBASE administrative user. | hbase | Required in order to provide access to select HDFS directories to this user ID.
+| HBASE_USER | Linux user name for the HBASE administrative user. | hbase | Required in order to provide access to select HDFS directories to this user ID.
+| HDFS_USER | Linux user name for the HDFS administrative user. | hdfs | The Trafodion Installer uses `sudo su` to make HDFS
+configuration changes under this user.
+| HOME_DIR | Root directory under which the `trafodion` home directory should be created. | /home | *Example* +
+ +
+If the home directory of the `trafodion` user is
+`/opt/home/trafodion`, then specify the root directory as `/opt/home`.
+| INIT_TRAFODION | Whether to automatically initialize the Trafodion database. | N | Does not apply to Recipe-Based Provisioning. Applies if $START=Y only.
+| INTERFACE | Interface type used for $FLOATING_IP. | None | Not needed if $ENABLE_HA = N.
+| JAVA_HOME | Location of Java 1.7.0_65 or higher (JDK). | $JAVA_HOME setting | Fully qualified path of the JDK. For example:
+`/usr/java/jdk1.7.0_67-cloudera`
+| LDAP_CERT^2^ | Full path to TLS certificate. | None | Required of $LDAP_LEVEL = 1 or 2.
+| LDAP_HOSTS^2^ | List of nodes where LDAP Identity Store servers are running. | None | Blank separated. FQDN format.
+| LDAP_ID^2^ | List of LDAP unique identifiers. | None | Blank separated.
+| LDAP_LEVEL^2^ | LDAP Encryption Level. | 0 | 0: Encryption not used, 1: SSL, 2: TLS
+| LDAP_PASSWORD^2^ | Password for LDAP_USER. | None | If LDAP_USER is required only.
+| LDAP_PORT^2^ | Port used to communicate with LDAP Identity Store. | None | Examples: 389 for no encryption or TLS, 636 for SSL.
+| LDAP_SECURITY^2^ | Whether to enable simple LDAP authentication. | N | If Y, then you need to provide LDAP_HOSTS.
+| LDAP_USER^2^ | LDAP Search user name. | None | If required. If so, must provide LDAP_PASSWORD, too.
+| LOCAL_WORKDIR | The directory where the Trafodion Installer is located. | None | Full path, no environmental variables.
+| MANAGEMENT_ENABLED | Whether your installation uses separate management nodes. | N | Y if using separate management nodes for Apache Ambari or Cloudera Manager.
+| MANAGEMENT_NODES | The FQDN names of management nodes, if any. | None | Provide a blank-separated list of node names.
+| NODE_LIST | The FQDN names of the nodes where Trafodion will be installed. | None | Provide a blank-separated list of node names. The Trafodion
+Provisioning ID must have passwordless and `sudo` access to these nodes.
+| PASSWORD | Administrator password for Apache Ambari or Cloudera Manager. | admin | A user that can change configuration and restart services via the
+distribution manager's REST API.
+| REST_BUILD | Tar file containing the REST component. | None | Not needed if using a Trafodion package installation tar file.
+| SQ_ROOT | Target directory for the Trafodion software. | $HOME_DIR/trafodion | Trafodion is installed in this directory on all nodes in `$NODE_LIST`.
+| START | Whether to start Trafodion after install/upgrade. | N | Does not apply to Recipe-Based Provisioning.
+| SUSE_LINUX | Whether your installing Trafodion on SUSE Linux. | false | Auto-detected by the Trafodion Installer.
+| TRAF_PACKAGE | The location of the Trafodion installation package tar file or core installation tar file. | None | The package file contains the Trafodion server,
+DCS, and REST software while the core installation file contains the Trafodion server software only. If you're using a core installation file, then you need to
+record the location of the DCS and REST installation tar files, too. Normally, you perform Trafodion provisioning using a Trafodion package installation tar file.
+| TRAF_USER | The Trafodion runtime user ID. | trafodion | Must be `trafodion` in this release.
+| TRAF_USER_PASSWORD | The password used for the `trafodion:trafodion` user ID. | traf123 | Must be 6-8 characters long.
+| URL | FQDN and port for the Distribution Manager's REST API. | None | Include `http://` or `https://` as applicable. Specify in the form:
+`<IP-address>:<port>` or `<node name>:<port>` Example: `https://susevm-1.yourcompany.local:8080`
+|===
+
+1. The ID matches the environmental variables used in the Trafodion Installation configuration file. Refer to <<install-trafodion-installer,Trafodion Installer>>
+for more information.
+2. Refer to <<enable-security,Enable Security>> for more information about these security settings.
+
+
+<<<
+[[prepare-install-required-software-packages]]
+== Install Required Software Packages
+
+[[prepare-download-and-install-packages]]
+=== Download and Install Packages
+
+The step is required if you're:
+
+* Installing Trafodion on SUSE.
+* Using Recipe-Based Provisioning.
+* Can't download the required software packages using the Internet.
+
+If none of these situations exist, then we highly recommend that you use the Trafodion Installer.
+
+You perform this step as a user with `root` or `sudo` access.
+
+Install the packages listed in <<requirements-software-packages,Software Packages>> above on all nodes in the cluster. Note the special
+handling for `log4c++`. See <<prepare-build-and-install-log4cplusplus, Build and Install log4c++>> below for more information.
--- End diff --
Yep. Markup-language limitations.
On Wed, Feb 17, 2016 at 1:17 PM, DaveBirdsall <no...@github.com>
wrote:
> In docs/provisioning_guide/src/asciidoc/_chapters/prepare.adoc
> <https://github.com/apache/incubator-trafodion/pull/326#discussion_r53224885>
> :
>
> > +
> > +[[prepare-download-and-install-packages]]
> > +=== Download and Install Packages
> > +
> > +The step is required if you're:
> > +
> > +* Installing Trafodion on SUSE.
> > +* Using Recipe-Based Provisioning.
> > +* Can't download the required software packages using the Internet.
> > +
> > +If none of these situations exist, then we highly recommend that you use the Trafodion Installer.
> > +
> > +You perform this step as a user with `root` or `sudo` access.
> > +
> > +Install the packages listed in <<requirements-software-packages,Software Packages>> above on all nodes in the cluster. Note the special
> > +handling for `log4c++`. See <<prepare-build-and-install-log4cplusplus, Build and Install log4c++>> below for more information.
>
> Three different ways to render "++"! :-)
>
> —
> Reply to this email directly or view it on GitHub
> <https://github.com/apache/incubator-trafodion/pull/326/files#r53224885>.
>
--
Thanks,
Gunnar
*If you think you can you can, if you think you can't you're right.*
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request: Trafodion 1822
Posted by DaveBirdsall <gi...@git.apache.org>.
Github user DaveBirdsall commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/326#discussion_r53224682
--- Diff: docs/provisioning_guide/src/asciidoc/_chapters/prepare.adoc ---
@@ -0,0 +1,379 @@
+////
+/**
+* @@@ START COPYRIGHT @@@
+*
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing,
+* software distributed under the License is distributed on an
+* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+* KIND, either express or implied. See the License for the
+* specific language governing permissions and limitations
+* under the License.
+*
+* @@@ END COPYRIGHT @@@
+ */
+////
+
+[[prepare]]
+= Prepare
+You need to prepare your Hadoop environment before installing Trafodion.
+
+1. <<prepare-install-optional-workstation-software,Install Optional Workstation Software>>
+2. <<configure-installation-user-id,Configure Installation User ID>>
+3. <<prepare-disable-requiretty,Disable requiretty>>
+4. <<prepare-verify-os-requirements-and-recommendations,Verify OS Requirements and Recommendations>>
+5. <<prepare-configure-ldap-identity-store,Configure LDAP Identity Store>>
+6. <<prepare-gather-configuration-information,Gather Configuration Information>>
+7. <<prepare-install-required-software-packages,Install Required Software Packages>>
+8. <<prepare-perform-recipe-based-provisioning-tasks,Perform Recipe-Based Provisioning Tasks>>
+
+[[prepare-install-optional-workstation-software]]
+== Install Optional Workstation Software
+
+If you are using a Windows workstation, then the following optional software helps installation process.
+We recommended that you pre-install the software before continuing with the Trafodion installation:
+
+* putty and puttygen (download from http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html[PuTTY web site])
+* VNC client (download from http://www.realvnc.com[RealVNC web site])
+* Firefox or Chrome browser
+* SFTP client to transfer files from your workstation to the Linux server: WinSCP or FileZilla
+
+[[configure-installation-user-id]]
+== Configure Installation User ID
+
+Trafodion installation requires a user ID with these attributes:
+
+* `sudo` access per the requirements documented in <<requirements-linux-installation-user,Linux Installation User>>.
+* passwordless ssh to all nodes on the cluster where Trafodion will be installed.
+
+NOTE: You may need to request permission from your cluster-management team to obtain this type of access.
+
+The following example shows how to set up your user ID to have "passwordless ssh" abilities.
+
+Do the following on the Provision Master Node:
+
+```
+echo -e 'y\n' | ssh-keygen -t rsa -N "" -f $HOME/.ssh/id_rsa
+cat $HOME/.ssh/id_rsa.pub >> $HOME/.ssh/authorized_keys
+echo localhost $(cat /etc/ssh/ssh_host_rsa_key.pub) >> $HOME/.ssh/known_hosts
+echo "NoHostAuthenticationForLocalhost=yes" >> $HOME/.ssh/config
+chmod 600 $HOME/.ssh/config
+```
+
+After running these commands, do the following:
+
+* Copy the content of the `$HOME/.ssh/id_rsa.pub` file (public key) and append it to the
+`$HOME/.ssh/authorized_keys` file on the other nodes in your cluster.
+* Copy the `$HOME/.ssh/id_rsa` file (private key) from the current node to the `$HOME/.ssh` directory on the other nodes in the cluster.
+* Secure `$HOME/.ssh/id_rsa` on all nodes as private to yourself (`chmod 700`).
+* `ssh` to the other nodes in the cluster. Answer `y` to the prompt asking you whether to continue the connection.
+This adds the node to the `$HOME/.ssh/known_hosts` file completing the passwordless ssh setup.
+
+
+[[prepare-disable-requiretty]]
+== Disable requiretty
+You need to disable `requiretty` in `/etc/sudoers` on all nodes in the cluster
+to ensure that `sudo` commands can be run from inside the installation scripts.
+
+Comment out the `Defaults requiretty` setting in the `/etc/sudoers` file to
+ensure that the `requiretty` option is NOT being used.
+
+[[prepare-verify-os-requirements-and-recommendations]]
+== Verify OS Requirements and Recommendations
+
+Please ensure that the <<requirements-os-requirements-and-recommendations,OS Requirements and Recommendations>>
+are met for each node in the cluster where you intend to install Trafodion.
+
+<<<
+[[prepare-configure-ldap-identity-store]]
+== Configure LDAP Identity Store
+
+If you plan to enable security in Trafodion, then you need to have an LDAP identity store available to perform authentication.
+The Trafodion Installer prompts you to set up an authentication configuration file that points to an LDAP server (or servers),
+which enables security (that is, authentication and authorization) in the Trafodion database.
+
+If you wish to manually set up the authentication configuration file and enable security, then refer to the section on
+<<enable-security,Enable Security>>.
+
+[[prepare-gather-configuration-information]]
+== Gather Configuration Information
+
+You need to gather/decide information about your environment to aid installation Trafodion, both for the Trafodion Installer
+and for recipe-based provisioning. (In alphabetical order to make it easier to find information later on in the provisioning process.)
--- End diff --
Grammar. The parenthetical remark is not a complete sentence. Possible wordsmith: "Do so in alphabetical order..."
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request: Trafodion 1822
Posted by gtapper <gi...@git.apache.org>.
Github user gtapper commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/326#discussion_r53245449
--- Diff: docs/provisioning_guide/src/asciidoc/_chapters/introduction.adoc ---
@@ -0,0 +1,466 @@
+////
+/**
+* @@@ START COPYRIGHT @@@
+*
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing,
+* software distributed under the License is distributed on an
+* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+* KIND, either express or implied. See the License for the
+* specific language governing permissions and limitations
+* under the License.
+*
+* @@@ END COPYRIGHT @@@
+ */
+////
+
+[[introduction]]
+= Introduction
+
+Trafodion is a Hadoop add-on service that provides transactional SQL on top of HBase. Typically, you
+use Trafodion as the database for applications that require Online Transaction Processing (OLTP),
+Operational Data Store (ODS), and/or strong reporting capabilities. You access Trafodion using
+standard JDBC and ODBC APIs.
+
+You may choose whether to add Trafodion to an existing Hadoop environment or to create a standalone
+Hadoop environment specifically for Hadoop.
+
+This guide assumes that a Hadoop environment exists upon which your provisioning Trafodion. Refer to
+<<requirements-hadoop-software,Hadoop Software>> for information about what Hadoop software is required
+Trafodion.
+
+[[introduction-security-considerations]]
+== Security Considerations
+
+The following users need be considered for Trafodion:
+
+* *Provisioning User*: A Linux-level user that performs the Trafodion provisioning tasks. This user ID
+requires `sudo` access and passwordless ssh among the nodes where Trafodion is installed. In addition,
+this user ID requires access to Hadoop distribution, HDFS, and HBase administrative users to change
+respective environment's configuration settings per Trafodion requirements. Refer to
+<<requirements-trafodion-provisioning-user,Trafodion Provisioning User>> for more information
+about the requirements and usage associated with this user ID.
+
+* *Runtime User*: A linux-level user under which the Trafodion software runs. This user ID must be registered
+as a user in the Hadoop Distributed File System (HDFS) to store and access objects in HDFS, HBase, and Hive.
+In addition, this user ID requires passwordless access among the nodes where Trafodion is installed.
+Refer to <<requirements-trafodion-runtime-user,Trafodion Runtime User>> for more information about this user ID.
+
+* *Trafodion Database Users*: Trafodion users are managed by the Trafodion security features (grant, revoke, etc.),
+which can be integrated with LDAP if so desired. These users are referred to as *database users* and
+do not have direct access to the operating system. Refer to
+http://trafodion.apache.org/docs/sql_reference/index.html#register_user_statement[Register User],
+http://trafodion.apache.org/docs/sql_reference/index.html#grant_statement[Grant], and other SQL statements
+in the http://trafodion.apache.org/docs/sql_reference/index.html[Trafodion SQL Reference Manual] for
+more information about managing Trafodion Database Users.
+ +
+ +
+Optionally, you can enable Trafodion Security. If you do not enable security in Trafodion, then a client interface
+to Trafodion may request a user name and password, but Trafodion ignores the user name and password entered in the
+client interface, and the session runs as the database *root* user, `DB__ROOT`, without restrictions. If you want
+to restrict users, restrict access to certain users only, or restrict access to an object or operation, then you must
+enable security, which enforces authentication and authorization. Refer to
+<<enable-security,Enable Security>> for more information about this option.
+
+[[introduction-provisioning-options]]
+== Provisioning Options
+
+Trafodion ships with a set of scripts (the Trafodion Installer) that takes care of many of the installation and upgrade
--- End diff --
Fixed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request: Trafodion 1822
Posted by DaveBirdsall <gi...@git.apache.org>.
Github user DaveBirdsall commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/326#discussion_r53225384
--- Diff: docs/provisioning_guide/src/asciidoc/_chapters/requirements.adoc ---
@@ -345,29 +386,28 @@ The following changes are required in `hdfs-site.xml`:
| dfs.namenode.acls.enabled | true | Enable HDFS POSIX Access Control Lists (ACLs).
|===
+[[requirements-hbase-changes]]
=== HBase Changes
-NOTE: These changes require a restart of ZooKeeper on all nodes in the cluster.
+NOTE: These changes require a restart of ZooKeeper HBase on all nodes in the cluster.
--- End diff --
Do you mean "ZooKeeper and HBase"?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request: Trafodion 1822
Posted by DaveBirdsall <gi...@git.apache.org>.
Github user DaveBirdsall commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/326#discussion_r53224031
--- Diff: docs/provisioning_guide/src/asciidoc/_chapters/introduction.adoc ---
@@ -0,0 +1,466 @@
+////
+/**
+* @@@ START COPYRIGHT @@@
+*
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing,
+* software distributed under the License is distributed on an
+* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+* KIND, either express or implied. See the License for the
+* specific language governing permissions and limitations
+* under the License.
+*
+* @@@ END COPYRIGHT @@@
+ */
+////
+
+[[introduction]]
+= Introduction
+
+Trafodion is a Hadoop add-on service that provides transactional SQL on top of HBase. Typically, you
+use Trafodion as the database for applications that require Online Transaction Processing (OLTP),
+Operational Data Store (ODS), and/or strong reporting capabilities. You access Trafodion using
+standard JDBC and ODBC APIs.
+
+You may choose whether to add Trafodion to an existing Hadoop environment or to create a standalone
+Hadoop environment specifically for Hadoop.
+
+This guide assumes that a Hadoop environment exists upon which your provisioning Trafodion. Refer to
+<<requirements-hadoop-software,Hadoop Software>> for information about what Hadoop software is required
+Trafodion.
+
+[[introduction-security-considerations]]
+== Security Considerations
+
+The following users need be considered for Trafodion:
+
+* *Provisioning User*: A Linux-level user that performs the Trafodion provisioning tasks. This user ID
+requires `sudo` access and passwordless ssh among the nodes where Trafodion is installed. In addition,
+this user ID requires access to Hadoop distribution, HDFS, and HBase administrative users to change
+respective environment's configuration settings per Trafodion requirements. Refer to
+<<requirements-trafodion-provisioning-user,Trafodion Provisioning User>> for more information
+about the requirements and usage associated with this user ID.
+
+* *Runtime User*: A linux-level user under which the Trafodion software runs. This user ID must be registered
+as a user in the Hadoop Distributed File System (HDFS) to store and access objects in HDFS, HBase, and Hive.
+In addition, this user ID requires passwordless access among the nodes where Trafodion is installed.
+Refer to <<requirements-trafodion-runtime-user,Trafodion Runtime User>> for more information about this user ID.
+
+* *Trafodion Database Users*: Trafodion users are managed by the Trafodion security features (grant, revoke, etc.),
+which can be integrated with LDAP if so desired. These users are referred to as *database users* and
+do not have direct access to the operating system. Refer to
+http://trafodion.apache.org/docs/sql_reference/index.html#register_user_statement[Register User],
+http://trafodion.apache.org/docs/sql_reference/index.html#grant_statement[Grant], and other SQL statements
+in the http://trafodion.apache.org/docs/sql_reference/index.html[Trafodion SQL Reference Manual] for
+more information about managing Trafodion Database Users.
+ +
+ +
+Optionally, you can enable Trafodion Security. If you do not enable security in Trafodion, then a client interface
+to Trafodion may request a user name and password, but Trafodion ignores the user name and password entered in the
+client interface, and the session runs as the database *root* user, `DB__ROOT`, without restrictions. If you want
+to restrict users, restrict access to certain users only, or restrict access to an object or operation, then you must
+enable security, which enforces authentication and authorization. Refer to
+<<enable-security,Enable Security>> for more information about this option.
+
+[[introduction-provisioning-options]]
+== Provisioning Options
+
+Trafodion ships with a set of scripts (the Trafodion Installer) that takes care of many of the installation and upgrade
+tasks associated with the Trafodion software and its requirements. There is a separate set of scripts to remove Trafodion,
+if needed.
+
+Currently, the Trafodion Installer is able to install Trafodion on select Cloudera and Hortonworks Hadoop distributions only.
+The Trafodion Installer limitations are noted as they apply in the different chapters below. For example, the Trafodion Installer
+is less capable on SUSE than it is on RedHat/CentOS; you have to install the prerequisite software packages outside the Trafodion Installer.
+
+The Trafodion Installer automates many of the tasks required to install/upgrade Trafodion, spanning from downloading and
+installing required software packages and making required changes to your Hadoop environment via creating
+the Trafodion runtime user ID to installing and starting Trafodion. It is, therefore, highly recommend that
+you use the Trafodion Installer for initial installation and upgrades of Trafodion. These steps are referred to as
+"Script-Based Provisioning" in this guide. Refer to <<introduction-trafodion-installer, Trafodion Installer>> provides
+usage information.
+
+If, for any reason, you choose not to use the Trafodion Installer, then separate chapters provide
+step-by-step recipes for the tasks required to install/upgrade Trafodion. These steps are referred to as
+*Recipe-Based Provisioning* in this guide. It is assumed that you are well-versed in Linux and Hadoop
+administrative tasks if using Recipe-Based Provisioning.
+
+[[introduction-provisioning-activities]]
+== Provisioning Activities
+
+Trafodion provisioning is divided into the following main activities:
+
+* *<<requirements,Requirements>>*: Activities and documentation required to install the Trafodion software.
+These activities include tasks such as understanding hardware and operating system requirements,
+Hadoop requirements, what software packages that need to be downloaded, configuration settings that need to be changed,
+user IDs requirements, and so on.
+
+* *<<prepare,Prepare>>*: Activities to prepare the operating system and the Hadoop ecosystem to run
+Trafodion. These activities include tasks such as installing required software packages, configure
+the Trafodion Installation User, gather information about the Hadoop environment, modify configuration
+for different Hadoop services, and so forth.
+
+* *<<install,Install>>*: Activities related to installing the Trafodion software. These activities
+include tasks such as unpacking the Trafodion tar files, creating the Trafodion Runtime User,
+creating Trafodion HDFS directories, installing the Trafodion software, and so forth.
+
+<<<
+* *<<upgrade,Upgrade>>*: Activities related to the upgrading the Trafodion software. These activities
+include tasks such as shutting down Trafodion, installing a new version of the Trafodion software,
+and so on. The upgrade tasks vary depending on the differences between the current and new release of
+Trafodion. For example, an upgrade may or may not include an upgrade of the Trafodion metadata.
+
+* *<<activate,Activate>>*: Activities related to starting the Trafodion software. These actives
+include basic management tasks such as starting and checking the status of the Trafodion components,
+performing basic smoke tests, and so forth.
+
+* *<<remove,Remove>>*: Activities related to removing Trafodion from your Hadoop cluster.
+
+[[introduction-provisioning-master-node]]
+== Provisioning Master Node
+All provisioning tasks are performed from a single node in the cluster, which must be part
+of the Hadoop environment you're adding Trafodion to. This node is referred to as the
+"*Provisioning Master Node*" in this guide.
+
+The Trafodion Provisioning User must have access to all other nodes from the Provisioning
+Master Node in order to perform provisioning tasks on the cluster.
+
+[[introduction-trafodion-installer]]
+== Trafodion Installer
+
+The Trafodion Installer is a set of scripts automates most of the tasks requires to install/upgrade Trafodion.
+You download the Trafodion Installer tar file from the Trafodion http://trafodion.apache.org/download.html[download] page.
+Next, you unpack the tar file.
+
+*Example*
+
+```
+$ mkdir $HOME/trafodion-installer
+$ cd $HOME/trafodion-downloads
+$ tar -zxf apache-trafodion-installer-1.3.0-incubating-bin.tar.gz -C $HOME/trafodion-installer
+$ ls $HOME/trafodion-installer/installer
+bashrc_default tools traf_config_check trafodion_apache_hadoop_install traf_package_setup
+build-version-1.3.0.txt traf_add_user traf_config_setup trafodion_config_default traf_setup
+dcs_installer traf_apache_hadoop_config_setup traf_create_systemdefaults trafodion_install traf_sqconfig
+rest_installer traf_authentication_conf_default traf_getHadoopNodes trafodion_license traf_start
+setup_known_hosts.exp traf_cloudera_mods98 traf_hortonworks_mods98 trafodion_uninstaller
+$
+```
+
+<<<
+The Trafodion Installer supports two different modes:
+
+1. *Guided Setup*: Prompts for information as it works through the installation/upgrade process. This mode is recommended for new users.
+2. *Automated Setup*: Required information is provided in a pre-formatted bash-script configuration file, which is provided
+via a command argument when running the Trafodion Installer thereby suppressing all prompts.
++
+A template of the configuration file is available here within the installer directory: `trafodion_config_default`.
+Make a copy of the file in your directory and populate the needed information.
++
+Automated Setup is recommended since it allows you to record the required provisioning information information ahead of time.
+Refer to <<introduction-trafodion-automated-setup,Automated Setup>> for information about how to
+populate this file.
+
+[[introduction-trafodion-installer-usage]]
+=== Usage
+
+The following shows help for the Trafodion Installer.
+
+```
+./trafodion_install --help
+
+This script will install Trafodion. It will create a configuration
+file (if one has not been created), setup of the environment needed
+for Trafodion, configure HBase with Hbase-trx and co-processors needed,
+and install a specified Trafodion build.
+
+Options:
+ --help Print this message and exit
+ --accept_license If provided, the user agrees to accept all the
+ provisions in the Trafodion license. This allows
+ for automation by skipping the display and prompt of
+ the Trafodion license.
+ --config_file If provided, all install prompts will be
+ taken from this file and not prompted for.
+```
+
+<<<
+[[introduction-trafodion-installer-install-vs-upgrade]]
+=== Install vs. Upgrade
+
+The Trafodion Installer automatically detects whether you're performing an install
+or an upgrade by looking for the Trafodion Runtime User in the `/etc/passwd` file.
+
+* If the user ID doesn't exists, then the Trafodion Installer runs in install mode.
--- End diff --
Typo, should be "doesn't exist"
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request: Trafodion 1822
Posted by gtapper <gi...@git.apache.org>.
Github user gtapper commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/326#discussion_r53246264
--- Diff: docs/provisioning_guide/src/asciidoc/_chapters/requirements.adoc ---
@@ -345,29 +386,28 @@ The following changes are required in `hdfs-site.xml`:
| dfs.namenode.acls.enabled | true | Enable HDFS POSIX Access Control Lists (ACLs).
|===
+[[requirements-hbase-changes]]
=== HBase Changes
-NOTE: These changes require a restart of ZooKeeper on all nodes in the cluster.
+NOTE: These changes require a restart of ZooKeeper HBase on all nodes in the cluster.
--- End diff --
I did. Fixed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request: Trafodion 1822
Posted by gtapper <gi...@git.apache.org>.
Github user gtapper commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/326#discussion_r53245545
--- Diff: docs/provisioning_guide/src/asciidoc/_chapters/introduction.adoc ---
@@ -0,0 +1,466 @@
+////
+/**
+* @@@ START COPYRIGHT @@@
+*
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing,
+* software distributed under the License is distributed on an
+* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+* KIND, either express or implied. See the License for the
+* specific language governing permissions and limitations
+* under the License.
+*
+* @@@ END COPYRIGHT @@@
+ */
+////
+
+[[introduction]]
+= Introduction
+
+Trafodion is a Hadoop add-on service that provides transactional SQL on top of HBase. Typically, you
+use Trafodion as the database for applications that require Online Transaction Processing (OLTP),
+Operational Data Store (ODS), and/or strong reporting capabilities. You access Trafodion using
+standard JDBC and ODBC APIs.
+
+You may choose whether to add Trafodion to an existing Hadoop environment or to create a standalone
+Hadoop environment specifically for Hadoop.
+
+This guide assumes that a Hadoop environment exists upon which your provisioning Trafodion. Refer to
+<<requirements-hadoop-software,Hadoop Software>> for information about what Hadoop software is required
+Trafodion.
+
+[[introduction-security-considerations]]
+== Security Considerations
+
+The following users need be considered for Trafodion:
+
+* *Provisioning User*: A Linux-level user that performs the Trafodion provisioning tasks. This user ID
+requires `sudo` access and passwordless ssh among the nodes where Trafodion is installed. In addition,
+this user ID requires access to Hadoop distribution, HDFS, and HBase administrative users to change
+respective environment's configuration settings per Trafodion requirements. Refer to
+<<requirements-trafodion-provisioning-user,Trafodion Provisioning User>> for more information
+about the requirements and usage associated with this user ID.
+
+* *Runtime User*: A linux-level user under which the Trafodion software runs. This user ID must be registered
+as a user in the Hadoop Distributed File System (HDFS) to store and access objects in HDFS, HBase, and Hive.
+In addition, this user ID requires passwordless access among the nodes where Trafodion is installed.
+Refer to <<requirements-trafodion-runtime-user,Trafodion Runtime User>> for more information about this user ID.
+
+* *Trafodion Database Users*: Trafodion users are managed by the Trafodion security features (grant, revoke, etc.),
+which can be integrated with LDAP if so desired. These users are referred to as *database users* and
+do not have direct access to the operating system. Refer to
+http://trafodion.apache.org/docs/sql_reference/index.html#register_user_statement[Register User],
+http://trafodion.apache.org/docs/sql_reference/index.html#grant_statement[Grant], and other SQL statements
+in the http://trafodion.apache.org/docs/sql_reference/index.html[Trafodion SQL Reference Manual] for
+more information about managing Trafodion Database Users.
+ +
+ +
+Optionally, you can enable Trafodion Security. If you do not enable security in Trafodion, then a client interface
+to Trafodion may request a user name and password, but Trafodion ignores the user name and password entered in the
+client interface, and the session runs as the database *root* user, `DB__ROOT`, without restrictions. If you want
+to restrict users, restrict access to certain users only, or restrict access to an object or operation, then you must
+enable security, which enforces authentication and authorization. Refer to
+<<enable-security,Enable Security>> for more information about this option.
+
+[[introduction-provisioning-options]]
+== Provisioning Options
+
+Trafodion ships with a set of scripts (the Trafodion Installer) that takes care of many of the installation and upgrade
+tasks associated with the Trafodion software and its requirements. There is a separate set of scripts to remove Trafodion,
+if needed.
+
+Currently, the Trafodion Installer is able to install Trafodion on select Cloudera and Hortonworks Hadoop distributions only.
+The Trafodion Installer limitations are noted as they apply in the different chapters below. For example, the Trafodion Installer
+is less capable on SUSE than it is on RedHat/CentOS; you have to install the prerequisite software packages outside the Trafodion Installer.
+
+The Trafodion Installer automates many of the tasks required to install/upgrade Trafodion, spanning from downloading and
+installing required software packages and making required changes to your Hadoop environment via creating
+the Trafodion runtime user ID to installing and starting Trafodion. It is, therefore, highly recommend that
+you use the Trafodion Installer for initial installation and upgrades of Trafodion. These steps are referred to as
+"Script-Based Provisioning" in this guide. Refer to <<introduction-trafodion-installer, Trafodion Installer>> provides
+usage information.
+
+If, for any reason, you choose not to use the Trafodion Installer, then separate chapters provide
+step-by-step recipes for the tasks required to install/upgrade Trafodion. These steps are referred to as
+*Recipe-Based Provisioning* in this guide. It is assumed that you are well-versed in Linux and Hadoop
+administrative tasks if using Recipe-Based Provisioning.
+
+[[introduction-provisioning-activities]]
+== Provisioning Activities
+
+Trafodion provisioning is divided into the following main activities:
+
+* *<<requirements,Requirements>>*: Activities and documentation required to install the Trafodion software.
+These activities include tasks such as understanding hardware and operating system requirements,
+Hadoop requirements, what software packages that need to be downloaded, configuration settings that need to be changed,
+user IDs requirements, and so on.
+
+* *<<prepare,Prepare>>*: Activities to prepare the operating system and the Hadoop ecosystem to run
+Trafodion. These activities include tasks such as installing required software packages, configure
+the Trafodion Installation User, gather information about the Hadoop environment, modify configuration
+for different Hadoop services, and so forth.
+
+* *<<install,Install>>*: Activities related to installing the Trafodion software. These activities
+include tasks such as unpacking the Trafodion tar files, creating the Trafodion Runtime User,
+creating Trafodion HDFS directories, installing the Trafodion software, and so forth.
+
+<<<
+* *<<upgrade,Upgrade>>*: Activities related to the upgrading the Trafodion software. These activities
+include tasks such as shutting down Trafodion, installing a new version of the Trafodion software,
+and so on. The upgrade tasks vary depending on the differences between the current and new release of
+Trafodion. For example, an upgrade may or may not include an upgrade of the Trafodion metadata.
+
+* *<<activate,Activate>>*: Activities related to starting the Trafodion software. These actives
+include basic management tasks such as starting and checking the status of the Trafodion components,
+performing basic smoke tests, and so forth.
+
+* *<<remove,Remove>>*: Activities related to removing Trafodion from your Hadoop cluster.
+
+[[introduction-provisioning-master-node]]
+== Provisioning Master Node
+All provisioning tasks are performed from a single node in the cluster, which must be part
+of the Hadoop environment you're adding Trafodion to. This node is referred to as the
+"*Provisioning Master Node*" in this guide.
+
+The Trafodion Provisioning User must have access to all other nodes from the Provisioning
+Master Node in order to perform provisioning tasks on the cluster.
+
+[[introduction-trafodion-installer]]
+== Trafodion Installer
+
+The Trafodion Installer is a set of scripts automates most of the tasks requires to install/upgrade Trafodion.
+You download the Trafodion Installer tar file from the Trafodion http://trafodion.apache.org/download.html[download] page.
+Next, you unpack the tar file.
+
+*Example*
+
+```
+$ mkdir $HOME/trafodion-installer
+$ cd $HOME/trafodion-downloads
+$ tar -zxf apache-trafodion-installer-1.3.0-incubating-bin.tar.gz -C $HOME/trafodion-installer
+$ ls $HOME/trafodion-installer/installer
+bashrc_default tools traf_config_check trafodion_apache_hadoop_install traf_package_setup
+build-version-1.3.0.txt traf_add_user traf_config_setup trafodion_config_default traf_setup
+dcs_installer traf_apache_hadoop_config_setup traf_create_systemdefaults trafodion_install traf_sqconfig
+rest_installer traf_authentication_conf_default traf_getHadoopNodes trafodion_license traf_start
+setup_known_hosts.exp traf_cloudera_mods98 traf_hortonworks_mods98 trafodion_uninstaller
+$
+```
+
+<<<
+The Trafodion Installer supports two different modes:
+
+1. *Guided Setup*: Prompts for information as it works through the installation/upgrade process. This mode is recommended for new users.
+2. *Automated Setup*: Required information is provided in a pre-formatted bash-script configuration file, which is provided
+via a command argument when running the Trafodion Installer thereby suppressing all prompts.
++
+A template of the configuration file is available here within the installer directory: `trafodion_config_default`.
+Make a copy of the file in your directory and populate the needed information.
++
+Automated Setup is recommended since it allows you to record the required provisioning information information ahead of time.
--- End diff --
Yep. We have a plan to add more discovery to reduce the number of items in the config file. 2.0 maybe.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request: Trafodion 1822
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/incubator-trafodion/pull/326
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request: Trafodion 1822
Posted by DaveBirdsall <gi...@git.apache.org>.
Github user DaveBirdsall commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/326#discussion_r53220722
--- Diff: docs/provisioning_guide/src/asciidoc/_chapters/about.adoc ---
@@ -24,19 +24,22 @@
////
= About This Document
-This guide describes how to provision Trafodion for end-user environments. This install allows you to store and query data using Trafodion, either via Trafodion clients
+This guide describes how to provision the end-user Trafodion binaries ontop of an existing Hadoop environment. This install allows you to store and query data using Trafodion, either via Trafodion clients
--- End diff --
Typo, "ontop"
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request: Trafodion 1822
Posted by DaveBirdsall <gi...@git.apache.org>.
Github user DaveBirdsall commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/326#discussion_r53224885
--- Diff: docs/provisioning_guide/src/asciidoc/_chapters/prepare.adoc ---
@@ -0,0 +1,379 @@
+////
+/**
+* @@@ START COPYRIGHT @@@
+*
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing,
+* software distributed under the License is distributed on an
+* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+* KIND, either express or implied. See the License for the
+* specific language governing permissions and limitations
+* under the License.
+*
+* @@@ END COPYRIGHT @@@
+ */
+////
+
+[[prepare]]
+= Prepare
+You need to prepare your Hadoop environment before installing Trafodion.
+
+1. <<prepare-install-optional-workstation-software,Install Optional Workstation Software>>
+2. <<configure-installation-user-id,Configure Installation User ID>>
+3. <<prepare-disable-requiretty,Disable requiretty>>
+4. <<prepare-verify-os-requirements-and-recommendations,Verify OS Requirements and Recommendations>>
+5. <<prepare-configure-ldap-identity-store,Configure LDAP Identity Store>>
+6. <<prepare-gather-configuration-information,Gather Configuration Information>>
+7. <<prepare-install-required-software-packages,Install Required Software Packages>>
+8. <<prepare-perform-recipe-based-provisioning-tasks,Perform Recipe-Based Provisioning Tasks>>
+
+[[prepare-install-optional-workstation-software]]
+== Install Optional Workstation Software
+
+If you are using a Windows workstation, then the following optional software helps installation process.
+We recommended that you pre-install the software before continuing with the Trafodion installation:
+
+* putty and puttygen (download from http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html[PuTTY web site])
+* VNC client (download from http://www.realvnc.com[RealVNC web site])
+* Firefox or Chrome browser
+* SFTP client to transfer files from your workstation to the Linux server: WinSCP or FileZilla
+
+[[configure-installation-user-id]]
+== Configure Installation User ID
+
+Trafodion installation requires a user ID with these attributes:
+
+* `sudo` access per the requirements documented in <<requirements-linux-installation-user,Linux Installation User>>.
+* passwordless ssh to all nodes on the cluster where Trafodion will be installed.
+
+NOTE: You may need to request permission from your cluster-management team to obtain this type of access.
+
+The following example shows how to set up your user ID to have "passwordless ssh" abilities.
+
+Do the following on the Provision Master Node:
+
+```
+echo -e 'y\n' | ssh-keygen -t rsa -N "" -f $HOME/.ssh/id_rsa
+cat $HOME/.ssh/id_rsa.pub >> $HOME/.ssh/authorized_keys
+echo localhost $(cat /etc/ssh/ssh_host_rsa_key.pub) >> $HOME/.ssh/known_hosts
+echo "NoHostAuthenticationForLocalhost=yes" >> $HOME/.ssh/config
+chmod 600 $HOME/.ssh/config
+```
+
+After running these commands, do the following:
+
+* Copy the content of the `$HOME/.ssh/id_rsa.pub` file (public key) and append it to the
+`$HOME/.ssh/authorized_keys` file on the other nodes in your cluster.
+* Copy the `$HOME/.ssh/id_rsa` file (private key) from the current node to the `$HOME/.ssh` directory on the other nodes in the cluster.
+* Secure `$HOME/.ssh/id_rsa` on all nodes as private to yourself (`chmod 700`).
+* `ssh` to the other nodes in the cluster. Answer `y` to the prompt asking you whether to continue the connection.
+This adds the node to the `$HOME/.ssh/known_hosts` file completing the passwordless ssh setup.
+
+
+[[prepare-disable-requiretty]]
+== Disable requiretty
+You need to disable `requiretty` in `/etc/sudoers` on all nodes in the cluster
+to ensure that `sudo` commands can be run from inside the installation scripts.
+
+Comment out the `Defaults requiretty` setting in the `/etc/sudoers` file to
+ensure that the `requiretty` option is NOT being used.
+
+[[prepare-verify-os-requirements-and-recommendations]]
+== Verify OS Requirements and Recommendations
+
+Please ensure that the <<requirements-os-requirements-and-recommendations,OS Requirements and Recommendations>>
+are met for each node in the cluster where you intend to install Trafodion.
+
+<<<
+[[prepare-configure-ldap-identity-store]]
+== Configure LDAP Identity Store
+
+If you plan to enable security in Trafodion, then you need to have an LDAP identity store available to perform authentication.
+The Trafodion Installer prompts you to set up an authentication configuration file that points to an LDAP server (or servers),
+which enables security (that is, authentication and authorization) in the Trafodion database.
+
+If you wish to manually set up the authentication configuration file and enable security, then refer to the section on
+<<enable-security,Enable Security>>.
+
+[[prepare-gather-configuration-information]]
+== Gather Configuration Information
+
+You need to gather/decide information about your environment to aid installation Trafodion, both for the Trafodion Installer
+and for recipe-based provisioning. (In alphabetical order to make it easier to find information later on in the provisioning process.)
+
+[cols="25%l,25%,15%l,35%",options="header"]
+|===
+| ID^1^ | Information | Default | Notes
+| ADMIN | Administrator user name for Apache Ambari or Cloudera Manager. | admin | A user that can change configuration and restart services via the
+distribution manager's REST API.
+| BACKUP_DCS_NODES | List of nodes where to start the backup DCS Master components. | None | Blank separated FQDN list. Not needed if $ENABLE_HA = N.
+| CLOUD_CONFIG | Whether you're installing Trafodion on a cloud environment. | N | N = bare-metal or VM installation.
+| CLOUD_TYPE | What type of cloud environment you're installing Trafodion on. | None | { AWS \| OpenStack \| Other }. Not applicable for bare-metal or VM installation.
+| CLUSTER_NAME | The name of the Hadoop Cluster. | None | From Apache Ambari or Cloudera Manager.
+| DCS_BUILD | Tar file containing the DCS component. | None | Not needed if using a Trafodion package installation tar file.
+| DCS_PRIMARY_MASTER_NODE | The node where the primary DCS should run. | None | The DCS Master handles JDBC and ODBC connection requests.
+| DCS_SERVER_PARM | Number of concurrent client sessions per node. | 16 | This number specifies the concurrent sessions per node to be supported. Each session could require up to 1GB of physical memory. The number can be changed post-installation. For more information,
+refer to the http://trafodion.apache.org/docs/client_install/index.html[Trafodion Client Installation Guide].
+| ENABLE_HA | Whether to run DCS in high-availability (HA) mode. | N | You need the floating IP address, the interface, and the backup nodes for DCS Master if enabling this feature.
+| EPEL_RPM | Location of EPEL RPM. | None | Specify if you don't have access to the Internet.
+Downloaded automatically by the Trafodion Installer.
+| FLOATING_IP | IP address if running DCS in HA mode. | None | Not needed if $ENABLE_HA = N. An FQDN name or IP address.
+| HADOOP_TYPE | The type of Hadoop distribution you're installing Trafodion on. | None | Lowercase. cloudera or hadoop.
+| HBASE_GROUP | Linux group name for the HBASE administrative user. | hbase | Required in order to provide access to select HDFS directories to this user ID.
+| HBASE_USER | Linux user name for the HBASE administrative user. | hbase | Required in order to provide access to select HDFS directories to this user ID.
+| HDFS_USER | Linux user name for the HDFS administrative user. | hdfs | The Trafodion Installer uses `sudo su` to make HDFS
+configuration changes under this user.
+| HOME_DIR | Root directory under which the `trafodion` home directory should be created. | /home | *Example* +
+ +
+If the home directory of the `trafodion` user is
+`/opt/home/trafodion`, then specify the root directory as `/opt/home`.
+| INIT_TRAFODION | Whether to automatically initialize the Trafodion database. | N | Does not apply to Recipe-Based Provisioning. Applies if $START=Y only.
+| INTERFACE | Interface type used for $FLOATING_IP. | None | Not needed if $ENABLE_HA = N.
+| JAVA_HOME | Location of Java 1.7.0_65 or higher (JDK). | $JAVA_HOME setting | Fully qualified path of the JDK. For example:
+`/usr/java/jdk1.7.0_67-cloudera`
+| LDAP_CERT^2^ | Full path to TLS certificate. | None | Required of $LDAP_LEVEL = 1 or 2.
+| LDAP_HOSTS^2^ | List of nodes where LDAP Identity Store servers are running. | None | Blank separated. FQDN format.
+| LDAP_ID^2^ | List of LDAP unique identifiers. | None | Blank separated.
+| LDAP_LEVEL^2^ | LDAP Encryption Level. | 0 | 0: Encryption not used, 1: SSL, 2: TLS
+| LDAP_PASSWORD^2^ | Password for LDAP_USER. | None | If LDAP_USER is required only.
+| LDAP_PORT^2^ | Port used to communicate with LDAP Identity Store. | None | Examples: 389 for no encryption or TLS, 636 for SSL.
+| LDAP_SECURITY^2^ | Whether to enable simple LDAP authentication. | N | If Y, then you need to provide LDAP_HOSTS.
+| LDAP_USER^2^ | LDAP Search user name. | None | If required. If so, must provide LDAP_PASSWORD, too.
+| LOCAL_WORKDIR | The directory where the Trafodion Installer is located. | None | Full path, no environmental variables.
+| MANAGEMENT_ENABLED | Whether your installation uses separate management nodes. | N | Y if using separate management nodes for Apache Ambari or Cloudera Manager.
+| MANAGEMENT_NODES | The FQDN names of management nodes, if any. | None | Provide a blank-separated list of node names.
+| NODE_LIST | The FQDN names of the nodes where Trafodion will be installed. | None | Provide a blank-separated list of node names. The Trafodion
+Provisioning ID must have passwordless and `sudo` access to these nodes.
+| PASSWORD | Administrator password for Apache Ambari or Cloudera Manager. | admin | A user that can change configuration and restart services via the
+distribution manager's REST API.
+| REST_BUILD | Tar file containing the REST component. | None | Not needed if using a Trafodion package installation tar file.
+| SQ_ROOT | Target directory for the Trafodion software. | $HOME_DIR/trafodion | Trafodion is installed in this directory on all nodes in `$NODE_LIST`.
+| START | Whether to start Trafodion after install/upgrade. | N | Does not apply to Recipe-Based Provisioning.
+| SUSE_LINUX | Whether your installing Trafodion on SUSE Linux. | false | Auto-detected by the Trafodion Installer.
+| TRAF_PACKAGE | The location of the Trafodion installation package tar file or core installation tar file. | None | The package file contains the Trafodion server,
+DCS, and REST software while the core installation file contains the Trafodion server software only. If you're using a core installation file, then you need to
+record the location of the DCS and REST installation tar files, too. Normally, you perform Trafodion provisioning using a Trafodion package installation tar file.
+| TRAF_USER | The Trafodion runtime user ID. | trafodion | Must be `trafodion` in this release.
+| TRAF_USER_PASSWORD | The password used for the `trafodion:trafodion` user ID. | traf123 | Must be 6-8 characters long.
+| URL | FQDN and port for the Distribution Manager's REST API. | None | Include `http://` or `https://` as applicable. Specify in the form:
+`<IP-address>:<port>` or `<node name>:<port>` Example: `https://susevm-1.yourcompany.local:8080`
+|===
+
+1. The ID matches the environmental variables used in the Trafodion Installation configuration file. Refer to <<install-trafodion-installer,Trafodion Installer>>
+for more information.
+2. Refer to <<enable-security,Enable Security>> for more information about these security settings.
+
+
+<<<
+[[prepare-install-required-software-packages]]
+== Install Required Software Packages
+
+[[prepare-download-and-install-packages]]
+=== Download and Install Packages
+
+The step is required if you're:
+
+* Installing Trafodion on SUSE.
+* Using Recipe-Based Provisioning.
+* Can't download the required software packages using the Internet.
+
+If none of these situations exist, then we highly recommend that you use the Trafodion Installer.
+
+You perform this step as a user with `root` or `sudo` access.
+
+Install the packages listed in <<requirements-software-packages,Software Packages>> above on all nodes in the cluster. Note the special
+handling for `log4c++`. See <<prepare-build-and-install-log4cplusplus, Build and Install log4c++>> below for more information.
--- End diff --
Three different ways to render "++"! :-)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request: Trafodion 1822
Posted by DaveBirdsall <gi...@git.apache.org>.
Github user DaveBirdsall commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/326#discussion_r53234517
--- Diff: docs/provisioning_guide/src/asciidoc/_chapters/script_install.adoc ---
@@ -0,0 +1,511 @@
+////
+/**
+* @@@ START COPYRIGHT @@@
+*
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing,
+* software distributed under the License is distributed on an
+* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+* KIND, either express or implied. See the License for the
+* specific language governing permissions and limitations
+* under the License.
+*
+* @@@ END COPYRIGHT @@@
+ */
+////
+
+[[install]]
+= Install
+
+This chapter describes how to use the Trafodion Installer to install Trafodion. You use the Trafodion Provisioning ID
+to run the Trafodion Installer.
+
+NOTE: You *must* install log4c++ on all nodes in the cluster prior to running the Trafodion Installer. Refer
+to <<prepare-build-and-install-log4cplusplus,Build and Install log4c++>> for instructions.
+
+[[install-unpack-installer]]
+== Unpack Installer
+
+You should already have downloaded the Trafodion Binaries per the instructions in the
+<<prepare-download-trafodion-binaries,Download Trafodion Binaries>> in the <<prepare,Prepare>>
+chapter. If not, please do so now.
+
+The first step in the installation process is to unpack the Trafodion Installer tar file.
+
+*Example*
+
+```
+$ mkdir $HOME/trafodion-installer
+$ cd $HOME/trafodion-downloads
+$ tar -zxf apache-trafodion-installer-1.3.0-incubating-bin.tar.gz -C $HOME/trafodion-installer
+$ ls $HOME/trafodion-installer/installer
+bashrc_default tools traf_config_check trafodion_apache_hadoop_install traf_package_setup
+build-version-1.3.0.txt traf_add_user traf_config_setup trafodion_config_default traf_setup
+dcs_installer traf_apache_hadoop_config_setup traf_create_systemdefaults trafodion_install traf_sqconfig
+rest_installer traf_authentication_conf_default traf_getHadoopNodes trafodion_license traf_start
+setup_known_hosts.exp traf_cloudera_mods98 traf_hortonworks_mods98 trafodion_uninstaller
+$
+```
+
+[[install-automated-install]]
+== Automated Install
+
+The `--config_file` option runs the Trafodion in Automated Setup mode. Refer to <<introduction-trafodion-installer,Trafodion Installer>>
+in the <<introduction,Introduction>> chapter for instructions of how you edit your configuration file.
+
+Edit your config file using the information you collected in the <<prepare-gather-configuration-information,Gather Configuration Information>>
+in the <<prepare,Prepare>> chapter.
--- End diff --
Do you want to say, "step" before "in the <<prepare,Prepare>> chapter"?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request: Trafodion 1822
Posted by DaveBirdsall <gi...@git.apache.org>.
Github user DaveBirdsall commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/326#discussion_r53223103
--- Diff: docs/provisioning_guide/src/asciidoc/_chapters/introduction.adoc ---
@@ -0,0 +1,466 @@
+////
+/**
+* @@@ START COPYRIGHT @@@
+*
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing,
+* software distributed under the License is distributed on an
+* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+* KIND, either express or implied. See the License for the
+* specific language governing permissions and limitations
+* under the License.
+*
+* @@@ END COPYRIGHT @@@
+ */
+////
+
+[[introduction]]
+= Introduction
+
+Trafodion is a Hadoop add-on service that provides transactional SQL on top of HBase. Typically, you
+use Trafodion as the database for applications that require Online Transaction Processing (OLTP),
+Operational Data Store (ODS), and/or strong reporting capabilities. You access Trafodion using
+standard JDBC and ODBC APIs.
+
+You may choose whether to add Trafodion to an existing Hadoop environment or to create a standalone
+Hadoop environment specifically for Hadoop.
+
+This guide assumes that a Hadoop environment exists upon which your provisioning Trafodion. Refer to
+<<requirements-hadoop-software,Hadoop Software>> for information about what Hadoop software is required
+Trafodion.
+
+[[introduction-security-considerations]]
+== Security Considerations
+
+The following users need be considered for Trafodion:
+
+* *Provisioning User*: A Linux-level user that performs the Trafodion provisioning tasks. This user ID
+requires `sudo` access and passwordless ssh among the nodes where Trafodion is installed. In addition,
+this user ID requires access to Hadoop distribution, HDFS, and HBase administrative users to change
+respective environment's configuration settings per Trafodion requirements. Refer to
+<<requirements-trafodion-provisioning-user,Trafodion Provisioning User>> for more information
+about the requirements and usage associated with this user ID.
+
+* *Runtime User*: A linux-level user under which the Trafodion software runs. This user ID must be registered
+as a user in the Hadoop Distributed File System (HDFS) to store and access objects in HDFS, HBase, and Hive.
+In addition, this user ID requires passwordless access among the nodes where Trafodion is installed.
+Refer to <<requirements-trafodion-runtime-user,Trafodion Runtime User>> for more information about this user ID.
+
+* *Trafodion Database Users*: Trafodion users are managed by the Trafodion security features (grant, revoke, etc.),
+which can be integrated with LDAP if so desired. These users are referred to as *database users* and
+do not have direct access to the operating system. Refer to
+http://trafodion.apache.org/docs/sql_reference/index.html#register_user_statement[Register User],
+http://trafodion.apache.org/docs/sql_reference/index.html#grant_statement[Grant], and other SQL statements
+in the http://trafodion.apache.org/docs/sql_reference/index.html[Trafodion SQL Reference Manual] for
+more information about managing Trafodion Database Users.
+ +
+ +
+Optionally, you can enable Trafodion Security. If you do not enable security in Trafodion, then a client interface
+to Trafodion may request a user name and password, but Trafodion ignores the user name and password entered in the
+client interface, and the session runs as the database *root* user, `DB__ROOT`, without restrictions. If you want
+to restrict users, restrict access to certain users only, or restrict access to an object or operation, then you must
+enable security, which enforces authentication and authorization. Refer to
+<<enable-security,Enable Security>> for more information about this option.
+
+[[introduction-provisioning-options]]
+== Provisioning Options
+
+Trafodion ships with a set of scripts (the Trafodion Installer) that takes care of many of the installation and upgrade
--- End diff --
Grammar, should be "take care"
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request: Trafodion 1822
Posted by DaveBirdsall <gi...@git.apache.org>.
Github user DaveBirdsall commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/326#discussion_r53224776
--- Diff: docs/provisioning_guide/src/asciidoc/_chapters/prepare.adoc ---
@@ -0,0 +1,379 @@
+////
+/**
+* @@@ START COPYRIGHT @@@
+*
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing,
+* software distributed under the License is distributed on an
+* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+* KIND, either express or implied. See the License for the
+* specific language governing permissions and limitations
+* under the License.
+*
+* @@@ END COPYRIGHT @@@
+ */
+////
+
+[[prepare]]
+= Prepare
+You need to prepare your Hadoop environment before installing Trafodion.
+
+1. <<prepare-install-optional-workstation-software,Install Optional Workstation Software>>
+2. <<configure-installation-user-id,Configure Installation User ID>>
+3. <<prepare-disable-requiretty,Disable requiretty>>
+4. <<prepare-verify-os-requirements-and-recommendations,Verify OS Requirements and Recommendations>>
+5. <<prepare-configure-ldap-identity-store,Configure LDAP Identity Store>>
+6. <<prepare-gather-configuration-information,Gather Configuration Information>>
+7. <<prepare-install-required-software-packages,Install Required Software Packages>>
+8. <<prepare-perform-recipe-based-provisioning-tasks,Perform Recipe-Based Provisioning Tasks>>
+
+[[prepare-install-optional-workstation-software]]
+== Install Optional Workstation Software
+
+If you are using a Windows workstation, then the following optional software helps installation process.
+We recommended that you pre-install the software before continuing with the Trafodion installation:
+
+* putty and puttygen (download from http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html[PuTTY web site])
+* VNC client (download from http://www.realvnc.com[RealVNC web site])
+* Firefox or Chrome browser
+* SFTP client to transfer files from your workstation to the Linux server: WinSCP or FileZilla
+
+[[configure-installation-user-id]]
+== Configure Installation User ID
+
+Trafodion installation requires a user ID with these attributes:
+
+* `sudo` access per the requirements documented in <<requirements-linux-installation-user,Linux Installation User>>.
+* passwordless ssh to all nodes on the cluster where Trafodion will be installed.
+
+NOTE: You may need to request permission from your cluster-management team to obtain this type of access.
+
+The following example shows how to set up your user ID to have "passwordless ssh" abilities.
+
+Do the following on the Provision Master Node:
+
+```
+echo -e 'y\n' | ssh-keygen -t rsa -N "" -f $HOME/.ssh/id_rsa
+cat $HOME/.ssh/id_rsa.pub >> $HOME/.ssh/authorized_keys
+echo localhost $(cat /etc/ssh/ssh_host_rsa_key.pub) >> $HOME/.ssh/known_hosts
+echo "NoHostAuthenticationForLocalhost=yes" >> $HOME/.ssh/config
+chmod 600 $HOME/.ssh/config
+```
+
+After running these commands, do the following:
+
+* Copy the content of the `$HOME/.ssh/id_rsa.pub` file (public key) and append it to the
+`$HOME/.ssh/authorized_keys` file on the other nodes in your cluster.
+* Copy the `$HOME/.ssh/id_rsa` file (private key) from the current node to the `$HOME/.ssh` directory on the other nodes in the cluster.
+* Secure `$HOME/.ssh/id_rsa` on all nodes as private to yourself (`chmod 700`).
+* `ssh` to the other nodes in the cluster. Answer `y` to the prompt asking you whether to continue the connection.
+This adds the node to the `$HOME/.ssh/known_hosts` file completing the passwordless ssh setup.
+
+
+[[prepare-disable-requiretty]]
+== Disable requiretty
+You need to disable `requiretty` in `/etc/sudoers` on all nodes in the cluster
+to ensure that `sudo` commands can be run from inside the installation scripts.
+
+Comment out the `Defaults requiretty` setting in the `/etc/sudoers` file to
+ensure that the `requiretty` option is NOT being used.
+
+[[prepare-verify-os-requirements-and-recommendations]]
+== Verify OS Requirements and Recommendations
+
+Please ensure that the <<requirements-os-requirements-and-recommendations,OS Requirements and Recommendations>>
+are met for each node in the cluster where you intend to install Trafodion.
+
+<<<
+[[prepare-configure-ldap-identity-store]]
+== Configure LDAP Identity Store
+
+If you plan to enable security in Trafodion, then you need to have an LDAP identity store available to perform authentication.
+The Trafodion Installer prompts you to set up an authentication configuration file that points to an LDAP server (or servers),
+which enables security (that is, authentication and authorization) in the Trafodion database.
+
+If you wish to manually set up the authentication configuration file and enable security, then refer to the section on
+<<enable-security,Enable Security>>.
+
+[[prepare-gather-configuration-information]]
+== Gather Configuration Information
+
+You need to gather/decide information about your environment to aid installation Trafodion, both for the Trafodion Installer
+and for recipe-based provisioning. (In alphabetical order to make it easier to find information later on in the provisioning process.)
+
+[cols="25%l,25%,15%l,35%",options="header"]
+|===
+| ID^1^ | Information | Default | Notes
+| ADMIN | Administrator user name for Apache Ambari or Cloudera Manager. | admin | A user that can change configuration and restart services via the
+distribution manager's REST API.
+| BACKUP_DCS_NODES | List of nodes where to start the backup DCS Master components. | None | Blank separated FQDN list. Not needed if $ENABLE_HA = N.
+| CLOUD_CONFIG | Whether you're installing Trafodion on a cloud environment. | N | N = bare-metal or VM installation.
+| CLOUD_TYPE | What type of cloud environment you're installing Trafodion on. | None | { AWS \| OpenStack \| Other }. Not applicable for bare-metal or VM installation.
+| CLUSTER_NAME | The name of the Hadoop Cluster. | None | From Apache Ambari or Cloudera Manager.
+| DCS_BUILD | Tar file containing the DCS component. | None | Not needed if using a Trafodion package installation tar file.
+| DCS_PRIMARY_MASTER_NODE | The node where the primary DCS should run. | None | The DCS Master handles JDBC and ODBC connection requests.
+| DCS_SERVER_PARM | Number of concurrent client sessions per node. | 16 | This number specifies the concurrent sessions per node to be supported. Each session could require up to 1GB of physical memory. The number can be changed post-installation. For more information,
+refer to the http://trafodion.apache.org/docs/client_install/index.html[Trafodion Client Installation Guide].
+| ENABLE_HA | Whether to run DCS in high-availability (HA) mode. | N | You need the floating IP address, the interface, and the backup nodes for DCS Master if enabling this feature.
+| EPEL_RPM | Location of EPEL RPM. | None | Specify if you don't have access to the Internet.
+Downloaded automatically by the Trafodion Installer.
+| FLOATING_IP | IP address if running DCS in HA mode. | None | Not needed if $ENABLE_HA = N. An FQDN name or IP address.
+| HADOOP_TYPE | The type of Hadoop distribution you're installing Trafodion on. | None | Lowercase. cloudera or hadoop.
+| HBASE_GROUP | Linux group name for the HBASE administrative user. | hbase | Required in order to provide access to select HDFS directories to this user ID.
+| HBASE_USER | Linux user name for the HBASE administrative user. | hbase | Required in order to provide access to select HDFS directories to this user ID.
+| HDFS_USER | Linux user name for the HDFS administrative user. | hdfs | The Trafodion Installer uses `sudo su` to make HDFS
+configuration changes under this user.
+| HOME_DIR | Root directory under which the `trafodion` home directory should be created. | /home | *Example* +
+ +
+If the home directory of the `trafodion` user is
+`/opt/home/trafodion`, then specify the root directory as `/opt/home`.
+| INIT_TRAFODION | Whether to automatically initialize the Trafodion database. | N | Does not apply to Recipe-Based Provisioning. Applies if $START=Y only.
+| INTERFACE | Interface type used for $FLOATING_IP. | None | Not needed if $ENABLE_HA = N.
+| JAVA_HOME | Location of Java 1.7.0_65 or higher (JDK). | $JAVA_HOME setting | Fully qualified path of the JDK. For example:
+`/usr/java/jdk1.7.0_67-cloudera`
+| LDAP_CERT^2^ | Full path to TLS certificate. | None | Required of $LDAP_LEVEL = 1 or 2.
+| LDAP_HOSTS^2^ | List of nodes where LDAP Identity Store servers are running. | None | Blank separated. FQDN format.
+| LDAP_ID^2^ | List of LDAP unique identifiers. | None | Blank separated.
+| LDAP_LEVEL^2^ | LDAP Encryption Level. | 0 | 0: Encryption not used, 1: SSL, 2: TLS
+| LDAP_PASSWORD^2^ | Password for LDAP_USER. | None | If LDAP_USER is required only.
+| LDAP_PORT^2^ | Port used to communicate with LDAP Identity Store. | None | Examples: 389 for no encryption or TLS, 636 for SSL.
+| LDAP_SECURITY^2^ | Whether to enable simple LDAP authentication. | N | If Y, then you need to provide LDAP_HOSTS.
+| LDAP_USER^2^ | LDAP Search user name. | None | If required. If so, must provide LDAP_PASSWORD, too.
+| LOCAL_WORKDIR | The directory where the Trafodion Installer is located. | None | Full path, no environmental variables.
+| MANAGEMENT_ENABLED | Whether your installation uses separate management nodes. | N | Y if using separate management nodes for Apache Ambari or Cloudera Manager.
+| MANAGEMENT_NODES | The FQDN names of management nodes, if any. | None | Provide a blank-separated list of node names.
+| NODE_LIST | The FQDN names of the nodes where Trafodion will be installed. | None | Provide a blank-separated list of node names. The Trafodion
+Provisioning ID must have passwordless and `sudo` access to these nodes.
+| PASSWORD | Administrator password for Apache Ambari or Cloudera Manager. | admin | A user that can change configuration and restart services via the
+distribution manager's REST API.
+| REST_BUILD | Tar file containing the REST component. | None | Not needed if using a Trafodion package installation tar file.
+| SQ_ROOT | Target directory for the Trafodion software. | $HOME_DIR/trafodion | Trafodion is installed in this directory on all nodes in `$NODE_LIST`.
+| START | Whether to start Trafodion after install/upgrade. | N | Does not apply to Recipe-Based Provisioning.
+| SUSE_LINUX | Whether your installing Trafodion on SUSE Linux. | false | Auto-detected by the Trafodion Installer.
+| TRAF_PACKAGE | The location of the Trafodion installation package tar file or core installation tar file. | None | The package file contains the Trafodion server,
+DCS, and REST software while the core installation file contains the Trafodion server software only. If you're using a core installation file, then you need to
+record the location of the DCS and REST installation tar files, too. Normally, you perform Trafodion provisioning using a Trafodion package installation tar file.
+| TRAF_USER | The Trafodion runtime user ID. | trafodion | Must be `trafodion` in this release.
+| TRAF_USER_PASSWORD | The password used for the `trafodion:trafodion` user ID. | traf123 | Must be 6-8 characters long.
+| URL | FQDN and port for the Distribution Manager's REST API. | None | Include `http://` or `https://` as applicable. Specify in the form:
+`<IP-address>:<port>` or `<node name>:<port>` Example: `https://susevm-1.yourcompany.local:8080`
+|===
+
+1. The ID matches the environmental variables used in the Trafodion Installation configuration file. Refer to <<install-trafodion-installer,Trafodion Installer>>
+for more information.
+2. Refer to <<enable-security,Enable Security>> for more information about these security settings.
+
+
+<<<
+[[prepare-install-required-software-packages]]
+== Install Required Software Packages
+
+[[prepare-download-and-install-packages]]
+=== Download and Install Packages
+
+The step is required if you're:
--- End diff --
Do you mean, "This step..."?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request: Trafodion 1822
Posted by DaveBirdsall <gi...@git.apache.org>.
Github user DaveBirdsall commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/326#discussion_r53225640
--- Diff: docs/provisioning_guide/src/asciidoc/_chapters/requirements.adoc ---
@@ -415,11 +456,34 @@ java.heapsize^1^ | 1073741824 (1GB) | Java Heap Size for the HDFS Secondary Name
1. Applies to Cloudera distributions only.
+[[requirements-recommended-configuration-changes]]
== Recommended Configuration Changes
The following configuration changes are recommended but not required.
NOTE: The Trafodion Installer does *not* make these changes.
+[[requirements-recommended-security-changes]]
+=== Recommended Security Changes
+
+The `trafodion` user ID should not be given other `sudo` privileges than what's specified in this manual. Also, we
+that this user ID is locked (`sudo passwd -l trafodion`) once the installation/upgrade activity has been completed.
--- End diff --
Missing verb? Did you mean, "Also, we checked that this user ID is locked..." Or perhaps you meant, "The Trafodion Installer checks that this user ID is locked..."
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request: Trafodion 1822
Posted by gtapper <gi...@git.apache.org>.
Github user gtapper commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/326#discussion_r53245872
--- Diff: docs/provisioning_guide/src/asciidoc/_chapters/introduction.adoc ---
@@ -0,0 +1,466 @@
+////
+/**
+* @@@ START COPYRIGHT @@@
+*
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing,
+* software distributed under the License is distributed on an
+* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+* KIND, either express or implied. See the License for the
+* specific language governing permissions and limitations
+* under the License.
+*
+* @@@ END COPYRIGHT @@@
+ */
+////
+
+[[introduction]]
+= Introduction
+
+Trafodion is a Hadoop add-on service that provides transactional SQL on top of HBase. Typically, you
+use Trafodion as the database for applications that require Online Transaction Processing (OLTP),
+Operational Data Store (ODS), and/or strong reporting capabilities. You access Trafodion using
+standard JDBC and ODBC APIs.
+
+You may choose whether to add Trafodion to an existing Hadoop environment or to create a standalone
+Hadoop environment specifically for Hadoop.
+
+This guide assumes that a Hadoop environment exists upon which your provisioning Trafodion. Refer to
+<<requirements-hadoop-software,Hadoop Software>> for information about what Hadoop software is required
+Trafodion.
+
+[[introduction-security-considerations]]
+== Security Considerations
+
+The following users need be considered for Trafodion:
+
+* *Provisioning User*: A Linux-level user that performs the Trafodion provisioning tasks. This user ID
+requires `sudo` access and passwordless ssh among the nodes where Trafodion is installed. In addition,
+this user ID requires access to Hadoop distribution, HDFS, and HBase administrative users to change
+respective environment's configuration settings per Trafodion requirements. Refer to
+<<requirements-trafodion-provisioning-user,Trafodion Provisioning User>> for more information
+about the requirements and usage associated with this user ID.
+
+* *Runtime User*: A linux-level user under which the Trafodion software runs. This user ID must be registered
+as a user in the Hadoop Distributed File System (HDFS) to store and access objects in HDFS, HBase, and Hive.
+In addition, this user ID requires passwordless access among the nodes where Trafodion is installed.
+Refer to <<requirements-trafodion-runtime-user,Trafodion Runtime User>> for more information about this user ID.
+
+* *Trafodion Database Users*: Trafodion users are managed by the Trafodion security features (grant, revoke, etc.),
+which can be integrated with LDAP if so desired. These users are referred to as *database users* and
+do not have direct access to the operating system. Refer to
+http://trafodion.apache.org/docs/sql_reference/index.html#register_user_statement[Register User],
+http://trafodion.apache.org/docs/sql_reference/index.html#grant_statement[Grant], and other SQL statements
+in the http://trafodion.apache.org/docs/sql_reference/index.html[Trafodion SQL Reference Manual] for
+more information about managing Trafodion Database Users.
+ +
+ +
+Optionally, you can enable Trafodion Security. If you do not enable security in Trafodion, then a client interface
+to Trafodion may request a user name and password, but Trafodion ignores the user name and password entered in the
+client interface, and the session runs as the database *root* user, `DB__ROOT`, without restrictions. If you want
+to restrict users, restrict access to certain users only, or restrict access to an object or operation, then you must
+enable security, which enforces authentication and authorization. Refer to
+<<enable-security,Enable Security>> for more information about this option.
+
+[[introduction-provisioning-options]]
+== Provisioning Options
+
+Trafodion ships with a set of scripts (the Trafodion Installer) that takes care of many of the installation and upgrade
+tasks associated with the Trafodion software and its requirements. There is a separate set of scripts to remove Trafodion,
+if needed.
+
+Currently, the Trafodion Installer is able to install Trafodion on select Cloudera and Hortonworks Hadoop distributions only.
+The Trafodion Installer limitations are noted as they apply in the different chapters below. For example, the Trafodion Installer
+is less capable on SUSE than it is on RedHat/CentOS; you have to install the prerequisite software packages outside the Trafodion Installer.
+
+The Trafodion Installer automates many of the tasks required to install/upgrade Trafodion, spanning from downloading and
+installing required software packages and making required changes to your Hadoop environment via creating
+the Trafodion runtime user ID to installing and starting Trafodion. It is, therefore, highly recommend that
+you use the Trafodion Installer for initial installation and upgrades of Trafodion. These steps are referred to as
+"Script-Based Provisioning" in this guide. Refer to <<introduction-trafodion-installer, Trafodion Installer>> provides
+usage information.
+
+If, for any reason, you choose not to use the Trafodion Installer, then separate chapters provide
+step-by-step recipes for the tasks required to install/upgrade Trafodion. These steps are referred to as
+*Recipe-Based Provisioning* in this guide. It is assumed that you are well-versed in Linux and Hadoop
+administrative tasks if using Recipe-Based Provisioning.
+
+[[introduction-provisioning-activities]]
+== Provisioning Activities
+
+Trafodion provisioning is divided into the following main activities:
+
+* *<<requirements,Requirements>>*: Activities and documentation required to install the Trafodion software.
+These activities include tasks such as understanding hardware and operating system requirements,
+Hadoop requirements, what software packages that need to be downloaded, configuration settings that need to be changed,
+user IDs requirements, and so on.
+
+* *<<prepare,Prepare>>*: Activities to prepare the operating system and the Hadoop ecosystem to run
+Trafodion. These activities include tasks such as installing required software packages, configure
+the Trafodion Installation User, gather information about the Hadoop environment, modify configuration
+for different Hadoop services, and so forth.
+
+* *<<install,Install>>*: Activities related to installing the Trafodion software. These activities
+include tasks such as unpacking the Trafodion tar files, creating the Trafodion Runtime User,
+creating Trafodion HDFS directories, installing the Trafodion software, and so forth.
+
+<<<
+* *<<upgrade,Upgrade>>*: Activities related to the upgrading the Trafodion software. These activities
+include tasks such as shutting down Trafodion, installing a new version of the Trafodion software,
+and so on. The upgrade tasks vary depending on the differences between the current and new release of
+Trafodion. For example, an upgrade may or may not include an upgrade of the Trafodion metadata.
+
+* *<<activate,Activate>>*: Activities related to starting the Trafodion software. These actives
+include basic management tasks such as starting and checking the status of the Trafodion components,
+performing basic smoke tests, and so forth.
+
+* *<<remove,Remove>>*: Activities related to removing Trafodion from your Hadoop cluster.
+
+[[introduction-provisioning-master-node]]
+== Provisioning Master Node
+All provisioning tasks are performed from a single node in the cluster, which must be part
+of the Hadoop environment you're adding Trafodion to. This node is referred to as the
+"*Provisioning Master Node*" in this guide.
+
+The Trafodion Provisioning User must have access to all other nodes from the Provisioning
+Master Node in order to perform provisioning tasks on the cluster.
+
+[[introduction-trafodion-installer]]
+== Trafodion Installer
+
+The Trafodion Installer is a set of scripts automates most of the tasks requires to install/upgrade Trafodion.
+You download the Trafodion Installer tar file from the Trafodion http://trafodion.apache.org/download.html[download] page.
+Next, you unpack the tar file.
+
+*Example*
+
+```
+$ mkdir $HOME/trafodion-installer
+$ cd $HOME/trafodion-downloads
+$ tar -zxf apache-trafodion-installer-1.3.0-incubating-bin.tar.gz -C $HOME/trafodion-installer
+$ ls $HOME/trafodion-installer/installer
+bashrc_default tools traf_config_check trafodion_apache_hadoop_install traf_package_setup
+build-version-1.3.0.txt traf_add_user traf_config_setup trafodion_config_default traf_setup
+dcs_installer traf_apache_hadoop_config_setup traf_create_systemdefaults trafodion_install traf_sqconfig
+rest_installer traf_authentication_conf_default traf_getHadoopNodes trafodion_license traf_start
+setup_known_hosts.exp traf_cloudera_mods98 traf_hortonworks_mods98 trafodion_uninstaller
+$
+```
+
+<<<
+The Trafodion Installer supports two different modes:
+
+1. *Guided Setup*: Prompts for information as it works through the installation/upgrade process. This mode is recommended for new users.
+2. *Automated Setup*: Required information is provided in a pre-formatted bash-script configuration file, which is provided
+via a command argument when running the Trafodion Installer thereby suppressing all prompts.
++
+A template of the configuration file is available here within the installer directory: `trafodion_config_default`.
+Make a copy of the file in your directory and populate the needed information.
++
+Automated Setup is recommended since it allows you to record the required provisioning information information ahead of time.
+Refer to <<introduction-trafodion-automated-setup,Automated Setup>> for information about how to
+populate this file.
+
+[[introduction-trafodion-installer-usage]]
+=== Usage
+
+The following shows help for the Trafodion Installer.
+
+```
+./trafodion_install --help
+
+This script will install Trafodion. It will create a configuration
+file (if one has not been created), setup of the environment needed
+for Trafodion, configure HBase with Hbase-trx and co-processors needed,
+and install a specified Trafodion build.
+
+Options:
+ --help Print this message and exit
+ --accept_license If provided, the user agrees to accept all the
+ provisions in the Trafodion license. This allows
+ for automation by skipping the display and prompt of
+ the Trafodion license.
+ --config_file If provided, all install prompts will be
+ taken from this file and not prompted for.
+```
+
+<<<
+[[introduction-trafodion-installer-install-vs-upgrade]]
+=== Install vs. Upgrade
+
+The Trafodion Installer automatically detects whether you're performing an install
+or an upgrade by looking for the Trafodion Runtime User in the `/etc/passwd` file.
+
+* If the user ID doesn't exists, then the Trafodion Installer runs in install mode.
--- End diff --
It should. Fixed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request: Trafodion 1822
Posted by gtapper <gi...@git.apache.org>.
Github user gtapper commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/326#discussion_r53245924
--- Diff: docs/provisioning_guide/src/asciidoc/_chapters/prepare.adoc ---
@@ -0,0 +1,379 @@
+////
+/**
+* @@@ START COPYRIGHT @@@
+*
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing,
+* software distributed under the License is distributed on an
+* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+* KIND, either express or implied. See the License for the
+* specific language governing permissions and limitations
+* under the License.
+*
+* @@@ END COPYRIGHT @@@
+ */
+////
+
+[[prepare]]
+= Prepare
+You need to prepare your Hadoop environment before installing Trafodion.
+
+1. <<prepare-install-optional-workstation-software,Install Optional Workstation Software>>
+2. <<configure-installation-user-id,Configure Installation User ID>>
+3. <<prepare-disable-requiretty,Disable requiretty>>
+4. <<prepare-verify-os-requirements-and-recommendations,Verify OS Requirements and Recommendations>>
+5. <<prepare-configure-ldap-identity-store,Configure LDAP Identity Store>>
+6. <<prepare-gather-configuration-information,Gather Configuration Information>>
+7. <<prepare-install-required-software-packages,Install Required Software Packages>>
+8. <<prepare-perform-recipe-based-provisioning-tasks,Perform Recipe-Based Provisioning Tasks>>
+
+[[prepare-install-optional-workstation-software]]
+== Install Optional Workstation Software
+
+If you are using a Windows workstation, then the following optional software helps installation process.
+We recommended that you pre-install the software before continuing with the Trafodion installation:
+
+* putty and puttygen (download from http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html[PuTTY web site])
+* VNC client (download from http://www.realvnc.com[RealVNC web site])
+* Firefox or Chrome browser
+* SFTP client to transfer files from your workstation to the Linux server: WinSCP or FileZilla
+
+[[configure-installation-user-id]]
+== Configure Installation User ID
+
+Trafodion installation requires a user ID with these attributes:
+
+* `sudo` access per the requirements documented in <<requirements-linux-installation-user,Linux Installation User>>.
+* passwordless ssh to all nodes on the cluster where Trafodion will be installed.
+
+NOTE: You may need to request permission from your cluster-management team to obtain this type of access.
+
+The following example shows how to set up your user ID to have "passwordless ssh" abilities.
+
+Do the following on the Provision Master Node:
+
+```
+echo -e 'y\n' | ssh-keygen -t rsa -N "" -f $HOME/.ssh/id_rsa
+cat $HOME/.ssh/id_rsa.pub >> $HOME/.ssh/authorized_keys
+echo localhost $(cat /etc/ssh/ssh_host_rsa_key.pub) >> $HOME/.ssh/known_hosts
+echo "NoHostAuthenticationForLocalhost=yes" >> $HOME/.ssh/config
+chmod 600 $HOME/.ssh/config
+```
+
+After running these commands, do the following:
+
+* Copy the content of the `$HOME/.ssh/id_rsa.pub` file (public key) and append it to the
+`$HOME/.ssh/authorized_keys` file on the other nodes in your cluster.
+* Copy the `$HOME/.ssh/id_rsa` file (private key) from the current node to the `$HOME/.ssh` directory on the other nodes in the cluster.
+* Secure `$HOME/.ssh/id_rsa` on all nodes as private to yourself (`chmod 700`).
+* `ssh` to the other nodes in the cluster. Answer `y` to the prompt asking you whether to continue the connection.
+This adds the node to the `$HOME/.ssh/known_hosts` file completing the passwordless ssh setup.
+
+
+[[prepare-disable-requiretty]]
+== Disable requiretty
+You need to disable `requiretty` in `/etc/sudoers` on all nodes in the cluster
+to ensure that `sudo` commands can be run from inside the installation scripts.
+
+Comment out the `Defaults requiretty` setting in the `/etc/sudoers` file to
+ensure that the `requiretty` option is NOT being used.
+
+[[prepare-verify-os-requirements-and-recommendations]]
+== Verify OS Requirements and Recommendations
+
+Please ensure that the <<requirements-os-requirements-and-recommendations,OS Requirements and Recommendations>>
+are met for each node in the cluster where you intend to install Trafodion.
+
+<<<
+[[prepare-configure-ldap-identity-store]]
+== Configure LDAP Identity Store
+
+If you plan to enable security in Trafodion, then you need to have an LDAP identity store available to perform authentication.
+The Trafodion Installer prompts you to set up an authentication configuration file that points to an LDAP server (or servers),
+which enables security (that is, authentication and authorization) in the Trafodion database.
+
+If you wish to manually set up the authentication configuration file and enable security, then refer to the section on
+<<enable-security,Enable Security>>.
+
+[[prepare-gather-configuration-information]]
+== Gather Configuration Information
+
+You need to gather/decide information about your environment to aid installation Trafodion, both for the Trafodion Installer
+and for recipe-based provisioning. (In alphabetical order to make it easier to find information later on in the provisioning process.)
--- End diff --
Listed in... fixed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request: Trafodion 1822
Posted by DaveBirdsall <gi...@git.apache.org>.
Github user DaveBirdsall commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/326#discussion_r53220855
--- Diff: docs/provisioning_guide/src/asciidoc/_chapters/about.adoc ---
@@ -24,19 +24,22 @@
////
= About This Document
-This guide describes how to provision Trafodion for end-user environments. This install allows you to store and query data using Trafodion, either via Trafodion clients
+This guide describes how to provision the end-user Trafodion binaries ontop of an existing Hadoop environment. This install allows you to store and query data using Trafodion, either via Trafodion clients
(see http://trafodion.incubator.apache.org/client_install/index.html[Trafodion Client Installation Guide] or via application code you write.
If you want to install a Trafodion developer-build environment, then please refer to the
http://trafodion.incubator.apache.org/contributing_redirect.html[Trafodion Contributor Guide] for instructions.
== Intended Audience
-This guide is intended for:
+This guide assumes that you are well-versed in Linux and Hadoop administration. If you don't have such experience, then
+you should consider going through the steps required to install a Hadoop environment before attempting to install Trafodion.
-* *Single-Node Environments*: Typically used when you want to evalulate Trafodion.
+The instructions contained herein applies to the following environments.
--- End diff --
Grammar: should be "apply"
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request: Trafodion 1822
Posted by gtapper <gi...@git.apache.org>.
Github user gtapper commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/326#discussion_r53246546
--- Diff: docs/provisioning_guide/src/asciidoc/_chapters/script_install.adoc ---
@@ -0,0 +1,511 @@
+////
+/**
+* @@@ START COPYRIGHT @@@
+*
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing,
+* software distributed under the License is distributed on an
+* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+* KIND, either express or implied. See the License for the
+* specific language governing permissions and limitations
+* under the License.
+*
+* @@@ END COPYRIGHT @@@
+ */
+////
+
+[[install]]
+= Install
+
+This chapter describes how to use the Trafodion Installer to install Trafodion. You use the Trafodion Provisioning ID
+to run the Trafodion Installer.
+
+NOTE: You *must* install log4c++ on all nodes in the cluster prior to running the Trafodion Installer. Refer
+to <<prepare-build-and-install-log4cplusplus,Build and Install log4c++>> for instructions.
+
+[[install-unpack-installer]]
+== Unpack Installer
+
+You should already have downloaded the Trafodion Binaries per the instructions in the
+<<prepare-download-trafodion-binaries,Download Trafodion Binaries>> in the <<prepare,Prepare>>
+chapter. If not, please do so now.
+
+The first step in the installation process is to unpack the Trafodion Installer tar file.
+
+*Example*
+
+```
+$ mkdir $HOME/trafodion-installer
+$ cd $HOME/trafodion-downloads
+$ tar -zxf apache-trafodion-installer-1.3.0-incubating-bin.tar.gz -C $HOME/trafodion-installer
+$ ls $HOME/trafodion-installer/installer
+bashrc_default tools traf_config_check trafodion_apache_hadoop_install traf_package_setup
+build-version-1.3.0.txt traf_add_user traf_config_setup trafodion_config_default traf_setup
+dcs_installer traf_apache_hadoop_config_setup traf_create_systemdefaults trafodion_install traf_sqconfig
+rest_installer traf_authentication_conf_default traf_getHadoopNodes trafodion_license traf_start
+setup_known_hosts.exp traf_cloudera_mods98 traf_hortonworks_mods98 trafodion_uninstaller
+$
+```
+
+[[install-automated-install]]
+== Automated Install
+
+The `--config_file` option runs the Trafodion in Automated Setup mode. Refer to <<introduction-trafodion-installer,Trafodion Installer>>
+in the <<introduction,Introduction>> chapter for instructions of how you edit your configuration file.
+
+Edit your config file using the information you collected in the <<prepare-gather-configuration-information,Gather Configuration Information>>
+in the <<prepare,Prepare>> chapter.
--- End diff --
I do! Fixed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request: Trafodion 1822
Posted by gtapper <gi...@git.apache.org>.
Github user gtapper commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/326#discussion_r53245276
--- Diff: docs/provisioning_guide/src/asciidoc/_chapters/activate.adoc ---
@@ -0,0 +1,147 @@
+////
+/**
+* @@@ START COPYRIGHT @@@
+*
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing,
+* software distributed under the License is distributed on an
+* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+* KIND, either express or implied. See the License for the
+* specific language governing permissions and limitations
+* under the License.
+*
+* @@@ END COPYRIGHT @@@
+ */
+////
+
+[[activate]]
+= Activate
+
+[[activate-manage-trafodion]]
+== Manage Trafodion
+
+You use the Trafodion runtime user ID to perform Trafodion management operations.
+
+The following table provides an overview of the different Trafodion management scripts.
+
+[cols="40%,20%l,20%l,20%l",options="header"]
+|===
+| Component | Start | Stop | Status
+| *All of Trafodion* | sqstart | sqstop | sqcheck
+| *RMS Server* | rmsstart | rmsstop | rmscheck
+| *REST Server* | reststart | reststop | -
+| *LOB Server* | lobstart | lobstop | -
+| *DCS (Database Connectivity Services)* | dcsstart | dcsstop | dcscheck
+|===
+
+*Example: Start Trafodion*
+
+```
+cd $MY_SQROOT/sql/scripts
+sqstart
+sqcheck
+```
+
+[[activate-validate-trafodion-installation]]
+== Validate Trafodion Installation
+
+You can use sqlci (part of the base product) or trafci (requires separate install; see the
+http://trafodion.apache.org/docs/client_install/index.html[Trafodion Client Installation Guide]) to validate your installation.
+
+=== Smoke Test
+A simple smoke test to validate that Trafodion is function.
--- End diff --
Fixed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request: Trafodion 1822
Posted by DaveBirdsall <gi...@git.apache.org>.
Github user DaveBirdsall commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/326#discussion_r53221205
--- Diff: docs/provisioning_guide/src/asciidoc/_chapters/activate.adoc ---
@@ -0,0 +1,147 @@
+////
+/**
+* @@@ START COPYRIGHT @@@
+*
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing,
+* software distributed under the License is distributed on an
+* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+* KIND, either express or implied. See the License for the
+* specific language governing permissions and limitations
+* under the License.
+*
+* @@@ END COPYRIGHT @@@
+ */
+////
+
+[[activate]]
+= Activate
+
+[[activate-manage-trafodion]]
+== Manage Trafodion
+
+You use the Trafodion runtime user ID to perform Trafodion management operations.
+
+The following table provides an overview of the different Trafodion management scripts.
+
+[cols="40%,20%l,20%l,20%l",options="header"]
+|===
+| Component | Start | Stop | Status
+| *All of Trafodion* | sqstart | sqstop | sqcheck
+| *RMS Server* | rmsstart | rmsstop | rmscheck
+| *REST Server* | reststart | reststop | -
+| *LOB Server* | lobstart | lobstop | -
+| *DCS (Database Connectivity Services)* | dcsstart | dcsstop | dcscheck
+|===
+
+*Example: Start Trafodion*
+
+```
+cd $MY_SQROOT/sql/scripts
+sqstart
+sqcheck
+```
+
+[[activate-validate-trafodion-installation]]
+== Validate Trafodion Installation
+
+You can use sqlci (part of the base product) or trafci (requires separate install; see the
+http://trafodion.apache.org/docs/client_install/index.html[Trafodion Client Installation Guide]) to validate your installation.
+
+=== Smoke Test
+A simple smoke test to validate that Trafodion is function.
--- End diff --
Typo, should be "functioning".
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request: Trafodion 1822
Posted by DaveBirdsall <gi...@git.apache.org>.
Github user DaveBirdsall commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/326#discussion_r53223914
--- Diff: docs/provisioning_guide/src/asciidoc/_chapters/introduction.adoc ---
@@ -0,0 +1,466 @@
+////
+/**
+* @@@ START COPYRIGHT @@@
+*
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing,
+* software distributed under the License is distributed on an
+* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+* KIND, either express or implied. See the License for the
+* specific language governing permissions and limitations
+* under the License.
+*
+* @@@ END COPYRIGHT @@@
+ */
+////
+
+[[introduction]]
+= Introduction
+
+Trafodion is a Hadoop add-on service that provides transactional SQL on top of HBase. Typically, you
+use Trafodion as the database for applications that require Online Transaction Processing (OLTP),
+Operational Data Store (ODS), and/or strong reporting capabilities. You access Trafodion using
+standard JDBC and ODBC APIs.
+
+You may choose whether to add Trafodion to an existing Hadoop environment or to create a standalone
+Hadoop environment specifically for Hadoop.
+
+This guide assumes that a Hadoop environment exists upon which your provisioning Trafodion. Refer to
+<<requirements-hadoop-software,Hadoop Software>> for information about what Hadoop software is required
+Trafodion.
+
+[[introduction-security-considerations]]
+== Security Considerations
+
+The following users need be considered for Trafodion:
+
+* *Provisioning User*: A Linux-level user that performs the Trafodion provisioning tasks. This user ID
+requires `sudo` access and passwordless ssh among the nodes where Trafodion is installed. In addition,
+this user ID requires access to Hadoop distribution, HDFS, and HBase administrative users to change
+respective environment's configuration settings per Trafodion requirements. Refer to
+<<requirements-trafodion-provisioning-user,Trafodion Provisioning User>> for more information
+about the requirements and usage associated with this user ID.
+
+* *Runtime User*: A linux-level user under which the Trafodion software runs. This user ID must be registered
+as a user in the Hadoop Distributed File System (HDFS) to store and access objects in HDFS, HBase, and Hive.
+In addition, this user ID requires passwordless access among the nodes where Trafodion is installed.
+Refer to <<requirements-trafodion-runtime-user,Trafodion Runtime User>> for more information about this user ID.
+
+* *Trafodion Database Users*: Trafodion users are managed by the Trafodion security features (grant, revoke, etc.),
+which can be integrated with LDAP if so desired. These users are referred to as *database users* and
+do not have direct access to the operating system. Refer to
+http://trafodion.apache.org/docs/sql_reference/index.html#register_user_statement[Register User],
+http://trafodion.apache.org/docs/sql_reference/index.html#grant_statement[Grant], and other SQL statements
+in the http://trafodion.apache.org/docs/sql_reference/index.html[Trafodion SQL Reference Manual] for
+more information about managing Trafodion Database Users.
+ +
+ +
+Optionally, you can enable Trafodion Security. If you do not enable security in Trafodion, then a client interface
+to Trafodion may request a user name and password, but Trafodion ignores the user name and password entered in the
+client interface, and the session runs as the database *root* user, `DB__ROOT`, without restrictions. If you want
+to restrict users, restrict access to certain users only, or restrict access to an object or operation, then you must
+enable security, which enforces authentication and authorization. Refer to
+<<enable-security,Enable Security>> for more information about this option.
+
+[[introduction-provisioning-options]]
+== Provisioning Options
+
+Trafodion ships with a set of scripts (the Trafodion Installer) that takes care of many of the installation and upgrade
+tasks associated with the Trafodion software and its requirements. There is a separate set of scripts to remove Trafodion,
+if needed.
+
+Currently, the Trafodion Installer is able to install Trafodion on select Cloudera and Hortonworks Hadoop distributions only.
+The Trafodion Installer limitations are noted as they apply in the different chapters below. For example, the Trafodion Installer
+is less capable on SUSE than it is on RedHat/CentOS; you have to install the prerequisite software packages outside the Trafodion Installer.
+
+The Trafodion Installer automates many of the tasks required to install/upgrade Trafodion, spanning from downloading and
+installing required software packages and making required changes to your Hadoop environment via creating
+the Trafodion runtime user ID to installing and starting Trafodion. It is, therefore, highly recommend that
+you use the Trafodion Installer for initial installation and upgrades of Trafodion. These steps are referred to as
+"Script-Based Provisioning" in this guide. Refer to <<introduction-trafodion-installer, Trafodion Installer>> provides
+usage information.
+
+If, for any reason, you choose not to use the Trafodion Installer, then separate chapters provide
+step-by-step recipes for the tasks required to install/upgrade Trafodion. These steps are referred to as
+*Recipe-Based Provisioning* in this guide. It is assumed that you are well-versed in Linux and Hadoop
+administrative tasks if using Recipe-Based Provisioning.
+
+[[introduction-provisioning-activities]]
+== Provisioning Activities
+
+Trafodion provisioning is divided into the following main activities:
+
+* *<<requirements,Requirements>>*: Activities and documentation required to install the Trafodion software.
+These activities include tasks such as understanding hardware and operating system requirements,
+Hadoop requirements, what software packages that need to be downloaded, configuration settings that need to be changed,
+user IDs requirements, and so on.
+
+* *<<prepare,Prepare>>*: Activities to prepare the operating system and the Hadoop ecosystem to run
+Trafodion. These activities include tasks such as installing required software packages, configure
+the Trafodion Installation User, gather information about the Hadoop environment, modify configuration
+for different Hadoop services, and so forth.
+
+* *<<install,Install>>*: Activities related to installing the Trafodion software. These activities
+include tasks such as unpacking the Trafodion tar files, creating the Trafodion Runtime User,
+creating Trafodion HDFS directories, installing the Trafodion software, and so forth.
+
+<<<
+* *<<upgrade,Upgrade>>*: Activities related to the upgrading the Trafodion software. These activities
+include tasks such as shutting down Trafodion, installing a new version of the Trafodion software,
+and so on. The upgrade tasks vary depending on the differences between the current and new release of
+Trafodion. For example, an upgrade may or may not include an upgrade of the Trafodion metadata.
+
+* *<<activate,Activate>>*: Activities related to starting the Trafodion software. These actives
+include basic management tasks such as starting and checking the status of the Trafodion components,
+performing basic smoke tests, and so forth.
+
+* *<<remove,Remove>>*: Activities related to removing Trafodion from your Hadoop cluster.
+
+[[introduction-provisioning-master-node]]
+== Provisioning Master Node
+All provisioning tasks are performed from a single node in the cluster, which must be part
+of the Hadoop environment you're adding Trafodion to. This node is referred to as the
+"*Provisioning Master Node*" in this guide.
+
+The Trafodion Provisioning User must have access to all other nodes from the Provisioning
+Master Node in order to perform provisioning tasks on the cluster.
+
+[[introduction-trafodion-installer]]
+== Trafodion Installer
+
+The Trafodion Installer is a set of scripts automates most of the tasks requires to install/upgrade Trafodion.
+You download the Trafodion Installer tar file from the Trafodion http://trafodion.apache.org/download.html[download] page.
+Next, you unpack the tar file.
+
+*Example*
+
+```
+$ mkdir $HOME/trafodion-installer
+$ cd $HOME/trafodion-downloads
+$ tar -zxf apache-trafodion-installer-1.3.0-incubating-bin.tar.gz -C $HOME/trafodion-installer
+$ ls $HOME/trafodion-installer/installer
+bashrc_default tools traf_config_check trafodion_apache_hadoop_install traf_package_setup
+build-version-1.3.0.txt traf_add_user traf_config_setup trafodion_config_default traf_setup
+dcs_installer traf_apache_hadoop_config_setup traf_create_systemdefaults trafodion_install traf_sqconfig
+rest_installer traf_authentication_conf_default traf_getHadoopNodes trafodion_license traf_start
+setup_known_hosts.exp traf_cloudera_mods98 traf_hortonworks_mods98 trafodion_uninstaller
+$
+```
+
+<<<
+The Trafodion Installer supports two different modes:
+
+1. *Guided Setup*: Prompts for information as it works through the installation/upgrade process. This mode is recommended for new users.
+2. *Automated Setup*: Required information is provided in a pre-formatted bash-script configuration file, which is provided
+via a command argument when running the Trafodion Installer thereby suppressing all prompts.
++
+A template of the configuration file is available here within the installer directory: `trafodion_config_default`.
+Make a copy of the file in your directory and populate the needed information.
++
+Automated Setup is recommended since it allows you to record the required provisioning information information ahead of time.
--- End diff --
Too bad the "Guided Setup" doesn't create a config file with the responses to all the prompts.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request: Trafodion 1822
Posted by gtapper <gi...@git.apache.org>.
Github user gtapper commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/326#discussion_r53245939
--- Diff: docs/provisioning_guide/src/asciidoc/_chapters/prepare.adoc ---
@@ -0,0 +1,379 @@
+////
+/**
+* @@@ START COPYRIGHT @@@
+*
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing,
+* software distributed under the License is distributed on an
+* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+* KIND, either express or implied. See the License for the
+* specific language governing permissions and limitations
+* under the License.
+*
+* @@@ END COPYRIGHT @@@
+ */
+////
+
+[[prepare]]
+= Prepare
+You need to prepare your Hadoop environment before installing Trafodion.
+
+1. <<prepare-install-optional-workstation-software,Install Optional Workstation Software>>
+2. <<configure-installation-user-id,Configure Installation User ID>>
+3. <<prepare-disable-requiretty,Disable requiretty>>
+4. <<prepare-verify-os-requirements-and-recommendations,Verify OS Requirements and Recommendations>>
+5. <<prepare-configure-ldap-identity-store,Configure LDAP Identity Store>>
+6. <<prepare-gather-configuration-information,Gather Configuration Information>>
+7. <<prepare-install-required-software-packages,Install Required Software Packages>>
+8. <<prepare-perform-recipe-based-provisioning-tasks,Perform Recipe-Based Provisioning Tasks>>
+
+[[prepare-install-optional-workstation-software]]
+== Install Optional Workstation Software
+
+If you are using a Windows workstation, then the following optional software helps installation process.
+We recommended that you pre-install the software before continuing with the Trafodion installation:
+
+* putty and puttygen (download from http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html[PuTTY web site])
+* VNC client (download from http://www.realvnc.com[RealVNC web site])
+* Firefox or Chrome browser
+* SFTP client to transfer files from your workstation to the Linux server: WinSCP or FileZilla
+
+[[configure-installation-user-id]]
+== Configure Installation User ID
+
+Trafodion installation requires a user ID with these attributes:
+
+* `sudo` access per the requirements documented in <<requirements-linux-installation-user,Linux Installation User>>.
+* passwordless ssh to all nodes on the cluster where Trafodion will be installed.
+
+NOTE: You may need to request permission from your cluster-management team to obtain this type of access.
+
+The following example shows how to set up your user ID to have "passwordless ssh" abilities.
+
+Do the following on the Provision Master Node:
+
+```
+echo -e 'y\n' | ssh-keygen -t rsa -N "" -f $HOME/.ssh/id_rsa
+cat $HOME/.ssh/id_rsa.pub >> $HOME/.ssh/authorized_keys
+echo localhost $(cat /etc/ssh/ssh_host_rsa_key.pub) >> $HOME/.ssh/known_hosts
+echo "NoHostAuthenticationForLocalhost=yes" >> $HOME/.ssh/config
+chmod 600 $HOME/.ssh/config
+```
+
+After running these commands, do the following:
+
+* Copy the content of the `$HOME/.ssh/id_rsa.pub` file (public key) and append it to the
+`$HOME/.ssh/authorized_keys` file on the other nodes in your cluster.
+* Copy the `$HOME/.ssh/id_rsa` file (private key) from the current node to the `$HOME/.ssh` directory on the other nodes in the cluster.
+* Secure `$HOME/.ssh/id_rsa` on all nodes as private to yourself (`chmod 700`).
+* `ssh` to the other nodes in the cluster. Answer `y` to the prompt asking you whether to continue the connection.
+This adds the node to the `$HOME/.ssh/known_hosts` file completing the passwordless ssh setup.
+
+
+[[prepare-disable-requiretty]]
+== Disable requiretty
+You need to disable `requiretty` in `/etc/sudoers` on all nodes in the cluster
+to ensure that `sudo` commands can be run from inside the installation scripts.
+
+Comment out the `Defaults requiretty` setting in the `/etc/sudoers` file to
+ensure that the `requiretty` option is NOT being used.
+
+[[prepare-verify-os-requirements-and-recommendations]]
+== Verify OS Requirements and Recommendations
+
+Please ensure that the <<requirements-os-requirements-and-recommendations,OS Requirements and Recommendations>>
+are met for each node in the cluster where you intend to install Trafodion.
+
+<<<
+[[prepare-configure-ldap-identity-store]]
+== Configure LDAP Identity Store
+
+If you plan to enable security in Trafodion, then you need to have an LDAP identity store available to perform authentication.
+The Trafodion Installer prompts you to set up an authentication configuration file that points to an LDAP server (or servers),
+which enables security (that is, authentication and authorization) in the Trafodion database.
+
+If you wish to manually set up the authentication configuration file and enable security, then refer to the section on
+<<enable-security,Enable Security>>.
+
+[[prepare-gather-configuration-information]]
+== Gather Configuration Information
+
+You need to gather/decide information about your environment to aid installation Trafodion, both for the Trafodion Installer
+and for recipe-based provisioning. (In alphabetical order to make it easier to find information later on in the provisioning process.)
+
+[cols="25%l,25%,15%l,35%",options="header"]
+|===
+| ID^1^ | Information | Default | Notes
+| ADMIN | Administrator user name for Apache Ambari or Cloudera Manager. | admin | A user that can change configuration and restart services via the
+distribution manager's REST API.
+| BACKUP_DCS_NODES | List of nodes where to start the backup DCS Master components. | None | Blank separated FQDN list. Not needed if $ENABLE_HA = N.
+| CLOUD_CONFIG | Whether you're installing Trafodion on a cloud environment. | N | N = bare-metal or VM installation.
+| CLOUD_TYPE | What type of cloud environment you're installing Trafodion on. | None | { AWS \| OpenStack \| Other }. Not applicable for bare-metal or VM installation.
+| CLUSTER_NAME | The name of the Hadoop Cluster. | None | From Apache Ambari or Cloudera Manager.
+| DCS_BUILD | Tar file containing the DCS component. | None | Not needed if using a Trafodion package installation tar file.
+| DCS_PRIMARY_MASTER_NODE | The node where the primary DCS should run. | None | The DCS Master handles JDBC and ODBC connection requests.
+| DCS_SERVER_PARM | Number of concurrent client sessions per node. | 16 | This number specifies the concurrent sessions per node to be supported. Each session could require up to 1GB of physical memory. The number can be changed post-installation. For more information,
+refer to the http://trafodion.apache.org/docs/client_install/index.html[Trafodion Client Installation Guide].
+| ENABLE_HA | Whether to run DCS in high-availability (HA) mode. | N | You need the floating IP address, the interface, and the backup nodes for DCS Master if enabling this feature.
+| EPEL_RPM | Location of EPEL RPM. | None | Specify if you don't have access to the Internet.
+Downloaded automatically by the Trafodion Installer.
+| FLOATING_IP | IP address if running DCS in HA mode. | None | Not needed if $ENABLE_HA = N. An FQDN name or IP address.
+| HADOOP_TYPE | The type of Hadoop distribution you're installing Trafodion on. | None | Lowercase. cloudera or hadoop.
+| HBASE_GROUP | Linux group name for the HBASE administrative user. | hbase | Required in order to provide access to select HDFS directories to this user ID.
+| HBASE_USER | Linux user name for the HBASE administrative user. | hbase | Required in order to provide access to select HDFS directories to this user ID.
+| HDFS_USER | Linux user name for the HDFS administrative user. | hdfs | The Trafodion Installer uses `sudo su` to make HDFS
+configuration changes under this user.
+| HOME_DIR | Root directory under which the `trafodion` home directory should be created. | /home | *Example* +
+ +
+If the home directory of the `trafodion` user is
+`/opt/home/trafodion`, then specify the root directory as `/opt/home`.
+| INIT_TRAFODION | Whether to automatically initialize the Trafodion database. | N | Does not apply to Recipe-Based Provisioning. Applies if $START=Y only.
+| INTERFACE | Interface type used for $FLOATING_IP. | None | Not needed if $ENABLE_HA = N.
+| JAVA_HOME | Location of Java 1.7.0_65 or higher (JDK). | $JAVA_HOME setting | Fully qualified path of the JDK. For example:
+`/usr/java/jdk1.7.0_67-cloudera`
+| LDAP_CERT^2^ | Full path to TLS certificate. | None | Required of $LDAP_LEVEL = 1 or 2.
+| LDAP_HOSTS^2^ | List of nodes where LDAP Identity Store servers are running. | None | Blank separated. FQDN format.
+| LDAP_ID^2^ | List of LDAP unique identifiers. | None | Blank separated.
+| LDAP_LEVEL^2^ | LDAP Encryption Level. | 0 | 0: Encryption not used, 1: SSL, 2: TLS
+| LDAP_PASSWORD^2^ | Password for LDAP_USER. | None | If LDAP_USER is required only.
+| LDAP_PORT^2^ | Port used to communicate with LDAP Identity Store. | None | Examples: 389 for no encryption or TLS, 636 for SSL.
+| LDAP_SECURITY^2^ | Whether to enable simple LDAP authentication. | N | If Y, then you need to provide LDAP_HOSTS.
+| LDAP_USER^2^ | LDAP Search user name. | None | If required. If so, must provide LDAP_PASSWORD, too.
+| LOCAL_WORKDIR | The directory where the Trafodion Installer is located. | None | Full path, no environmental variables.
+| MANAGEMENT_ENABLED | Whether your installation uses separate management nodes. | N | Y if using separate management nodes for Apache Ambari or Cloudera Manager.
+| MANAGEMENT_NODES | The FQDN names of management nodes, if any. | None | Provide a blank-separated list of node names.
+| NODE_LIST | The FQDN names of the nodes where Trafodion will be installed. | None | Provide a blank-separated list of node names. The Trafodion
+Provisioning ID must have passwordless and `sudo` access to these nodes.
+| PASSWORD | Administrator password for Apache Ambari or Cloudera Manager. | admin | A user that can change configuration and restart services via the
+distribution manager's REST API.
+| REST_BUILD | Tar file containing the REST component. | None | Not needed if using a Trafodion package installation tar file.
+| SQ_ROOT | Target directory for the Trafodion software. | $HOME_DIR/trafodion | Trafodion is installed in this directory on all nodes in `$NODE_LIST`.
+| START | Whether to start Trafodion after install/upgrade. | N | Does not apply to Recipe-Based Provisioning.
+| SUSE_LINUX | Whether your installing Trafodion on SUSE Linux. | false | Auto-detected by the Trafodion Installer.
+| TRAF_PACKAGE | The location of the Trafodion installation package tar file or core installation tar file. | None | The package file contains the Trafodion server,
+DCS, and REST software while the core installation file contains the Trafodion server software only. If you're using a core installation file, then you need to
+record the location of the DCS and REST installation tar files, too. Normally, you perform Trafodion provisioning using a Trafodion package installation tar file.
+| TRAF_USER | The Trafodion runtime user ID. | trafodion | Must be `trafodion` in this release.
+| TRAF_USER_PASSWORD | The password used for the `trafodion:trafodion` user ID. | traf123 | Must be 6-8 characters long.
+| URL | FQDN and port for the Distribution Manager's REST API. | None | Include `http://` or `https://` as applicable. Specify in the form:
+`<IP-address>:<port>` or `<node name>:<port>` Example: `https://susevm-1.yourcompany.local:8080`
+|===
+
+1. The ID matches the environmental variables used in the Trafodion Installation configuration file. Refer to <<install-trafodion-installer,Trafodion Installer>>
+for more information.
+2. Refer to <<enable-security,Enable Security>> for more information about these security settings.
+
+
+<<<
+[[prepare-install-required-software-packages]]
+== Install Required Software Packages
+
+[[prepare-download-and-install-packages]]
+=== Download and Install Packages
+
+The step is required if you're:
--- End diff --
Absolutely. Fixed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request: Trafodion 1822
Posted by gtapper <gi...@git.apache.org>.
Github user gtapper commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/326#discussion_r53245396
--- Diff: docs/provisioning_guide/src/asciidoc/_chapters/introduction.adoc ---
@@ -0,0 +1,466 @@
+////
+/**
+* @@@ START COPYRIGHT @@@
+*
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing,
+* software distributed under the License is distributed on an
+* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+* KIND, either express or implied. See the License for the
+* specific language governing permissions and limitations
+* under the License.
+*
+* @@@ END COPYRIGHT @@@
+ */
+////
+
+[[introduction]]
+= Introduction
+
+Trafodion is a Hadoop add-on service that provides transactional SQL on top of HBase. Typically, you
+use Trafodion as the database for applications that require Online Transaction Processing (OLTP),
+Operational Data Store (ODS), and/or strong reporting capabilities. You access Trafodion using
+standard JDBC and ODBC APIs.
+
+You may choose whether to add Trafodion to an existing Hadoop environment or to create a standalone
+Hadoop environment specifically for Hadoop.
+
+This guide assumes that a Hadoop environment exists upon which your provisioning Trafodion. Refer to
+<<requirements-hadoop-software,Hadoop Software>> for information about what Hadoop software is required
+Trafodion.
+
+[[introduction-security-considerations]]
+== Security Considerations
+
+The following users need be considered for Trafodion:
+
+* *Provisioning User*: A Linux-level user that performs the Trafodion provisioning tasks. This user ID
+requires `sudo` access and passwordless ssh among the nodes where Trafodion is installed. In addition,
+this user ID requires access to Hadoop distribution, HDFS, and HBase administrative users to change
+respective environment's configuration settings per Trafodion requirements. Refer to
+<<requirements-trafodion-provisioning-user,Trafodion Provisioning User>> for more information
+about the requirements and usage associated with this user ID.
+
+* *Runtime User*: A linux-level user under which the Trafodion software runs. This user ID must be registered
--- End diff --
Good catch. Fixed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-trafodion pull request: Trafodion 1822
Posted by gtapper <gi...@git.apache.org>.
Github user gtapper commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/326#discussion_r53245261
--- Diff: docs/provisioning_guide/src/asciidoc/_chapters/about.adoc ---
@@ -24,19 +24,22 @@
////
= About This Document
-This guide describes how to provision Trafodion for end-user environments. This install allows you to store and query data using Trafodion, either via Trafodion clients
+This guide describes how to provision the end-user Trafodion binaries ontop of an existing Hadoop environment. This install allows you to store and query data using Trafodion, either via Trafodion clients
(see http://trafodion.incubator.apache.org/client_install/index.html[Trafodion Client Installation Guide] or via application code you write.
If you want to install a Trafodion developer-build environment, then please refer to the
http://trafodion.incubator.apache.org/contributing_redirect.html[Trafodion Contributor Guide] for instructions.
== Intended Audience
-This guide is intended for:
+This guide assumes that you are well-versed in Linux and Hadoop administration. If you don't have such experience, then
+you should consider going through the steps required to install a Hadoop environment before attempting to install Trafodion.
-* *Single-Node Environments*: Typically used when you want to evalulate Trafodion.
+The instructions contained herein applies to the following environments.
--- End diff --
Fixed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---