You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hawq.apache.org by yo...@apache.org on 2017/04/25 00:03:56 UTC
[03/50] [abbrv] incubator-hawq-docs git commit: Adding draft
documentation for Ranger integration feature
Adding draft documentation for Ranger integration feature
Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/commit/a7e32e0c
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/tree/a7e32e0c
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/diff/a7e32e0c
Branch: refs/heads/master
Commit: a7e32e0ce4f8cd15001f7b4900218bbfa2ba8d45
Parents: 9175d25
Author: David Yozie <yo...@apache.org>
Authored: Thu Mar 30 12:33:22 2017 -0700
Committer: David Yozie <yo...@apache.org>
Committed: Thu Mar 30 12:33:30 2017 -0700
----------------------------------------------------------------------
markdown/admin/logfiles.html.md.erb | 6 +
markdown/images/hawqpolicydetails.png | Bin 0 -> 165359 bytes
markdown/ranger/ranger-auditing.html.md.erb | 156 ++++++
.../ranger-integration-config.html.md.erb | 131 +++++
markdown/ranger/ranger-overview.html.md.erb | 64 +++
.../ranger/ranger-policy-creation.html.md.erb | 508 +++++++++++++++++++
.../cli/admin_utilities/hawqstate.html.md.erb | 1 +
7 files changed, 866 insertions(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a7e32e0c/markdown/admin/logfiles.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/admin/logfiles.html.md.erb b/markdown/admin/logfiles.html.md.erb
index 482caef..882ecd9 100644
--- a/markdown/admin/logfiles.html.md.erb
+++ b/markdown/admin/logfiles.html.md.erb
@@ -282,6 +282,12 @@ PXF provides both service- and database-level logging. Refer to [PXF Logging](..
Ambari log files may be useful in helping diagnose general cluster problems. The Ambari server log files are located in the `/var/log/ambari-server/` directory. Ambari agent log files are located in `/var/log/ambari-agent/`. Refer to [Reviewing Ambari Log Files](https://docs.hortonworks.com/HDPDocuments/Ambari-2.2.1.1/bk_ambari_troubleshooting/content/_reviewing_ambari_log_files.html) for additional information.
+## <a id="rangerlogs"></a> Ranger Log Files
+
+The HAWQ Ranger Plug-in Service log files may be useful in helping diagnose Ranger connectivity and authorization problems. You will find these log files in the `$GPHOME/ranger/plugin-service/logs/` directory. In addition to HAWQ Ranger Plug-in service-related logs, this directory includes the `log4j` provider `audit.log` file. (Refer to [Auditing Authorization Events](../ranger/ranger-auditing.html) for information on configuring HAWQ Ranger audit logging.)
+
+Ranger log files are located in the `/var/log/ranger/admin/` directory.
+
## <a id="logging_other"></a>Hadoop Log Files
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a7e32e0c/markdown/images/hawqpolicydetails.png
----------------------------------------------------------------------
diff --git a/markdown/images/hawqpolicydetails.png b/markdown/images/hawqpolicydetails.png
new file mode 100644
index 0000000..4c7945f
Binary files /dev/null and b/markdown/images/hawqpolicydetails.png differ
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a7e32e0c/markdown/ranger/ranger-auditing.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/ranger/ranger-auditing.html.md.erb b/markdown/ranger/ranger-auditing.html.md.erb
new file mode 100644
index 0000000..7f6fed0
--- /dev/null
+++ b/markdown/ranger/ranger-auditing.html.md.erb
@@ -0,0 +1,156 @@
+---
+title: Auditing Authorization Events
+---
+
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+The HAWQ Ranger Plug-in Service supports storing auditing records in any of the Ranger auditing framework audit destinations, referred to as *audit sink*s. The `/usr/local/hawq/ranger/etc/ranger-hawq-audit.xml` file specifies the audit configuration. It contains sample definitions for the HDFS, Solr, and Log4j audit sinks.
+
+As a best practice, configure one or more audit sinks in `ranger-hawq-audit.xml` before you register the HAWQ Ranger Plug-in Service. By default only the Log4j sink is enabled. Production deployments should use both a Solr and an HDFS audit sink, with the Solr destination configured to automatically purge audit records after some period of time. This configuration enables you to search the most recent audit records, while a keeping a full history of auditing records in HDFS.
+
+If you modify `ranger-hawq-audit.xml` after you have registered the HAWQ Ranger Plug-in, you must restart the plug-in for the changes to take effect.
+
+Full documentation for the Ranger auditing configuration properties and the Ranger auditing framework is available at [Ranger 0.5 Audit Configuration](https://cwiki.apache.org/confluence/display/RANGER/Ranger+0.5+Audit+Configuration).
+
+## <a id="solr"></a>Configuring Solr Auditing
+To configure a Solr audit sink, you define a different set of properties in `ranger-hawq-audit.xml` depending on whether you use Zookeeper or a direct URL connect to your Solr destination. For a production environment, use Zookeeper instead of a direct URL.
+
+If you use Zookeeper to connect to Solr, configure these auditing properties in `ranger-hawq-audit.xml`:
+
+Table 1. Properties for Zookeeper Configuration
+
+| Property | Value | Description |
+| -------- | ----- | ----------- |
+| xasecure.audit.destination.solr | true | Use this property to enable or disable the Solr sink. |
+| xasecure.audit.destination.solr.zookeepers | <zookeeper connect string> | Specify the Zookeeper connection string for the Solr destination. |
+| xasecure.audit.destination.solr.collection | <collection name> | Specify the Solr collection name to use for indexing the HAWQ audit records. By default HAWQ uses the `ranger_audits` collection. |
+| xasecure.audit.destination.solr.batch.filespool.* | Multiple Properties | See [Configuration related to File spooling](https://cwiki.apache.org/confluence/display/RANGER/Ranger+0.5+Audit+Configuration#Ranger0.5AuditConfiguration-ConfigurationrelatedtoFilespooling) in the Ranger documentation if you want to configure spooling of auditing events to disk when the in-memory buffer is full. |
+| xasecure.audit.destination.solr.urls | NONE | Leave this property value empty or set it to `NONE` when using Zookeeper to connect to Solr. |
+
+For example:
+
+```
+<!-- ********************************* -->
+<!-- SOLR audit provider configuration -->
+<!-- ********************************* -->
+<property>
+ <name>xasecure.audit.destination.solr</name>
+ <value>true</value>
+</property>
+
+<property>
+ <name>xasecure.audit.destination.solr.zookeepers</name>
+ <value>zkhost1:2181,zkhost2:2181/infra-solr</value>
+</property>
+
+<property>
+ <name>xasecure.audit.destination.solr.collection</name>
+ <value>ranger_audits</value>
+</property>
+
+<property>
+ <name>xasecure.audit.destination.solr.urls</name>
+ <value>NONE</value>
+</property>
+
+<property>
+ <name>xasecure.audit.destination.solr.batch.filespool.enabled</name>
+ <value>true</value>
+</property>
+
+<property>
+ <name>xasecure.audit.destination.solr.batch.filespool.dir</name>
+ <value>/usr/local/hawq_2_2_0_0/ranger/plugin-service/logs/spool/audit/solr</value>
+</property>
+```
+## <a id="hdfs"></a>Configuring HDFS Auditing
+To configure an HDFS audit sink, define these auditing properties in `ranger-hawq-audit.xml`:
+
+Table 2. Properties for HDFS Configuration
+
+| Property | Value | Description |
+| -------- | ----- | ----------- |
+| xasecure.audit.destination.hdfs | true | Use this property to enable or disable the HDFS sink. |
+| xasecure.audit.destination.hdfs.dir | <HDFS directory> | Specify the HDFS directory in which the plug-in records audit events. |
+| xasecure.audit.destination.hdfs.batch.filespool.* | Multiple Properties | See [Configuration related to File spooling](https://cwiki.apache.org/confluence/display/RANGER/Ranger+0.5+Audit+Configuration#Ranger0.5AuditConfiguration-ConfigurationrelatedtoFilespooling) in the Ranger documentation if you want to configure spooling of auditing events to disk when the in-memory buffer is full. |
+
+For example:
+
+```
+<!-- ********************************* -->
+<!-- HDFS audit provider configuration -->
+<!-- ********************************* -->
+<property>
+ <name>xasecure.audit.destination.hdfs</name>
+ <value>true</value>
+</property>
+
+<property>
+ <name>xasecure.audit.destination.hdfs.dir</name>
+ <value>hdfs://localhost:8020/ranger/audit</value>
+</property>
+
+<property>
+ <name>xasecure.audit.destination.hdfs.batch.filespool.enabled</name>
+ <value>true</value>
+</property>
+
+<property>
+ <name>xasecure.audit.destination.hdfs.batch.filespool.dir</name>
+ <value>/usr/local/hawq_2_2_0_0/ranger/plugin-service/logs/spool/audit/hdfs</value>
+</property>
+```
+## <a id="log4j"></a>Configuring Log4j Auditing
+To configure a Log4j audit sink, define these auditing properties in `ranger-hawq-audit.xml`:
+
+Table 3. Properties for Log4j Configuration
+
+| Property | Value | Description |
+| -------- | ----- | ----------- |
+| xasecure.audit.destination.log4j | true | Use this property to enable or disable the Log4j sink. |
+| xasecure.audit.destination.log4j.logger | <Logger Name> | Specify the name of the logger to use for sending audit events. |
+| xasecure.audit.destination.log4j.batch.filespool.* | Multiple Properties | See [Configuration related to File spooling](https://cwiki.apache.org/confluence/display/RANGER/Ranger+0.5+Audit+Configuration#Ranger0.5AuditConfiguration-ConfigurationrelatedtoFilespooling) in the Ranger documentation if you want to configure spooling of auditing events to disk when the in-memory buffer is full. |
+
+For example:
+
+```
+<!-- ********************************** -->
+<!-- Log4j audit provider configuration -->
+<!-- ********************************** -->
+<property>
+ <name>xasecure.audit.destination.log4j</name>
+ <value>true</value>
+</property>
+
+<property>
+ <name>xasecure.audit.destination.log4j.logger</name>
+ <value>ranger_audit_logger</value>
+</property>
+```
+
+## <a id="reconfigure"></a>Changing the Plug-in Auditing Settings
+If you modify `ranger-hawq-audit.xml` after you have registered the HAWQ Ranger Plug-in, you must either restart the HAWQ cluster or restart the plug-in for the changes to take effect.
+
+To restart only the HAWQ Ranger Plug-in:
+
+``` bash
+$ /usr/local/hawq_2_2_0_0/ranger/bin/rps.sh stop
+$ /usr/local/hawq_2_2_0_0/ranger/bin/rps.sh start
+```
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a7e32e0c/markdown/ranger/ranger-integration-config.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/ranger/ranger-integration-config.html.md.erb b/markdown/ranger/ranger-integration-config.html.md.erb
new file mode 100644
index 0000000..8b687b5
--- /dev/null
+++ b/markdown/ranger/ranger-integration-config.html.md.erb
@@ -0,0 +1,131 @@
+---
+title: Configuring HAWQ to use Ranger Policy Management
+---
+
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+Your HAWQ 2.2.0 installation includes the following HAWQ-related Ranger components:
+
+- Ranger Administrative UI
+- HAWQ Ranger Plug-in Service
+
+The Ranger Administrative UI is installed when you install HDP. You configure the Ranger service itself through Ambari. You configure HAWQ-Ranger authorization policies through the Ranger Administrative UI, which you can access at `http://<ranger-admin-node>:6080`.
+
+Installing or upgrading to HAWQ 2.2.0 installs the HAWQ Ranger Plug-in Service, but neither configures nor registers the plug-in.
+
+To use Ranger for managing HAWQ authentication events, you must first install and register several HAWQ JAR files on the Ranger Administration host. This one-time configuration establishes connectivity to your HAWQ cluster from the Ranger Administration host.
+
+After registering the JAR files, you enable or disable Ranger integration in HAWQ by setting the `hawq_acl_type` configuration parameter. After Ranger integration is enabled, you must use the Ranger interface to create all security policies to manage access to HAWQ resources. Ranger is only pre-populated with policies to allow `gpadmin` superuser access to default resources. See [Creating HAWQ Authorization Policies in Ranger](ranger-policy-creation.html) for information about creating policies in Ranger. When Ranger is enabled, all access to HAWQ resources is controlled by security policies on Ranger.
+
+Use the following procedures to register the HAWQ Ranger Plug-in Service and enable Ranger authorization for HAWQ..
+
+## <a id="prereq"></a>Prerequisites
+To use HAWQ Ranger integration, install a compatible Hadoop distribution and Apache Ranger 0.6. You must also have `admin` access to the **Ranger Admin UI**.
+
+## <a id="jar"></a>Step 1: Install Ranger Connectivity to HAWQ
+1. `ssh` into the Ranger Administration host as a user with root privileges:
+
+ ``` bash
+ $ ssh root@<ranger-admin-node>
+ root@ranger-admin-node$
+ ```
+2. Create the directory for the HAWQ JAR files:
+
+ ``` bash
+ root@ranger-admin-node$ cd /usr/hdp/current/ranger-admin/ews/webapp/WEB-INF/classes/ranger-plugins
+ root@ranger-admin-node$ mkdir hawq
+ ```
+3. Copy the necessary HAWQ JAR files (`postgresql-9.1-901-1.jdbc4.jar` and `ranger-plugin-admin-2.2.0.0.jar`) from the HAWQ master node to the new directory:
+
+ ``` bash
+ root@ranger-admin-node$ scp <hawq-master>:/usr/local/hawq/ranger/lib/*.jar ./hawq
+ ```
+4. Change the ownership of the new folder and JAR files to the `ranger` user:
+
+ ``` bash
+ root@ranger-admin-node$ chown -R ranger:ranger hawq
+ ```
+5. The `enable-ranger-plugin.sh` script configures Ranger connectivity to your HAWQ cluster. The command has the syntax:
+
+ ``` pre
+ enable-ranger-plugin.sh -r <ranger_admin_node>:<ranger_port> -u <ranger_user> -p <ranger_password> -h <hawq_master>:<hawq_port> -w <hawq_user> -q <hawq_password>
+ ```
+
+ Log in to the HAWQ master node as the `gpadmin` user and execute the `enable-ranger-plugin.sh` script. Ensure \<hawq_master\> identifies the fully qualified domain name of the HAWQ master node. For example:
+
+ ``` bash
+ sudo su - gpadmin
+ gpadmin@master$ cd /usr/local/hawq/ranger/bin
+ gpadmin@master$ ./enable-ranger-plugin.sh -r ranger_host:6080 -u admin -p admin -h hawq_master:5432 -w gpadmin -q gpadmin
+ ```
+
+ ***Note*** You can also enter the short form of the command: `./enable-ranger-plugin.sh -r` and the script will prompt you for entries.
+
+ When the script completes, the default HAWQ service definition is registered in the Ranger Admin UI. This service definition is named `hawq`.
+
+6. Locate the `pg_hba.conf` file on the HAWQ master node, for example:
+
+ ``` bash
+ gpadmin@master$ hawq config --show hawq_master_directory
+ GUC : hawq_master_directory
+ Value : /data/hawq/master
+
+ ```
+
+ Edit the `pg_hba.conf` file on the HAWQ master node to configure HAWQ access for \<hawq_user\> on the \<ranger-admin-node\>. For example, you would add an entry similar to the following for the example `enable-ranger-plugin.sh` call above:
+
+ ``` bash
+ host all gpadmin ranger_host/32 trust
+ ```
+
+ And reload HAWQ configuration:
+
+ ``` bash
+ gpadmin@master$ hawq stop cluster --reload
+ ```
+
+7. When setup is complete, use the fully-qualified domain name to log into the Ambari server. Use the Ranger link in the left nav to bring up the Ranger Summary pane in the HAWQ Ambari interface. Use the Quick Links to access Ranger. This link will take you to the Ranger Login interface.
+
+8. Log into the Ranger Access Manager. You will see a list of icons under the Service Manager. Click the **Edit** icon on the right, under the HAWQ service icon. Ensure that the Active Status is set to Enabled, and click the **Test Connection** button. You should receive a message that Ranger connected successfully. If it fails to connect, you may need to edit your Ranger connection in `pg_hba.conf,` perform
+
+ ``` bash
+ gpadmin@masterhawq stop cluster --reload
+ ```
+ and re-test the connection.
+
+
+## <a id="enable"></a>Step 2: Configure HAWQ to Use Ranger Policy Management
+
+The default Ranger service definition for HAWQ assigns the HAWQ administrator (typically `gpadmin`) all privileges to all objects.
+
+Once the connection between HAWQ and Ranger is configured, you can either set up policies for the HAWQ users according to the procedures in [Creating HAWQ Authorization Policies in Ranger](ranger-policy-creation.html) or enable Ranger with only the default policies.
+
+**Note**: Any authorization defined using GRANT commands will no longer apply after enabling HAWQ Ranger. Only gpadmin access is allowed when Ranger is first initialized.
+
+1. On Ambari, select the **HAWQ** Service, and then select the **Configs** tab.
+2. Select the **Advanced** tab, and then expand **Custom hawq-site**.
+4. Click **Add Property...** and add the new property, `hawq_acl_type=ranger` property. (If the property already exists, change its value from `standalone` (the default) to `ranger`.)
+5. Click **Save** to save your changes.
+6. Select **Service Actions > Restart All** and confirm that you want to restart the HAWQ cluster.
+
+
+## <a id="caching"></a>Changing the Frequency of Policy Caching
+
+You may wish to change the frequency of policy caching to suit your individual needs.
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a7e32e0c/markdown/ranger/ranger-overview.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/ranger/ranger-overview.html.md.erb b/markdown/ranger/ranger-overview.html.md.erb
new file mode 100644
index 0000000..56b45be
--- /dev/null
+++ b/markdown/ranger/ranger-overview.html.md.erb
@@ -0,0 +1,64 @@
+---
+title: Overview of Ranger Policy Management
+---
+
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+HAWQ supports using Apache Ranger for authorizing user access to HAWQ resources. Using Ranger enables you to manage all of your Hadoop components' authorization policies using the same user interface, policy store, and auditing stores.
+
+
+## <a id="arch"></a>Policy Management Architecture
+Each HAWQ installation includes a Ranger plug-in service to support Ranger Policy management. The Ranger plug-in service implements the Ranger REST API to bridge all requests between the Ranger Policy Manager and a HAWQ instance.
+
+HAWQ also provides a JAR library that enables the Ranger Policy Manager to lookup HAWQ metadata (the names of databases, schemas, tables, and so forth) to populate the user interface and assist in creating new policies. This JAR uses a JDBC connection to HAWQ, and requires a one-time registration with the Ranger Policy Manager.
+
+A single configuration parameter, `hawq_acl_type` determines whether HAWQ defers all policy management to Ranger via the plug-in service, or whether HAWQ handles authorization natively using catalog tables. By default, HAWQ uses SQL commands to create all access policies, and the policy information is stored in catalog tables. When you enable Ranger integration for policy management, any authorization policies that you have configured in HAWQ using SQL no longer apply to your installation; you must create new policies using the Ranger interface. See [Creating HAWQ Authorization Policies in Ranger](ranger-policy-creation.html)
+
+The Ranger plug-in service caches Ranger policies locally on each HAWQ node to avoid unnecessary round trips between the HAWQ node and the Ranger Policy Manager server. You can use the configuration parameter `that` to control how frequently the plug-in service contacts the Ranger Policy Manager to refresh cached policies. See [Changing the Frequency of Policy Caching](ranger-integration-config.html#caching).
+
+## <a id="limitations"></a>Limitations of Ranger Policy Management
+Neither Kerberos authentication nor SSL encryption is supported between a HAWQ node and the Ranger plug-in service, or between the plug-in service and the Ranger Policy Manager.
+
+The Ranger plug-in service is not compatible Highly-Available Ranger deployments. The plug-in will not connect to another Ranger Policy Manager if a failure occurs.
+
+HAWQ supports setting user-level authorization policies with Ranger. These correspond to access policies that would typically be applied using the SQL `GRANT` command, and include authorization events for:
+
+- Databases
+- Schemas
+- Tables
+- Sequences
+- Functions
+- Languages
+- Protocols
+
+All authorization checks for superuser-restricted authorization events are handled by HAWQ natively, even when Ranger integration is enabled. These superuser-restricted events include:
+
+- `CREATE CAST` command
+- `CREATE FILESPACE` command
+- `CREATE`, `DROP`, or `ALTER` commands that involve a foreign-data wrapper
+- `CREATE FUNCTION` command for untrusted languages.
+- `CREATE` or `DROP` commands for procedural Languages
+- `CREATE`, `DROP`, or `ALTER` commands for resource queues
+- `CREATE TABLESPACE` command. Note that Ranger does manage authorization for creating databases, tables, indexes, and so forth _within_ an existing tablespace.
+- `CREATE EXTERNAL TABLE` commands that include the `EXECUTE` clause.
+- `CREATE OPERATOR CLASS` command
+- `COPY` command. Use of the `COPY` command is always limited to the superuser. When Ranger policy management is enabled, the superuser must have `SELECT` or `INSERT` privileges on a table in order to `COPY` from or to that table.
+- Built-in functions such as pg_logdir_ls, pg_ls_dir, pg_read_file, pg_reload_conf, pg_rotate_logfile, pg_signal_backend, pg_start_backup, pg_stat_file, pg_stat_get_activity, pg_stat_get_backend_activity_start, pg_stat_get_backend_activity, pg_stat_get_backend_client_addr, pg_stat_get_backend_client_port, pg_stat_get_backend_start, pg_stat_get_backend_waiting, pg_stop_backup, pg_switch_xlog, and pg_stat_reset.
+
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a7e32e0c/markdown/ranger/ranger-policy-creation.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/ranger/ranger-policy-creation.html.md.erb b/markdown/ranger/ranger-policy-creation.html.md.erb
new file mode 100644
index 0000000..c66f5ba
--- /dev/null
+++ b/markdown/ranger/ranger-policy-creation.html.md.erb
@@ -0,0 +1,508 @@
+---
+title: Creating HAWQ Authorization Policies in Ranger
+---
+
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+Ranger secures your Hadoop services, providing a centralized console to manage user access to the data in your HAWQ cluster.
+
+Native HAWQ authorization provides SQL standard authorization at the database and table level for specific users/roles using `GRANT` and `REVOKE` SQL commands. HAWQ integration with Ranger provides policy-based authorization, enabling you to identify the conditions under which a user and/or group can access individual HAWQ resources, including the operations permitted on those resources.
+
+**Note**: The HAWQ `GRANT` and `REVOKE` operations are not permitted when Ranger authorization is enabled for HAWQ; you must configure all user and object access through Ranger policies.
+
+You will configure HAWQ-Ranger authorization through the Ranger Administrative UI, which you can access at `http://<ranger-admin-node>:6080`.
+
+
+## <a id="userrole"></a>User/Role Mapping
+
+When configuring your HAWQ cluster, you identify the HAWQ database objects to which you want specific users to have access. This configuration is required for both HAWQ-Native and HAWQ-Ranger authorization.
+
+You create HAWQ users with the `createuser` command line utility or `CREATE ROLE` SQL command. These HAWQ users may or may not reflect an underlying operating system user.
+
+Ranger includes a `UserSync` process to synchronize users and groups on the \<ranger-admin-node\>. You can sync users and groups from the operating system (default), a file, or from LDAP/AD services. Once the sync source is identified, Ranger `UserSync` automatically detects new users provisioned on the \<ranger-admin-node\>.
+
+If your HAWQ cluster includes HAWQ-only roles (i.e. roles with no associated OS user), you must manually configure a Ranger user for each such role. You would use the Ranger Admin UI **Settings > Users/Groups** page for this purpose.
+
+
+
+## <a id="authchecks"></a>HAWQ Authorization
+
+
+### <a id="pghbaconf"></a> pg_hba.conf
+The `pg_hba.conf` file on the HAWQ master node identifies the users you permit to access the HAWQ cluster, and the hosts from which the access may be initiated. This authentication is the first line of defense for both HAWQ-Native and HAWQ-Ranger authorization.
+
+
+### <a id="alwaysnative"></a> HAWQ-Native Authorization
+HAWQ *always* employs its native authorization for operations on its catalog. HAWQ also uses only native authorization for the following HAWQ operations, *even when Ranger is enabled*. These operations are available to superusers and may be available those non-admin users to which access was specifically configured:
+
+- operations on HAWQ catalog
+- HAWQ catalog-related built-in functions
+- `CREATE CAST` command when function is NULL
+- `CREATE DATABASE`, `DROP DATABASE`, `createdb`, `dropdb`
+- `hawq filespace`
+- `CREATE`, `DROP`, or `ALTER` commands for resource queues
+- `CREATE ROLE`, `DROP ROLE`, `SET ROLE`, `createuser`, `dropuser`
+- `CREATE TABLESPACE`, `DROP TABLESPACE` (Ranger does manage authorization for creating tables and indexes _within_ an existing tablespace.)
+
+The following SQL operations do not require any authorization checks:
+
+- `DEALLOCATE`
+- `SET`, `RESET`
+
+
+### <a id="rangersuperuser"></a> HAWQ-Ranger Authorization
+When Ranger is enabled, HAWQ-Ranger authorization is employed for access to user database objects outside of the operations mentioned above. HAWQ will deny an operation if no policy exists providing the appropriate permissions for the requesting user to access the specific resource(s).
+
+In cases where an operation requires super-user privileges, HAWQ first performs a super-user check, then requests the Ranger access check. Those operations requiring super-user checks include:
+
+- `CREATE`, `DROP`, or `ALTER` commands that involve a foreign-data wrapper
+- `CREATE LANGUAGE`, `DROP LANGUAGE` for non-built-in languages
+- `CREATE FUNCTION` command for untrusted languages.
+- `CREATE EXTERNAL TABLE` commands that include the `EXECUTE` clause.
+- `CREATE OPERATOR CLASS` command
+- `COPY` command. Use of the `COPY` command is always limited to the superuser. When Ranger policy management is enabled, the superuser must have `SELECT` or `INSERT` privileges on a table in order to `COPY` from or to that table.
+
+
+### <a id="authalgorithm"></a> Access Check Algorithm
+
+A simple algorithm describing the HAWQ access checks follows:
+
+``` pre
+1. Confirm user access allowed by pg_hba.conf file
+2. Perform HAWQ access checks
+ user-is-superuser = (is user superuser?)
+ op-needs-superuser = (does operation require superuser?)
+ use-hawq-native-auth = (does operation require hawq-native authorization?)
+ ranger-enabled = (is ranger enabled?)
+ - If( op-needs-superuser && !user-is-superuser )
+ Denied
+ - If( use-hawq-native-auth || !ranger-enabled )
+ HAWQ-Native authorization check
+ Else
+ HAWQ-Ranger policy check
+```
+
+## <a id="policyeval"></a> Ranger Policy Evaluation
+Ranger evaluates policies from most to least restrictive, searching for a policy with sufficient privileges allowing the requesting user access to the identified resource(s). Deny conditions are evaluated before allow conditions. And policies for specific resources are evaluated before those identifying a wildcard `*` resource.
+
+Refer to the [Ranger User Guide ??apache or hortonworks??](https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.4.0/bk_Ranger_User_Guide/bk_Ranger_User_Guide-20160301.pdf) and [Deny-conditions and excludes in Ranger policies](https://cwiki.apache.org/confluence/display/RANGER/Deny-conditions+and+excludes+in+Ranger+policies) for detailed information on the Ranger Admin UI and Ranger policy evaluation.
+
+
+## <a id="policydef"></a> HAWQ Policy Definition
+
+When configuring a HAWQ-Ranger authorization policy, you:
+
+- Name and provide a description for the policy
+- Identify the HAWQ resource(s) to which the policy applies
+- Identify the conditions under which access to the HAWQ resource(s) should be allowed
+- Enable/Disable audit logging for the policy
+
+
+
+
+### <a id="createpoliciesresource"></a> HAWQ Ranger Resources
+
+You configure the resources to which a HAWQ policy applies in the **Create Policy > Policy Details** pane of the Ranger HAWQ Policy editor. HAWQ resources whose access is managed by Ranger include:
+
+| Resource | Description |
+|-------------|------------------------|
+| database | The database to which you want to provide access |
+| schema | The schema in which you want to provide access |
+| table | The table to which you want to provide access |
+| sequence | The sequence to which you want to provide access |
+| function | The user-defined function to which you want to provide access |
+| language | The language to which you want to provide access |
+| tablespace | The tablespace to which you want to provide access to create databases and tables |
+| protocol | The protocol to which you want to provide access |
+
+The HAWQ Ranger service definition supports only the combinations of resources that reflect the scoping of database objects with HAWQ:
+
+- database/schema/table
+- database/schema/sequence
+- database/schema/function
+- database/language
+- tablespace
+- protocol
+
+The Ranger policy editor provides resource name look-up. That is, when you start entering data into a resource field, HAWQ populates a pop-up list with all existing HAWQ object names matching your text.
+
+The policy editor also allows you to wildcard (`*`) resources in policy details. More restrictive policies will not use wildcarding, but rather will identify specific resource names.
+
+When specifying resources and permissions in your set of policy definitions, you will want to take into consideration the operations you wish to permit on a resource itself, as well as the operations you may wish to allow on subordinate resources.
+
+
+### <a id="createpoliciesconditions"></a> Resource Access Conditions
+
+When defining a HAWQ policy via the Ranger Admin UI, you identify the Groups/Users to which to permit or deny access to the specified HAWQ resource(s). You also identify the permissions for the resource(s) that you wish to assign or deny to these users. You provide this information in the **Create Policy > Allow Conditions** and **Deny Conditions** panes of the Ranger HAWQ Policy editor.
+
+#### <a id="conditionusergroup"></a> Identifying Users and Groups
+
+You may identify one or more users and/or groups to which to provide or deny access to HAWQ resources in the Allow/Deny Conditions of a HAWQ policy. These users/groups must be known to Ranger.
+
+| Field | Value | Description |
+|-------------|----------------------|------------------------|
+| Group | \<group-name\> | The group(s) to which you want to provide or deny access. All groups sync'd from \<ranger-admin-node\> are available in the picklist. |
+| User | \<user-name\> | The user(s) to which you want to provide or deny access. All users sync'd from \<ranger-admin-node\> or explicitly registered via the Ranger Admin UI are available in the picklist. |
+
+
+#### <a id="conditionperms"></a> Identifying Permissions
+
+You can assign users/groups the following permissions when allowing or denying access to specific HAWQ resources:
+
+| Permission | Description |
+|-------------|-----------------------|
+| select | Select from a table or sequence, or through a protocol |
+| insert | Insert or copy into a table, or insert through a protocol |
+| update | Update a sequence value |
+| delete | This permission is not used by HAWQ |
+| references | This permission is not used by HAWQ |
+| usage | Use a language or sequence |
+| create | Create a table, function, sequence, etc. |
+| connect | Connect to a specific database |
+| execute | ?Create and? Execute a function |
+| temp | Create a temporary table or sequence |
+| create-schema | Create a schema |
+| usage-schema | Use a schema |
+
+These permissions map pretty closely to the privileges you assign when using specific HAWQ `GRANT` commands when configuring HAWQ-Native authorization.
+
+**Note**: The HAWQ Ranger policy editor always displays the complete list of HAWQ permissions. This list is not filtered on the operations supported by the specific resource(s) you identify in the **Policy Details**.
+
+## <a id="createpolicies"></a>Creating HAWQ Policies
+
+You will configure HAWQ-Ranger authorization policies through the Ranger Administrative UI, which you access at `http://<ranger-admin-node>:6080`.
+
+Define more restrictive HAWQ policies first to ensure that you do not accidentally provide unwanted access to specific resources.
+
+It may take a collection of policies to provide access to a specific HAWQ database resource.
+
+MORE HERE
+
+
+### <a id="wildcardinpolicies"></a> Wildcarding in HAWQ Policies
+
+When defining a HAWQ policy, wildcarding (`*`) a leaf node resource will scope the policy at two levels:
+
+1. `*` = no resource - permissions you identify are assigned to the parent resource
+2. `*` = all resources - permissions you identify are assigned to all instances of the resource at that level
+
+For example, consider the following policies assigned to user `hawquser1` for a table named `table99` in the `public` schema of database `testdb`:
+
+ Policy 1: testdb/public/*(table), usage-schema permission
+ Policy 2: testdb/public/table99, select permission
+
+Policies 1 and 2 collectively permit `hawquser1` to access the `public` schema of `testdb` and select from `table99` residing in that schema. In Policy 1, wildcarding is used to scope the permissions to those operations you can perform within the schema (`usage-schema`). `*`\(table\) in this context effectively acts as no tables. Policy 2 restricts the `select` operation to the specific table named `table99`.
+
+Contrast this with the single policy below:
+
+ Policy 10: testdb/public/*(table), usage-schema and select permissions
+
+Policy 10 permits the policy holder to use the `public` schema and select from *any* table in the schema. In this policy, you use wildcarding and a subordinate object privilege (`select`) to apply a permission to **all** instances of the resource. `*`\(table\) in this context effectively applies to all tables.
+
+
+### <a id="dbops"></a> Policies for Database Operations
+
+The database operations governed by HAWQ-Ranger authorization are those that you perform at the purely database-level. These operations include connecting to the database, creating schemas, and creating temporary tables and sequences. Use the following HAWQ Ranger Policy Details to assign permissions for these operations:
+
+| Resource | Value | Description |
+|-------------|----------------------|------------------------|
+| database | \<db-name\> | The database to which you want to provide access |
+| schema | `*` | No schema |
+| table | `*` | No table |
+
+specifying the permissions you wish to assign:
+
+| Permission | Allows SQL Commands | Equivalent GRANT Command |
+|-------------|----------------------|------------------------|
+| connect | CONNECT | GRANT CONNECT ON DATABASE \<db-name\> TO \<user-name\> |
+| create-schema | CREATE SCHEMA | GRANT CREATE ON DATABASE \<db-name\> TO \<user-name\> |
+| temp| CREATE TEMP TABLE<p>CREATE TEMP SEQUENCE | GRANT TEMP ON DATABASE \<db-name\> TO \<user-name\> |
+
+
+### <a id="dbschemaops"></a> Policies for Schema Operations
+
+You perform many HAWQ operations within the scope of a specific database and schema, including creating/dropping/altering database objects. These operations will require permission to use the specified schema.
+
+The HAWQ schema named `public` is the default schema. When HAWQ-Native authorization is in effect, users are automatically granted access to this schema. When Ranger is enabled, users must be explicitly assigned the `usage-schema` permission to the `public` schema.
+
+Use these HAWQ Ranger Policy Details to assign permission for schema-related operations:
+
+| Resource | Value | Description |
+|-------------|----------------------|------------------------|
+| database | \<db-name\> | The database to which you want to provide access |
+| schema | \<schema-name\> | The schema in which you want to provide access |
+| table | sequence | function | `* `| No table/sequence/function |
+
+specifying these permissions:
+
+| Permission | Allows SQL Commands | Equivalent GRANT Command |
+|-------------|------------------------------|----------------------|
+| usage-schema | TOO MANY TO LIST?, built-in HAWQ functions | GRANT USAGE ON SCHEMA \<schema-name\> TO \<user-name\> |
+| create | CREATE [EXTERNAL] TABLE, CREATE SEQUENCE, CREATE FUNCTION, CREATE OPERATOR, CREATE OPERATOR CLASS (superuser only), CREATE AGGREGATE, CREATE VIEW, CREATE TYPE, SELECT INTO, ?MORE? | GRANT CREATE ON SCHEMA \<schema-name\> TO \<user-name\> |
+
+
+### <a id="tblops"></a> Policies for Table Operations
+
+You can insert data into and select a table within schemas in which you have `usage-schema` permissions. Use the following HAWQ Ranger Policy Details to assign permission for these operations:
+
+| Resource | Value | Description |
+|-------------|---------------|-------------------|
+| database | \<db-name\> | The database to which you want to provide access |
+| schema | \<schema-name\> | The schema in which you want to provide access |
+| table | \<table-name\> | The table to which you want to provide access |
+
+specifying the permissions you wish to assign:
+
+| Permission | Allows SQL Commands | Equivalent GRANT Command |
+|-------------|---------------------------|-----------------|
+| select | ANALYZE, COPY INTO, SELECT, VACUUM ANALYZE | GRANT SELECT ON TABLE \<table-name\> TO \<user-name\> |
+| insert | COPY FROM, INSERT | GRANT INSERT ON TABLE \<table-name\> TO \<user-name\> |
+
+
+### <a id="sequenceops"></a> Policies for Sequence Operations
+
+You can use and select sequences and update sequence values in schemas in which you have `usage-schema` permissions. You can also use the `nextval()` and `setval()` HAWQ built-in functions to return and set sequence values. Use the following HAWQ Ranger Policy Details to assign permission for these operations:
+
+| Resource | Value | Description |
+|-------------|----------------------|------------------------|
+| database | \<db-name\> | The database to which you want to provide access |
+| schema | \<schema-name\> | The schema in which you want to provide access |
+| sequence | \<sequence-name\> | The sequence to which you want to provide access |
+
+specifying the permissions you wish to assign:
+
+| Permission | Allows SQL Commands, built-in functions | Equivalent GRANT Command |
+|-------------|---------------------------|------|
+| select | SELECT \<sequence-name\> | GRANT SELECT ON SEQUENCE \<sequence-name\> TO \<user-name\> |
+| usage, update | nextval() | GRANT USAGE, UPDATE ON SEQUENCE \<sequence-name\> TO \<user-name\> |
+| update | setval() | GRANT UPDATE ON SEQUENCE \<sequence-name\> TO \<user-name\> |
+
+
+### <a id="functionops"></a> Policies for Function Operations
+
+You can execute user-defined functions in schemas in which you have `usage-schema` permissions. Use the following HAWQ Ranger Policy Details to assign permission for this operation:
+
+| Resource | Value | Description |
+|-------------|----------------------|------------------------|
+| database | \<db-name\> | The database to which you want to provide access |
+| schema | \<schema-name\> | The schema in which you want to provide access |
+| function | \<function-name\> | The user-defined function to which you want to provide access |
+
+specifying the permissions you wish to assign:
+
+| Permission | Allows SQL Commands | Equivalent GRANT Command |
+|-------------|---------------------------|-------|
+| execute | SELECT \<function-name\>() | GRANT EXECUTE ON FUNCTION \<function-name\> TO \<user-name\> |
+
+**Note**: Functions typically access database objects such as tables, views, sequences, etc and other functions. When setting up your HAWQ policies, ensure you have also provided access to all database resources referenced within the function (recursively).
+
+
+### <a id="dblangops"></a> Policies for Language Operations
+
+Only super-users may register and drop languages for a specific database. These operations are governed by HAWQ-Native authorization.
+
+You may choose to permit users to use a specific language to create user-defined functions. Use these HAWQ Ranger Policy Details to assign such permission:
+
+| Resource | Value | Description |
+|-------------|----------------------|------------------------|
+| database | \<db-name\> | The database to which you want to provide access |
+| language | \<language-name\> | The language to which you want to provide access (plpgsql, sql, other languages explicitly registered in the database) |
+
+specifying these permissions:
+
+| Permission | Allows SQL Commands | Equivalent GRANT Command |
+|-------------|------------------------------|----------------------|
+| usage | CREATE FUNCTION ... LANGUAGE \<language-name\> | GRANT USAGE ON LANGUAGE\<language-name\> TO \<user-name\> |
+
+
+### <a id="dbtblspaceops"></a> Policies for Tablespace Operations
+
+Only super-users may create and drop tablespaces. These operations are governed by HAWQ-Native authorization.
+
+You may choose to allow specific users to create tables within an existing tablespace. Use these HAWQ Ranger Policy Details to assign such permissions:
+
+| Resource | Value | Description |
+|-------------|----------------------|------------------------|
+| tablespace | \<tablespace-name\> | The tablespace to which you want to provide access |
+
+specifying these permissions:
+
+| Permission | Allows SQL Commands | Equivalent GRANT Command |
+|-------------|------------------------------|----------------------|
+| create | CREATE TABLE ... TABLESPACE | GRANT CREATE ON \<tablespace-name\> TO \<user-name\> |
+
+
+### <a id="dbprotocolops"></a> Policies for Protocol Operations
+
+??gpfdist(s) and http protocols - hawq-native or ranger? super-user?
+
+You may choose to permit access to the `pxf` protocol to create readable and writable external tables. Use the following HAWQ Ranger Policy Details to assign permission for these operations:
+
+| Resource | Value | Description |
+|-------------|---------------|-------------------|
+| protocol | \<protocol-name\> | The protocol to which you want to provide access \(pxf\) |
+
+specifying the permissions you wish to assign:
+
+| Permission | Allows SQL Commands | Equivalent GRANT Command |
+|-------------|---------------------------|-----------------|
+| select | CREATE READABLE EXTERNAL TABLE | GRANT SELECT ON PROTOCOL \<protocol-name\> TO \<user-name\> |
+| insert | CREATE WRITABLE EXTERNAL TABLE | GRANT INSERT ON PROTOCOL \<protocol-name\> TO \<user-name\> |
+
+Refer to [Using PXF with Ranger Authorization](#pxfranger) later in this topic for additional considerations when accessing HAWQ PXF external tables with Ranger enabled.
+
+
+## <a id="admintasks"></a>Policy Creation for HAWQ Maintenance Tasks
+
+Administrators periodically perform maintentance tasks on the HAWQ cluster, including vacuuming and analyzing databases. Users performing these operations must be the owner of the databases, and must be explicitly assigned the permissions to do so, just as you would for general database operations.
+
+The `ANALYZE` operation requires `select` permission on the table to be analyzed, as well as `usage-schema` permissions on the schema in which the table resides.
+
+The `VACUUM ANALYZE` operation requires `select` permission on all table(s) specified, as well as `usage-schema` permissions on the schema(s) in which the tables reside.
+
+The `VACUUM` and `TRUNCATE` operations require `usage-schema` permissions on the schema in which the table resides.
+
+
+## <a id="specialconsider"></a>Special Considerations
+
+- The `psql` `search_path` session configuration parameter affects Ranger access control checks for `CREATE` operations. (?all of them?) The object will be created under the *first* schema in the `search_path` in which `usage-schema` permissions were assigned to the user. The schema `search_path` does not affect `SELECT` or other operations.
+
+- When Ranger authorization is enabled for HAWQ, members of HAWQ roles assigned create database permissions must be provided `pg_hba.conf` access to the `postgres` database to use the `createdb` command line utility. This configuration step is not required for `CREATE DATABASE` operations invoked within the `psql` shell.
+
+- `CREATE LANGUAGE` commands (super-user-only) issued for non-built-in languages (pljava, plpython, ..) require the `usage` permission for the `c` language.
+
+- The HAWQ Ranger service definition includes identifying information for the HAWQ master hostname and port. Should you need to activate the standby master in your HAWQ cluster, you must update the HAWQ Ranger service definition with the new master node identifying information. ?WHEN - before or after activating the standby master?.
+
+
+## <a id="permsummary"></a>Summary of Permissions per SQL Command
+
+| SQL Command | Permission | Resource |
+|-------------|----------------------|------------------------|
+| \d | usage-schema | \<db-name\>/public/`*` |
+| ANALYZE \<table-name\>| usage-schema<p>select | \<db-name\>/\<schema-name\>/`*`<p>\<db-name\>/\<schema-name\>/\<table-name\> |
+| ALTER AGGREGATE ... RENAME TO | usage-schema, create | \<db-name\>/\<schema-name\>/`*` |
+| ALTER SEQUENCE | usage-schema | \<db-name\>/\<schema-name\>/`*` |
+| ALTER TABLE ... RENAME | usage-schema | \<db-name\>/\<schema-name\>/`*` |
+| ALTER TABLE \<table-name\> SET DISTRIBUTED BY | usage-schema, create<p>select | \<db-name\>/\<schema-name\>/`*`<p>\<db-name\>/\<schema-name\>/\<table-name\> |
+| BEGIN ... COMMIT | usage-schema | \<db-name\>/\<schema-name\>/`*`<p>## |
+| \c, CONNECT \<db-name\>| connect | \<db-name\>/`*`/`*` |
+| COPY \<table-name\> FROM ** | usage-schema<p>insert, select | \<db-name\>/\<schema-name\>/`*`<p>\<db-name\>/\<schema-name\>/\<table-name\> |
+| COPY \<table-name\> TO | usage-schema<p>select | \<db-name\>/\<schema-name\>/`*`<p>\<db-name\>/\<schema-name\>/\<table-name\> |
+| CREATE AGGREGATE | usage-schema, create<p>execute | \<db-name\>/\<schema-name\>/`*`<p>\<db-name\>/\<schema-name\>/\<sfunc-name\> |
+| CREATE EXTERNAL TABLE (pxf protocol) | usage-schema, create<p>select | \<db-name\>/\<schema-name\>/`*`<p>pxf |
+| CREATE FUNCTION \<function-name\> (trusted \<language-name\>) | usage-schema, create<p>usage<p>execute | \<db-name\>/\<schema-name\>/`*`<p><p>\<db-name\>/\<language-name\><p>\<db-name\>/\<schema-name\>/\<function-name\><p>## |
+| CREATE FUNCTION \<function-name\> (untrusted \<language-name\>) ** | usage-schema, create<p>usage<p>execute | \<db-name\>/\<schema-name\>/`*`<p><p>\<db-name\>/\<language-name\><p>\<db-name\>/\<schema-name\>/\<function-name\><p>## |
+| CREATE LANGUAGE ** | usage | \<db-name\>/c |
+| CREATE OPERATOR | usage-schema, create | \<db-name\>/\<schema-name\>/`*` |
+| CREATE OPERATOR CLASS * | usage-schema, create | \<db-name\>/\<schema-name\>/`*` |
+| CREATE SCHEMA | create-schema | \<db-name\>/`*`/`*` |
+| CREATE SEQUENCE | usage-schema, create | \<db-name\>/\<schema-name\>/`*` |
+| CREATE TABLE | usage-schema, create | \<db-name\>/\<schema-name\>/`*` |
+| CREATE TABLE (in <\private-schema\>) | create | \<db-name\>/\<private-schema\>/`*` |
+| CREATE TABLE ... AS | usage-schema, create<p>select | \<db-name\>/\<schema-name\>/`*`<p>\<db-name\>/\<schema-name\>/\<table-name\> |
+| CREATE ... TABLESPACE \<tablespace-name\> | usage-schema, create<p>create | \<db-name\>/\<schema-name\>/`*`<p>\<tablespace-name\> |
+| CREATE TEMP SEQUENCE | temp | \<db-name\>/`*`/`*` |
+| CREATE TEMP TABLE | temp | \<db-name\>/`*`/`*` |
+| CREATE TYPE | usage-schema, create | \<db-name\>/\<schema-name\>/`*` |
+| CREATE VIEW | usage-schema, create | \<db-name\>/\<schema-name\>/`*` |
+| CREATE WRITABLE EXTERNAL TABLE (pxf protocol) | usage-schema, create<p>insert | \<db-name\>/\<schema-name\>/`*`<p>pxf |
+| DROP AGGREGATE | usage-schema | \<db-name\>/\<schema-name\>/`*` |
+| DROP FUNCTION | usage-schema | \<db-name\>/\<schema-name\>/`*` |
+| DROP SCHEMA | usage-schema | \<db-name\>/\<schema-name\>/`*` |
+| DROP TABLE | usage-schema | \<db-name\>/\<schema-name\>/`*` |
+| DROP VIEW | usage-schema | \<db-name\>/\<schema-name\>/`*` |
+| DROP OPERATOR | usage-schema | \<db-name\>/\<schema-name\>/`*` |
+| DROP OPERATOR CLASS ** | usage-schema | \<db-name\>/\<schema-name\>/`*` |
+| EXECUTE | usage-schema | \<db-name\>/\<schema-name\>/`*`<p>## |
+| EXPLAIN | usage-schema | \<db-name\>/\<schema-name\>/`*`<p>## |
+| INSERT INTO \<table-name\> | usage-schema<p>insert | \<db-name\>/\<schema-name\>/`*`<p>\<db-name\>/\<schema-name\>/\<table-name\> |
+| PREPARE | usage-schema | \<db-name\>/\<schema-name\>/`*` |
+| SELECT \<aggregate-name\> | usage-schema<p>execute<p>execute | \<db-name\>/\<schema-name\>/`*`<p>\<db-name\>/\<schema-name\>/\<aggregate-name\> <p>\<db-name\>/\<schema-name\>/\<sfunc-name\> <p>##|
+| SELECT \<built-in-function\> | usage-schema | \<db-name\>/\<schema-name\>/`*` |
+| SELECT \<function-name\> (trusted ??) | usage-schema<p>execute | \<db-name\>/\<schema-name\>/`*`<p>\<db-name\>/\<schema-name\>/\<function-name\> <p>##|
+| SELECT (using operator) | execute | \<db-name\>/\<schema-name\>/\<operator-procedure\> <p>## |
+| SELECT ... FROM \<table-name\> | usage-schema<p>select | \<db-name\>/\<schema-name\>/`*`<p>\<db-name\>/\<schema-name\>/\<table-name\> |
+| SELECT ... INTO ... FROM \<table-name\> | usage-schema, create<p>select | \<db-name\>/\<schema-name\>/`*`<p>\<db-name\>/\<schema-name\>/\<table-name\> |
+| SELECT ... FROM \<view-name\> | usage-schema<p>select | \<db-name\>/\<schema-name\>/`*`<p>\<db-name\>/\<schema-name\>/\<view-name\><p>## |
+| TRUNCATE | usage-schema | \<db-name\>/\<schema-name\>/`*` |
+| VACUUM | usage-schema | \<db-name\>/\<schema-name\>/`*` |
+| VACUUM ANALYZE \<table-name\>| usage-schema<p>select | \<db-name\>/\<schema-name\>/`*`<p>\<db-name\>/\<schema-name\>/\<table-name\> |
+
+
+**Notes**:
+
+- A `**` in **SQL Command** column identifies a super-user operation.
+
+- A `##` in the **Resource** column signifies that additional policies may be required to provide access to resources used within the operation(s).
+
+
+## <a id="pxfranger"></a>Using PXF with Ranger Authorization
+
+### <a id="pxfrangerhive"></a>Accessing Hive Data
+
+If Ranger is enabled for Hive authorization, you must create Hive policies that allow user `pxf` to access the desired Hive tables.
+
+The HAWQ policies providing access to PXF HCatalog integration must identify database `hcatalog`, schema `<hive-schema-name>`, and table `<hive-table-name>` resources. These privileges are required in addition to any Hive policies for user `pxf` when Ranger is enabled for Hive authorization.
+
+**Note**: When creating HAWQ policies for PXF HCatalog authorization, resource name look up is not available for Hive schema and table names.
+
+### <a id="pxfrangerhdfs"></a>Accessing HDFS Data
+
+If Ranger is enabled for HDFS authorization, you must create HDFS policies that allow user `pxf` to access the HDFS directories backing the PXF tables.
+
+
+## <a id="madlibranger"></a>Using MADLib with Ranger Authorization
+
+You can use MADlib, an open source library for in-database analytics, with your HAWQ installation. MADlib functions typically operate on source, output, and model tables. When Ranger is enabled for HAWQ authorization, you will need to provide access to all MADLib-related databases, schemas, tables, and functions to the appropriate users.
+
+Consider the following when setting up HAWQ policies for MADlib access:
+
+- Assign `temp` permission to the database on which users will run MADlib functions.
+- MADlib users often share their output tables. If this is the case in your deployment, create a shared schema dedicated to output tables, assigning `usage-schema` and `create` privileges for all MADlib users to this shared schema.
+- Assign `create-schema` database permission to those MADlib users that do not choose to share their output tables.
+
+- `madlib` Schema-Level Permissions
+ - Assign `usage-schema` and `create` privileges to the `madlib` schema.
+ - Assign `execute` permissions on all functions within the `madlib` schema, including any functions called within.
+ - Assign `insert` and `select` permissions to all tables within the `madlib` schema.
+ - Assign the `usage-schema` and `create` permissions for the current schema, and any schema in which the source, output, and model tables may reside.
+
+- Function-Specific Permissions
+ - Assign `insert` and `select` permissions for the source, output, and model tables.
+ - Assign `insert` and `select` permissions for the output \_summary and \__group tables.
+
+
+## <a id="bestpractices"></a>Best Practices
+
+- Create policies *before* enabling HAWQ-Ranger authorization. This will ensure access is available to users without any downtime.
+- Define more restrictive HAWQ policies first to ensure that you do not accidentally provide unwanted access to specific resources.
+- Identify and configure your Ranger auditing requirements *before* enabling HAWQ-Ranger authorization.
+- If you use Ranger authorization for Hive, create Hive policy(s) providing the user `pxf` access to any Hive tables you want to expose via PXF HCatalog integration or HAWQ PXF external tables.
+- If you have enabled Ranger authorization for HDFS:
+ - Create an HDFS policy(s) providing user `gpadmin` access to the HDFS HAWQ filespace (?hawq\_dfs\_url?).
+ - If you plan to use PXF external tables to read and write HDFS data, create HDFS policies providing user `pxf` access to the HDFS files backing your PXF external tables.
+
+
+## <a id="troubleshooting"></a>Troubleshooting
+
+| Problem/Error | Discussion |
+|-------------|---------------------------|
+| HAWQ object lookup in Ranger Admin UI not working | If object lookup is not working:<p> 1. Verify that the HAWQ Ranger plug-in JARs and JDBC driver have been copied to \<ranger-admin-node\>.<p> 2. Test the connection between the Ranger Admin UI and the HAWQ master node by clicking the edit icon associated with the active HAWQ service definition, then clicking the **Config Properties: > Test Connection** button.<p> 3. Verify that the HAWQ master node `pg_hba.conf` file includes a `host` entry for \<ranger-admin-node\>, HAWQ user (typically `gpadmin`).|
+
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a7e32e0c/markdown/reference/cli/admin_utilities/hawqstate.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/cli/admin_utilities/hawqstate.html.md.erb b/markdown/reference/cli/admin_utilities/hawqstate.html.md.erb
index e56ecd1..a8c505f 100644
--- a/markdown/reference/cli/admin_utilities/hawqstate.html.md.erb
+++ b/markdown/reference/cli/admin_utilities/hawqstate.html.md.erb
@@ -43,6 +43,7 @@ The `hawq state` utility displays information about a running HAWQ instance. A H
- Master and segment configuration information (hosts, data directories, etc.).
- The ports used by the system.
- Whether a standby master is present, and if it is active.
+- Whether Ranger authorization is enabled for HAWQ, and if so, the status of the HAWQ Ranger Plug-in Service.
## <a id="topic1__section4"></a>Options