Added: incubator/argus/site/trunk/ch_XA-configure.html
URL: http://svn.apache.org/viewvc/incubator/argus/site/trunk/ch_XA-configure.html?rev=1628612&view=auto
==============================================================================
--- incubator/argus/site/trunk/ch_XA-configure.html (added)
+++ incubator/argus/site/trunk/ch_XA-configure.html Wed Oct 1 06:40:31 2014
@@ -0,0 +1,1914 @@
+<!DOCTYPE html>
+<!--
+ | Generated by Apache Maven Doxia
+ | Rendered using Apache Maven Fluido Skin 1.3.1
+-->
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+ <head>
+ <meta charset="UTF-8" />
+ <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+ <meta name="Date-Revision-yyyymmdd" content="20140930" />
+ <meta http-equiv="Content-Language" content="en" />
+ <title>Apache Argus - </title>
+ <link rel="stylesheet" href="./css/apache-maven-fluido-1.3.1.min.css" />
+ <link rel="stylesheet" href="./css/site.css" />
+ <link rel="stylesheet" href="./css/print.css" media="print" />
+
+
+ <script type="text/javascript" src="./js/apache-maven-fluido-1.3.1.min.js"></script>
+
+
+ </head>
+ <body class="topBarDisabled">
+
+
+
+
+ <div class="container-fluid">
+ <div id="banner">
+ <div class="pull-left">
+ <a href="./" id="bannerLeft">
+ <img src="arguslogo/slide1.png" alt="Argus logo" width="400px" height="200px"/>
+ </a>
+ </div>
+ <div class="pull-right"> <div id="bannerRight">
+ <img src="" />
+ </div>
+ </div>
+ <div class="clear"><hr/></div>
+ </div>
+
+ <div id="breadcrumbs">
+ <ul class="breadcrumb">
+
+
+ <li id="publishDate">Last Published: 2014-09-30
+ <span class="divider">|</span>
+ </li>
+ <li id="projectVersion">Version: 0.4
+ </li>
+
+
+
+
+ </ul>
+ </div>
+
+
+ <div class="row-fluid">
+ <div id="leftColumn" class="span3">
+ <div class="well sidebar-nav">
+
+
+ <ul class="nav nav-list">
+ <li class="nav-header">Overview</li>
+
+ <li>
+
+ <a href="index.html" title="Introduction">
+ <i class="none"></i>
+ Introduction</a>
+ </li>
+
+ <li>
+
+ <a href="faq.html" title="FAQ">
+ <i class="none"></i>
+ FAQ</a>
+ </li>
+ <li class="nav-header">Resources</li>
+
+ <li>
+
+ <a href="wiki.html" title="Wiki">
+ <i class="none"></i>
+ Wiki</a>
+ </li>
+
+ <li>
+
+ <a href="http://www.apache.org/licenses/" class="externalLink" title="License">
+ <i class="none"></i>
+ License</a>
+ </li>
+ <li class="nav-header">Project Information</li>
+
+ <li>
+
+ <a href="project-summary.html" title="Project Summary">
+ <i class="none"></i>
+ Project Summary</a>
+ </li>
+
+ <li>
+
+ <a href="mail-lists.html" title="Mailing Lists">
+ <i class="none"></i>
+ Mailing Lists</a>
+ </li>
+
+ <li>
+
+ <a href="team-list.html" title="Team">
+ <i class="none"></i>
+ Team</a>
+ </li>
+ </ul>
+
+
+
+ <hr />
+
+ <div id="poweredBy">
+ <div class="clear"></div>
+
+
+
+
+ <iframe src="http://www.facebook.com/plugins/like.php?href=http://argus.incubator.apache.org/&send=false&layout=box_count&show-faces=false&action=like&colorscheme=light"
+ scrolling="no" frameborder="0"
+ style="border:none; width:48px; height:63px; margin-top: 10px;" ></iframe>
+ <div class="clear"></div>
+
+
+
+ <div id="twitter">
+
+ <a href="https://twitter.com/apacheargus" class="twitter-follow-button" data-show-count="true" data-align="left" data-size="medium" data-show-screen-name="true" data-lang="en">Follow apacheargus</a>
+ <script type="text/javascript">!function(d,s,id){var js,fjs=d.getElementsByTagName(s)[0];if(!d.getElementById(id)){js=d.createElement(s);js.id=id;js.src="//platform.twitter.com/widgets.js";fjs.parentNode.insertBefore(js,fjs);}}(document,"script","twitter-wjs");</script>
+
+ </div>
+ <div class="clear"></div>
+ <div class="clear"></div>
+ <a href="http://maven.apache.org/" title="Maven" class="builtBy">
+ <img class="builtBy" alt="Maven" src="http://maven.apache.org/images/logos/maven-feather.png" />
+ </a>
+ </div>
+ </div>
+ </div>
+
+
+ <div id="bodyColumn" class="span9" >
+
+
+ Configure Repositories and Install Security Agents
+
+ Argus Administration tools allow you to audit activity and
+ enforce access policies for up to ten different Hadoop clusters. Access Policies and Audited
+ events are created and stored in the Argus Administration
+ server and pushed to Security Agents installed on Hadoop cluster nodes.
+ The Security Agents integrate with data services in the Hadoop cluster to enforce access
+ policies and audit activity. The agents are installed on cluster nodes as follows:
+
+
+
+ HDFS Security Agent is installed on the NameNode host and in HA (High
+ Availability) clusters also on the stand-by NN.
+
+
+ Hive Security Agent is installed on the HiveServer2 host.
+
+
+ HBase Security Agents are installed on each HBase Regional Server host.
+
+
+
+
+ <div class="section">
+<h2><a name="null"></a></h2>
+
+ The HDFS repository contains access policies for the Hadoop cluster HDFS. The Security
+ Agent integrates with the NameNode service on the NameNode host. The agent enforces the
+ policy's configured in the Argus Administration Web UI and sends HDFS audit
+ information to the portal where it can be viewed and reported on from a central
+ location.
+
+ In Apache Ambari managed environments additional configuration is required. Ensure that
+ you carefully follow the steps outlined in the <link linkend="ch_XA-conf-nn-conf-ambari">Configure Hadoop Agent to run in Ambari
+ Environments</link>.
+
+
+<div class="section">
+<h2><a name="null"></a></h2>
+
+ Add HDFS repositories after the Hadoop environment is fully operational. During
+ the initial set up of the repository, Hortonworks recommends testing the connection
+ from the Argus Administration Web UI to the NameNode to ensure that the agent
+ will be able to connect to the server after installation is complete.
+
+<div class="section">
+<h2><a name="null"></a></h2>
+
+ Before installing the agent on the NameNode, create a HDFS Repository as follows:
+
+ Sign in to the Argus Administration Web UI and click
+ Policy Manager.
+
+
+
+
+
+
+
+
+ Next to HDFS, click the + (plus symbol).
+ The Create Repository page displays.
+
+
+
+
+
+
+
+
+ Complete the Repository Details:
+<table border="0" class="table table-striped" frame="all">
+
+
+
+
+
+ <thead>
+
+ Label
+ Value
+ Description
+
+ </thead>
+ <tbody>
+
+ Repository Name
+ $name
+ Specify a unique name for the repository, you
+ will need to specify the same repository name in
+ the agent installation properties. For example,
+ clustername_hdfs.
+
+
+ Description
+ $description-of-repo
+ Enter a description up to 150
+ characters.
+
+
+ Active Status
+ Enabled or
+ Disabled
+ Enable or disable policy enforcement for the
+ repository.
+
+
+ Repository type
+ HDFS,
+ Hive, or
+ HBase
+ Select the type of repository, HDFS.
+
+
+ User name
+
+ $user
+ Specify a user name on the remote system with
+ permission to establish the connection, for
+ example hdfs.
+
+
+ Password
+ $password
+ Specify the password of the user account for
+ connection.
+
+ </tbody>
+
+ </table>
+
+
+ Complete the security settings for the Hadoop cluster, the
+ settings must match the values specified in the
+ core-site.xml file as follows:
+<table border="0" class="table table-striped" frame="all">
+
+
+
+
+
+ <thead>
+
+ Label
+ Value
+ Description
+
+ </thead>
+ <tbody>
+
+ fs.default.name
+ $hdfs-url
+ HDFS URL, should match the setting in the
+ Hadoop core-site.xml file.
+ For example,
+ hdfs://sandbox.hortonworks.com:8020
+
+
+ hadoop.security.authorization
+ true or
+ false
+ Specify the same setting found in the
+ core-site.xml.
+
+
+ hadoop.security.authentication
+ simple or
+ kerberos
+ Specify the type indicated in the
+ core-site.xml.
+
+
+ hadoop.security.auth_to_local
+ $usermapping
+ Must match the setting in the core-site.xml
+ file. For example:
+ RULE:[2:$1@$0]([rn]m@.*)s/.*/yarn/
+ RULE:[2:$1@$0](jhs@.*)s/.*/mapred/
+ RULE:[2:$1@$0]([nd]n@.*)s/.*/hdfs/
+ RULE:[2:$1@$0](hm@.*)s/.*/hbase/
+ RULE:[2:$1@$0](rs@.*)s/.*/hbase/
+ DEFAULT
+
+
+ dfs.datanode.kerberos.principal
+ $dn-principal
+ Specify the Kerberos DataNode principal
+ name.
+
+
+ dfs.namenode.kerberos.principal
+ $nn-principal
+ Specify the Kerberos NameNode principal
+ name.
+
+
+ dfs.secondary.namenode.kerberos.principal
+ $secondary-nn-principal
+ Specify the Kerberos Secondary NN principal
+ name.
+
+
+ Common Name For
+ Certificate
+ $cert-name
+ Specify the name of the certificate.
+
+ </tbody>
+
+ </table>
+
+
+ Click Test Connection.
+ If the server can connect to HDFS, the connection successful
+ message displays. If the connection fails, go to the troubleshooting
+ appendix.
+
+
+ After making a successful connection, click
+ Save.
+
+
+ </div>
+
+<div class="section">
+<h2><a name="null"></a></h2>
+
+ Install the agent on the NameNode Host as root (or sudo
+ privileges). In HA Hadoop clusters, you must also install an agent on the
+ Secondary NN.
+
+<div class="section">
+<h2><a name="null"></a></h2>
+
+ Perform the following steps on the Hadoop
+ NameNode host.
+
+ Log on to the host as
+ root.
+
+
+ Create a temporary directory,
+ such as
+ /tmp/xasecure:mkdir /tmp/xasecure
+
+
+ Move the package into the
+ temporary directory along with the
+ MySQL Connector Jar.
+
+
+ Extract the
+ contents:tar xvf $xasecureinstallation.tar
+
+
+ Go to the directory where you
+ extracted the installation
+ files:cd /tmp/xasecure/xasecure-$name-$build-version
+
+
+ Open the
+ install.properties
+ file for editing.
+
+
+ Change the following parameters
+ for your environment:
+<table border="0" class="table table-striped" frame="all">
+
+
+
+
+
+ <thead>
+
+ Parameter
+ Value
+ Description
+
+ </thead>
+ <tbody>
+
+ POLICY_MGR_URL
+ $url
+ Specify the full URL to
+ access the Policy
+ Manager Web UI. For
+ example,
+ http://pm-host:6080.
+
+
+ MYSQL_CONNECTOR_JAR
+ $path-to-mysql-connector
+ Absolute path on the local
+ host to the JDBC driver for mysql
+ including filename.
+ Download the JAR from <link xlink:href="http://www.mysql.com/products/connector/" xlink:show="new">here</link>.
+ For example,
+ /tmp/xasecure/
+
+
+ REPOSITORY_NAME
+ $Policy-Manager-Repo-Name
+ Name of the HDFS Repository
+ in the Policy Manager that this
+ agent connects to after
+ installation.
+
+
+ XAAUDIT.DB.HOSTNAME
+ $XAsecure-db-host
+ Specify the host name of the
+ MySQL database.
+
+
+ XAAUDIT.DB.DATABASE_NAME
+ $auditdb
+ Specify the audit database
+ name that matches the
+ audit_db_name
+ specified during the web
+ application server
+ installation.
+
+
+ XAAUDIT.DB.USER_NAME
+ $auditdbuser
+ Specify the audit database
+ name that matches the
+ audit_db_user
+ specified during the web
+ application server
+ installation
+
+
+ XAAUDIT.DB.PASSWORD
+ $auditdbupw
+ Specify the audit database
+ name that matches the
+ audit_db_password
+ specified during the web
+ application server
+ installation.
+
+ </tbody>
+
+ </table>
+
+
+ Save the
+ install.properties
+ file.
+
+
+ If your environment is configured to
+ use SSL, modify the properties
+ following the instructions in
+ <link xlink:href="http://dev.hortonworks.com.s3.amazonaws.com/HDPDocuments/HDP2/HDP-2-trunk/bk_HDPSecure_Admin/content/ch_ssl-hdfsagent.html">Set Up SSL for HDFS Security
+ Agent</link>.
+
+
+ </div>
+
+
+
+<div class="section">
+<h2><a name="null"></a></h2>
+
+ On Hadoop clusters managed by Ambari, change the default HDFS settings to
+ allow the agent to enforce policies and report auditing events.
+ Additionally, Ambari uses its own startup scripts to start and stop the
+ NameNode server. Therefore, modify the Hadoop configuration script to
+ include the Security Agent with a NameNode restart.
+ To configure HDFS properties and NameNode startup scripts:
+
+ Update HDFS properties from the Ambari Web Interface as
+ follows:
+
+
+ On the Dashboard, click
+ HDFS.
+ The HDFS Service page displays.
+
+
+ Go to the Configs tab.
+
+
+ In Filter, type
+ dfs.permissions.enabled and press
+ enter.
+ The results display. This property is located under
+ Advanced.
+
+
+
+
+
+
+
+
+ Expand Advanced, then change
+ dfs.permissions.enabled to
+ true.
+
+
+ In Filter, type
+ hadoop.security.authorization and
+ press enter.
+ Under the already expanded Advanced option, the
+ parameter displays.
+
+
+
+
+
+
+
+
+ Change
+ hadoop.security.authorization to
+ true.
+
+
+ Scroll to the bottom of the page and click
+ Save.
+ At the top of the page, a message displays indicating
+ the services that need to be restarted.
+
+ Do not restart the services until after you
+ perform the next step.
+
+
+
+
+
+ Change the Hadoop configuration script to start the Security
+ Agent with the NameNode service:
+
+ In the Ambari Administrator Portal, click
+ HDFS and then
+ NameNode.
+ The NameNode Hosts page displays.
+
+
+ Click Host Actions and choose
+ Turn on Maintenance
+ Mode.
+
+
+
+
+
+
+ Wait for the cluster to enter maintenance mode.
+
+
+
+ SSH to the NameNode as the root
+ user.
+
+
+ Open the hadoop-config.sh
+ script for editing and go to the end of the file.
+ For
+ example:vi /usr/lib/hadoop/libexec/hadoop-config.sh
+
+
+ At the end of the file paste the following
+ statement:if [ -f ${HADOOP_CONF_DIR}/xasecure-hadoop-env.sh ]
+then
+ . ${HADOOP_CONF_DIR}/xasecure-hadoop-env.sh
+fi
+ This adds the Security Agent for Hadoop to the
+ start script for Hadoop.
+
+
+ Save the changes.
+
+
+
+
+ In the Ambari Administrative Portal, click
+ Services > Service
+ Actions > Restart All.
+
+
+
+
+
+
+ Wait for the services to completely restart.
+
+
+ Click Services > Service
+ Actions > Turn off Maintenance
+ Mode.
+ It may take several minutes for the process to complete. After
+ confirming all the services restart as expected, perform a few
+ simple HDFS comments such as browsing the file system from
+ Hue.
+
+
+ </div>
+
+ </div>
+
+<div class="section">
+<h2><a name="null"></a></h2>
+
+ After completing the setup of the HDFS Repository and agent, perform a few
+ simple tests to ensure that the agent is auditing and reporting events to the
+ Argus Administration Web UI. By default, the repository allows all access
+ and has auditing enabled.
+
+ Log into the Hadoop cluster.
+
+
+ Type the following command to display a list of items at the root
+ folder of
+ HDFS:hadoop fs -ls /
+Found 6 items
+drwxrwxrwx - yarn hadoop 0 2014-04-21 07:21 /app-logs
+drwxr-xr-x - hdfs hdfs 0 2014-04-21 07:23 /apps
+drwxr-xr-x - mapred hdfs 0 2014-04-21 07:16 /mapred
+drwxr-xr-x - hdfs hdfs 0 2014-04-21 07:16 /mr-history
+drwxrwxrwx - hdfs hdfs 0 2014-06-17 15:05 /tmp
+drwxr-xr-x - hdfs hdfs 0 2014-04-22 07:21 /user
+
+
+ Sign in to the Web UI and click
+ Audit.
+ The Big Data page displays a list of events for the configured
+ Repositories.
+
+
+ Click Search > Repository
+ Type > HDFS.
+ The list filters as you make selections.
+
+
+ </div>
+ </div>
+ </div>
+
+<div class="section">
+<h2><a name="null"></a></h2>
+
+ Argus Administration tools support access control and auditing for Hive
+ repositories in Hadoop clusters.
+
+<div class="section">
+<h2><a name="null"></a></h2>
+
+ Before installing the agent on the HiveServer2 host set up a repository in the
+ Policy Manager.
+
+ For Hive connection information, see <link xlink:href=" https://cwiki.apache.org/confluence/display/Hive/HiveServer2+Clients#HiveServer2Clients-JDBC">HiveServer2 Clients, JDBC</link>.
+
+ To create a Hive Repository:
+
+ Sign in to the Argus Administrator Web UI as an
+ administrator.
+
+
+ Click Policy Manager.
+ The Manage Repository page displays.
+
+
+
+
+
+
+
+ Next to Hive, click the green plus symbol.
+ The Create Repository page displays.
+
+
+
+
+
+
+
+
+ Complete the required settings with the following information:
+<table border="0" class="table table-striped" frame="all">
+
+
+
+
+
+ <thead>
+
+ Label
+ Value
+ Description
+
+ </thead>
+ <tbody>
+
+ Repository Name
+ $name
+ Specify a unique name for the repository, you
+ will need to specify the repository name in the
+ agent installation properties. For example,
+ clustername_hive.
+
+
+ Description
+ $description-of-repo
+ Enter a description up to 150 characters.
+
+
+ Active Status
+ Enabled or
+ Disabled
+ Enable or disable policy enforcement for the
+ repository.
+
+
+ Repository type
+ HDFS, Hive,
+ or HBase
+ Select the type of repository, Hive.
+
+
+ User name
+
+ $user
+ Specify a user name on the remote system with
+ permission to establish the connection with the
+ hive, for example hive.
+
+
+ Password
+ $password
+ Specify the password of the user account for
+ connection.
+
+
+ jdbc.driverClassName
+ $classname
+ Specify the full classname of the driver used for
+ Hive connections. The default HiveServer2 classname
+ is
+ org.apache.hive.jdbc.HiveDriver.
+
+
+ jdbc.url
+ $jdbc:hive2://hiveserver-host:port/db
+ Specify the complete connection URL, including
+ port (default port is 10000) and database name. For
+ example on sandbox,
+ jdbc:hive2://sandbox:10000/.
+
+
+ </tbody>
+
+ </table>
+
+
+ Click Test Connection.
+ If the server can establish a connection with HiveServer using the
+ information you provided a success message displays.
+
+
+ After the connection is successful, click
+ Save.
+
+
+ </div>
+
+<div class="section">
+<h2><a name="null"></a></h2>
+
+ After creating the Hive Repository in the Policy Manager, install the agent on the
+ HiveServer2 host.
+
+<div class="section">
+<h2><a name="null"></a></h2>
+
+ Perform the following steps on the HiveServer2 host.
+
+ Log on to the host as
+ root.
+
+
+ Create a temporary directory, such
+ as
+ /tmp/xasecure:mkdir /tmp/xasecure
+
+
+ Move the package into the temporary
+ directory along with the MySQL
+ Connector Jar.
+
+
+ Extract the
+ contents:tar xvf $xasecureinstallation.tar
+
+
+ Go to the directory where you
+ extracted the installation
+ files:cd /tmp/xasecure/xasecure-$name-$build-version
+
+
+ Open the
+ install.properties
+ file for editing.
+
+
+ Change the following parameters for
+ your environment:
+<table border="0" class="table table-striped" frame="all">
+
+
+
+
+
+ <thead>
+
+ Parameter
+ Value
+ Description
+
+ </thead>
+ <tbody>
+
+ POLICY_MGR_URL
+ $url
+ Specify the full URL to
+ access the Policy
+ Manager Web UI. For
+ example,
+ http://pm-host:6080.
+
+
+ MYSQL_CONNECTOR_JAR
+ $path-to-mysql-connector
+ Absolute path on the local
+ host to the JDBC driver for mysql
+ including filename.
+ Download the JAR from <link xlink:href="http://www.mysql.com/products/connector/" xlink:show="new">here</link>.
+ For example,
+ /tmp/xasecure/
+
+
+ REPOSITORY_NAME
+ $Policy-Manager-Repo-Name
+ Name of the HDFS Repository
+ in the Policy Manager that this
+ agent connects to after
+ installation.
+
+
+ XAAUDIT.DB.HOSTNAME
+ $XAsecure-db-host
+ Specify the host name of the
+ MySQL database.
+
+
+ XAAUDIT.DB.DATABASE_NAME
+ $auditdb
+ Specify the audit database
+ name that matches the
+ audit_db_name
+ specified during
+ installation.
+
+
+ XAAUDIT.DB.USER_NAME
+ $auditdbuser
+ Specify the audit database
+ name that matches the
+ audit_db_user
+ specified during
+ installation
+
+
+ XAAUDIT.DB.PASSWORD
+ $auditdbupw
+ Specify the audit database
+ name that matches the
+ audit_db_password
+ specified during
+ installation
+
+ </tbody>
+
+ </table>
+
+
+ Save the
+ install.properties
+ file.
+
+
+ If your environment is configured to use
+ SSL, modify the properties following the
+ instructions in <link xlink:href="http://dev.hortonworks.com.s3.amazonaws.com/HDPDocuments/HDP2/HDP-2-trunk/bk_HDPSecure_Admin/content/ch_ssl-hiveagent.html">Set Up SSL for Hive Security
+ Agent</link>.
+
+ </div>
+
+
+<div class="section">
+<h2><a name="null"></a></h2>
+
+ After installing the agent in an environment that does NOT have Ambari,
+ manually restart the Hive services as follows:
+
+
+ Stop Hive. Execute this command on the Hive Metastore and Hive Server2
+ host
+ machine.ps aux | awk '{print $1,$2}' | grep hive | awk '{print $2}' | xargs kill >/dev/null 2>&1
+
+
+ Start Hive Metastore. On the Hive Metastore host machine, execute the
+ following command:
+
+ su - hive -c "env HADOOP_HOME=/usr JAVA_HOME=/usr/jdk64/jdk1.6.0_31 /tmp/startMetastore.sh /var/log/hive/hive.out /var/log/hive/hive.log /var/run/hive/hive.pid /etc/hive/conf.server"
+
+ where, $HIVE_LOG_DIR is the directory
+ where Hive server logs are stored. For example,
+ /var/logs/hive.
+
+
+ Start HiveServer2. On the Hive Server2 host machine, execute the
+ following
+ command:su - hive -c "env JAVA_HOME=/usr/jdk64/jdk1.6.0_31 /tmp/startHiveserver2.sh /var/log/hive/hive-server2.out /var/log/hive/hive-server2.log /var/run/hive/hive-server.pid /etc/hive/conf.server"
+ where $HIVE_LOG_DIR is the directory
+ where Hive server logs are stored. For example,
+ /var/logs/hive.
+
+
+ </div>
+ </div>
+
+<div class="section">
+<h2><a name="null"></a></h2>
+
+ Follow the configuration steps in environments where Hive is managed by the Ambari Server:
+
+ <link linkend="ch_XA-configure-ambari-hivestart">Modify the Ambari
+ Hive Startup Script</link>
+
+
+ <link linkend="ch_XA-configure-ambari-hiveconf">Configure
+ Hive</link>
+
+
+
+<div class="section">
+<h2><a name="null"></a></h2>
+
+ Remove the HiveServer configuration string from the Ambari Hive startup
+ script.
+
+ Ambari starts and stops the HiveServer2 using a built in script. In order
+ to start and stop HiveServer2 with the integrated Security Agent, you must
+ comment out the HiveServer configuration string.
+
+
+
+ Log into the Ambari Server Linux host using the Ambari account.
+
+
+ Open the Ambari Hive startup script for editing:
+
+ cd /var/lib/ambari-server/resources/stacks/HDP/2.0.6/services/HIVE/package/templates
+vi startHiveserver2.sh.j2
+
+
+
+ Comment out the following line by prepending a # at the beginning of
+ the line as
+ follows:# HIVE_SERVER2_OPTS="${HIVE_SERVER2_OPTS} –hiveconf hive.security.authenticator.manager=org.apache.hadoop.hive.ql.security.SessionStateUserAuthenticator -hiveconf hive.security.authorization.manager=org.apache.hadoop.hive.ql.security.authorization.plugin.sqlstd.SQLStdHiveAuthorizerFactory"
+
+
+ Restart the Ambari Server from the command line as
+ follows:su -l ambari -c "/etc/init.d/ambari-server stop"
+su -l ambari -c "/etc/init.d/ambari-server start"
+
+
+ On each node in the cluster, restart the Ambari
+ Agents:su -l ambari -c "/etc/init.d/ambari-agent stop"
+su -l ambari -c "/etc/init.d/ambari-agent start"
+
+
+ After the Ambari Server and Agents finish rebooting, update the Hive
+ Configuration with the required settings.
+ </div>
+
+<div class="section">
+<h2><a name="null"></a></h2>
+
+ After changing the Ambari Hive startup script and restarting the Ambari Server
+ from the command line, perform the following steps to configure Hive server for
+ agent integration.
+
+
+ Log into the Ambari Web UI, and click Hive >
+ Config.
+
+ To find a property, type the name in the Filter field and press
+ enter; if the parameter exists, it is returned under the common or
+ advanced list. Click the arrow key to expand the lists to see the
+ settings.
+
+
+
+ Update the following properties as
+ follows:
+
+
+
+ Property name:
+ hive.security.authorization.manager
+
+ New
+ Value:
+ com.xasecure.authorization.hive.authorizer.XaSecureAuthorizer
+
+
+ Property name:
+ hive.security.authorization.enabled
+
+ New
+ Value:
+ true
+
+
+
+
+
+ Filter for the hive.exec.pre.hooks property.
+ Add the Argus hook after the existing value by inserting a
+ comma followed by
+ com.xasecure.authorization.hive.hooks.XaSecureHivePreExecuteRunHook.
+ For example, if the existing value is
+ org.apache.hadoop.hive.ql.hooks.ATSHook the new
+ value with the Argus hook is:
+ org.apache.hadoop.hive.ql.hooks.ATSHook,com.xasecure.authorization.hive.hooks.XaSecureHivePreExecuteRunHook
+
+
+ Search for the hive.exec.post.hooks property.
+ Add the Argus hook after the existing value by inserting a
+ comma followed by
+ com.xasecure.authorization.hive.hooks.XaSecureHivePostExecuteRunHook.
+ For example if the existing value is
+ org.apache.hadoop.hive.ql.hooks.ATSHook the new
+ value with the Argus hook
+ is:org.apache.hadoop.hive.ql.hooks.ATSHook,com.xasecure.authorization.hive.hooks.XaSecureHivePostExecuteRunHook
+
+
+ Expand Custom hive-site.xml, and add the
+ following properties:
+<table border="0" class="table table-striped" frame="all">
+
+
+
+
+ <thead>
+
+ Key
+ Value
+
+ </thead>
+ <tbody>
+
+ hive.semantic.analyzer.hook
+ com.xasecure.authorization.hive.hooks.XaSecureSemanticAnalyzerHook
+
+
+ hive.server2.custom.authentication.class
+ com.xasecure.authentication.hive.LoginNameAuthenticator
+
+
+ hive.conf.restricted.list
+ hive.exec.driver.run.hooks,
+ hive.server2.authentication,
+ hive.metastore.pre.event.listeners,
+ hive.security.authorization.enabled,hive.security.authorization.manager,
+ hive.semantic.analyzer.hook,
+ hive.exec.post.hooks
+
+ </tbody>
+
+ </table>
+
+ For each property, click Add Property,
+ enter Key and Value shown in the table above, then click
+ Add.
+
+
+
+ After all the properties have been updated and added, scroll to the
+ bottom and click Save.
+ The settings display under Custom
+ hive-site.xml.
+
+
+
+
+
+
+ When properties change, the affected services must be restarted. A
+ Restart option displays.
+
+
+ Click Restart > Restart
+ all.
+
+
+ </div>
+ </div>
+
+ </div>
+
+<div class="section">
+<h2><a name="null"></a></h2>
+
+ HBase agents integrate with the HBase Master and HBase
+ Regional Servers.
+
+ When adding an HBase Repository you must install the
+ Security Agent for HBase on the HBase Master and each
+ of the HBase Regional Servers in your cluster and
+ ensure that the configuration settings are the
+ same.
+
+
+<div class="section">
+<h2><a name="null"></a></h2>
+
+ Argus Administration requires that the following properties are set in the
+ hbase-site.xml. Configure these properties and restart
+ Hbase before creating a repository in the Policy Manager.
+
+
+<table border="0" class="table table-striped" frame="all">
+
+
+
+
+ <thead>
+
+ Key
+ Value
+
+ </thead>
+ <tbody>
+
+ hbase.security.authorization
+ true
+
+
+ hbase.coprocessor.master.classes
+ com.xasecure.authorization.hbase.XaSecureAuthorizationCoprocessor
+
+
+ hbase.coprocessor.region.classes
+ org.apache.hadoop.hbase.security.token.TokenProvider,org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint,com.xasecure.authorization.hbase.XaSecureAuthorizationCoprocessor
+
+
+ hbase.rpc.engine
+ org.apache.hadoop.hbase.ipc.SecureRpcEngine
+
+
+ hbase.rpc.protection
+ PRIVACY
+
+ </tbody>
+
+ </table>
+
+
+<div class="section">
+<h2><a name="null"></a></h2>
+
+ Use these instructions to update the Hbase properties in the Ambari UI.
+
+
+ Log into the Ambari Web UI, and click HBase >
+ Config.
+
+ To find a parameter, type the parameter name in the Filter field
+ and press enter; if the parameter exists, it is returned under list.
+ Click the arrow key to expand the lists and see the parameter
+ settings.
+
+
+
+ Update the following properties from the Ambari Default Value to the
+ Argus required values:
+<table border="0" class="table table-striped" frame="all">
+
+
+
+
+
+ <thead>
+
+ HBase Property
+ Ambari Default Value
+ Argus Required Value
+
+ </thead>
+ <tbody>
+
+ hbase.security.authorization
+ false
+ true
+
+ </tbody>
+
+ </table>
+
+
+ Expand Custom hbase-site.xml, and add the
+ following properties:
+<table border="0" class="table table-striped" frame="all">
+
+
+
+
+ <thead>
+
+ Key
+ Value
+
+ </thead>
+ <tbody>
+
+ hbase.coprocessor.master.classes
+ com.xasecure.authorization.hbase.XaSecureAuthorizationCoprocessor
+
+
+ hbase.coprocessor.region.classes
+ org.apache.hadoop.hbase.security.token.TokenProvider,org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint,com.xasecure.authorization.hbase.XaSecureAuthorizationCoprocessor
+
+
+ hbase.rpc.protection
+ PRIVACY
+
+ </tbody>
+
+ </table>
+
+ For each property, click Add Property,
+ enter Key and Value shown in the table above, then click
+ Add.
+
+
+
+ After all the properties have been updated or added, click
+ Save.
+ The Custom hbase-site.xml properties
+ display.
+
+
+
+
+
+
+ When properties change, the affected services must be restarted. A
+ Restart option displays.
+
+
+ Click Restart > Restart
+ all.
+
+
+ </div>
+ </div>
+
+<div class="section">
+<h2><a name="null"></a></h2>
+
+ Before installing the agent on the HBase Regional Servers, create an HBase
+ Repository as follows:
+
+ Sign in to the Argus Administration Web UI.
+
+
+ Click Policy Manager.
+ The Manage Repository page displays.
+
+
+
+
+
+
+
+ Next to HBase, click the + (plus symbol).
+ The Create Repository page displays.
+
+
+
+
+
+
+
+
+ Complete the Repository Details with the following information:
+<table border="0" class="table table-striped" frame="all">
+
+
+
+
+
+ <thead>
+
+ Label
+ Value
+ Description
+
+ </thead>
+ <tbody>
+
+ Repository Name
+ $name
+ Specify a unique name for the repository, you
+ will need to specify the same repository name in the
+ agent installation properties. For example,
+ clustername_hbase.
+
+
+ Description
+ $description-of-repo
+ Enter a description up to 150 characters.
+
+
+ Active Status
+ Enabled or
+ Disabled
+ Enable or disable policy enforcement for the
+ repository.
+
+
+ Repository type
+ HDFS, Hive,
+ or HBase
+ Select the type of repository, HBase.
+
+
+ User name
+
+ $user
+ Specify a user name on the remote system with
+ permission to establish the connection, for example
+ hbase.
+
+
+ Password
+ $password
+ Specify the password of the user account for
+ connection.
+
+ </tbody>
+
+ </table>
+
+
+ Complete the HBase Configuration:
+
+
+
+
+
+
+ The settings must match the values specified in the
+ core-site.xml and
+ hbase-site.xml file as follows:
+<table border="0" class="table table-striped" frame="all">
+
+
+
+
+
+ <thead>
+
+ Label
+ Value
+ File
+
+ </thead>
+ <tbody>
+
+ fs.default.name
+ $hdfs-url
+ core-site.xml
+ For example,
+ hdfs://sandbox.hortonworks.com:8020
+
+
+ hadoop.security.authorization
+ true
+ core-site.xml
+ If this field is false, then change
+ to true in core-site before you
+ continue.
+
+
+ hadoop.security.authentication
+ simple or
+ kerberos
+ core-site.xml
+
+
+ hadoop.security.auth_to_local
+ $usermapping
+ core-site.xmlFor example:
+ RULE:[2:$1@$0]([rn]m@.*)s/.*/yarn/
+ RULE:[2:$1@$0](jhs@.*)s/.*/mapred/
+ RULE:[2:$1@$0]([nd]n@.*)s/.*/hdfs/
+ RULE:[2:$1@$0](hm@.*)s/.*/hbase/
+ RULE:[2:$1@$0](rs@.*)s/.*/hbase/
+ DEFAULT
+
+
+ dfs.datanode.kerberos.principal
+ $dn-principal
+ Specify the Kerberos DataNode principal
+ name.
+
+
+ dfs.namenode.kerberos.principal
+ $nn-principal
+ Specify the Kerberos NameNode principal
+ name.
+
+
+ dfs.secondary.namenode.kerberos.principal
+ $secondary-nn-principal
+ Specify the Kerberos Secondary NN principal
+ name.
+
+
+ hbase.master.kerberos.principal
+ $hbase-principal
+ Specify the Kerberos principal for the HBase
+ Master.
+
+
+ hbase.rpc.engine
+ org.apache.hadoop.hbase.ipc.SecureRpcEngine
+ hbase-site.xml
+
+
+ hbase.rpc.protection
+ PRIVACY
+ hbase-site.xml
+
+
+ hbase.security.authentication
+ simple
+ hbase-site.xml
+
+
+ hbase.zoopkeeper.property.clientPort
+ 2181
+ hbase-site.xml
+
+
+ hbase.zookeeper.quorom
+
+ hbase-site.xml
+
+
+ zookeeper.znode.parent
+ /hbase
+ hbase-site.xml
+
+
+ Common Name For
+ Certificate
+ $cert-name
+ Specify the name of the certificate.
+
+ </tbody>
+
+ </table>
+
+ The blank fields are optional.
+
+
+
+ Click Test Connection.
+ If the server can connect to HBase, the connection successful message
+ displays.
+ Argus Administration server connects to HBase and lists the
+ tables. Hortonworks recommends creating the repository and installing
+ the agent after HBase contains data. If HBase connection fails (and
+ tables exist), go to the troubleshooting appendix.
+
+
+ After making a successful connection, click
+ Save.
+
+
+ The repository is created with an open access Policy, that is auditing is enabled
+ and all users are allowed to access the resources. Complete the installation of the
+ agent and do a few simple access test before configuring policies to ensure that the
+ solution is working properly.
+ </div>
+
+<div class="section">
+<h2><a name="null"></a></h2>
+
+ Use same installation properties file to install the
+ Security Agent for HBase. Install the agent on
+ all of the
+ following HBase hosts:
+
+ HBase Master host
+
+
+ All HBase Regional Server host
+
+
+
+
+ Log on to the host as
+ root.
+
+
+ Create a temporary directory, such as
+ /tmp/xasecure:mkdir /tmp/xasecure
+
+
+ Move the package into the temporary
+ directory along with the MySQL Connector
+ Jar.
+
+
+ Extract the
+ contents:tar xvf $xasecureinstallation.tar
+
+
+ Go to the directory where you extracted the
+ installation
+ files:cd /tmp/xasecure/xasecure-$name-$build-version
+
+
+ Open the
+ install.properties
+ file for editing.
+
+
+ Change the following parameters for your
+ environment:
+<table border="0" class="table table-striped" frame="all">
+
+
+
+
+
+ <thead>
+
+ Parameter
+ Value
+ Description
+
+ </thead>
+ <tbody>
+
+ POLICY_MGR_URL
+ $url
+ Specify the full URL to
+ access the Policy
+ Manager Web UI. For
+ example,
+ http://pm-host:6080.
+
+
+ MYSQL_CONNECTOR_JAR
+ $path-to-mysql-connector
+ Absolute path on the local
+ host to the JDBC driver for mysql
+ including filename.
+ Download the JAR from <link xlink:href="http://www.mysql.com/products/connector/" xlink:show="new">here</link>.
+ For example,
+ /tmp/xasecure/
+
+
+ REPOSITORY_NAME
+ $Policy-Manager-Repo-Name
+ Name of the HDFS Repository
+ in the Policy Manager that this
+ agent connects to after
+ installation.
+
+
+ XAAUDIT.DB.HOSTNAME
+ $XAsecure-db-host
+ Specify the host name of the
+ MySQL database.
+
+
+ XAAUDIT.DB.DATABASE_NAME
+ $auditdb
+ Specify the audit database
+ name that matches the
+ audit_db_name
+ specified during
+ installation.
+
+
+ XAAUDIT.DB.USER_NAME
+ $auditdbuser
+ Specify the audit database
+ name that matches the
+ audit_db_user
+ specified during
+ installation.
+
+
+ XAAUDIT.DB.PASSWORD
+ $auditdbupw
+ Specify the audit database
+ name that matches the
+ audit_db_password
+ specified during
+ installation.
+
+ </tbody>
+
+ </table>
+
+
+ Save the
+ install.properties
+ file.
+
+
+
+ If your environment is configured to use SSL,
+ modify the properties following the instructions
+ in <link xlink:href="http://dev.hortonworks.com.s3.amazonaws.com/HDPDocuments/HDP2/HDP-2-trunk/bk_HDPSecure_Admin/content/ch_ssl-hbaseagent.html">Set Up SSL for HBase Security
+ Agents</link>.
+
+ The following is an example of the HBase
+ install.properties:#
+# Location of Policy Manager URL
+#
+#
+# Example:
+# POLICY_MGR_URL=http://policymanager.xasecure.net:6080
+#
+
+POLICY_MGR_URL=http://policymgr:6080
+
+#
+# Location of mysql client library (please check the location of the jar file)
+#
+MYSQL_CONNECTOR_JAR=/usr/share/java/mysql-connector-java.jar
+
+#
+# This is the repository name created within policy manager
+#
+# Example:
+# REPOSITORY_NAME=hbasedev
+#
+
+REPOSITORY_NAME=sandbox_2_hbase
+
+#
+# AUDIT DB Configuration
+#
+# This information should match with the one you specified during the PolicyManager Installation
+#
+# Example:
+# XAAUDIT.DB.HOSTNAME=localhost
+# XAAUDIT.DB.DATABASE_NAME=xasecure
+# XAAUDIT.DB.USER_NAME=xalogger
+# XAAUDIT.DB.PASSWORD=
+#
+#
+
+XAAUDIT.DB.HOSTNAME=xasecure
+XAAUDIT.DB.DATABASE_NAME=xasecure
+XAAUDIT.DB.USER_NAME=xasecure
+XAAUDIT.DB.PASSWORD=hadoop
+
+
+#
+# SSL Client Certificate Information
+#
+# Example:
+# SSL_KEYSTORE_FILE_PATH=/etc/xasecure/conf/xasecure-hadoop-client.jks
+# SSL_KEYSTORE_PASSWORD=clientdb01
+# SSL_TRUSTSTORE_FILE_PATH=/etc/xasecure/conf/xasecure-truststore.jks
+# SSL_TRUSTSTORE_PASSWORD=changeit
+
+#
+# IF YOU DO NOT DEFINE SSL parameters, the installation script will automatically generate necessary key(s) and assign appropriate values
+# ONLY If you want to assign manually, please uncomment the following variables and assign appropriate values.
+
+ </div>
+
+
+<div class="section">
+<h2><a name="null"></a></h2>
+
+ Changes to the properties require a restart of the HBase services.
+ To restart HBase:
+
+ Execute this command on the HBase Master host machine:
+ su -l hbase -c "/usr/lib/hbase/bin/hbase-daemon.sh --config /etc/hbase/conf stop master; sleep 25"
+
+
+ Execute this command on all RegionServers:
+ su -l hbase -c "/usr/lib/hbase/bin/hbase-daemon.sh --config /etc/hbase/conf stop regionserver"
+
+
+ Execute this command on the HBase Master host machine:
+ su -l hbase -c "/usr/lib/hbase/bin/hbase-daemon.sh --config /etc/hbase/conf start master; sleep 25"
+
+
+ Execute this command on all RegionServers:
+ su -l hbase -c "/usr/lib/hbase/bin/hbase-daemon.sh --config /etc/hbase/conf start regionserver"
+
+
+ </div>
+
+
+<div class="section">
+<h2><a name="null"></a></h2>
+
+ After the repository is set up and you have verified that the agent is connected
+ to the server, perform a few basic HBase test as outlined below:
+
+ Open a browser and go to
+ http://hue-host:8888.
+
+
+ Click on the Hue Shell icon in the navigation
+ pane.
+
+
+ Click HBase Shell.
+ The prompt displays.hbase(main):001:0>
+
+
+ At the prompt type list.
+ hbase(main):001:0> list
+list
+TABLE
+SLF4J: Class path contains multiple SLF4J bindings.
+SLF4J: Found binding in [jar:file:/usr/lib/hadoop/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
+SLF4J: Found binding in [jar:file:/usr/lib/zookeeper/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
+SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
+ambarismoketest
+test
+2 row(s) in 4.9490 seconds
+
+=> ["ambarismoketest", "test"]The
+ XASecure HBase agent reports the activity to the server.
+
+ If the HBase command fails with the following Zookeeper error,
+ restart HBase with the root user account from the command line and
+ retest.ERROR: Can't get master address from ZooKeeper; znode data == null
+
+
+
+ Sign in to the Web UI and click Audit.
+ The Big Data page displays a list of events for the configured
+ Repositories.
+
+
+ Click Search > Repository
+ Type > HBase.
+ The list filters as you make selections.
+
+
+
+ </div>
+ </div>
+
+<div class="section">
+<h2><a name="null"></a></h2>
+
+ You can edit repository details, including the configuration properties using the edit
+ icon next to a repository name.
+ To change the settings:
+
+ Sign in to the Argus Administration Web UI.
+
+
+ Click Policy Manager.
+ The Repository Details page displays.
+
+
+
+
+
+
+
+
+ Click the Edit icon next to the Repository
+ name.
+ The Repository Details page displays.
+
+
+ Change the settings and click Save.
+
+
+
+ Changing the repository name does not break the link between the agent and the
+ repository. The name is updated on the corresponding Audit and Reporting
+ pages.
+
+ </div>
+
+<div class="section">
+<h2><a name="null"></a></h2>
+
+ Deleting a repository only hides it from the Administration Web UI and does not
[... 44 lines stripped ...]