You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hbase.apache.org by st...@apache.org on 2012/06/03 23:59:52 UTC
svn commit: r1345788 [7/10] - in /hbase/trunk: ./ hbase-assembly/
hbase-common/ hbase-server/ hbase-site/ src/ src/assembly/ src/docbkx/
src/site/ src/site/resources/ src/site/resources/css/
src/site/resources/images/ src/site/xdoc/ src/xslt/
Added: hbase/trunk/src/docbkx/security.xml
URL: http://svn.apache.org/viewvc/hbase/trunk/src/docbkx/security.xml?rev=1345788&view=auto
==============================================================================
--- hbase/trunk/src/docbkx/security.xml (added)
+++ hbase/trunk/src/docbkx/security.xml Sun Jun 3 21:59:50 2012
@@ -0,0 +1,509 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<chapter version="5.0" xml:id="security"
+ xmlns="http://docbook.org/ns/docbook"
+ xmlns:xlink="http://www.w3.org/1999/xlink"
+ xmlns:xi="http://www.w3.org/2001/XInclude"
+ xmlns:svg="http://www.w3.org/2000/svg"
+ xmlns:m="http://www.w3.org/1998/Math/MathML"
+ xmlns:html="http://www.w3.org/1999/xhtml"
+ xmlns:db="http://docbook.org/ns/docbook">
+<!--
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+-->
+<title>Secure HBase</title>
+<section xml:id="hbase.secure.configuration">
+ <title>Secure Client Access to HBase</title>
+ <para>Newer releases of HBase (>= 0.92) support optional SASL authentication of clients.</para>
+ <para>This describes how to set up HBase and HBase clients for connection to secure HBase resources. </para>
+
+ <section><title>Prerequisites</title>
+ <para>
+ HBase must have been built using the new maven profile for secure Hadoop/HBase: <code>-P security</code>. Secure Hadoop dependent classes are separated under a pseudo-module in the security/ directory and are only included if built with the secure Hadoop profile.
+ </para>
+ <para>
+ You need to have a working Kerberos KDC.
+ </para>
+ <para>
+ A HBase configured for secure client access is expected to be running
+ on top of a secured HDFS cluster. HBase must be able to authenticate
+ to HDFS services. HBase needs Kerberos credentials to interact with
+ the Kerberos-enabled HDFS daemons. Authenticating a service should be
+ done using a keytab file. The procedure for creating keytabs for HBase
+ service is the same as for creating keytabs for Hadoop. Those steps
+ are omitted here. Copy the resulting keytab files to wherever HBase
+ Master and RegionServer processes are deployed and make them readable
+ only to the user account under which the HBase daemons will run.
+ </para>
+ <para>
+ A Kerberos principal has three parts, with the form
+ <code>username/fully.qualified.domain.name@YOUR-REALM.COM</code>. We
+ recommend using <code>hbase</code> as the username portion.
+ </para>
+ <para>
+ The following is an example of the configuration properties for
+ Kerberos operation that must be added to the
+ <code>hbase-site.xml</code> file on every server machine in the
+ cluster. Required for even the most basic interactions with a
+ secure Hadoop configuration, independent of HBase security.
+ </para>
+ <programlisting><![CDATA[
+ <property>
+ <name>hbase.regionserver.kerberos.principal</name>
+ <value>hbase/_HOST@YOUR-REALM.COM</value>
+ </property>
+ <property>
+ <name>hbase.regionserver.keytab.file</name>
+ <value>/etc/hbase/conf/keytab.krb5</value>
+ </property>
+ <property>
+ <name>hbase.master.kerberos.principal</name>
+ <value>hbase/_HOST@YOUR-REALM.COM</value>
+ </property>
+ <property>
+ <name>hbase.master.keytab.file</name>
+ <value>/etc/hbase/conf/keytab.krb5</value>
+ </property>
+ ]]></programlisting>
+ <para>
+ Each HBase client user should also be given a Kerberos principal. This
+ principal should have a password assigned to it (as opposed to a
+ keytab file). The client principal's <code>maxrenewlife</code> should
+ be set so that it can be renewed enough times for the HBase client
+ process to complete. For example, if a user runs a long-running HBase
+ client process that takes at most 3 days, we might create this user's
+ principal within <code>kadmin</code> with: <code>addprinc -maxrenewlife
+ 3days</code>
+ </para>
+ <para>
+ Long running daemons with indefinite lifetimes that require client
+ access to HBase can instead be configured to log in from a keytab. For
+ each host running such daemons, create a keytab with
+ <code>kadmin</code> or <code>kadmin.local</code>. The procedure for
+ creating keytabs for HBase service is the same as for creating
+ keytabs for Hadoop. Those steps are omitted here. Copy the resulting
+ keytab files to where the client daemon will execute and make them
+ readable only to the user account under which the daemon will run.
+ </para>
+ </section>
+
+ <section><title>Server-side Configuration for Secure Operation</title>
+ <para>
+ Add the following to the <code>hbase-site.xml</code> file on every server machine in the cluster:
+ </para>
+ <programlisting><![CDATA[
+ <property>
+ <name>hbase.security.authentication</name>
+ <value>kerberos</value>
+ </property>
+ <property>
+ <name>hbase.security.authorization</name>
+ <value>true</value>
+ </property>
+ <property>
+ <name>hbase.rpc.engine</name>
+ <value>org.apache.hadoop.hbase.ipc.SecureRpcEngine</value>
+ </property>
+ <property>
+ <name>hbase.coprocessor.region.classes</name>
+ <value>org.apache.hadoop.hbase.security.token.TokenProvider</value>
+ </property>
+ ]]></programlisting>
+ <para>
+ A full shutdown and restart of HBase service is required when deploying
+ these configuration changes.
+ </para>
+ </section>
+
+ <section><title>Client-side Configuration for Secure Operation</title>
+ <para>
+ Add the following to the <code>hbase-site.xml</code> file on every client:
+ </para>
+ <programlisting><![CDATA[
+ <property>
+ <name>hbase.security.authentication</name>
+ <value>kerberos</value>
+ </property>
+ <property>
+ <name>hbase.rpc.engine</name>
+ <value>org.apache.hadoop.hbase.ipc.SecureRpcEngine</value>
+ </property>
+ ]]></programlisting>
+ <para>
+ The client environment must be logged in to Kerberos from KDC or
+ keytab via the <code>kinit</code> command before communication with
+ the HBase cluster will be possible.
+ </para>
+ <para>
+ Be advised that if the <code>hbase.security.authentication</code>
+ and <code>hbase.rpc.engine</code> properties in the client- and
+ server-side site files do not match, the client will not be able to
+ communicate with the cluster.
+ </para>
+ <para>
+ Once HBase is configured for secure RPC it is possible to optionally
+ configure encrypted communication. To do so, add the following to the
+ <code>hbase-site.xml</code> file on every client:
+ </para>
+ <programlisting><![CDATA[
+ <property>
+ <name>hbase.rpc.protection</name>
+ <value>privacy</value>
+ </property>
+ ]]></programlisting>
+ <para>
+ This configuration property can also be set on a per connection basis.
+ Set it in the <code>Configuration</code> supplied to
+ <code>HTable</code>:
+ </para>
+ <programlisting>
+ Configuration conf = HBaseConfiguration.create();
+ conf.set("hbase.rpc.protection", "privacy");
+ HTable table = new HTable(conf, tablename);
+ </programlisting>
+ <para>
+ Expect a ~10% performance penalty for encrypted communication.
+ </para>
+ </section>
+
+ <section><title>Client-side Configuration for Secure Operation - Thrift Gateway</title>
+ <para>
+ Add the following to the <code>hbase-site.xml</code> file for every Thrift gateway:
+ <programlisting><![CDATA[
+ <property>
+ <name>hbase.thrift.keytab.file</name>
+ <value>/etc/hbase/conf/hbase.keytab</value>
+ </property>
+ <property>
+ <name>hbase.thrift.kerberos.principal</name>
+ <value>$USER/_HOST@HADOOP.LOCALDOMAIN</value>
+ </property>
+ ]]></programlisting>
+ </para>
+ <para>
+ Substitute the appropriate credential and keytab for $USER and $KEYTAB
+ respectively.
+ </para>
+ <para>
+ The Thrift gateway will authenticate with HBase using the supplied
+ credential. No authentication will be performed by the Thrift gateway
+ itself. All client access via the Thrift gateway will use the Thrift
+ gateway's credential and have its privilege.
+ </para>
+ </section>
+
+ <section><title>Client-side Configuration for Secure Operation - REST Gateway</title>
+ <para>
+ Add the following to the <code>hbase-site.xml</code> file for every REST gateway:
+ <programlisting><![CDATA[
+ <property>
+ <name>hbase.rest.keytab.file</name>
+ <value>$KEYTAB</value>
+ </property>
+ <property>
+ <name>hbase.rest.kerberos.principal</name>
+ <value>$USER/_HOST@HADOOP.LOCALDOMAIN</value>
+ </property>
+ ]]></programlisting>
+ </para>
+ <para>
+ Substitute the appropriate credential and keytab for $USER and $KEYTAB
+ respectively.
+ </para>
+ <para>
+ The REST gateway will authenticate with HBase using the supplied
+ credential. No authentication will be performed by the REST gateway
+ itself. All client access via the REST gateway will use the REST
+ gateway's credential and have its privilege.
+ </para>
+ <para>
+ It should be possible for clients to authenticate with the HBase
+ cluster through the REST gateway in a pass-through manner via SPEGNO
+ HTTP authentication. This is future work.
+ </para>
+ </section>
+
+</section> <!-- Secure Client Access to HBase -->
+
+
+<section xml:id="hbase.accesscontrol.configuration">
+ <title>Access Control</title>
+ <para>
+ Newer releases of HBase (>= 0.92) support optional access control
+ list (ACL-) based protection of resources on a column family and/or
+ table basis.
+ </para>
+ <para>
+ This describes how to set up Secure HBase for access control, with an
+ example of granting and revoking user permission on table resources
+ provided.
+ </para>
+
+ <section><title>Prerequisites</title>
+ <para>
+ You must configure HBase for secure operation. Refer to the section
+ "Secure Client Access to HBase" and complete all of the steps described
+ there.
+ </para>
+ <para>
+ You must also configure ZooKeeper for secure operation. Changes to ACLs
+ are synchronized throughout the cluster using ZooKeeper. Secure
+ authentication to ZooKeeper must be enabled or otherwise it will be
+ possible to subvert HBase access control via direct client access to
+ ZooKeeper. Refer to the section on secure ZooKeeper configuration and
+ complete all of the steps described there.
+ </para>
+ </section>
+
+ <section><title>Overview</title>
+ <para>
+ With Secure RPC and Access Control enabled, client access to HBase is
+ authenticated and user data is private unless access has been
+ explicitly granted. Access to data can be granted at a table or per
+ column family basis.
+ </para>
+ <para>
+ However, the following items have been left out of the initial
+ implementation for simplicity:
+ </para>
+ <orderedlist>
+ <listitem>
+ <para>Row-level or per value (cell): This would require broader changes for storing the ACLs inline with rows. It is a future goal.</para>
+ </listitem>
+ <listitem>
+ <para>Push down of file ownership to HDFS: HBase is not designed for the case where files may have different permissions than the HBase system principal. Pushing file ownership down into HDFS would necessitate changes to core code. Also, while HDFS file ownership would make applying quotas easy, and possibly make bulk imports more straightforward, it is not clear that it would offer a more secure setup.</para>
+ </listitem>
+ <listitem>
+ <para>HBase managed "roles" as collections of permissions: We will not model "roles" internally in HBase to begin with. We instead allow group names to be granted permissions, which allows external modeling of roles via group membership. Groups are created and manipulated externally to HBase, via the Hadoop group mapping service.</para>
+ </listitem>
+ </orderedlist>
+ <para>
+Access control mechanisms are mature and fairly standardized in the relational database world. The HBase implementation approximates current convention, but HBase has a simpler feature set than relational databases, especially in terms of client operations. We don't distinguish between an insert (new record) and update (of existing record), for example, as both collapse down into a Put. Accordingly, the important operations condense to four permissions: READ, WRITE, CREATE, and ADMIN.
+ </para>
+ <table>
+ <title>Operation To Permission Mapping</title>
+ <thead>
+ <row>
+ <entry>Permission</entry>
+ <entry>Operation</entry>
+ </row>
+ </thead>
+ <tbody>
+ <!-- READ -->
+ <row>
+ <entry>Read</entry>
+ <entry>Get</entry>
+ </row>
+ <row>
+ <entry></entry>
+ <entry>Exists</entry>
+ </row>
+ <row>
+ <entry></entry>
+ <entry>Scan</entry>
+ </row>
+ <!-- WRITE -->
+ <row>
+ <entry>Write</entry>
+ <entry>Put</entry>
+ </row>
+ <row>
+ <entry></entry>
+ <entry>Delete</entry>
+ </row>
+ <row>
+ <entry></entry>
+ <entry>Lock/UnlockRow</entry>
+ </row>
+ <row>
+ <entry></entry>
+ <entry>IncrementColumnValue</entry>
+ </row>
+ <row>
+ <entry></entry>
+ <entry>CheckAndDelete/Put</entry>
+ </row>
+ <row>
+ <entry></entry>
+ <entry>Flush</entry>
+ </row>
+ <row>
+ <entry></entry>
+ <entry>Compact</entry>
+ </row>
+ <!-- CREATE -->
+ <row>
+ <entry>Create</entry>
+ <entry>Create</entry>
+ </row>
+ <row>
+ <entry></entry>
+ <entry>Alter</entry>
+ </row>
+ <row>
+ <entry></entry>
+ <entry>Drop</entry>
+ </row>
+ <!-- ADMIN -->
+ <row>
+ <entry>Admin</entry>
+ <entry>Enable/Disable</entry>
+ </row>
+ <row>
+ <entry></entry>
+ <entry>Split</entry>
+ </row>
+ <row>
+ <entry></entry>
+ <entry>Major Compact</entry>
+ </row>
+ <row>
+ <entry></entry>
+ <entry>Grant</entry>
+ </row>
+ <row>
+ <entry></entry>
+ <entry>Revoke</entry>
+ </row>
+ <row>
+ <entry></entry>
+ <entry>Shutdown</entry>
+ </row>
+ </tbody>
+ </table>
+ <para>
+ Permissions can be granted in any of the following scopes, though
+ CREATE and ADMIN permissions are effective only at table scope.
+ </para>
+ <para>
+ <itemizedlist>
+ <listitem>
+ <para>Table</para>
+ <para>
+ <itemizedlist>
+ <listitem><para>Read: User can read from any column family in table</para></listitem>
+ <listitem><para>Write: User can write to any column family in table</para></listitem>
+ <listitem><para>Create: User can alter table attributes; add, alter, or drop column families; and drop the table.</para></listitem>
+ <listitem><para>Admin: User can alter table attributes; add, alter, or drop column families; and enable, disable, or drop the table. User can also trigger region (re)assignments or relocation.</para></listitem>
+ </itemizedlist>
+ </para>
+ </listitem>
+ <listitem>
+ <para>Column Family</para>
+ <para>
+ <itemizedlist>
+ <listitem><para>Read: User can read from the column family</para></listitem>
+ <listitem><para>Write: User can write to the column family</para></listitem>
+ </itemizedlist>
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ There is also an implicit global scope for the superuser.
+ </para>
+ <para>
+ The superuser is a principal, specified in the HBase site configuration
+ file, that has equivalent access to HBase as the 'root' user would on a
+ UNIX derived system. Normally this is the principal that the HBase
+ processes themselves authenticate as. Although future versions of HBase
+ Access Control may support multiple superusers, the superuser privilege
+ will always include the principal used to run the HMaster process. Only
+ the superuser is allowed to create tables, switch the balancer on or
+ off, or take other actions with global consequence. Furthermore, the
+ superuser has an implicit grant of all permissions to all resources.
+ </para>
+ <para>
+ Tables have a new metadata attribute: OWNER, the user principal who owns
+ the table. By default this will be set to the user principal who creates
+ the table, though it may be changed at table creation time or during an
+ alter operation by setting or changing the OWNER table attribute. Only a
+ single user principal can own a table at a given time. A table owner will
+ have all permissions over a given table.
+ </para>
+ </section>
+
+ <section><title>Server-side Configuration for Access Control</title>
+ <para>
+ Enable the AccessController coprocessor in the cluster configuration
+ and restart HBase. The restart can be a rolling one. Complete the
+ restart of all Master and RegionServer processes before setting up
+ ACLs.
+ </para>
+ <para>
+ To enable the AccessController, modify the <code>hbase-site.xml</code> file on every server machine in the cluster to look like:
+ </para>
+ <programlisting><![CDATA[
+ <property>
+ <name>hbase.coprocessor.master.classes</name>
+ <value>org.apache.hadoop.hbase.security.access.AccessController</value>
+ </property>
+ <property>
+ <name>hbase.coprocessor.region.classes</name>
+ <value>org.apache.hadoop.hbase.security.token.TokenProvider,
+ org.apache.hadoop.hbase.security.access.AccessController</value>
+ </property>
+ ]]></programlisting>
+ </section>
+
+ <section><title>Shell Enhancements for Access Control</title>
+ <para>
+The HBase shell has been extended to provide simple commands for editing and updating user permissions. The following commands have been added for access control list management:
+ </para>
+ Grant
+ <para>
+ <programlisting>
+ grant <user> <permissions> <table> [ <column family> [ <column qualifier> ] ]
+ </programlisting>
+ </para>
+ <para>
+ <code><permissions></code> is zero or more letters from the set "RWCA": READ('R'), WRITE('W'), CREATE('C'), ADMIN('A').
+ </para>
+ <para>
+ Note: Grants and revocations of individual permissions on a resource are both accomplished using the <code>grant</code> command. A separate <code>revoke</code> command is also provided by the shell, but this is for fast revocation of all of a user's access rights to a given resource only.
+ </para>
+ <para>
+ Revoke
+ </para>
+ <para>
+ <programlisting>
+ revoke <user> <table> [ <column family> [ <column qualifier> ] ]
+ </programlisting>
+ </para>
+ <para>
+ Alter
+ </para>
+ <para>
+ The <code>alter</code> command has been extended to allow ownership assignment:
+ <programlisting>
+ alter 'tablename', {OWNER => 'username'}
+ </programlisting>
+ </para>
+ <para>
+ User Permission
+ </para>
+ <para>
+ The <code>user_permission</code> command shows all access permissions for the current user for a given table:
+ <programlisting>
+ user_permission <table>
+ </programlisting>
+ </para>
+ </section>
+
+</section> <!-- Access Control -->
+</chapter>
Added: hbase/trunk/src/docbkx/shell.xml
URL: http://svn.apache.org/viewvc/hbase/trunk/src/docbkx/shell.xml?rev=1345788&view=auto
==============================================================================
--- hbase/trunk/src/docbkx/shell.xml (added)
+++ hbase/trunk/src/docbkx/shell.xml Sun Jun 3 21:59:50 2012
@@ -0,0 +1,108 @@
+<?xml version="1.0"?>
+ <chapter xml:id="shell"
+ version="5.0" xmlns="http://docbook.org/ns/docbook"
+ xmlns:xlink="http://www.w3.org/1999/xlink"
+ xmlns:xi="http://www.w3.org/2001/XInclude"
+ xmlns:svg="http://www.w3.org/2000/svg"
+ xmlns:m="http://www.w3.org/1998/Math/MathML"
+ xmlns:html="http://www.w3.org/1999/xhtml"
+ xmlns:db="http://docbook.org/ns/docbook">
+<!--
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+-->
+ <title>The HBase Shell</title>
+
+ <para>
+ The HBase Shell is <link xlink:href="http://jruby.org">(J)Ruby</link>'s
+ IRB with some HBase particular commands added. Anything you can do in
+ IRB, you should be able to do in the HBase Shell.</para>
+ <para>To run the HBase shell,
+ do as follows:
+ <programlisting>$ ./bin/hbase shell</programlisting>
+ </para>
+ <para>Type <command>help</command> and then <command><RETURN></command>
+ to see a listing of shell
+ commands and options. Browse at least the paragraphs at the end of
+ the help emission for the gist of how variables and command
+ arguments are entered into the
+ HBase shell; in particular note how table names, rows, and
+ columns, etc., must be quoted.</para>
+ <para>See <xref linkend="shell_exercises" />
+ for example basic shell operation.</para>
+
+ <section xml:id="scripting"><title>Scripting</title>
+ <para>For examples scripting HBase, look in the
+ HBase <filename>bin</filename> directory. Look at the files
+ that end in <filename>*.rb</filename>. To run one of these
+ files, do as follows:
+ <programlisting>$ ./bin/hbase org.jruby.Main PATH_TO_SCRIPT</programlisting>
+ </para>
+ </section>
+
+ <section xml:id="shell_tricks"><title>Shell Tricks</title>
+ <section><title><filename>irbrc</filename></title>
+ <para>Create an <filename>.irbrc</filename> file for yourself in your
+ home directory. Add customizations. A useful one is
+ command history so commands are save across Shell invocations:
+ <programlisting>
+ $ more .irbrc
+ require 'irb/ext/save-history'
+ IRB.conf[:SAVE_HISTORY] = 100
+ IRB.conf[:HISTORY_FILE] = "#{ENV['HOME']}/.irb-save-history"</programlisting>
+ See the <application>ruby</application> documentation of
+ <filename>.irbrc</filename> to learn about other possible
+ confiurations.
+ </para>
+ </section>
+ <section><title>LOG data to timestamp</title>
+ <para>
+ To convert the date '08/08/16 20:56:29' from an hbase log into a timestamp, do:
+ <programlisting>
+ hbase(main):021:0> import java.text.SimpleDateFormat
+ hbase(main):022:0> import java.text.ParsePosition
+ hbase(main):023:0> SimpleDateFormat.new("yy/MM/dd HH:mm:ss").parse("08/08/16 20:56:29", ParsePosition.new(0)).getTime() => 1218920189000</programlisting>
+ </para>
+ <para>
+ To go the other direction:
+ <programlisting>
+ hbase(main):021:0> import java.util.Date
+ hbase(main):022:0> Date.new(1218920189000).toString() => "Sat Aug 16 20:56:29 UTC 2008"</programlisting>
+ </para>
+ <para>
+ To output in a format that is exactly like that of the HBase log format will take a little messing with
+ <link xlink:href="http://download.oracle.com/javase/6/docs/api/java/text/SimpleDateFormat.html">SimpleDateFormat</link>.
+ </para>
+ </section>
+ <section><title>Debug</title>
+ <section><title>Shell debug switch</title>
+ <para>You can set a debug switch in the shell to see more output
+ -- e.g. more of the stack trace on exception --
+ when you run a command:
+ <programlisting>hbase> debug <RETURN></programlisting>
+ </para>
+ </section>
+ <section><title>DEBUG log level</title>
+ <para>To enable DEBUG level logging in the shell,
+ launch it with the <command>-d</command> option.
+ <programlisting>$ ./bin/hbase shell -d</programlisting>
+ </para>
+ </section>
+ </section>
+ </section>
+ </chapter>