You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@knox.apache.org by mo...@apache.org on 2021/07/20 15:16:30 UTC
svn commit: r1891689 [12/14] - in /knox: site/ site/books/knox-0-12-0/
site/books/knox-0-13-0/ site/books/knox-0-14-0/ site/books/knox-1-0-0/
site/books/knox-1-1-0/ site/books/knox-1-2-0/ site/books/knox-1-3-0/
site/books/knox-1-4-0/ site/books/knox-1-...
Added: knox/trunk/books/1.6.0/knoxshell-guide/knoxshell_user_guide.md
URL: http://svn.apache.org/viewvc/knox/trunk/books/1.6.0/knoxshell-guide/knoxshell_user_guide.md?rev=1891689&view=auto
==============================================================================
--- knox/trunk/books/1.6.0/knoxshell-guide/knoxshell_user_guide.md (added)
+++ knox/trunk/books/1.6.0/knoxshell-guide/knoxshell_user_guide.md Tue Jul 20 15:16:28 2021
@@ -0,0 +1,346 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ https://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+<<../../common/header.md>>
+
+<img src="knox-logo.gif" alt="Knox"/>
+<img src="apache-logo.gif" align="right" alt="Apache"/>
+
+# Apache Knox - KnoxShell 1.5.x User Guide #
+
+
+* #[Introduction]
+* #[Representing and Working with Tabular Data]
+* #[KnoxShellTable]
+ * #[Builders]
+* #[Usecases]
+ * #[JDBC Resultset Representations]
+ * #[CSV Representations]
+ * #[General Table Operations]
+ * #[Persistence and Publishing]
+ * #[KnoxLine SQL Shell]
+ * #[Custom GroovySh Commands]
+* #[JDBC Resultset Representations]
+* #[CSV Representations]
+* #[General Table Operations]
+ * #[Sorting]
+ * #[Selecting]
+ * #[Filtering]
+ * #[Fluent API]
+ * #[Aggregating]
+* #[KnoxLine SQL Shell]
+* #[Custom GroovySh Commands]
+ * #[KnoxShell Commands:]
+* #[EXAMPLE: COVID19 Data Flow into DataLake]
+ * #[Build Table from Public CSV File]
+ * #[Select Columns, Filter and Sort by Column]
+ * #[Aggregate Calculations on Columns of Table]
+ * #[Persist Tables to Local Disk]
+ * #[Add Tables to DataLake]
+ * #[Building KnoxShell Truststore]
+ * #[Mount a WebHDFS Filesystem]
+ * #[Accessing a Filesystem]
+ * #[Put Tables into DataLake]
+ * #[Pull CSV Files from WebHDFS and Create Tables]
+
+## Introduction
+
+The KnoxShell environment has been extended to provide more of an interactive experience through the use of custom commands and the newly added KnoxShellTable rendering and dataset representation class. This is provided through by integrating the power of groovysh extensions and the KnoxShell client classes/SDK and make for some really powerful command line capabilities that would otherwise require the user to SSH to a node within the cluster and use CLIs of different tools or components.
+
+This document will cover the various KnoxShell extentions and how to use them on their own and describe combinations of them as flows for working with tabular data from various sources.
+
+## Representing and Working with Tabular Data
+The ability to read, write and work with tabular data formats such as CSV files, JDBC resultsets and others is core to the motivations of this KnoxShell oriented work. Intentions include: the ability to read arbitrary data from sources from inside a proxied cluster or from external sources, the ability to render the resulting tables, sort the table, filter it for specific subsets of the data and do some interesting calculations that can provide simple insights into your data.
+
+KnoxShellTable represents those core capabilties with its simple representation of a table, operation methods and builder classes.
+
+## KnoxShellTable
+
+KnoxShellTable has a number of dedicated builders that have a fluent API for building table representations from various sources.
+
+### Builders
+The following builders aid in the creation of tables from various types of data sources.
+
+#### JDBC
+```
+
+ ports = KnoxShellTable.builder().jdbc().
+ connect("jdbc:hive2://knox-host:8443/;ssl=true;transportMode=http;httpPath=topology/cdp-proxy-api/hive").
+ driver("org.apache.hive.jdbc.HiveDriver").
+ username("lmccay").pwd("xxxx").
+ sql("select * FROM ports");
+```
+
+Running the above within KnoxShell will submit the provided SQL to HS2, create and assign a new KnoxShellTable instance to the "ports" variable representing the border ports of entry data.
+#### CSV
+
+```
+crossings = KnoxShellTable.builder().csv().
+ withHeaders().
+ url("file:///home/lmccay/Border_Crossing_Entry_Data.csv")
+```
+Running the above within KnoxShell will import a CSV file from local disk, create and assign a new KnoxShellTable instance to the "result" variable.
+
+A higher level KnoxShell Custom Command allows for easier use of the builder through more natural syntax and hides the use of the lower level classes and syntax.
+
+#### Join
+
+```
+crossings = KnoxShellTable.builder().join().
+ left(ports).
+ right(crossings).
+ on("code","Port Code"
+```
+
+Running the above within KnoxShell will import a join the two tables with a simple match of the values in left and right tables on each row that matches.
+#### JSON
+```
+tornados = KnoxShellTable.builder().json().
+ url("file:///home/lmccay/.knoxshell/.tables/tornados.json")
+```
+Running the above within KnoxShell will rematerialize a table that was persisted as JSON and assign it to a local "tornados" variable.
+
+#### Persistence and Publishing
+
+Being able to create tables, combine them with other datasets, filter them and add new cols based on calculations between cols, etc is all great for creating tables in memory and working with them.
+
+We also want to be able to persist these tables in a KnoxShellTable canonical JSON format of its own and be able to reload the same datasets later.
+
+We also want to be able to take a given dataset and publish it as a brand new CSV file that can be pushed into HDFS, saved to local disk, written to cloud storage, etc.
+
+In addition, we may want to be able to write it directly to Hive or another JDBC datasource.
+##### JSON
+
+```
+tornados.toJSON()
+```
+The above will return and render a JSON representation of the tornados KnoxShellTable including: headers, rows, optionally title and optionally callHistory.
+##### CSV
+
+```
+tornados.toCSV()
+```
+The above will return and render a CSV representation of the tornados KnoxShellTable including: headers (if present), and all rows.
+
+Note that title and callhistory which are KnoxShellTable specifics are excluded and lost unless also saved as JSON.
+
+
+## Usecases
+
+* JDBC Resultset Representations
+* CSV Representations
+* General Table Operations
+- Joining
+- Sorting, Selecting, Filtering, Calculations
+* Persistence and Publishing
+* KnoxLine SQL Shell
+* Custom GroovySh Commands
+
+Let's take a look at each usecase.
+
+## JDBC Resultset Representations
+
+KnoxLine SQL Client requires a tabular representation of the data from a SQL/JDBC Resultset. This requirement led to the creation of the KnoxShellTable JDBC Builder. It may be used outside of KnoxLine within your own Java clients or groovy scripts leveraging the KnoxShell classes.
+
+```
+ports = KnoxShellTable.builder().jdbc().
+ connect("jdbc:hive2://knox-host:8443/;ssl=true;transportMode=http;httpPath=topology/datalake-api/hive").
+ driver("org.apache.hive.jdbc.HiveDriver").
+ username("lmccay").pwd("xxxx").
+ sql("select * FROM ports");
+```
+
+It can create the cols based on the metadata of the resultset and accurately represent the data and perform type specific operations, sorts, etc.
+
+A higher level KnoxShell Custom Command allows for the use of this builder with Datasources that are managed within the KnoxShell environment and persisted to the users' home directory to allow continued use across sessions. This command hides the use of the underlying classes and syntax and allows the user to concentrate on SQL.
+
+## CSV Representations
+
+Another dedicated table builder is provided for creating a table from a CSV file that is imported via URL.
+
+Combined with all the general table operations and ability to join them with other KnoxShellTable representations, this allows for CSV data to be combined with JDBC datasets, filtered and republished as a new dataset or report to be rendered or even reexecuted later.
+
+## General Table Operations
+In addition to the builders described above, there are a number of operations that may be executed on the table itself.
+
+### Sorting
+```
+tornados.sort("state")
+```
+
+When a column is of String type values but they are numerics, you may also sort numerically.
+
+```
+tornados.sortNumeric("count")
+```
+
+The above will sort the tornados table by the "state" column.
+### Selecting
+```
+tornados.select("state,cat,inj,fat,date,month,day,year")
+```
+The above will return and render a new table with only the subset of cols selected.
+### Filtering
+```
+tornados.filter().name("fat").greaterThan(0)
+```
+The above will return and render a table with only those tornados that resulted in one or more fatalities.
+### Fluent API
+
+The above operations can be combined in a natural, fluent manner
+```
+tornados.select("state,cat,inj,fat,date,month,day,year").
+
+ filter().name("fat").greaterThan(0).
+
+ sort("state")
+```
+### Aggregating
+The following method allows for the use of table column calculations to build an aggregate view of helpful calculations for multiple columns in a table and summarizes them in a new table representation.
+
+```
+table.aggregate().columns("col1, col2, col3").functions("min,max,mean,median,mode,sum")
+```
+
+The above allows you to combine them by streaming them into each other in one line the select of only certain cols, the filtering of only those events with more than 0 fatalities and the much more efficient sort of the resulting table.
+
+## KnoxLine SQL Shell
+
+KnoxLine is a beeline like facility built into the KnoxShell client toolbox with basic datasource management and simple SQL client capabilities.
+ResultSets are rendered via KnoxShellTable but further table based manipulations are not available within the knoxline shell. This is purely
+dedicated to SQL interactions and table renderings.
+
+For leveraging the SQL builder of KnoxShellTable to be able to operate on the results locally, see the custom KnoxShell command 'SQL'.
+
+
+ 
+
+Once connected to the datasource, SQL commands may be invoked via the command line directly.
+
+
+## Custom GroovySh Commands
+
+Groovy shell has the ability to extend the commands available to help automate scripting or coding that you would otherwise need to do programmatically over and over.
+
+By providing custom commands for KnoxShellTable operations, builders and manipulation we can greatly simplify what would need to be done with the fluent API of KnoxShellTable and groovy/java code for saving state, etc.
+
+### KnoxShell Commands:
+
+1. **Datasources** (:datasource|:ds) CRUD and select operations for a set of JDBC datasources that are persisted to disk (KNOX-2128)
+2. **SQL** (:SQL|:sql) SQL query execution with persisted SQL history per datasource (KNOX-2128)
+3. **CSV** (:CSV|:csv) Import and Export from CSV and JSON formats
+4. **Filesystem** (:Filesystem|:fs) POSIX style commands for HDFS and cloud storage (mount, unmount, mounts, ls, rm, mkdir, cat, put, etc)
+
+
+ 
+
+
+## EXAMPLE: COVID19 Data Flow into DataLake
+
+Let's start to put the commands and table capabilities together to consume some public tabular data and usher it into our datalake or cluster.
+
+### Build Table from Public CSV File
+
+ 
+
+The use of the CSV KnoxShell command above can be easily correlated to the CSV builder of KnoxShellTable. It is obviously less verbose and more natural than using the fluent API of KnoxShellTable directly and also leverages a separate capability for KnoxShell to assign the resulting table to a KnoxShell variable that can be references and manipulated afterward.
+
+As you can see the result of creating the table from a CSV file is a rendering of the entire table and often does not fit the screen propertly.
+This is where the operations on the resulting table come in handy for explorer the dataset.
+Let's filter the above dataset of COVID19 across the world to only a subset of columns and for only New Jersey by selecting, filtering and sorting numerically by number of Confirmed cases.
+
+### Select Columns, Filter and Sort by Column
+
+First we will interrogate the table for its column names or headers. Then we will select only those columns that we want in order to fit it to the screen, filter it for only New Jersey information and sort numerically by the number of Confirmed cases per county.
+
+ 
+
+From the above operation, we can now see the COVID19 data for New Jersey counties for 4/10/2020 sorted by the number of Confirmed cases and the subset of cols of the most interest and tailored to fit our screen. From the above table, we can visually see a number of insights in terms of the most affected counties across the state of New Jersey but it may be more interesting to be able to see an aggregation of some of the calculations available for numeric columns through KnoxShellTable. Let's take a look at an aggregate table for this dataset.
+
+### Aggregate Calculations on Columns of Table
+
+Since the KnoxShellTable fluent API allows us to chain such operations together easily, we will just hit the up arrow to get the previous table operation command and add the aggregate operation to the chain.
+
+ 
+
+Now, by using both tables above, we can see that my county of Camden is both visually in approximately the center of the counties in terms of Confirmed case numbers but how it stands related to both the average and the median calculations. You can also see the sum of all of New Jersey and the number of those that belong to my county.
+
+### Persist Tables to Local Disk
+
+Next, we will persist these tables to our local disk and then push them into our HDFS based datalake for access by cluster resources and other users.
+
+ 
+
+### Add Tables to DataLake
+
+Now that we have these tables persisted to local disk, we can use our KnoxShell Filesystem commands to add them to the datalake.
+
+### Building KnoxShell Truststore
+
+Before we can access resources from datalake behind Knox we need to insure that the cert presented by the Knox instance is trusted. If the deployment is using certs signed by a well-known ca, then we generally don't have to do anything. If we are using Knox self-signed certs or certs signed by an internal ca of some sort then we must import them into the KnoxShell truststore. While this can be located in arbitrary places and configured via system properties and environment variables, the most common approach is to use then default location.
+
+```
+// exit knoxshell
+^C
+
+bin/knoxshell.sh buildTrustStore https://nightly7x-1.nightly7x.root.hwx.site:8443/gateway/datalake-api
+
+ls -l ~/gateway-client-trust.jks
+
+// to reenter knoxshell
+bin/knoxshell.sh
+```
+
+With the default password of 'changeit'.
+
+### Mount a WebHDFS Filesystem
+
+We may now mount a filesystem from the remote Knox instance by mounting the topology that hosts the WebHDFS API endpoint.
+
+```
+:fs mount https://nightly7x-1.nightly7x.root.hwx.site:8443/gateway/datalake-api nightly
+```
+
+### Accessing a Filesystem
+
+Once we have the desired mount, we may now access it by specifying the mountpoint name as the path prefix into the HDFS filesystem.
+Upon mounting or first access, the KnoxShell will prompt for user credentials for use as HTTP Basic credentials while accessing WebHDFS API.
+
+ 
+
+Once we authenticate to the mounted filesystem, we reference it by mountpoint and never concern ourselves with the actual URL to the endpoint.
+
+### Put Tables into DataLake
+
+ 
+
+Above, we have put the previously persisted CSV files into the tmp directory of the mounted filesystem to be available to other datalake users.
+
+We can now also access them from any other KnoxShell instance that has mounted this filesystem with appropriate credentials.
+Let's now cat the contents of one of the CSV files into the KnoxShell and then render it as a table from the raw CSV format.
+
+### Pull CSV Files from WebHDFS and Create Tables
+
+ 
+
+Note that the cat command returns the CSV file contents as a string to the KnoxShell environment as a variable called '_' .
+
+This is true of any command in groovysh or KnoxShell. The previous result is always available as this variable.
+Here we pass the contents of the variable to the CSV KnoxShellTable builder string() method. This is a very convenient way to render tabular data from a cat'd file from your remote datalake.
+
+Also note that tables that are assigned to variables within KnoxShell will render themselves just by typing the variable name.
+
Added: knox/trunk/books/1.6.0/likeised
URL: http://svn.apache.org/viewvc/knox/trunk/books/1.6.0/likeised?rev=1891689&view=auto
==============================================================================
--- knox/trunk/books/1.6.0/likeised (added)
+++ knox/trunk/books/1.6.0/likeised Tue Jul 20 15:16:28 2021
@@ -0,0 +1,47 @@
+# This sed script must be kept in sync with the table of contents
+
+#wrap the entire page and the banner
+s@<p><br> <img src="knox-logo.gif"@<div id="page-wrap"><div id="banner"><p><br> <img src="knox-logo.gif"@
+
+# close the banner and the start the sidebar
+s@<h2><a id="Table+Of+Contents"></a>Table Of Contents</h2>@</div><div id="sidebar">@
+
+#close the sidebar, start main content section and start the first of the chapers
+s@<h2><a id="Introduction@</div><div id="content"><div id="Introduction"><h2><a id="Introduction@
+s@<h2><a id="Quick+Start@</div><div id="Quick+Start"><h2><a id="Quick+Start@
+s@<h2><a id="Apache+Knox+Details@</div><div id="Apache+Knox+Details"><h2><a id="Apache+Knox+Details@
+# subchapters...
+s@<h4><a id="Apache+Knox+Directory+Layout@</div><div id="Apache+Knox+Directory+Layout"><h4><a id="Layout@
+s@<h3><a id="Supported+Services@</div><div id="Supported+Services"><h3><a id="Supported+Services@
+s@<h4><a id="Configure+Sandbox+port+mapping+for+VirtualBox@</div><div id="Configure+Sandbox+port+mapping+for+VirtualBox"><h4><a id="Configure+Sandbox+port+mapping+for+VirtualBox@
+s@<h2><a id="Gateway+Details@</div><div id="Gateway+Details"><h2><a id="Gateway+Details@
+s@<h3><a id="Configuration@</div><div id="Configuration"><h3><a id="Configuration@
+s@<h3><a id="Knox+CLI@</div><div id="Knox+CLI"><h3><a id="Knox+CLI@
+s@<h3><a id="Authentication@</div><div id="Authentication"><h3><a id="Authentication@
+s@<h3><a id="LDAP+Group+Lookup@</div><div id="LDAP+Group+Lookup"><h3><a id="LDAP+Group+Lookup@
+s@<h3><a id="Identity+Assertion@</div><div id="Identity+Assertion"><h3><a id="Identity+Assertion@
+s@<h3><a id="Authorization@</div><div id="Authorization"><h3><a id="Authorization@
+s@<h2><a id="Configuration@</div><div id="Configuration"><h2><a id="Configuration@
+s@<h3><a id="Secure+Clusters@</div><div id="Secure+Clusters"><h3><a id="Secure+Clusters@
+s@<h3><a id="High+Availability@</div><div id="High+Availability"><h3><a id="High+Availability@
+s@<h3><a id="Web+App+Security+Provider@</div><div id="Web+App+Security+Provider"><h3><a id="Web+App+Security+Provider@
+s@<h3><a id="Preauthenticated+SSO+Provider@</div><div id="Preauthenticated+SSO+Provider"><h3><a id="Preauthenticated+SSO+Provider@
+s@<h3><a id="Mutual+Authentication+with+SSL@</div><div id="Mutual+Authentication+with+SSL"><h3><a id="Mutual+Authentication+with+SSL@
+s@<h3><a id="Audit@</div><div id="Audit"><h3><a id="Audit@
+s@<h2><a id="Client+Details@</div><div id="Client+Details"><h2><a id="Client+Details@
+s@<h2><a id="Service+Details@</div><div id="Service+Details"><h2><a id="Service+Details@
+s@<h3><a id="WebHDFS@</div><div id="WebHDFS"><h3><a id="WebHDFS@
+s@<h3><a id="WebHCat@</div><div id="WebHCat"><h3><a id="WebHCat@
+s@<h3><a id="Oozie@</div><div id="Oozie"><h3><a id="Oozie@
+s@<h3><a id="HBase@</div><div id="HBase"><h3><a id="HBase@
+s@<h3><a id="Hive@</div><div id="Hive"><h3><a id="Hive@
+s@<h3><a id="Storm@</div><div id="Storm"><h3><a id="Storm@
+s@<h3><a id="Default+Service+HA+support@</div><div id="Default+Service+HA+support"><h3><a id="Default+Service+HA+support@
+s@<h2><a id="Limitations@</div><div id="Limitations"><h2><a id="Limitations@
+s@<h2><a id="Troubleshooting@</div><div id="Troubleshooting"><h2><a id="Troubleshooting@
+s@<h2><a id="Export+Controls@</div><div id="Export+Controls"><h2><a id="Export+Controls@
+
+# closing the last chapter section, page-wrap and content sections is done outside of this script
+# using cat >> filename
+
+# sed -f likeised knox-incubating-0-4-0.html > knox-incubating-0-4-0-new.html && echo "</div></div></div>" >> knox-incubating-0-4-0-new.html
Added: knox/trunk/books/1.6.0/quick_start.md
URL: http://svn.apache.org/viewvc/knox/trunk/books/1.6.0/quick_start.md?rev=1891689&view=auto
==============================================================================
--- knox/trunk/books/1.6.0/quick_start.md (added)
+++ knox/trunk/books/1.6.0/quick_start.md Tue Jul 20 15:16:28 2021
@@ -0,0 +1,207 @@
+<!---
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ https://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+## Quick Start ##
+
+Here are the steps to have Apache Knox up and running against a Hadoop Cluster:
+
+1. Verify system requirements
+1. Download a virtual machine (VM) with Hadoop
+1. Download Apache Knox Gateway
+1. Start the virtual machine with Hadoop
+1. Install Knox
+1. Start the LDAP embedded within Knox
+1. Start the Knox Gateway
+1. Do Hadoop with Knox
+
+
+
+### 1 - Requirements ###
+
+#### Java ####
+
+Java 1.8 is required for the Knox Gateway runtime.
+Use the command below to check the version of Java installed on the system where Knox will be running.
+
+ java -version
+
+#### Hadoop ####
+
+Knox 1.6.0 supports Hadoop 2.x and 3.x, the quick start instructions assume a Hadoop 2.x virtual machine based environment.
+
+
+### 2 - Download Hadoop 2.x VM ###
+The quick start provides a link to download Hadoop 2.0 based Hortonworks virtual machine [Sandbox](http://hortonworks.com/products/hdp-2/#install). Please note Knox supports other Hadoop distributions and is configurable against a full-blown Hadoop cluster.
+Configuring Knox for Hadoop 2.x version, or Hadoop deployed in EC2 or a custom Hadoop cluster is documented in advance deployment guide.
+
+
+### 3 - Download Apache Knox Gateway ###
+
+Download one of the distributions below from the [Apache mirrors][mirror].
+
+* Source archive: [knox-1.6.0-src.zip][src-zip] ([PGP signature][src-pgp], [SHA1 digest][src-sha], [MD5 digest][src-md5])
+* Binary archive: [knox-1.6.0.zip][bin-zip] ([PGP signature][bin-pgp], [SHA1 digest][bin-sha], [MD5 digest][bin-md5])
+
+[keys]: https://dist.apache.org/repos/dist/release/knox/KEYS
+[src-zip]: http://www.apache.org/dyn/closer.cgi/knox/1.6.0/knox-1.6.0-src.zip
+[src-sha]: https://downloads.apache.org//knox/1.6.0/knox-1.6.0-src.zip.sha1
+[src-pgp]: https://downloads.apache.org//knox/1.6.0/knox-1.6.0-src.zip.asc
+[src-md5]: https://downloads.apache.org//knox/1.6.0/knox-1.6.0-src.zip.md5
+[bin-zip]: http://www.apache.org/dyn/closer.cgi/knox/1.6.0/knox-1.6.0.zip
+[bin-pgp]: https://downloads.apache.org//knox/1.6.0/knox-1.6.0.zip.asc
+[bin-sha]: https://downloads.apache.org//knox/1.6.0/knox-1.6.0.zip.sha1
+[bin-md5]: https://downloads.apache.org//knox/1.6.0/knox-1.6.0.zip.md5
+
+Apache Knox Gateway releases are available under the [Apache License, Version 2.0][asl].
+See the NOTICE file contained in each release artifact for applicable copyright attribution notices.
+
+
+### Verify ###
+
+While recommended, verification of signatures is an optional step. You can verify the integrity of any downloaded files using the PGP signatures.
+Please read [Verifying Apache HTTP Server Releases](http://httpd.apache.org/dev/verification.html) for more information on why you should verify our releases.
+
+The PGP signatures can be verified using PGP or GPG.
+First download the [KEYS][keys] file as well as the `.asc` signature files for the relevant release packages.
+Make sure you get these files from the main distribution directory linked above, rather than from a mirror.
+Then verify the signatures using one of the methods below.
+
+ % pgpk -a KEYS
+ % pgpv knox-1.6.0.zip.asc
+
+or
+
+ % pgp -ka KEYS
+ % pgp knox-1.6.0.zip.asc
+
+or
+
+ % gpg --import KEYS
+ % gpg --verify knox-1.6.0.zip.asc
+
+### 4 - Start Hadoop virtual machine ###
+
+Start the Hadoop virtual machine.
+
+### 5 - Install Knox ###
+
+The steps required to install the gateway will vary depending upon which distribution format (zip | rpm) was downloaded.
+In either case you will end up with a directory where the gateway is installed.
+This directory will be referred to as your `{GATEWAY_HOME}` throughout this document.
+
+#### ZIP ####
+
+If you downloaded the Zip distribution you can simply extract the contents into a directory.
+The example below provides a command that can be executed to do this.
+Note the `{VERSION}` portion of the command must be replaced with an actual Apache Knox Gateway version number.
+This might be 1.6.0 for example.
+
+ unzip knox-{VERSION}.zip
+
+This will create a directory `knox-{VERSION}` in your current directory.
+The directory `knox-{VERSION}` will considered your `{GATEWAY_HOME}`
+
+### 6 - Start LDAP embedded in Knox ###
+
+Knox comes with an LDAP server for demonstration purposes.
+Note: If the tool used to extract the contents of the Tar or tar.gz file was not capable of
+making the files in the bin directory executable
+
+ cd {GATEWAY_HOME}
+ bin/ldap.sh start
+
+### 7 - Create the Master Secret
+
+Run the `knoxcli.sh create-master` command in order to persist the master secret
+that is used to protect the key and credential stores for the gateway instance.
+
+ cd {GATEWAY_HOME}
+ bin/knoxcli.sh create-master
+
+The CLI will prompt you for the master secret (i.e. password).
+
+### 7 - Start Knox ###
+
+The gateway can be started using the provided shell script.
+
+The server will discover the persisted master secret during start up and complete the setup process for demo installs.
+A demo install will consist of a Knox gateway instance with an identity certificate for localhost.
+This will require clients to be on the same machine or to turn off hostname verification.
+For more involved deployments, See the Knox CLI section of this document for additional configuration options,
+including the ability to create a self-signed certificate for a specific hostname.
+
+ cd {GATEWAY_HOME}
+ bin/gateway.sh start
+
+When starting the gateway this way the process will be run in the background.
+The log files will be written to `{GATEWAY_HOME}/logs` and the process ID files (PIDs) will be written to `{GATEWAY_HOME}/pids`.
+
+In order to stop a gateway that was started with the script use this command:
+
+ cd {GATEWAY_HOME}
+ bin/gateway.sh stop
+
+If for some reason the gateway is stopped other than by using the command above you may need to clear the tracking PID:
+
+ cd {GATEWAY_HOME}
+ bin/gateway.sh clean
+
+__NOTE: This command will also clear any `.out` and `.err` file from the `{GATEWAY_HOME}/logs` directory so use this with caution.__
+
+
+### 8 - Access Hadoop with Knox
+
+#### Invoke the LISTSTATUS operation on WebHDFS via the gateway.
+This will return a directory listing of the root (i.e. `/`) directory of HDFS.
+
+ curl -i -k -u guest:guest-password -X GET \
+ 'https://localhost:8443/gateway/sandbox/webhdfs/v1/?op=LISTSTATUS'
+
+The results of the above command should result in something to along the lines of the output below.
+The exact information returned is subject to the content within HDFS in your Hadoop cluster.
+Successfully executing this command at a minimum proves that the gateway is properly configured to provide access to WebHDFS.
+It does not necessarily mean that any of the other services are correctly configured to be accessible.
+To validate that see the sections for the individual services in #[Service Details].
+
+ HTTP/1.1 200 OK
+ Content-Type: application/json
+ Content-Length: 760
+ Server: Jetty(6.1.26)
+
+ {"FileStatuses":{"FileStatus":[
+ {"accessTime":0,"blockSize":0,"group":"hdfs","length":0,"modificationTime":1350595859762,"owner":"hdfs","pathSuffix":"apps","permission":"755","replication":0,"type":"DIRECTORY"},
+ {"accessTime":0,"blockSize":0,"group":"mapred","length":0,"modificationTime":1350595874024,"owner":"mapred","pathSuffix":"mapred","permission":"755","replication":0,"type":"DIRECTORY"},
+ {"accessTime":0,"blockSize":0,"group":"hdfs","length":0,"modificationTime":1350596040075,"owner":"hdfs","pathSuffix":"tmp","permission":"777","replication":0,"type":"DIRECTORY"},
+ {"accessTime":0,"blockSize":0,"group":"hdfs","length":0,"modificationTime":1350595857178,"owner":"hdfs","pathSuffix":"user","permission":"755","replication":0,"type":"DIRECTORY"}
+ ]}}
+
+#### Put a file in HDFS via Knox.
+
+ curl -i -k -u guest:guest-password -X PUT \
+ 'https://localhost:8443/gateway/sandbox/webhdfs/v1/tmp/LICENSE?op=CREATE'
+
+ curl -i -k -u guest:guest-password -T LICENSE -X PUT \
+ '{Value of Location header from response above}'
+
+#### Get a file in HDFS via Knox.
+
+ curl -i -k -u guest:guest-password -X GET \
+ 'https://localhost:8443/gateway/sandbox/webhdfs/v1/tmp/LICENSE?op=OPEN'
+
+ curl -i -k -u guest:guest-password -X GET \
+ '{Value of Location header from command response above}'
+
Added: knox/trunk/books/1.6.0/service_avatica.md
URL: http://svn.apache.org/viewvc/knox/trunk/books/1.6.0/service_avatica.md?rev=1891689&view=auto
==============================================================================
--- knox/trunk/books/1.6.0/service_avatica.md (added)
+++ knox/trunk/books/1.6.0/service_avatica.md Tue Jul 20 15:16:28 2021
@@ -0,0 +1,100 @@
+<!---
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ https://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+--->
+
+### Avatica ###
+
+Knox provides gateway functionality for access to all Apache Avatica-based servers.
+The gateway can be used to provide authentication and encryption for clients to
+servers like the Apache Phoenix Query Server.
+
+#### Gateway configuration ####
+
+The Gateway can be configured for Avatica by modifying the topology XML file
+and providing a new service XML file.
+
+In the topology XML file, add the following with the correct hostname:
+
+ <service>
+ <role>AVATICA</role>
+ <url>http://avatica:8765</url>
+ </service>
+
+Your installation likely already contains the following service files. Ensure
+that they are present in your installation. In `services/avatica/1.9.0/rewrite.xml`:
+
+ <rules>
+ <rule dir="IN" name="AVATICA/avatica/inbound/root" pattern="*://*:*/**/avatica/">
+ <rewrite template="{$serviceUrl[AVATICA]}/"/>
+ </rule>
+ <rule dir="IN" name="AVATICA/avatica/inbound/path" pattern="*://*:*/**/avatica/{**}">
+ <rewrite template="{$serviceUrl[AVATICA]}/{**}"/>
+ </rule>
+ </rules>
+
+And in `services/avatica/1.9.0/service.xml`:
+
+ <service role="AVATICA" name="avatica" version="1.9.0">
+ <policies>
+ <policy role="webappsec"/>
+ <policy role="authentication"/>
+ <policy role="rewrite"/>
+ <policy role="authorization"/>
+ </policies>
+ <routes>
+ <route path="/avatica">
+ <rewrite apply="AVATICA/avatica/inbound/root" to="request.url"/>
+ </route>
+ <route path="/avatica/**">
+ <rewrite apply="AVATICA/avatica/inbound/path" to="request.url"/>
+ </route>
+ </routes>
+ </service>
+
+#### JDBC Drivers ####
+
+In most cases, users only need to modify the hostname of the Avatica server to
+instead be the Knox Gateway. To enable authentication, some of the Avatica
+property need to be added to the Properties object used when constructing the
+`Connection` or to the JDBC URL directly.
+
+The JDBC URL can be modified like:
+
+ jdbc:avatica:remote:url=https://knox_gateway.domain:8443/gateway/sandbox/avatica;avatica_user=username;avatica_password=password;authentication=BASIC
+
+Or, using the `Properties` class:
+
+ Properties props = new Properties();
+ props.setProperty("avatica_user", "username");
+ props.setProperty("avatica_password", "password");
+ props.setProperty("authentication", "BASIC");
+ DriverManager.getConnection(url, props);
+
+Additionally, when the TLS certificate of the Knox Gateway is not trusted by your JVM installation,
+it will be necessary for you to pass in a custom truststore and truststore password to perform the
+necessary TLS handshake. This can be realized with the `truststore` and `truststore_password` properties
+using the same approaches as above.
+
+Via the JDBC URL:
+
+ jdbc:avatica:remote:url=https://...;authentication=BASIC;truststore=/tmp/knox_truststore.jks;truststore_password=very_secret
+
+Using Java code:
+
+ ...
+ props.setProperty("truststore", "/tmp/knox_truststore.jks");
+ props.setProperty("truststore_password", "very_secret");
+ DriverManager.getConnection(url, props);
Added: knox/trunk/books/1.6.0/service_cloudera_manager.md
URL: http://svn.apache.org/viewvc/knox/trunk/books/1.6.0/service_cloudera_manager.md?rev=1891689&view=auto
==============================================================================
--- knox/trunk/books/1.6.0/service_cloudera_manager.md (added)
+++ knox/trunk/books/1.6.0/service_cloudera_manager.md Tue Jul 20 15:16:28 2021
@@ -0,0 +1,33 @@
+<!---
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ https://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+--->
+
+### Cloudera Manager ###
+
+Knox provides proxied access to Cloudera Manager API.
+
+#### Gateway configuration ####
+
+The Gateway can be configured for Cloudera Manager API by modifying the topology XML file
+and providing a new service XML file.
+
+In the topology XML file, add the following with the correct hostname:
+
+ <service>
+ <role>CM-API</role>
+ <url>http://<cloudera-manager>:7180/api</url>
+ </service>
+
Added: knox/trunk/books/1.6.0/service_config.md
URL: http://svn.apache.org/viewvc/knox/trunk/books/1.6.0/service_config.md?rev=1891689&view=auto
==============================================================================
--- knox/trunk/books/1.6.0/service_config.md (added)
+++ knox/trunk/books/1.6.0/service_config.md Tue Jul 20 15:16:28 2021
@@ -0,0 +1,41 @@
+<!---
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ https://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+--->
+
+### Common Service Config ###
+
+It is possible to override a few of the global configuration settings provided in gateway-site.xml at the service level.
+These overrides are specified as name/value pairs within the \<service> elements of a particular service.
+The overridden settings apply only to that service.
+
+The following table shows the common configuration settings available at the service level via service level parameters.
+Individual services may support additional service level parameters.
+
+Property | Description | Default
+---------|-------------|---------
+httpclient.maxConnections | The maximum number of connections that a single httpclient will maintain to a single host:port. | 32
+httpclient.connectionTimeout | The amount of time to wait when attempting a connection. The natural unit is milliseconds, but a 's' or 'm' suffix may be used for seconds or minutes respectively. The default timeout is system dependent. | 20s
+httpclient.socketTimeout | The amount of time to wait for data on a socket before aborting the connection. The natural unit is milliseconds, but a 's' or 'm' suffix may be used for seconds or minutes respectively. The default timeout is system dependent but is likely to be indefinite. | 20s
+
+The example below demonstrates how these service level parameters are used.
+
+ <service>
+ <role>HIVE</role>
+ <param>
+ <name>httpclient.socketTimeout</name>
+ <value>180s</value>
+ </param>
+ </service>
Added: knox/trunk/books/1.6.0/service_default_ha.md
URL: http://svn.apache.org/viewvc/knox/trunk/books/1.6.0/service_default_ha.md?rev=1891689&view=auto
==============================================================================
--- knox/trunk/books/1.6.0/service_default_ha.md (added)
+++ knox/trunk/books/1.6.0/service_default_ha.md Tue Jul 20 15:16:28 2021
@@ -0,0 +1,136 @@
+<!---
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ https://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+--->
+
+### Default Service HA support ###
+
+Knox provides connectivity based failover functionality for service calls that can be made to more than one server
+instance in a cluster. To enable this functionality HaProvider configuration needs to be enabled for the service and
+the service itself needs to be configured with more than one URL in the topology file.
+
+The default HA functionality works on a simple round robin algorithm which can be configured to run in the following modes
+
+* The top of the list of URLs is used to route all of a service's REST calls until a connection error occurs (Default).
+* Round robin all the requests, distributing the load evenly across all the HA url's (`enableLoadBalancing`)
+* Round robin with sticky session, requires cookies. Here, only new sessions will round robin ensuring sticky sessions. In case of failure next configured HA url is picked up to dispatch the request which might cause loss of session depending on the backend implementation (`enableStickySession`).
+* Round robin with sticky session and no fallback, depends on `enableStickySession` to be true. Here, new sessions will round robin ensuring sticky sessions. In case of failure, Knox returns http status code 502. By default noFallback is turned off (`noFallback`).
+* Turn off HA round robin feature for a request based on user-agent header property (`disableLoadBalancingForUserAgents`).
+
+This goes on until the setting of 'maxFailoverAttempts' is reached.
+
+At present the following services can use this default High Availability functionality and have been tested for the
+same:
+
+* WEBHCAT
+* HBASE
+* OOZIE
+* HIVE
+
+To enable HA functionality for a service in Knox the following configuration has to be added to the topology file.
+
+ <provider>
+ <role>ha</role>
+ <name>HaProvider</name>
+ <enabled>true</enabled>
+ <param>
+ <name>{SERVICE}</name>
+ <value>maxFailoverAttempts=3;failoverSleep=1000;enabled=true;enableStickySession=true;</value>
+ </param>
+ </provider>
+
+The role and name of the provider above must be as shown. The name in the 'param' section i.e. `{SERVICE}` must match
+that of the service role name that is being configured for HA and the value in the 'param' section is the configuration
+for that particular service in HA mode. For example, the value of `{SERVICE}` can be 'WEBHCAT', 'HBASE' or 'OOZIE'.
+
+To configure multiple services in HA mode, additional 'param' sections can be added.
+
+For example,
+
+ <provider>
+ <role>ha</role>
+ <name>HaProvider</name>
+ <enabled>true</enabled>
+ <param>
+ <name>OOZIE</name>
+ <value>maxFailoverAttempts=3;failoverSleep=1000;enabled=true</value>
+ </param>
+ <param>
+ <name>HBASE</name>
+ <value>maxFailoverAttempts=3;failoverSleep=1000;enabled=true</value>
+ </param>
+ <param>
+ <name>WEBHCAT</name>
+ <value>maxFailoverAttempts=3;failoverSleep=1000;enabled=true</value>
+ </param>
+ </provider>
+
+The various configuration parameters are described below:
+
+* maxFailoverAttempts -
+This is the maximum number of times a failover will be attempted. The failover strategy at this time is very simplistic
+in that the next URL in the list of URLs provided for the service is used and the one that failed is put at the bottom
+of the list. If the list is exhausted and the maximum number of attempts is not reached then the first URL will be tried
+again.
+
+* failoverSleep -
+The amount of time in millis that the process will wait or sleep before attempting to failover.
+
+* enabled -
+Flag to turn the particular service on or off for HA.
+
+* enableLoadBalancing -
+Round robin all the requests, distributing the load evenly across all the HA url's (no sticky sessions)
+
+* enableStickySession -
+Round robin with sticky session.
+
+* noFallback -
+Round robin with sticky session and no fallback, requires `enableStickySession` to be true.
+
+* stickySessionCookieName -
+Customize sticky session cookie name, default is 'KNOX_BACKEND-{serviceName}'.
+
+And for the service configuration itself the additional URLs should be added to the list.
+
+ <service>
+ <role>{SERVICE}</role>
+ <url>http://host1:port1</url>
+ <url>http://host2:port2</url>
+ </service>
+
+For example,
+
+ <service>
+ <role>OOZIE</role>
+ <url>http://sandbox1:11000/oozie</url>
+ <url>http://sandbox2:11000/oozie</url>
+ </service>
+
+* disableLoadBalancingForUserAgents -
+HA round robin feature can be turned off based on client user-agent header property. To turn it off for a specific user-agent add the user-agent value to parameter `disableLoadBalancingForUserAgents` which takes a list of user-agents (NOTE: user-agent value does not have to be an exact match, partial match will work). Default value is `ClouderaODBCDriverforApacheHive`
+
+example:
+
+ <provider>
+ <role>ha</role>
+ <name>HaProvider</name>
+ <enabled>true</enabled>
+ <param>
+ <name>HIVE</name>
+ <value>enableStickySession=true;enableLoadBalancing=true;enabled=true;disableLoadBalancingForUserAgents=Test User Agent, Test User Agent2,Test User Agent3 ,Test User Agent4 ;retrySleep=1000</value>
+ </param>
+ </provider>
+
Added: knox/trunk/books/1.6.0/service_elasticsearch.md
URL: http://svn.apache.org/viewvc/knox/trunk/books/1.6.0/service_elasticsearch.md?rev=1891689&view=auto
==============================================================================
--- knox/trunk/books/1.6.0/service_elasticsearch.md (added)
+++ knox/trunk/books/1.6.0/service_elasticsearch.md Tue Jul 20 15:16:28 2021
@@ -0,0 +1,159 @@
+<!---
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ https://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+--->
+
+### Elasticsearch ###
+
+Elasticsearch provides a REST API for communicating with Elasticsearch via JSON over HTTP. Elasticsearch uses X-Pack to do its own security (authentication and authorization). Therefore, the Knox Gateway is to forward the user credentials to Elasticsearch, and treats the Elasticsearch-authenticated user as "anonymous" to the backend service via a doas query param while Knox will authenticate to backend services as itself.
+
+#### Gateway configuration ####
+
+The Gateway can be configured for Elasticsearch by modifying the topology XML file and providing a new service XML file.
+
+In the topology XML file, add the following new service named "ELASTICSEARCH" with the correct elasticsearch-rest-server hostname and port number (e.g., 9200):
+
+ <service>
+ <role>ELASTICSEARCH</role>
+ <url>http://<elasticsearch-rest-server>:9200/</url>
+ <name>elasticsearch</name>
+ </service>
+
+#### Elasticsearch via Knox Gateway ####
+
+After adding the above to a topology, you can make a cURL request similar to the following structures:
+
+##### 1. Elasticsearch Node Root Query #####
+
+ curl -i -k -u username:password -H "Accept: application/json" -X GET "https://{gateway-hostname}:{gateway-port}/gateway/{topology-name}/elasticsearch"
+
+ or
+
+ curl -i -k -u username:password -H "Accept: application/json" -X GET "https://{gateway-hostname}:{gateway-port}/gateway/{topology-name}/elasticsearch/"
+
+The quotation marks around the URL, can be single quotes or double quotes on both sides, and can also be omitted (Note: This is true for all other Elasticsearch queries via Knox). Below is an example response:
+
+ HTTP/1.1 200 OK
+ Date: Wed, 23 May 2018 16:36:34 GMT
+ Content-Type: application/json; charset=UTF-8
+ Content-Length: 356
+ Server: Jetty(9.2.15.v20160210)
+
+ {"name":"w0A80p0","cluster_name":"elasticsearch","cluster_uuid":"poU7j48pSpu5qQONr64HLQ","version":{"number":"6.2.4","build_hash":"ccec39f","build_date":"2018-04-12T20:37:28.497551Z","build_snapshot":false,"lucene_version":"7.2.1","minimum_wire_compatibility_version":"5.6.0","minimum_index_compatibility_version":"5.0.0"},"tagline":"You Know, for Search"}
+
+##### 2. Elasticsearch Index - Creation, Deletion, Refreshing and Data Operations - Writing, Updating and Retrieval #####
+
+###### (1) Index Creation ######
+
+ curl -i -k -u username:password -H "Content-Type: application/json" -X PUT "https://{gateway-hostname}:{gateway-port}/gateway/{topology-name}/elasticsearch/{index-name}" -d '{
+ "settings" : {
+ "index" : {
+ "number_of_shards" : {index-shards-number},
+ "number_of_replicas" : {index-replicas-number}
+ }
+ }
+ }'
+
+Below is an example response:
+
+ HTTP/1.1 200 OK
+ Date: Wed, 23 May 2018 16:51:31 GMT
+ Content-Type: application/json; charset=UTF-8
+ Content-Length: 65
+ Server: Jetty(9.2.15.v20160210)
+
+ {"acknowledged":true,"shards_acknowledged":true,"index":"estest"}
+
+###### (2) Index Data Writing ######
+
+For adding a "Hello Joe Smith" document:
+
+ curl -i -k -u username:password -H "Content-Type: application/json" -X PUT "https://{gateway-hostname}:{gateway-port}/gateway/{topology-name}/elasticsearch/{index-name}/{document-type-name}/{document-id}" -d '{
+ "title":"Hello Joe Smith"
+ }'
+
+Below is an example response:
+
+ HTTP/1.1 201 Created
+ Date: Wed, 23 May 2018 17:00:17 GMT
+ Location: /estest/greeting/1
+ Content-Type: application/json; charset=UTF-8
+ Content-Length: 158
+ Server: Jetty(9.2.15.v20160210)
+
+ {"_index":"estest","_type":"greeting","_id":"1","_version":1,"result":"created","_shards":{"total":1,"successful":1,"failed":0},"_seq_no":0,"_primary_term":1}
+
+###### (3) Index Refreshing ######
+
+ curl -i -k -u username:password -X POST "https://{gateway-hostname}:{gateway-port}/gateway/{topology-name}/elasticsearch/{index-name}/_refresh"
+
+Below is an example response:
+
+ HTTP/1.1 200 OK
+ Date: Wed, 23 May 2018 17:02:32 GMT
+ Content-Type: application/json; charset=UTF-8
+ Content-Length: 49
+ Server: Jetty(9.2.15.v20160210)
+
+ {"_shards":{"total":1,"successful":1,"failed":0}}
+
+###### (4) Index Data Upgrading ######
+
+For changing the Person Joe Smith to Tom Smith:
+
+ curl -i -k -u username:password -H "Content-Type: application/json" -X PUT "https://{gateway-hostname}:{gateway-port}/gateway/{topology-name}/elasticsearch/{index-name}/{document-type-name}/{document-id}" -d '{
+ "title":"Hello Tom Smith"
+ }'
+
+Below is an example response:
+
+ HTTP/1.1 200 OK
+ Date: Wed, 23 May 2018 17:09:59 GMT
+ Content-Type: application/json; charset=UTF-8
+ Content-Length: 158
+ Server: Jetty(9.2.15.v20160210)
+
+ {"_index":"estest","_type":"greeting","_id":"1","_version":2,"result":"updated","_shards":{"total":1,"successful":1,"failed":0},"_seq_no":1,"_primary_term":1}
+
+###### (5) Index Data Retrieval or Search ######
+
+For finding documents with "title":"Hello" in a specified document-type:
+
+ curl -i -k -u username:password -H "Accept: application/json" -X GET "https://{gateway-hostname}:{gateway-port}/gateway/{topology-name}/elasticsearch/{index-name}/{document-type-name}/ _search?pretty=true;q=title:Hello"
+
+Below is an example response:
+
+ HTTP/1.1 200 OK
+ Date: Wed, 23 May 2018 17:13:08 GMT
+ Content-Type: application/json; charset=UTF-8
+ Content-Length: 244
+ Server: Jetty(9.2.15.v20160210)
+
+ {"took":0,"timed_out":false,"_shards":{"total":1,"successful":1,"skipped":0,"failed":0},"hits":{"total":1,"max_score":0.2876821,"hits":[{"_index":"estest","_type":"greeting","_id":"1","_score":0.2876821,"_source":{"title":"Hello Tom Smith"}}]}}
+
+###### (6) Index Deleting ######
+
+ curl -i -k -u username:password -X DELETE "https://{gateway-hostname}:{gateway-port}/gateway/{topology-name}/elasticsearch/{index-name}"
+
+Below is an example response:
+
+ HTTP/1.1 200 OK
+ Date: Wed, 23 May 2018 17:20:19 GMT
+ Content-Type: application/json; charset=UTF-8
+ Content-Length: 21
+ Server: Jetty(9.2.15.v20160210)
+
+ {"acknowledged":true}
+
Added: knox/trunk/books/1.6.0/service_hbase.md
URL: http://svn.apache.org/viewvc/knox/trunk/books/1.6.0/service_hbase.md?rev=1891689&view=auto
==============================================================================
--- knox/trunk/books/1.6.0/service_hbase.md (added)
+++ knox/trunk/books/1.6.0/service_hbase.md Tue Jul 20 15:16:28 2021
@@ -0,0 +1,720 @@
+<!---
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ https://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+--->
+
+### HBase ###
+
+HBase provides an optional REST API (previously called Stargate).
+See the HBase REST Setup section below for getting started with the HBase REST API and Knox with the Hortonworks Sandbox environment.
+
+The gateway by default includes a sample topology descriptor file `{GATEWAY_HOME}/deployments/sandbox.xml`. The value in this sample is configured to work with an installed Sandbox VM.
+
+ <service>
+ <role>WEBHBASE</role>
+ <url>http://localhost:60080</url>
+ <param>
+ <name>replayBufferSize</name>
+ <value>8</value>
+ </param>
+ </service>
+
+By default the gateway is configured to use port 60080 for Hbase in the Sandbox. Please see the steps to configure the port mapping below.
+
+A default replayBufferSize of 8KB is shown in the sample topology file above. This may need to be increased if your query size is larger.
+
+#### HBase URL Mapping ####
+
+| ------- | ----------------------------------------------------------------------------- |
+| Gateway | `https://{gateway-host}:{gateway-port}/{gateway-path}/{cluster-name}/hbase` |
+| Cluster | `http://{hbase-rest-host}:8080/` |
+
+#### HBase Examples ####
+
+The examples below illustrate the set of basic operations with HBase instance using the REST API.
+Use following link to get more details about HBase REST API: http://hbase.apache.org/book.html#_rest.
+
+Note: Some HBase examples may not work due to enabled [Access Control](http://hbase.apache.org/book.html#_securing_access_to_your_data). User may not be granted access for performing operations in the samples. In order to check if Access Control is configured in the HBase instance verify `hbase-site.xml` for a presence of `org.apache.hadoop.hbase.security.access.AccessController` in `hbase.coprocessor.master.classes` and `hbase.coprocessor.region.classes` properties.
+To grant the Read, Write, Create permissions to `guest` user execute the following command:
+
+ echo grant 'guest', 'RWC' | hbase shell
+
+If you are using a cluster secured with Kerberos you will need to have used `kinit` to authenticate to the KDC.
+
+#### HBase REST API Setup ####
+
+#### Launch REST API ####
+
+The command below launches the REST daemon on port 8080 (the default)
+
+ sudo {HBASE_BIN}/hbase-daemon.sh start rest
+
+Where `{HBASE_BIN}` is `/usr/hdp/current/hbase-master/bin/` in the case of a HDP install.
+
+To use a different port use the `-p` option:
+
+ sudo {HBASE_BIN/hbase-daemon.sh start rest -p 60080
+
+#### Configure Sandbox port mapping for VirtualBox ####
+
+1. Select the VM
+2. Select menu Machine>Settings...
+3. Select tab Network
+4. Select Adapter 1
+5. Press Port Forwarding button
+6. Press Plus button to insert new rule: Name=HBASE REST, Host Port=60080, Guest Port=60080
+7. Press OK to close the rule window
+8. Press OK to Network window save the changes
+
+#### HBase Restart ####
+
+If it becomes necessary to restart HBase you can log into the hosts running HBase and use these steps.
+
+ sudo {HBASE_BIN}/hbase-daemon.sh stop rest
+ sudo -u hbase {HBASE_BIN}/hbase-daemon.sh stop regionserver
+ sudo -u hbase {HBASE_BIN}/hbase-daemon.sh stop master
+ sudo -u hbase {HBASE_BIN}/hbase-daemon.sh stop zookeeper
+
+ sudo -u hbase {HBASE_BIN}/hbase-daemon.sh start regionserver
+ sudo -u hbase {HBASE_BIN}/hbase-daemon.sh start master
+ sudo -u hbase {HBASE_BIN}/hbase-daemon.sh start zookeeper
+ sudo {HBASE_BIN}/hbase-daemon.sh start rest -p 60080
+
+Where `{HBASE_BIN}` is `/usr/hdp/current/hbase-master/bin/` in the case of a HDP Sandbox install.
+
+#### HBase client DSL ####
+
+For more details about client DSL usage please look at the chapter about the client DSL in this guide.
+
+After launching the shell, execute the following command to be able to use the snippets below.
+`import org.apache.knox.gateway.shell.hbase.HBase;`
+
+#### systemVersion() - Query Software Version.
+
+* Request
+ * No request parameters.
+* Response
+ * BasicResponse
+* Example
+ * `HBase.session(session).systemVersion().now().string`
+
+#### clusterVersion() - Query Storage Cluster Version.
+
+* Request
+ * No request parameters.
+* Response
+ * BasicResponse
+* Example
+ * `HBase.session(session).clusterVersion().now().string`
+
+#### status() - Query Storage Cluster Status.
+
+* Request
+ * No request parameters.
+* Response
+ * BasicResponse
+* Example
+ * `HBase.session(session).status().now().string`
+
+#### table().list() - Query Table List.
+
+* Request
+ * No request parameters.
+* Response
+ * BasicResponse
+* Example
+ * `HBase.session(session).table().list().now().string`
+
+#### table(String tableName).schema() - Query Table Schema.
+
+* Request
+ * No request parameters.
+* Response
+ * BasicResponse
+* Example
+ * `HBase.session(session).table().schema().now().string`
+
+#### table(String tableName).create() - Create Table Schema.
+
+* Request
+ * attribute(String name, Object value) - the table's attribute.
+ * family(String name) - starts family definition. Has sub requests:
+ * attribute(String name, Object value) - the family's attribute.
+ * endFamilyDef() - finishes family definition.
+* Response
+ * EmptyResponse
+* Example
+
+
+ HBase.session(session).table(tableName).create()
+ .attribute("tb_attr1", "value1")
+ .attribute("tb_attr2", "value2")
+ .family("family1")
+ .attribute("fm_attr1", "value3")
+ .attribute("fm_attr2", "value4")
+ .endFamilyDef()
+ .family("family2")
+ .family("family3")
+ .endFamilyDef()
+ .attribute("tb_attr3", "value5")
+ .now()
+
+#### table(String tableName).update() - Update Table Schema.
+
+* Request
+ * family(String name) - starts family definition. Has sub requests:
+ * attribute(String name, Object value) - the family's attribute.
+ * endFamilyDef() - finishes family definition.
+* Response
+ * EmptyResponse
+* Example
+
+
+ HBase.session(session).table(tableName).update()
+ .family("family1")
+ .attribute("fm_attr1", "new_value3")
+ .endFamilyDef()
+ .family("family4")
+ .attribute("fm_attr3", "value6")
+ .endFamilyDef()
+ .now()```
+
+#### table(String tableName).regions() - Query Table Metadata.
+
+* Request
+ * No request parameters.
+* Response
+ * BasicResponse
+* Example
+ * `HBase.session(session).table(tableName).regions().now().string`
+
+#### table(String tableName).delete() - Delete Table.
+
+* Request
+ * No request parameters.
+* Response
+ * EmptyResponse
+* Example
+ * `HBase.session(session).table(tableName).delete().now()`
+
+#### table(String tableName).row(String rowId).store() - Cell Store.
+
+* Request
+ * column(String family, String qualifier, Object value, Long time) - the data to store; "qualifier" may be "null"; "time" is optional.
+* Response
+ * EmptyResponse
+* Example
+
+
+ HBase.session(session).table(tableName).row("row_id_1").store()
+ .column("family1", "col1", "col_value1")
+ .column("family1", "col2", "col_value2", 1234567890l)
+ .column("family2", null, "fam_value1")
+ .now()
+
+
+ HBase.session(session).table(tableName).row("row_id_2").store()
+ .column("family1", "row2_col1", "row2_col_value1")
+ .now()
+
+#### table(String tableName).row(String rowId).query() - Cell or Row Query.
+
+* rowId is optional. Querying with null or empty rowId will select all rows.
+* Request
+ * column(String family, String qualifier) - the column to select; "qualifier" is optional.
+ * startTime(Long) - the lower bound for filtration by time.
+ * endTime(Long) - the upper bound for filtration by time.
+ * times(Long startTime, Long endTime) - the lower and upper bounds for filtration by time.
+ * numVersions(Long) - the maximum number of versions to return.
+* Response
+ * BasicResponse
+* Example
+
+
+ HBase.session(session).table(tableName).row("row_id_1")
+ .query()
+ .now().string
+
+
+ HBase.session(session).table(tableName).row().query().now().string
+
+
+ HBase.session(session).table(tableName).row().query()
+ .column("family1", "row2_col1")
+ .column("family2")
+ .times(0, Long.MAX_VALUE)
+ .numVersions(1)
+ .now().string
+
+#### table(String tableName).row(String rowId).delete() - Row, Column, or Cell Delete.
+
+* Request
+ * column(String family, String qualifier) - the column to delete; "qualifier" is optional.
+ * time(Long) - the upper bound for time filtration.
+* Response
+ * EmptyResponse
+* Example
+
+
+ HBase.session(session).table(tableName).row("row_id_1")
+ .delete()
+ .column("family1", "col1")
+ .now()```
+
+
+ HBase.session(session).table(tableName).row("row_id_1")
+ .delete()
+ .column("family2")
+ .time(Long.MAX_VALUE)
+ .now()```
+
+#### table(String tableName).scanner().create() - Scanner Creation.
+
+* Request
+ * startRow(String) - the lower bound for filtration by row id.
+ * endRow(String) - the upper bound for filtration by row id.
+ * rows(String startRow, String endRow) - the lower and upper bounds for filtration by row id.
+ * column(String family, String qualifier) - the column to select; "qualifier" is optional.
+ * batch(Integer) - the batch size.
+ * startTime(Long) - the lower bound for filtration by time.
+ * endTime(Long) - the upper bound for filtration by time.
+ * times(Long startTime, Long endTime) - the lower and upper bounds for filtration by time.
+ * filter(String) - the filter XML definition.
+ * maxVersions(Integer) - the maximum number of versions to return.
+* Response
+ * scannerId : String - the scanner ID of the created scanner. Consumes body.
+* Example
+
+
+ HBase.session(session).table(tableName).scanner().create()
+ .column("family1", "col2")
+ .column("family2")
+ .startRow("row_id_1")
+ .endRow("row_id_2")
+ .batch(1)
+ .startTime(0)
+ .endTime(Long.MAX_VALUE)
+ .filter("")
+ .maxVersions(100)
+ .now()```
+
+#### table(String tableName).scanner(String scannerId).getNext() - Scanner Get Next.
+
+* Request
+ * No request parameters.
+* Response
+ * BasicResponse
+* Example
+ * `HBase.session(session).table(tableName).scanner(scannerId).getNext().now().string`
+
+#### table(String tableName).scanner(String scannerId).delete() - Scanner Deletion.
+
+* Request
+ * No request parameters.
+* Response
+ * EmptyResponse
+* Example
+ * `HBase.session(session).table(tableName).scanner(scannerId).delete().now()`
+
+### HBase via Client DSL ###
+
+This example illustrates sequence of all basic HBase operations:
+1. get system version
+2. get cluster version
+3. get cluster status
+4. create the table
+5. get list of tables
+6. get table schema
+7. update table schema
+8. insert single row into table
+9. query row by id
+10. query all rows
+11. delete cell from row
+12. delete entire column family from row
+13. get table regions
+14. create scanner
+15. fetch values using scanner
+16. drop scanner
+17. drop the table
+
+There are several ways to do this depending upon your preference.
+
+You can use the Groovy interpreter provided with the distribution.
+
+ java -jar bin/shell.jar samples/ExampleHBase.groovy
+
+You can manually type in the KnoxShell DSL script into the interactive Groovy interpreter provided with the distribution.
+
+ java -jar bin/shell.jar
+
+Each line from the file below will need to be typed or copied into the interactive shell.
+
+ /**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+ package org.apache.knox.gateway.shell.hbase
+
+ import org.apache.knox.gateway.shell.Hadoop
+
+ import static java.util.concurrent.TimeUnit.SECONDS
+
+ gateway = "https://localhost:8443/gateway/sandbox"
+ username = "guest"
+ password = "guest-password"
+ tableName = "test_table"
+
+ session = Hadoop.login(gateway, username, password)
+
+ println "System version : " + HBase.session(session).systemVersion().now().string
+
+ println "Cluster version : " + HBase.session(session).clusterVersion().now().string
+
+ println "Status : " + HBase.session(session).status().now().string
+
+ println "Creating table '" + tableName + "'..."
+
+ HBase.session(session).table(tableName).create() \
+ .attribute("tb_attr1", "value1") \
+ .attribute("tb_attr2", "value2") \
+ .family("family1") \
+ .attribute("fm_attr1", "value3") \
+ .attribute("fm_attr2", "value4") \
+ .endFamilyDef() \
+ .family("family2") \
+ .family("family3") \
+ .endFamilyDef() \
+ .attribute("tb_attr3", "value5") \
+ .now()
+
+ println "Done"
+
+ println "Table List : " + HBase.session(session).table().list().now().string
+
+ println "Schema for table '" + tableName + "' : " + HBase.session(session) \
+ .table(tableName) \
+ .schema() \
+ .now().string
+
+ println "Updating schema of table '" + tableName + "'..."
+
+ HBase.session(session).table(tableName).update() \
+ .family("family1") \
+ .attribute("fm_attr1", "new_value3") \
+ .endFamilyDef() \
+ .family("family4") \
+ .attribute("fm_attr3", "value6") \
+ .endFamilyDef() \
+ .now()
+
+ println "Done"
+
+ println "Schema for table '" + tableName + "' : " + HBase.session(session) \
+ .table(tableName) \
+ .schema() \
+ .now().string
+
+ println "Inserting data into table..."
+
+ HBase.session(session).table(tableName).row("row_id_1").store() \
+ .column("family1", "col1", "col_value1") \
+ .column("family1", "col2", "col_value2", 1234567890l) \
+ .column("family2", null, "fam_value1") \
+ .now()
+
+ HBase.session(session).table(tableName).row("row_id_2").store() \
+ .column("family1", "row2_col1", "row2_col_value1") \
+ .now()
+
+ println "Done"
+
+ println "Querying row by id..."
+
+ println HBase.session(session).table(tableName).row("row_id_1") \
+ .query() \
+ .now().string
+
+ println "Querying all rows..."
+
+ println HBase.session(session).table(tableName).row().query().now().string
+
+ println "Querying row by id with extended settings..."
+
+ println HBase.session(session).table(tableName).row().query() \
+ .column("family1", "row2_col1") \
+ .column("family2") \
+ .times(0, Long.MAX_VALUE) \
+ .numVersions(1) \
+ .now().string
+
+ println "Deleting cell..."
+
+ HBase.session(session).table(tableName).row("row_id_1") \
+ .delete() \
+ .column("family1", "col1") \
+ .now()
+
+ println "Rows after delete:"
+
+ println HBase.session(session).table(tableName).row().query().now().string
+
+ println "Extended cell delete"
+
+ HBase.session(session).table(tableName).row("row_id_1") \
+ .delete() \
+ .column("family2") \
+ .time(Long.MAX_VALUE) \
+ .now()
+
+ println "Rows after delete:"
+
+ println HBase.session(session).table(tableName).row().query().now().string
+
+ println "Table regions : " + HBase.session(session).table(tableName) \
+ .regions() \
+ .now().string
+
+ println "Creating scanner..."
+
+ scannerId = HBase.session(session).table(tableName).scanner().create() \
+ .column("family1", "col2") \
+ .column("family2") \
+ .startRow("row_id_1") \
+ .endRow("row_id_2") \
+ .batch(1) \
+ .startTime(0) \
+ .endTime(Long.MAX_VALUE) \
+ .filter("") \
+ .maxVersions(100) \
+ .now().scannerId
+
+ println "Scanner id=" + scannerId
+
+ println "Scanner get next..."
+
+ println HBase.session(session).table(tableName).scanner(scannerId) \
+ .getNext() \
+ .now().string
+
+ println "Dropping scanner with id=" + scannerId
+
+ HBase.session(session).table(tableName).scanner(scannerId).delete().now()
+
+ println "Done"
+
+ println "Dropping table '" + tableName + "'..."
+
+ HBase.session(session).table(tableName).delete().now()
+
+ println "Done"
+
+ session.shutdown(10, SECONDS)
+
+### HBase via cURL
+
+#### Get software version
+
+Set Accept Header to "text/plain", "text/xml", "application/json" or "application/x-protobuf"
+
+ % curl -ik -u guest:guest-password\
+ -H "Accept: application/json"\
+ -X GET 'https://localhost:8443/gateway/sandbox/hbase/version'
+
+#### Get version information regarding the HBase cluster backing the REST API instance
+
+Set Accept Header to "text/plain", "text/xml" or "application/x-protobuf"
+
+ % curl -ik -u guest:guest-password\
+ -H "Accept: text/xml"\
+ -X GET 'https://localhost:8443/gateway/sandbox/hbase/version/cluster'
+
+#### Get detailed status on the HBase cluster backing the REST API instance.
+
+Set Accept Header to "text/plain", "text/xml", "application/json" or "application/x-protobuf"
+
+ curl -ik -u guest:guest-password\
+ -H "Accept: text/xml"\
+ -X GET 'https://localhost:8443/gateway/sandbox/hbase/status/cluster'
+
+#### Get the list of available tables.
+
+Set Accept Header to "text/plain", "text/xml", "application/json" or "application/x-protobuf"
+
+ curl -ik -u guest:guest-password\
+ -H "Accept: text/xml"\
+ -X GET 'https://localhost:8443/gateway/sandbox/hbase'
+
+#### Create table with two column families using xml input
+
+ curl -ik -u guest:guest-password\
+ -H "Accept: text/xml" -H "Content-Type: text/xml"\
+ -d '<?xml version="1.0" encoding="UTF-8"?><TableSchema name="table1"><ColumnSchema name="family1"/><ColumnSchema name="family2"/></TableSchema>'\
+ -X PUT 'https://localhost:8443/gateway/sandbox/hbase/table1/schema'
+
+#### Create table with two column families using JSON input
+
+ curl -ik -u guest:guest-password\
+ -H "Accept: application/json" -H "Content-Type: application/json"\
+ -d '{"name":"table2","ColumnSchema":[{"name":"family3"},{"name":"family4"}]}'\
+ -X PUT 'https://localhost:8443/gateway/sandbox/hbase/table2/schema'
+
+#### Get table metadata
+
+ curl -ik -u guest:guest-password\
+ -H "Accept: text/xml"\
+ -X GET 'https://localhost:8443/gateway/sandbox/hbase/table1/regions'
+
+#### Insert single row table
+
+ curl -ik -u guest:guest-password\
+ -H "Content-Type: text/xml"\
+ -H "Accept: text/xml"\
+ -d '<?xml version="1.0" encoding="UTF-8" standalone="yes"?><CellSet><Row key="cm93MQ=="><Cell column="ZmFtaWx5MTpjb2wx" >dGVzdA==</Cell></Row></CellSet>'\
+ -X POST 'https://localhost:8443/gateway/sandbox/hbase/table1/row1'
+
+#### Insert multiple rows into table
+
+ curl -ik -u guest:guest-password\
+ -H "Content-Type: text/xml"\
+ -H "Accept: text/xml"\
+ -d '<?xml version="1.0" encoding="UTF-8" standalone="yes"?><CellSet><Row key="cm93MA=="><Cell column=" ZmFtaWx5Mzpjb2x1bW4x" >dGVzdA==</Cell></Row><Row key="cm93MQ=="><Cell column=" ZmFtaWx5NDpjb2x1bW4x" >dGVzdA==</Cell></Row></CellSet>'\
+ -X POST 'https://localhost:8443/gateway/sandbox/hbase/table2/false-row-key'
+
+#### Get all data from table
+
+Set Accept Header to "text/plain", "text/xml", "application/json" or "application/x-protobuf"
+
+ curl -ik -u guest:guest-password\
+ -H "Accept: text/xml"\
+ -X GET 'https://localhost:8443/gateway/sandbox/hbase/table1/*'
+
+#### Execute cell or row query
+
+Set Accept Header to "text/plain", "text/xml", "application/json" or "application/x-protobuf"
+
+ curl -ik -u guest:guest-password\
+ -H "Accept: text/xml"\
+ -X GET 'https://localhost:8443/gateway/sandbox/hbase/table1/row1/family1:col1'
+
+#### Delete entire row from table
+
+ curl -ik -u guest:guest-password\
+ -H "Accept: text/xml"\
+ -X DELETE 'https://localhost:8443/gateway/sandbox/hbase/table2/row0'
+
+#### Delete column family from row
+
+ curl -ik -u guest:guest-password\
+ -H "Accept: text/xml"\
+ -X DELETE 'https://localhost:8443/gateway/sandbox/hbase/table2/row0/family3'
+
+#### Delete specific column from row
+
+ curl -ik -u guest:guest-password\
+ -H "Accept: text/xml"\
+ -X DELETE 'https://localhost:8443/gateway/sandbox/hbase/table2/row0/family3'
+
+#### Create scanner
+
+Scanner URL will be in Location response header
+
+ curl -ik -u guest:guest-password\
+ -H "Content-Type: text/xml"\
+ -d '<Scanner batch="1"/>'\
+ -X PUT 'https://localhost:8443/gateway/sandbox/hbase/table1/scanner'
+
+#### Get the values of the next cells found by the scanner
+
+ curl -ik -u guest:guest-password\
+ -H "Accept: application/json"\
+ -X GET 'https://localhost:8443/gateway/sandbox/hbase/table1/scanner/13705290446328cff5ed'
+
+#### Delete scanner
+
+ curl -ik -u guest:guest-password\
+ -H "Accept: text/xml"\
+ -X DELETE 'https://localhost:8443/gateway/sandbox/hbase/table1/scanner/13705290446328cff5ed'
+
+#### Delete table
+
+ curl -ik -u guest:guest-password\
+ -X DELETE 'https://localhost:8443/gateway/sandbox/hbase/table1/schema'
+
+
+### HBase REST HA ###
+
+Please look at #[Default Service HA support] if you wish to explicitly list the URLs under the service definition.
+
+If you run the HBase REST Server from the HBase Region Server nodes, you can utilize more advanced HA support. The HBase
+REST Server does not register itself with ZooKeeper. So the Knox HA component looks in ZooKeeper for instances of HBase Region
+Servers and then performs a light weight ping for the presence of the REST Server on the same hosts. The user should not supply URLs
+in the service definition.
+
+Note: Users of Ambari must manually startup the HBase REST Server.
+
+To enable HA functionality for HBase in Knox the following configuration has to be added to the topology file.
+
+ <provider>
+ <role>ha</role>
+ <name>HaProvider</name>
+ <enabled>true</enabled>
+ <param>
+ <name>WEBHBASE</name>
+ <value>maxFailoverAttempts=3;failoverSleep=1000;enabled=true;zookeeperEnsemble=machine1:2181,machine2:2181,machine3:2181</value>
+ </param>
+ </provider>
+
+The role and name of the provider above must be as shown. The name in the 'param' section must match that of the service
+role name that is being configured for HA and the value in the 'param' section is the configuration for that particular
+service in HA mode. In this case the name is 'WEBHBASE'.
+
+The various configuration parameters are described below:
+
+* maxFailoverAttempts -
+This is the maximum number of times a failover will be attempted. The failover strategy at this time is very simplistic
+in that the next URL in the list of URLs provided for the service is used and the one that failed is put at the bottom
+of the list. If the list is exhausted and the maximum number of attempts is not reached then the first URL will be tried
+again after the list is fetched again from Zookeeper (a refresh of the list is done at this point)
+
+* failoverSleep -
+The amount of time in millis that the process will wait or sleep before attempting to failover.
+
+* enabled -
+Flag to turn the particular service on or off for HA.
+
+* zookeeperEnsemble -
+A comma separated list of host names (or IP addresses) of the ZooKeeper hosts that consist of the ensemble that the HBase
+servers register their information with.
+
+And for the service configuration itself the URLs need NOT be added to the list. For example:
+
+ <service>
+ <role>WEBHBASE</role>
+ </service>
+
+Please note that there is no `<url>` tag specified here as the URLs for the Kafka servers are obtained from ZooKeeper.