You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hawq.apache.org by yo...@apache.org on 2017/02/22 22:10:55 UTC
[01/18] incubator-hawq-docs git commit: HAWQ-1261 - add discussion of
HAWQ administrative log files (closes #88)
Repository: incubator-hawq-docs
Updated Branches:
refs/heads/release/2.1.0.0-incubating c8cb30276 -> 67ca8cc93
HAWQ-1261 - add discussion of HAWQ administrative log files (closes #88)
Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/commit/2351d288
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/tree/2351d288
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/diff/2351d288
Branch: refs/heads/release/2.1.0.0-incubating
Commit: 2351d2880ababe170ea273c1c1cfb6d440eb467e
Parents: c8cb302
Author: Lisa Owen <lo...@pivotal.io>
Authored: Fri Jan 20 17:07:34 2017 -0800
Committer: David Yozie <yo...@apache.org>
Committed: Fri Jan 20 17:08:50 2017 -0800
----------------------------------------------------------------------
.../source/subnavs/apache-hawq-nav-210.erb | 3 +
markdown/admin/logfiles.html.md.erb | 269 +++++++++++++++++++
markdown/admin/maintain.html.md.erb | 22 +-
markdown/admin/monitor.html.md.erb | 59 ----
.../guc/parameter_definitions.html.md.erb | 14 +-
.../HAWQResourceManagement.html.md.erb | 2 +-
6 files changed, 277 insertions(+), 92 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/2351d288/book/master_middleman/source/subnavs/apache-hawq-nav-210.erb
----------------------------------------------------------------------
diff --git a/book/master_middleman/source/subnavs/apache-hawq-nav-210.erb b/book/master_middleman/source/subnavs/apache-hawq-nav-210.erb
index baab7b7..1b997b8 100644
--- a/book/master_middleman/source/subnavs/apache-hawq-nav-210.erb
+++ b/book/master_middleman/source/subnavs/apache-hawq-nav-210.erb
@@ -120,6 +120,9 @@
<li>
<a href="/docs/userguide/2.1.0.0-incubating/admin/monitor.html">Monitoring a HAWQ System</a>
</li>
+ <li>
+ <a href="/docs/userguide/2.1.0.0-incubating/admin/logfiles.html">HAWQ Administrative Log Files</a>
+ </li>
</ul>
</li>
<li class="has_submenu">
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/2351d288/markdown/admin/logfiles.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/admin/logfiles.html.md.erb b/markdown/admin/logfiles.html.md.erb
new file mode 100644
index 0000000..3202ed3
--- /dev/null
+++ b/markdown/admin/logfiles.html.md.erb
@@ -0,0 +1,269 @@
+---
+title: HAWQ Administrative Log Files
+---
+
+Log files are files that include messages and other information about your HAWQ deployment, including the database and utilities. HAWQ administrative log files reside in pre-defined or configured locations on the local file system of the HAWQ node. These log files are distinctly located, formatted, configured, and managed.
+
+Every database instance in HAWQ \(master, standby, and segments\) runs a PostgreSQL database server with its own server log file. You generate log files when you invoke HAWQ management utilities directly, or indirectly via Ambari management operations. Additionally, other components in your HAWQ cluster (PXF, HDFS) generate log files of their own.
+
+Configurable logging parameters may affect what, when, and where messages are logged. You configure HAWQ administrative logging options via HAWQ server configuration parameters or command line options.
+
+Log files may be created and/or rotated at pre-defined or configured intervals. Note that administrative log files are not automatically truncated or deleted. The administrator must implement and periodically run scripts to clean up these log files.
+
+## <a id="topic28"></a>HAWQ Database Server Log Files
+
+
+### <a id="logfile_locate_db"></a>Locating HAWQ Log Files
+
+Each HAWQ master, standby, and segment database instance has its own server log file. Daily log files are created in the `pg_log` subdirectory of the master and segment data directory on the respective HAWQ node. You can obtain the master data directory location from the `hawq_master_directory` property value set in the `$GPHOME/etc/`[`hawq-site.xml`](../reference/HAWQSampleSiteConfig.html) configuration file. Similarly, you can obtain the segment data directory location from the `hawq_segment_directory` property value from `hawq-site.xml`.
+
+The naming convention for HAWQ database server log files is `hawq-<date>_<time>.[csv|log]`. For example, `hawq-2017-01-02_061611.csv` or `hawq-2017-01-03_001704.log`. The number and size of log files present for a given \<date\> is dependent upon the values of certain HAWQ server configuration parameters (discussed later in this document).
+
+### <a id="logfile_format_db"></a>HAWQ Log Format
+
+The HAWQ server log files are written in text or comma-separated values \(CSV\) format.
+
+HAWQ log entries may include the following fields:
+
+<table>
+ <tr><th>#</th><th>Field Name</th><th>Data Type</th><th>Description</th></tr>
+ <tr><td>1</td><td>event_time</td><td>timestamp with time zone</td><td>Th time that the log entry was written to the log</td></tr>
+ <tr><td>2</td><td>user_name</td><td>varchar(100)</td><td>The database user name</td></tr>
+ <tr><td>3</td><td>database_name</td><td>varchar(100)</td><td>The database name</td></tr>
+ <tr><td>4</td><td>process_id</td><td>varchar(10)</td><td>The system process ID (prefixed with "p")</td></tr>
+ <tr><td>5</td><td>thread_id</td><td>varchar(50)</td><td>The thread count (prefixed with "th-")</td></tr>
+ <tr><td>6</td><td>remote_host</td><td>varchar(100)</td><td>The hostname/address of the client machine (if on master node). The hostname/address of the master (if on segment node).</td></tr>
+ <tr><td>7</td><td>remote_port</td><td>varchar(10)</td><td>The segment or master port number</td></tr>
+ <tr><td>8</td><td>session_start_time</td><td>timestamp with time zone</td><td>The time the session connection was opened</td></tr>
+ <tr><td>9</td><td>transaction_id</td><td>int</td><td>The top-level transaction ID on the master; this ID is the parent of any subtransactions.</td></tr>
+ <tr><td>10</td><td>gp_session_id</td><td>text</td><td>The session identifier number (prefixed with "con")</td></tr>
+ <tr><td>11</td><td>gp_command_count</td><td>text</td><td>The command number within a session (prefixed with "cmd")</td></tr>
+ <tr><td>12</td><td>gp_segment</td><td>text</td><td>The segment content identifier. The master always has a content ID of -1.</td></tr>
+ <tr><td>13</td><td>slice_id</td><td>text</td><td>The slice ID (portion of the query plan being executed)</td></tr>
+ <tr><td>14</td><td>distr_tranx_id</td><td>text</td><td>The distributed transaction identifier</td></tr>
+ <tr><td>15</td><td>local_tranx_id</td><td>text</td><td>The local transaction identifier</td></tr>
+ <tr><td>16</td><td>sub_tranx_id</td><td>text</td><td>The subtransaction identifier</td></tr>
+ <tr><td>17</td><td>event_severity</td><td>varchar(10)</td><td>The event severity; values include: LOG, ERROR, FATAL, PANIC, DEBUG1, DEBUG2</td></tr>
+ <tr><td>18</td><td>sql_state_code</td><td>varchar(10)</td><td>The SQL state code associated with the log message</td></tr>
+ <tr><td>19</td><td>event_message</td><td>text</td><td>The log or error message text</td></tr>
+ <tr><td>20</td><td>event_detail</td><td>text</td><td>The detail message text associated with an error or warning message</td></tr>
+ <tr><td>21</td><td>event_hint</td><td>text</td><td>The hint message text associated with an error or warning message</td></tr>
+ <tr><td>22</td><td>internal_query</td><td>text</td><td>The internally-generated query text</td></tr>
+ <tr><td>23</td><td>internal_query_pos</td><td>int</td><td>The cursor index into the internally-generated query text</td></tr>
+ <tr><td>24</td><td>event_context</td><td>text</td><td>The context in which this message is generated</td></tr>
+ <tr><td>25</td><td>debug_query_string</td><td>text</td><td>User-supplied query string with full detail for debugging. This string can be modified for internal use.</td></tr>
+ <tr><td>26</td><td>error_cursor_pos</td><td>int</td><td>The cursor index into the query string</td></tr>
+ <tr><td>27</td><td>func_name</td><td>text</td><td>The function in which this message is generated</td></tr>
+ <tr><td>28</td><td>file_name</td><td>text</td><td>The name of the source file in which the message originated</td></tr>
+ <tr><td>29</td><td>file_line</td><td>int</td><td>The line number of the source file in which the message originated</td></tr>
+ <tr><td>30</td><td>stack_trace</td><td>text</td><td>The stack trace text associated with this message</td></tr>
+</table>
+
+**Note**: Log entries may not include values for all log fields. For example, the `slice_id` field is populated only for log entries associated with a query worker process.
+
+Example HAWQ server log file entries in a `.csv` file:
+
+``` pre
+2017-01-02 01:19:55.293050 PST,"gpadmin","testdb",p381034,th-1259067104,"[local]",,2017-01-02 01:19:21 PST,28083,con3595,cmd7,seg-1,,,x28083,sx1,"LOG","00000","ConnID 207. Acquired resource from resource manager, (256 MB, 0.062500 CORE) x 1.",,,,,,"INSERT INTO extblwstrings VALUES ('my string');",0,,"rmcomm_QD2RM.c",868,
+2017-01-02 06:16:06.383827 PST,,,p360256,th-1259067104,,,,0,,,seg-10000,,,,,"LOG","00000","database system is shut down",,,,,,,0,,"xlog.c",7882,
+```
+
+Example HAWQ server log file entries in a `.log` file:
+
+``` pre
+2017-01-03 00:17:04.848268 PST|||706424|startup||:-LOG: database system was shut down at 2017-01-03 00:16:58 PST
+2017-01-03 00:17:04.988778 PST|gpadmin|template1|706437|cmd1|x31410|:-LOG: statement: SELECT oid,rolname,rolsuper,rolresqueue FROM pg_authid
+2017-01-03 00:17:04.990520 PST|||706432|master resource manager||:-LOG: Resource manager successfully loaded role specifications.
+```
+
+### <a id="logfile_examine_db"></a>Examining HAWQ Log Files
+
+You will examine the HAWQ log files to obtain information about your HAWQ deployment, as well as diagnose problems.
+
+Identify log entries of related transactions using the `transaction_id`. You can identify related log entries of a particular query by the query's session identifier \(`gp_session_id`\) and command identifier \(`gp_command_count`\).
+
+Alternatively, you can use the [*hawq_toolkit*](../reference/toolkit/hawq_toolkit.html#topic16) administrative schema to query HAWQ log files when the [`gp_log_format`](../reference/guc/parameter_definitions.html#gp_log_format) server configuration parameter value specifies `csv` format log files.
+
+For example, the following `hawq_toolkit` query displays the time and message for all `ERROR` logs related to the database named `testdb`:
+
+``` sql
+=# SELECT logtime, logmessage FROM hawq_toolkit.hawq_log_master_concise
+ WHERE logdatabase='testdb' AND logseverity='ERROR';
+```
+
+### <a id="topic30"></a>Searching HAWQ Log Files
+
+Use the [`gplogfilter`](../reference/cli/admin_utilities/gplogfilter.html) HAWQ utility to search through a HAWQ log file for entries matching specific criteria. By default, this utility searches through the HAWQ master log file in the default location.
+
+For example, to display the entries logged to the master log file after 2:00pm on January 18, 2016:
+
+``` shell
+$ gplogfilter -b '2016-01-18 14:00'
+```
+
+Run `gplogfilter` using the [`hawq ssh`](../reference/cli/admin_utilities/hawqssh.html) utility to search through all segment log files simultaneously. For example, create a \<seg\_hosts\> file that includes all segment hosts of interest, then invoke `gplogfilter` to display the last three lines of each segment log file on each segment host. (Note: enter the commands after the `=>` prompt, do not include the `=>`.):
+
+``` shell
+$ hawq ssh -f <seg_hosts>
+=> source /usr/local/hawq/greenplum_path.sh
+=> gplogfilter -n 3 /data/hawq/segment/pg_log/hawq*.csv
+```
+
+### <a id="logging_config"></a>Configuring HAWQ Logging
+
+You can configure the logging characteristics of HAWQ server operations using HAWQ server configuration [Logging Parameters](../reference/guc/guc_category-list.html#topic29). These configuration parameters work in conjunction with each other to determine what information is logged to the HAWQ server log file, and when.
+
+HAWQ server logging-related configuration parameters include:
+
+| Server Configuration Parameter | Description |
+|-----------------------------------|------------------------|
+| [`client_min_messages`](../reference/guc/parameter_definitions.html#client_min_messages) | Identify log level for client messages |
+| [`debug_pretty_print`](../reference/guc/parameter_definitions.html#debug_pretty_print) | Indent/format debug output to make it more readable |
+| [`gp_log_format`](../reference/guc/parameter_definitions.html#gp_log_format) | Identify format of server log files |
+| [`gp_max_csv_line_length`](../reference/guc/parameter_definitions.html#gp_max_csv_line_length) | Set the maximum line length of csv-formatted file |
+| [`log_autostats`](../reference/guc/parameter_definitions.html#log_autostats) | Log information about automatic statistics generation |
+| [`log_connections`](../reference/guc/parameter_definitions.html#log_connections) | Log each client connection |
+| [`log_disconnections`](../reference/guc/parameter_definitions.html#log_disconnections) | Log each client connection termination |
+| [`log_dispatch_stats`](../reference/guc/parameter_definitions.html#log_dispatch_stats) | Log information related to statement dispatch |
+| [`log_duration`](../reference/guc/parameter_definitions.html#log_duration) | Log the duration of completed statements satisfying `log_statement` |
+| [`log_error_verbosity`](../reference/guc/parameter_definitions.html#log_error_verbosity) | Govern how much detail is written to the log |
+| [`log_hostname`](../reference/guc/parameter_definitions.html#log_hostname) | Log the hostname of the connecting host |
+| [`log_min_duration_statement`](../reference/guc/parameter_definitions.html#log_min_duration_statement) | Configure runtime duration at which a statement should be logged |
+| [`log_min_error_statement`](../reference/guc/parameter_definitions.html#log_min_error_statement) | Write sql statement causing the error condition |
+| [`log_min_messages`](../reference/guc/parameter_definitions.html#log_min_messages) | Identify log level for writing to log file |
+| [`log_statement`](../reference/guc/parameter_definitions.html#log_statement)| Control which SQL statements are logged |
+| [`log_timezone`](../reference/guc/parameter_definitions.html#log_timezone)| Set the timezone used in log file timestamps |
+
+
+#### <a id="logging_config_query"></a>Configuring Log Rotation
+
+When log file rotation is enabled, you can control the rotation options with these HAWQ server configuration parameters:
+
+| Server Configuration Parameter | Description |
+|-----------------------------------|------------------------|
+| [`log_rotation_age`](../reference/guc/parameter_definitions.html#log_rotation_age) | Configure the lifetime of log file |
+| [`log_rotation_size`](../reference/guc/parameter_definitions.html#log_rotation_size) | Configure the maximum size of log file |
+| [`log_truncate_on_rotation`](../reference/guc/parameter_definitions.html#log_truncate_on_rotation) | Identify whether to truncate or append to the log file |
+
+#### <a id="logging_config_query"></a>Configuring Query Logging
+
+HAWQ exposes a set of server configuration parameters dealing exclusively with GPORCA query executor and optimizer logging. These include:
+
+| Server Configuration Parameter | Description |
+|-----------------------------------|------------------------|
+| [`debug_print_parse`](../reference/guc/parameter_definitions.html#debug_print_parse) | Log the query parse tree |
+| [`debug_print_plan`](../reference/guc/parameter_definitions.html#debug_print_plan) | Log the query plan |
+| [`debug_print_prelim_plan`](../reference/guc/parameter_definitions.html#debug_print_prelim_plan) | Log the preliminary query plan |
+| [`debug_print_rewritten`](../reference/guc/parameter_definitions.html#debug_print_rewritten) | Log the query rewriter output |
+| [`debug_print_slice_table`](../reference/guc/parameter_definitions.html#debug_print_slice_table) | Log the query slice plan |
+| [`log_executor_stats`](../reference/guc/parameter_definitions.html#log_executor_stats) | Log query executor performance statistics |
+| [`log_parser_stats`](../reference/guc/parameter_definitions.html#log_parser_stats) | Log query parser performance statistics |
+| [`log_planner_status`](../reference/guc/parameter_definitions.html#log_planner_stats) | Log the performance statistics of the legacy query optimizer (planner) |
+| [`log_statement_stats`](../reference/guc/parameter_definitions.html#log_statement_stats) | Log the total performance statistics of the query parser, planner, and executor |
+
+[Query Minidump Log Files](#query_log) identifies configuration options and additional information related to generating query optimizer minidump log files.
+
+### <a id="logfile_manage_db"></a>Managing HAWQ Log Files
+
+HAWQ log output tends to be voluminous, especially at higher debug levels. You should not need to save this information indefinitely. HAWQ administrators typically configure HAWQ to rotate the log files periodically so new log files are created.
+
+Daily log files are created in the `pg_log` subdirectory of the master and each segment data directory. Although log files are rolled over daily, they are not automatically truncated or deleted. Administrators must implement and run scripts to periodically clean up old log files in the `pg_log` directory of the master, standby, and every segment instance.
+
+
+## <a id="mgmtutil_log"></a>Management Utility Log Files
+You invoke HAWQ management utilities both from the command line and when you perform cluster management activities using the Ambari user interface. Regardless of source, these utilities log information about command execution status and results. The messages are logged to both `stdout` and a log file.
+
+Each management utility, when invoked, creates/manages a daily log file. The logs for a particular execution of a utility are added or appended to its daily log file each time that utility is run that day.
+
+**Note**: Some management utilities are aliases for invoking one or more other management utilities. For example, `hawq restart` invokes `hawq stop` and `hawq start`. Logs are written to the daily log files associated with the two invoked utilities.
+
+### <a id="logfile_locate_mu"></a>Locating Management Utility Log Files
+
+The default location of HAWQ management utility log files is `/home/gpadmin/hawqAdminLogs/`. You may choose to specify an alternate log file directory.
+
+The naming convention for HAWQ management utility log files is `<cmdname>_<date>.log`. For example, `hawq_state_20170102.log` or `hawq_start_20161228.log`.
+
+### <a id="logfile_format_mu"></a>Management Utility Log Format
+
+HAWQ management utility log files are written in text format.
+
+Management utility commands' log entries are formed as follows:
+
+ `<date>:<time>:<pid> <cmdname>:<host>:<user>-[<loglevel>]:-<message>`
+
+Log entry fields are described in the table below.
+
+| Log Entry Field | Description |
+|----------------------|------------------------|
+| date | The date (month, day, year) the entry was logged |
+| time | The time the entry was logged |
+| pid | The process id associated with the command invocation |
+| cmdname | The name of the HAWQ management utility command |
+| host | The name of the host on which the command was executed |
+| user | The name of the user invoking the command |
+| loglevel | The log level, one of `DEBUG`, `INFO`, `WARN`, or `FATAL`. The default logging level will log `INFO` and more severe messages. |
+| message | The log message |
+
+
+Example management utility log entries:
+
+ `20170102:06:16:06:451225 hawq_stop:hmast1:gpadmin-[INFO]:-Master host=hmast1`
+
+ `20170102:06:16:15:451826 hawq_start:hmast1:gpadmin-[INFO]:-Start hawq with args: ['start', 'segment']`
+
+
+**Note**: The log files for certain management utility commands, for example `hawq init`, diverge from these conventions.
+
+
+### <a id="logfile_examine_mu"></a>Examining Management Utility Log Entries
+
+You will examine the HAWQ management utility log files to get more detailed information about the execution of the command. Additionally, the most recent log file for a particular command provides date/time and status information about the last time you invoked the command.
+
+
+### <a id="logging_config_mu"></a>Configuring Management Utility Logging
+
+Most management utilities support options for verbose logging and identifying alternate log file directories.
+
+Management utility logging-related configuration command options include:
+
+| Option | Description |
+|----------------------|------------------------|
+| -l | --logdir \<dir\> | Use alternate log file directory \<dir\> for this command invocation. |
+| -v | --verbose | Include `DEBUG` log messages in output. The default logging level will log `INFO` and more severe messages. |
+
+Invoke the management utility command with the `--help` option to determine support for changing the log directory and/or enabling verbose output.
+
+### <a id="logfile_manage_mu"></a>Managing Management Utility Log Files
+
+Although Management Utility log files are rolled over daily, they are not automatically truncated or deleted. Administrators should implement and run scripts or programs to periodically clean up old Management Utility log files.
+
+
+## <a id="query_log"></a>Query Minidump Log Files
+
+You can configure the HAWQ GPORCA query optimizer to generate output minidump files describing the optimization context for a given query. (The information in a minidump file is not in a format that you can easily understand. Rather, you would generate and provide the minidump file to the HAWQ development team or support.)
+
+GPORCA minidump files include the following query-related information:
+
+- Catalog objects including data types, tables, operators, and statistics required by GPORCA.
+- An internal representation (DXL) of the query.
+- An internal representation (DXL) of the query plan produced by GPORCA.
+- System configuration information passed to GPORCA, including server configuration parameters, cost and statistics configuration, and number of segments.
+- A stack trace of errors generated while optimizing the query.
+
+Query minidump files are generated in a `minidumps/` subdirectory in the HAWQ master data directory. The minidump file naming convention is `Minidump_<date>_<time>_<gp_session_id>_<gp_command_count>.mdp`, for example `Minidump_20170102_213701_904_12.mdp`.
+
+Use the [optimizer_minidump](../reference/guc/parameter_definitions.html#optimizer_minidump) HAWQ server configuration parameter to configure minidump generation. The default setting generates minidump files only on error.
+
+
+## <a id="logging_pxf"></a>PXF Log Files
+
+PXF provides both service- and database-level logging. Refer to [PXF Logging](../pxf/TroubleshootingPXF.html#pxflogging) for specific PXF logging configuration and management information.
+
+## <a id="logging_ambari"></a>Ambari Log Files
+
+Ambari log files may be useful in helping diagnose general cluster problems. The Ambari server log files are located in the `/var/log/ambari-server/` directory. Ambari agent log files are located in `/var/log/ambari-agent/`. Refer to [Reviewing Ambari Log Files](https://docs.hortonworks.com/HDPDocuments/Ambari-2.2.1.1/bk_ambari_troubleshooting/content/_reviewing_ambari_log_files.html) for additional information.
+
+
+## <a id="logging_other"></a>Hadoop Log Files
+
+Hadoop log files are managed by your Hadoop distribution. If you built Hadoop from source, your log file locations and configuration may be governed by the build and/or install options you specified during the build process.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/2351d288/markdown/admin/maintain.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/admin/maintain.html.md.erb b/markdown/admin/maintain.html.md.erb
index f4b1491..b3e8ab6 100644
--- a/markdown/admin/maintain.html.md.erb
+++ b/markdown/admin/maintain.html.md.erb
@@ -8,24 +8,8 @@ To keep a HAWQ system running efficiently, the database must be regularly cleare
HAWQ requires that certain tasks be performed regularly to achieve optimal performance. The tasks discussed here are required, but database administrators can automate them using standard UNIX tools such as `cron` scripts. An administrator sets up the appropriate scripts and checks that they execute successfully. See [Recommended Monitoring and Maintenance Tasks](RecommendedMonitoringTasks.html) for additional suggested maintenance activities you can implement to keep your HAWQ system running optimally.
-## <a id="topic10"></a>Database Server Log Files
+## <a id="topic10"></a>Log File Maintenance
-HAWQ log output tends to be voluminous, especially at higher debug levels, and you do not need to save it indefinitely. Administrators rotate the log files periodically so new log files are started and old ones are removed.
+Every database instance in HAWQ \(master and segments\) runs a PostgreSQL database server with its own server log file. For information about managing these log files, refer to [HAWQ Database Server Log Files](logfiles.html#topic28).
-HAWQ has log file rotation enabled on the master and all segment instances. Daily log files are created in the `pg_log` subdirectory of the master and each segment data directory using the following naming convention: <code>hawq-<i>YYYY-MM-DD\_hhmmss</i>.csv</code>. Although log files are rolled over daily, they are not automatically truncated or deleted. Administrators need to implement scripts or programs to periodically clean up old log files in the `pg_log` directory of the master and of every segment instance.
-
-For information about viewing the database server log files, see [Viewing the Database Server Log Files](monitor.html).
-
-## <a id="topic11"></a>Management Utility Log Files
-
-Log files for the HAWQ management utilities are written to `~/hawqAdminLogs` by default. The naming convention for management log files is:
-
-<pre><code><i>script_name_date</i>.log
-</code></pre>
-
-The log entry format is:
-
-<pre><code><i>timestamp:utility:host:user</i>:[INFO|WARN|FATAL]:<i>message</i>
-</code></pre>
-
-The log file for a particular utility execution is appended to its daily log file each time that utility is run.
+Log files are also generated when you invoke HAWQ management utilities such as `hawq start` and `gpfdist`. [Management Utility Log Files](logfiles.html#mgmtutil_log) provides information and maintenance strategies for these log files.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/2351d288/markdown/admin/monitor.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/admin/monitor.html.md.erb b/markdown/admin/monitor.html.md.erb
index 418c8c3..c6e96f1 100644
--- a/markdown/admin/monitor.html.md.erb
+++ b/markdown/admin/monitor.html.md.erb
@@ -145,65 +145,6 @@ Views in the *hawq\_toolkit* schema include:
- *hawq\_workfile\_usage\_per\_segment* - one row for each segment where each row displays the total amount of disk space currently in use for workfiles on the segment
-## <a id="topic28"></a>Viewing the Database Server Log Files
-
-Every database instance in HAWQ \(master and segments\) runs a PostgreSQL database server with its own server log file. Daily log files are created in the `pg_log` directory of the master and each segment data directory.
-
-### <a id="topic29"></a>Log File Format
-
-The server log files are written in comma-separated values \(CSV\) format. Log entries may not include values for all log fields. For example, only log entries associated with a query worker process will have the `slice_id` populated. You can identify related log entries of a particular query by the query's session identifier \(`gp_session_id`\) and command identifier \(`gp_command_count`\).
-
-Log entries may include the following fields:
-
-<table>
- <tr><th>#</th><th>Field Name</th><th>Data Type</th><th>Description</th></tr>
- <tr><td>1</td><td>event_time</td><td>timestamp with time zone</td><td>Time that the log entry was written to the log</td></tr>
- <tr><td>2</td><td>user_name</td><td>varchar(100)</td><td>The database user name</td></tr>
- <tr><td>3</td><td>database_name</td><td>varchar(100)</td><td>The database name</td></tr>
- <tr><td>4</td><td>process_id</td><td>varchar(10)</td><td>The system process ID (prefixed with "p")</td></tr>
- <tr><td>5</td><td>thread_id</td><td>varchar(50)</td><td>The thread count (prefixed with "th")</td></tr>
- <tr><td>6</td><td>remote_host</td><td>varchar(100)</td><td>On the master, the hostname/address of the client machine. On the segment, the hostname/address of the master.</td></tr>
- <tr><td>7</td><td>remote_port</td><td>varchar(10)</td><td>The segment or master port number</td></tr>
- <tr><td>8</td><td>session_start_time</td><td>timestamp with time zone</td><td>Time session connection was opened</td></tr>
- <tr><td>9</td><td>transaction_id</td><td>int</td><td>Top-level transaction ID on the master. This ID is the parent of any subtransactions.</td></tr>
- <tr><td>10</td><td>gp_session_id</td><td>text</td><td>Session identifier number (prefixed with "con")</td></tr>
- <tr><td>11</td><td>gp_command_count</td><td>text</td><td>The command number within a session (prefixed with "cmd")</td></tr>
- <tr><td>12</td><td>gp_segment</td><td>text</td><td>The segment content identifier. The master always has a content ID of -1.</td></tr>
- <tr><td>13</td><td>slice_id</td><td>text</td><td>The slice ID (portion of the query plan being executed)</td></tr>
- <tr><td>14</td><td>distr_tranx_id</td><td>text</td><td>Distributed transaction ID</td></tr>
- <tr><td>15</td><td>local_tranx_id</td><td>text</td><td>Local transaction ID</td></tr>
- <tr><td>16</td><td>sub_tranx_id</td><td>text</td><td>Subtransaction ID</td></tr>
- <tr><td>17</td><td>event_severity</td><td>varchar(10)</td><td>Values include: LOG, ERROR, FATAL, PANIC, DEBUG1, DEBUG2</td></tr>
- <tr><td>18</td><td>sql_state_code</td><td>varchar(10)</td><td>SQL state code associated with the log message</td></tr>
- <tr><td>19</td><td>event_message</td><td>text</td><td>Log or error message text</td></tr>
- <tr><td>20</td><td>event_detail</td><td>text</td><td>Detail message text associated with an error or warning message</td></tr>
- <tr><td>21</td><td>event_hint</td><td>text</td><td>Hint message text associated with an error or warning message</td></tr>
- <tr><td>22</td><td>internal_query</td><td>text</td><td>The internally-generated query text</td></tr>
- <tr><td>23</td><td>internal_query_pos</td><td>int</td><td>The cursor index into the internally-generated query text</td></tr>
- <tr><td>24</td><td>event_context</td><td>text</td><td>The context in which this message gets generated</td></tr>
- <tr><td>25</td><td>debug_query_string</td><td>text</td><td>User-supplied query string with full detail for debugging. This string can be modified for internal use.</td></tr>
- <tr><td>26</td><td>error_cursor_pos</td><td>int</td><td>The cursor index into the query string</td></tr>
- <tr><td>27</td><td>func_name</td><td>text</td><td>The function in which this message is generated</td></tr>
- <tr><td>28</td><td>file_name</td><td>text</td><td>The internal code file where the message originated</td></tr>
- <tr><td>29</td><td>file_line</td><td>int</td><td>The line of the code file where the message originated</td></tr>
- <tr><td>30</td><td>stack_trace</td><td>text</td><td>Stack trace text associated with this message</td></tr>
-</table>
-### <a id="topic30"></a>Searching the HAWQ Server Log Files
-
-You can use the `gplogfilter` HAWQ utility to search through a HAWQ log file for entries matching specific criteria. By default, this utility searches through the HAWQ master log file in the default logging location. For example, to display the entries to the master log file starting after 2 pm on a certain date:
-
-``` shell
-$ gplogfilter -b '2016-01-18 14:00'
-```
-
-To search through all segment log files simultaneously, run `gplogfilter` through the `hawq ssh` utility. For example, specify a \<seg\_hosts\> file that includes all segment hosts of interest, then invoke `gplogfilter` to display the last three lines of each segment log file on each segment host. (Note: enter the commands after the `=>` prompt, do not include the `=>`.):
-
-``` shell
-$ hawq ssh -f <seg_hosts>
-=> source /usr/local/hawq/greenplum_path.sh
-=> gplogfilter -n 3 /data/hawq/segment/pg_log/hawq*.csv
-```
-
## <a id="topic_jx2_rqg_kp"></a>HAWQ Error Codes
The following section describes SQL error codes for certain database events.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/2351d288/markdown/reference/guc/parameter_definitions.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/guc/parameter_definitions.html.md.erb b/markdown/reference/guc/parameter_definitions.html.md.erb
index b568da8..1420793 100644
--- a/markdown/reference/guc/parameter_definitions.html.md.erb
+++ b/markdown/reference/guc/parameter_definitions.html.md.erb
@@ -2593,19 +2593,7 @@ For information about the legacy query optimizer and GPORCA, see [About GPORCA](
## <a name="optimizer_minidump"></a>optimizer\_minidump
-GPORCA generates minidump files to describe the optimization context for a given query. Use the minidump files to analyze HAWQ issues. The minidump file is located under the master data directory and uses the following naming format:
-
-`Minidump_date_time.mdp`
-
-The minidump file contains this query related information:
-
-- Catalog objects including data types, tables, operators, and statistics required by GPORCA
-- An internal representation (DXL) of the query
-- An internal representation (DXL) of the plan produced by GPORCA
-- System configuration information passed to GPORCA such as server configuration parameters, cost and statistics configuration, and number of segments
-- A stack trace of errors generated while optimizing the query
-
-Setting this parameter to `ALWAYS` generates a minidump for all queries.
+GPORCA generates minidump files to describe the optimization context for a given query. Set this parameter to `ALWAYS` to generate a minidump for all queries.
**Note:** Set this parameter to `ONERROR` in production environments to minimize total optimization time.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/2351d288/markdown/resourcemgmt/HAWQResourceManagement.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/resourcemgmt/HAWQResourceManagement.html.md.erb b/markdown/resourcemgmt/HAWQResourceManagement.html.md.erb
index dd5c9b3..cfa9942 100644
--- a/markdown/resourcemgmt/HAWQResourceManagement.html.md.erb
+++ b/markdown/resourcemgmt/HAWQResourceManagement.html.md.erb
@@ -66,4 +66,4 @@ Resource manager adjusts segment localhost original resource capacity from (8192
Resource manager adjusts segment localhost original global resource manager resource capacity from (8192 MB, 5 CORE) to (5120 MB, 5 CORE)
```
-See [Viewing the Database Server Log Files](../admin/monitor.html#topic28) for more information on working with HAWQ log files.
+See [HAWQ Database Server Log Files](../admin/logfiles.html#topic28) for more information on working with HAWQ database server log files.
[09/18] incubator-hawq-docs git commit: HAWQ-1305 Add apache license
header to doc source
Posted by yo...@apache.org.
HAWQ-1305 Add apache license header to doc source
Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/commit/a42c05a9
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/tree/a42c05a9
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/diff/a42c05a9
Branch: refs/heads/release/2.1.0.0-incubating
Commit: a42c05a957a128ed7fd5f492544681d548f826ca
Parents: 18e72d6
Author: David Yozie <yo...@apache.org>
Authored: Wed Feb 1 09:32:04 2017 -0800
Committer: David Yozie <yo...@apache.org>
Committed: Wed Feb 1 09:32:12 2017 -0800
----------------------------------------------------------------------
...ackingUpandRestoringHAWQDatabases.html.md.erb | 19 +++++++++++++++++++
markdown/admin/ClusterExpansion.html.md.erb | 19 +++++++++++++++++++
markdown/admin/ClusterShrink.html.md.erb | 19 +++++++++++++++++++
markdown/admin/FaultTolerance.html.md.erb | 19 +++++++++++++++++++
...cesandHighAvailabilityEnabledHDFS.html.md.erb | 19 +++++++++++++++++++
markdown/admin/HighAvailability.html.md.erb | 19 +++++++++++++++++++
markdown/admin/MasterMirroring.html.md.erb | 19 +++++++++++++++++++
.../admin/RecommendedMonitoringTasks.html.md.erb | 19 +++++++++++++++++++
markdown/admin/RunningHAWQ.html.md.erb | 19 +++++++++++++++++++
markdown/admin/logfiles.html.md.erb | 19 +++++++++++++++++++
markdown/admin/maintain.html.md.erb | 19 +++++++++++++++++++
markdown/admin/monitor.html.md.erb | 19 +++++++++++++++++++
markdown/admin/setuphawqopenv.html.md.erb | 19 +++++++++++++++++++
markdown/admin/startstop.html.md.erb | 19 +++++++++++++++++++
.../HAWQBestPracticesOverview.html.md.erb | 19 +++++++++++++++++++
.../general_bestpractices.html.md.erb | 19 +++++++++++++++++++
.../managing_data_bestpractices.html.md.erb | 19 +++++++++++++++++++
.../managing_resources_bestpractices.html.md.erb | 19 +++++++++++++++++++
.../operating_hawq_bestpractices.html.md.erb | 19 +++++++++++++++++++
.../querying_data_bestpractices.html.md.erb | 19 +++++++++++++++++++
.../secure_bestpractices.html.md.erb | 19 +++++++++++++++++++
markdown/clientaccess/client_auth.html.md.erb | 19 +++++++++++++++++++
.../clientaccess/disable-kerberos.html.md.erb | 19 +++++++++++++++++++
.../g-connecting-with-psql.html.md.erb | 19 +++++++++++++++++++
...g-database-application-interfaces.html.md.erb | 19 +++++++++++++++++++
...g-establishing-a-database-session.html.md.erb | 19 +++++++++++++++++++
...hawq-database-client-applications.html.md.erb | 19 +++++++++++++++++++
.../g-supported-client-applications.html.md.erb | 19 +++++++++++++++++++
...oubleshooting-connection-problems.html.md.erb | 19 +++++++++++++++++++
markdown/clientaccess/index.md.erb | 19 +++++++++++++++++++
markdown/clientaccess/kerberos.html.md.erb | 19 +++++++++++++++++++
markdown/clientaccess/ldap.html.md.erb | 19 +++++++++++++++++++
markdown/clientaccess/roles_privs.html.md.erb | 19 +++++++++++++++++++
.../datamgmt/BasicDataOperations.html.md.erb | 19 +++++++++++++++++++
markdown/datamgmt/ConcurrencyControl.html.md.erb | 19 +++++++++++++++++++
.../HAWQInputFormatforMapReduce.html.md.erb | 19 +++++++++++++++++++
markdown/datamgmt/Transactions.html.md.erb | 19 +++++++++++++++++++
markdown/datamgmt/about_statistics.html.md.erb | 19 +++++++++++++++++++
markdown/datamgmt/dml.html.md.erb | 19 +++++++++++++++++++
.../datamgmt/load/client-loadtools.html.md.erb | 19 +++++++++++++++++++
...creating-external-tables-examples.html.md.erb | 19 +++++++++++++++++++
...out-gpfdist-setup-and-performance.html.md.erb | 19 +++++++++++++++++++
.../load/g-character-encoding.html.md.erb | 19 +++++++++++++++++++
...command-based-web-external-tables.html.md.erb | 19 +++++++++++++++++++
.../load/g-configuration-file-format.html.md.erb | 19 +++++++++++++++++++
...g-controlling-segment-parallelism.html.md.erb | 19 +++++++++++++++++++
...-table-and-declare-a-reject-limit.html.md.erb | 19 +++++++++++++++++++
...ing-and-using-web-external-tables.html.md.erb | 19 +++++++++++++++++++
...e-with-single-row-error-isolation.html.md.erb | 19 +++++++++++++++++++
...based-writable-external-web-table.html.md.erb | 19 +++++++++++++++++++
...ile-based-writable-external-table.html.md.erb | 19 +++++++++++++++++++
...termine-the-transformation-schema.html.md.erb | 19 +++++++++++++++++++
...r-web-or-writable-external-tables.html.md.erb | 19 +++++++++++++++++++
...g-escaping-in-csv-formatted-files.html.md.erb | 19 +++++++++++++++++++
...-escaping-in-text-formatted-files.html.md.erb | 19 +++++++++++++++++++
markdown/datamgmt/load/g-escaping.html.md.erb | 19 +++++++++++++++++++
...se-publications-in-demo-directory.html.md.erb | 19 +++++++++++++++++++
...-example-hawq-file-server-gpfdist.html.md.erb | 19 +++++++++++++++++++
...s-mef-xml-files-in-demo-directory.html.md.erb | 19 +++++++++++++++++++
...le-witsml-files-in-demo-directory.html.md.erb | 19 +++++++++++++++++++
.../g-examples-read-fixed-width-data.html.md.erb | 19 +++++++++++++++++++
.../datamgmt/load/g-external-tables.html.md.erb | 19 +++++++++++++++++++
.../load/g-formatting-columns.html.md.erb | 19 +++++++++++++++++++
.../load/g-formatting-data-files.html.md.erb | 19 +++++++++++++++++++
.../datamgmt/load/g-formatting-rows.html.md.erb | 19 +++++++++++++++++++
.../datamgmt/load/g-gpfdist-protocol.html.md.erb | 19 +++++++++++++++++++
.../load/g-gpfdists-protocol.html.md.erb | 19 +++++++++++++++++++
.../g-handling-errors-ext-table-data.html.md.erb | 19 +++++++++++++++++++
.../load/g-handling-load-errors.html.md.erb | 19 +++++++++++++++++++
...lid-csv-files-in-error-table-data.html.md.erb | 19 +++++++++++++++++++
...ng-and-exporting-fixed-width-data.html.md.erb | 19 +++++++++++++++++++
.../load/g-installing-gpfdist.html.md.erb | 19 +++++++++++++++++++
.../datamgmt/load/g-load-the-data.html.md.erb | 19 +++++++++++++++++++
.../g-loading-and-unloading-data.html.md.erb | 19 +++++++++++++++++++
...-and-writing-non-hdfs-custom-data.html.md.erb | 19 +++++++++++++++++++
...ding-data-using-an-external-table.html.md.erb | 19 +++++++++++++++++++
.../load/g-loading-data-with-copy.html.md.erb | 19 +++++++++++++++++++
.../g-loading-data-with-hawqload.html.md.erb | 19 +++++++++++++++++++
.../g-moving-data-between-tables.html.md.erb | 19 +++++++++++++++++++
...g-data-load-and-query-performance.html.md.erb | 19 +++++++++++++++++++
.../datamgmt/load/g-register_files.html.md.erb | 19 +++++++++++++++++++
.../load/g-representing-null-values.html.md.erb | 19 +++++++++++++++++++
...n-single-row-error-isolation-mode.html.md.erb | 19 +++++++++++++++++++
.../g-starting-and-stopping-gpfdist.html.md.erb | 19 +++++++++++++++++++
.../g-transfer-and-store-the-data.html.md.erb | 19 +++++++++++++++++++
.../load/g-transforming-with-gpload.html.md.erb | 19 +++++++++++++++++++
...ming-with-insert-into-select-from.html.md.erb | 19 +++++++++++++++++++
.../load/g-transforming-xml-data.html.md.erb | 19 +++++++++++++++++++
.../load/g-troubleshooting-gpfdist.html.md.erb | 19 +++++++++++++++++++
...unloading-data-from-hawq-database.html.md.erb | 19 +++++++++++++++++++
...a-using-a-writable-external-table.html.md.erb | 19 +++++++++++++++++++
.../load/g-unloading-data-using-copy.html.md.erb | 19 +++++++++++++++++++
.../g-url-based-web-external-tables.html.md.erb | 19 +++++++++++++++++++
.../load/g-using-a-custom-format.html.md.erb | 19 +++++++++++++++++++
...ng-the-hawq-file-server--gpfdist-.html.md.erb | 19 +++++++++++++++++++
...orking-with-file-based-ext-tables.html.md.erb | 19 +++++++++++++++++++
.../load/g-write-a-transform.html.md.erb | 19 +++++++++++++++++++
...g-write-the-gpfdist-configuration.html.md.erb | 19 +++++++++++++++++++
.../g-xml-transformation-examples.html.md.erb | 19 +++++++++++++++++++
markdown/ddl/ddl-database.html.md.erb | 19 +++++++++++++++++++
markdown/ddl/ddl-partition.html.md.erb | 19 +++++++++++++++++++
markdown/ddl/ddl-schema.html.md.erb | 19 +++++++++++++++++++
markdown/ddl/ddl-storage.html.md.erb | 19 +++++++++++++++++++
markdown/ddl/ddl-table.html.md.erb | 19 +++++++++++++++++++
markdown/ddl/ddl-tablespace.html.md.erb | 19 +++++++++++++++++++
markdown/ddl/ddl-view.html.md.erb | 19 +++++++++++++++++++
markdown/ddl/ddl.html.md.erb | 19 +++++++++++++++++++
markdown/install/aws-config.html.md.erb | 19 +++++++++++++++++++
markdown/install/select-hosts.html.md.erb | 19 +++++++++++++++++++
markdown/overview/ElasticSegments.html.md.erb | 19 +++++++++++++++++++
markdown/overview/HAWQArchitecture.html.md.erb | 19 +++++++++++++++++++
markdown/overview/HAWQOverview.html.md.erb | 19 +++++++++++++++++++
markdown/overview/HDFSCatalogCache.html.md.erb | 19 +++++++++++++++++++
markdown/overview/ManagementTools.html.md.erb | 19 +++++++++++++++++++
markdown/overview/RedundancyFailover.html.md.erb | 19 +++++++++++++++++++
markdown/overview/ResourceManagement.html.md.erb | 19 +++++++++++++++++++
.../TableDistributionStorage.html.md.erb | 19 +++++++++++++++++++
markdown/overview/system-overview.html.md.erb | 19 +++++++++++++++++++
.../plext/UsingProceduralLanguages.html.md.erb | 19 +++++++++++++++++++
markdown/plext/builtin_langs.html.md.erb | 19 +++++++++++++++++++
markdown/plext/using_pgcrypto.html.md.erb | 19 +++++++++++++++++++
markdown/plext/using_pljava.html.md.erb | 19 +++++++++++++++++++
markdown/plext/using_plperl.html.md.erb | 19 +++++++++++++++++++
markdown/plext/using_plpgsql.html.md.erb | 19 +++++++++++++++++++
markdown/plext/using_plpython.html.md.erb | 19 +++++++++++++++++++
markdown/plext/using_plr.html.md.erb | 19 +++++++++++++++++++
markdown/pxf/ConfigurePXF.html.md.erb | 19 +++++++++++++++++++
markdown/pxf/HBasePXF.html.md.erb | 19 +++++++++++++++++++
markdown/pxf/HDFSFileDataPXF.html.md.erb | 19 +++++++++++++++++++
markdown/pxf/HDFSWritablePXF.html.md.erb | 19 +++++++++++++++++++
.../pxf/HawqExtensionFrameworkPXF.html.md.erb | 19 +++++++++++++++++++
markdown/pxf/HivePXF.html.md.erb | 19 +++++++++++++++++++
markdown/pxf/InstallPXFPlugins.html.md.erb | 19 +++++++++++++++++++
markdown/pxf/JsonPXF.html.md.erb | 19 +++++++++++++++++++
.../PXFExternalTableandAPIReference.html.md.erb | 19 +++++++++++++++++++
markdown/pxf/ReadWritePXF.html.md.erb | 19 +++++++++++++++++++
markdown/pxf/TroubleshootingPXF.html.md.erb | 19 +++++++++++++++++++
markdown/query/HAWQQueryProcessing.html.md.erb | 19 +++++++++++++++++++
markdown/query/defining-queries.html.md.erb | 19 +++++++++++++++++++
markdown/query/functions-operators.html.md.erb | 19 +++++++++++++++++++
.../gporca/query-gporca-changed.html.md.erb | 19 +++++++++++++++++++
.../query/gporca/query-gporca-enable.html.md.erb | 19 +++++++++++++++++++
.../gporca/query-gporca-fallback.html.md.erb | 19 +++++++++++++++++++
.../gporca/query-gporca-features.html.md.erb | 19 +++++++++++++++++++
.../gporca/query-gporca-limitations.html.md.erb | 19 +++++++++++++++++++
.../query/gporca/query-gporca-notes.html.md.erb | 19 +++++++++++++++++++
.../gporca/query-gporca-optimizer.html.md.erb | 19 +++++++++++++++++++
.../gporca/query-gporca-overview.html.md.erb | 19 +++++++++++++++++++
markdown/query/query-performance.html.md.erb | 19 +++++++++++++++++++
markdown/query/query-profiling.html.md.erb | 19 +++++++++++++++++++
markdown/query/query.html.md.erb | 19 +++++++++++++++++++
.../CharacterSetSupportReference.html.md.erb | 19 +++++++++++++++++++
markdown/reference/HAWQDataTypes.html.md.erb | 19 +++++++++++++++++++
.../HAWQEnvironmentVariables.html.md.erb | 19 +++++++++++++++++++
.../reference/HAWQSampleSiteConfig.html.md.erb | 19 +++++++++++++++++++
markdown/reference/HAWQSiteConfig.html.md.erb | 19 +++++++++++++++++++
...FSConfigurationParameterReference.html.md.erb | 19 +++++++++++++++++++
.../reference/SQLCommandReference.html.md.erb | 19 +++++++++++++++++++
.../catalog/catalog_ref-html.html.md.erb | 19 +++++++++++++++++++
.../catalog/catalog_ref-tables.html.md.erb | 19 +++++++++++++++++++
.../catalog/catalog_ref-views.html.md.erb | 19 +++++++++++++++++++
.../reference/catalog/catalog_ref.html.md.erb | 19 +++++++++++++++++++
.../catalog/gp_configuration_history.html.md.erb | 19 +++++++++++++++++++
.../catalog/gp_distribution_policy.html.md.erb | 19 +++++++++++++++++++
.../catalog/gp_global_sequence.html.md.erb | 19 +++++++++++++++++++
.../catalog/gp_master_mirroring.html.md.erb | 19 +++++++++++++++++++
.../gp_persistent_database_node.html.md.erb | 19 +++++++++++++++++++
.../gp_persistent_filespace_node.html.md.erb | 19 +++++++++++++++++++
.../gp_persistent_relation_node.html.md.erb | 19 +++++++++++++++++++
.../gp_persistent_relfile_node.html.md.erb | 19 +++++++++++++++++++
.../gp_persistent_tablespace_node.html.md.erb | 19 +++++++++++++++++++
.../catalog/gp_relfile_node.html.md.erb | 19 +++++++++++++++++++
.../catalog/gp_segment_configuration.html.md.erb | 19 +++++++++++++++++++
.../catalog/gp_version_at_initdb.html.md.erb | 19 +++++++++++++++++++
.../reference/catalog/pg_aggregate.html.md.erb | 19 +++++++++++++++++++
markdown/reference/catalog/pg_am.html.md.erb | 19 +++++++++++++++++++
markdown/reference/catalog/pg_amop.html.md.erb | 19 +++++++++++++++++++
markdown/reference/catalog/pg_amproc.html.md.erb | 19 +++++++++++++++++++
.../reference/catalog/pg_appendonly.html.md.erb | 19 +++++++++++++++++++
.../reference/catalog/pg_attrdef.html.md.erb | 19 +++++++++++++++++++
.../reference/catalog/pg_attribute.html.md.erb | 19 +++++++++++++++++++
.../catalog/pg_attribute_encoding.html.md.erb | 19 +++++++++++++++++++
.../catalog/pg_auth_members.html.md.erb | 19 +++++++++++++++++++
markdown/reference/catalog/pg_authid.html.md.erb | 19 +++++++++++++++++++
markdown/reference/catalog/pg_cast.html.md.erb | 19 +++++++++++++++++++
markdown/reference/catalog/pg_class.html.md.erb | 19 +++++++++++++++++++
.../reference/catalog/pg_compression.html.md.erb | 19 +++++++++++++++++++
.../reference/catalog/pg_constraint.html.md.erb | 19 +++++++++++++++++++
.../reference/catalog/pg_conversion.html.md.erb | 19 +++++++++++++++++++
.../reference/catalog/pg_database.html.md.erb | 19 +++++++++++++++++++
markdown/reference/catalog/pg_depend.html.md.erb | 19 +++++++++++++++++++
.../reference/catalog/pg_description.html.md.erb | 19 +++++++++++++++++++
.../reference/catalog/pg_exttable.html.md.erb | 19 +++++++++++++++++++
.../reference/catalog/pg_filespace.html.md.erb | 19 +++++++++++++++++++
.../catalog/pg_filespace_entry.html.md.erb | 19 +++++++++++++++++++
markdown/reference/catalog/pg_index.html.md.erb | 19 +++++++++++++++++++
.../reference/catalog/pg_inherits.html.md.erb | 19 +++++++++++++++++++
.../reference/catalog/pg_language.html.md.erb | 19 +++++++++++++++++++
.../reference/catalog/pg_largeobject.html.md.erb | 19 +++++++++++++++++++
.../reference/catalog/pg_listener.html.md.erb | 19 +++++++++++++++++++
markdown/reference/catalog/pg_locks.html.md.erb | 19 +++++++++++++++++++
.../reference/catalog/pg_namespace.html.md.erb | 19 +++++++++++++++++++
.../reference/catalog/pg_opclass.html.md.erb | 19 +++++++++++++++++++
.../reference/catalog/pg_operator.html.md.erb | 19 +++++++++++++++++++
.../reference/catalog/pg_partition.html.md.erb | 19 +++++++++++++++++++
.../catalog/pg_partition_columns.html.md.erb | 19 +++++++++++++++++++
.../catalog/pg_partition_encoding.html.md.erb | 19 +++++++++++++++++++
.../catalog/pg_partition_rule.html.md.erb | 19 +++++++++++++++++++
.../catalog/pg_partition_templates.html.md.erb | 19 +++++++++++++++++++
.../reference/catalog/pg_partitions.html.md.erb | 19 +++++++++++++++++++
.../reference/catalog/pg_pltemplate.html.md.erb | 19 +++++++++++++++++++
markdown/reference/catalog/pg_proc.html.md.erb | 19 +++++++++++++++++++
.../reference/catalog/pg_resqueue.html.md.erb | 19 +++++++++++++++++++
.../catalog/pg_resqueue_status.html.md.erb | 19 +++++++++++++++++++
.../reference/catalog/pg_rewrite.html.md.erb | 19 +++++++++++++++++++
markdown/reference/catalog/pg_roles.html.md.erb | 19 +++++++++++++++++++
.../reference/catalog/pg_shdepend.html.md.erb | 19 +++++++++++++++++++
.../catalog/pg_shdescription.html.md.erb | 19 +++++++++++++++++++
.../catalog/pg_stat_activity.html.md.erb | 19 +++++++++++++++++++
.../catalog/pg_stat_last_operation.html.md.erb | 19 +++++++++++++++++++
.../catalog/pg_stat_last_shoperation.html.md.erb | 19 +++++++++++++++++++
.../catalog/pg_stat_operations.html.md.erb | 19 +++++++++++++++++++
.../pg_stat_partition_operations.html.md.erb | 19 +++++++++++++++++++
.../reference/catalog/pg_statistic.html.md.erb | 19 +++++++++++++++++++
markdown/reference/catalog/pg_stats.html.md.erb | 19 +++++++++++++++++++
.../reference/catalog/pg_tablespace.html.md.erb | 19 +++++++++++++++++++
.../reference/catalog/pg_trigger.html.md.erb | 19 +++++++++++++++++++
markdown/reference/catalog/pg_type.html.md.erb | 19 +++++++++++++++++++
.../catalog/pg_type_encoding.html.md.erb | 19 +++++++++++++++++++
markdown/reference/catalog/pg_window.html.md.erb | 19 +++++++++++++++++++
.../cli/admin_utilities/analyzedb.html.md.erb | 19 +++++++++++++++++++
.../cli/admin_utilities/gpfdist.html.md.erb | 19 +++++++++++++++++++
.../cli/admin_utilities/gplogfilter.html.md.erb | 19 +++++++++++++++++++
.../cli/admin_utilities/hawqactivate.html.md.erb | 19 +++++++++++++++++++
.../cli/admin_utilities/hawqcheck.html.md.erb | 19 +++++++++++++++++++
.../admin_utilities/hawqcheckperf.html.md.erb | 19 +++++++++++++++++++
.../cli/admin_utilities/hawqconfig.html.md.erb | 19 +++++++++++++++++++
.../cli/admin_utilities/hawqextract.html.md.erb | 19 +++++++++++++++++++
.../admin_utilities/hawqfilespace.html.md.erb | 19 +++++++++++++++++++
.../cli/admin_utilities/hawqinit.html.md.erb | 19 +++++++++++++++++++
.../cli/admin_utilities/hawqload.html.md.erb | 19 +++++++++++++++++++
.../cli/admin_utilities/hawqregister.html.md.erb | 19 +++++++++++++++++++
.../cli/admin_utilities/hawqrestart.html.md.erb | 19 +++++++++++++++++++
.../cli/admin_utilities/hawqscp.html.md.erb | 19 +++++++++++++++++++
.../admin_utilities/hawqssh-exkeys.html.md.erb | 19 +++++++++++++++++++
.../cli/admin_utilities/hawqssh.html.md.erb | 19 +++++++++++++++++++
.../cli/admin_utilities/hawqstart.html.md.erb | 19 +++++++++++++++++++
.../cli/admin_utilities/hawqstate.html.md.erb | 19 +++++++++++++++++++
.../cli/admin_utilities/hawqstop.html.md.erb | 19 +++++++++++++++++++
.../cli/client_utilities/createdb.html.md.erb | 19 +++++++++++++++++++
.../cli/client_utilities/createuser.html.md.erb | 19 +++++++++++++++++++
.../cli/client_utilities/dropdb.html.md.erb | 19 +++++++++++++++++++
.../cli/client_utilities/dropuser.html.md.erb | 19 +++++++++++++++++++
.../cli/client_utilities/pg_dump.html.md.erb | 19 +++++++++++++++++++
.../cli/client_utilities/pg_dumpall.html.md.erb | 19 +++++++++++++++++++
.../cli/client_utilities/pg_restore.html.md.erb | 19 +++++++++++++++++++
.../cli/client_utilities/psql.html.md.erb | 19 +++++++++++++++++++
.../cli/client_utilities/vacuumdb.html.md.erb | 19 +++++++++++++++++++
.../reference/cli/management_tools.html.md.erb | 19 +++++++++++++++++++
.../reference/guc/guc_category-list.html.md.erb | 19 +++++++++++++++++++
markdown/reference/guc/guc_config.html.md.erb | 19 +++++++++++++++++++
.../guc/parameter_definitions.html.md.erb | 19 +++++++++++++++++++
markdown/reference/hawq-reference.html.md.erb | 19 +++++++++++++++++++
markdown/reference/sql/ABORT.html.md.erb | 19 +++++++++++++++++++
.../reference/sql/ALTER-AGGREGATE.html.md.erb | 19 +++++++++++++++++++
.../reference/sql/ALTER-CONVERSION.html.md.erb | 19 +++++++++++++++++++
.../reference/sql/ALTER-DATABASE.html.md.erb | 19 +++++++++++++++++++
.../reference/sql/ALTER-FUNCTION.html.md.erb | 19 +++++++++++++++++++
.../sql/ALTER-OPERATOR-CLASS.html.md.erb | 19 +++++++++++++++++++
.../reference/sql/ALTER-OPERATOR.html.md.erb | 19 +++++++++++++++++++
.../sql/ALTER-RESOURCE-QUEUE.html.md.erb | 19 +++++++++++++++++++
markdown/reference/sql/ALTER-ROLE.html.md.erb | 19 +++++++++++++++++++
markdown/reference/sql/ALTER-TABLE.html.md.erb | 19 +++++++++++++++++++
.../reference/sql/ALTER-TABLESPACE.html.md.erb | 19 +++++++++++++++++++
markdown/reference/sql/ALTER-TYPE.html.md.erb | 19 +++++++++++++++++++
markdown/reference/sql/ALTER-USER.html.md.erb | 19 +++++++++++++++++++
markdown/reference/sql/ANALYZE.html.md.erb | 19 +++++++++++++++++++
markdown/reference/sql/BEGIN.html.md.erb | 19 +++++++++++++++++++
markdown/reference/sql/CHECKPOINT.html.md.erb | 19 +++++++++++++++++++
markdown/reference/sql/CLOSE.html.md.erb | 19 +++++++++++++++++++
markdown/reference/sql/COMMIT.html.md.erb | 19 +++++++++++++++++++
markdown/reference/sql/COPY.html.md.erb | 19 +++++++++++++++++++
.../reference/sql/CREATE-AGGREGATE.html.md.erb | 19 +++++++++++++++++++
markdown/reference/sql/CREATE-CAST.html.md.erb | 19 +++++++++++++++++++
.../reference/sql/CREATE-CONVERSION.html.md.erb | 19 +++++++++++++++++++
.../reference/sql/CREATE-DATABASE.html.md.erb | 19 +++++++++++++++++++
.../sql/CREATE-EXTERNAL-TABLE.html.md.erb | 19 +++++++++++++++++++
.../reference/sql/CREATE-FUNCTION.html.md.erb | 19 +++++++++++++++++++
markdown/reference/sql/CREATE-GROUP.html.md.erb | 19 +++++++++++++++++++
.../reference/sql/CREATE-LANGUAGE.html.md.erb | 19 +++++++++++++++++++
.../sql/CREATE-OPERATOR-CLASS.html.md.erb | 19 +++++++++++++++++++
.../reference/sql/CREATE-OPERATOR.html.md.erb | 19 +++++++++++++++++++
.../sql/CREATE-RESOURCE-QUEUE.html.md.erb | 19 +++++++++++++++++++
markdown/reference/sql/CREATE-ROLE.html.md.erb | 19 +++++++++++++++++++
markdown/reference/sql/CREATE-SCHEMA.html.md.erb | 19 +++++++++++++++++++
.../reference/sql/CREATE-SEQUENCE.html.md.erb | 19 +++++++++++++++++++
.../reference/sql/CREATE-TABLE-AS.html.md.erb | 19 +++++++++++++++++++
markdown/reference/sql/CREATE-TABLE.html.md.erb | 19 +++++++++++++++++++
.../reference/sql/CREATE-TABLESPACE.html.md.erb | 19 +++++++++++++++++++
markdown/reference/sql/CREATE-TYPE.html.md.erb | 19 +++++++++++++++++++
markdown/reference/sql/CREATE-USER.html.md.erb | 19 +++++++++++++++++++
markdown/reference/sql/CREATE-VIEW.html.md.erb | 19 +++++++++++++++++++
markdown/reference/sql/DEALLOCATE.html.md.erb | 19 +++++++++++++++++++
markdown/reference/sql/DECLARE.html.md.erb | 19 +++++++++++++++++++
.../reference/sql/DROP-AGGREGATE.html.md.erb | 19 +++++++++++++++++++
markdown/reference/sql/DROP-CAST.html.md.erb | 19 +++++++++++++++++++
.../reference/sql/DROP-CONVERSION.html.md.erb | 19 +++++++++++++++++++
markdown/reference/sql/DROP-DATABASE.html.md.erb | 19 +++++++++++++++++++
.../sql/DROP-EXTERNAL-TABLE.html.md.erb | 19 +++++++++++++++++++
.../reference/sql/DROP-FILESPACE.html.md.erb | 19 +++++++++++++++++++
markdown/reference/sql/DROP-FUNCTION.html.md.erb | 19 +++++++++++++++++++
markdown/reference/sql/DROP-GROUP.html.md.erb | 19 +++++++++++++++++++
markdown/reference/sql/DROP-LANGUAGE.html.md.erb | 19 +++++++++++++++++++
.../sql/DROP-OPERATOR-CLASS.html.md.erb | 19 +++++++++++++++++++
markdown/reference/sql/DROP-OPERATOR.html.md.erb | 19 +++++++++++++++++++
markdown/reference/sql/DROP-OWNED.html.md.erb | 19 +++++++++++++++++++
.../sql/DROP-RESOURCE-QUEUE.html.md.erb | 19 +++++++++++++++++++
markdown/reference/sql/DROP-ROLE.html.md.erb | 19 +++++++++++++++++++
markdown/reference/sql/DROP-SCHEMA.html.md.erb | 19 +++++++++++++++++++
markdown/reference/sql/DROP-SEQUENCE.html.md.erb | 19 +++++++++++++++++++
markdown/reference/sql/DROP-TABLE.html.md.erb | 19 +++++++++++++++++++
.../reference/sql/DROP-TABLESPACE.html.md.erb | 19 +++++++++++++++++++
markdown/reference/sql/DROP-TYPE.html.md.erb | 19 +++++++++++++++++++
markdown/reference/sql/DROP-USER.html.md.erb | 19 +++++++++++++++++++
markdown/reference/sql/DROP-VIEW.html.md.erb | 19 +++++++++++++++++++
markdown/reference/sql/END.html.md.erb | 19 +++++++++++++++++++
markdown/reference/sql/EXECUTE.html.md.erb | 19 +++++++++++++++++++
markdown/reference/sql/EXPLAIN.html.md.erb | 19 +++++++++++++++++++
markdown/reference/sql/FETCH.html.md.erb | 19 +++++++++++++++++++
markdown/reference/sql/GRANT.html.md.erb | 19 +++++++++++++++++++
markdown/reference/sql/INSERT.html.md.erb | 19 +++++++++++++++++++
markdown/reference/sql/PREPARE.html.md.erb | 19 +++++++++++++++++++
.../reference/sql/REASSIGN-OWNED.html.md.erb | 19 +++++++++++++++++++
.../reference/sql/RELEASE-SAVEPOINT.html.md.erb | 19 +++++++++++++++++++
markdown/reference/sql/RESET.html.md.erb | 19 +++++++++++++++++++
markdown/reference/sql/REVOKE.html.md.erb | 19 +++++++++++++++++++
.../sql/ROLLBACK-TO-SAVEPOINT.html.md.erb | 19 +++++++++++++++++++
markdown/reference/sql/ROLLBACK.html.md.erb | 19 +++++++++++++++++++
markdown/reference/sql/SAVEPOINT.html.md.erb | 19 +++++++++++++++++++
markdown/reference/sql/SELECT-INTO.html.md.erb | 19 +++++++++++++++++++
markdown/reference/sql/SELECT.html.md.erb | 19 +++++++++++++++++++
markdown/reference/sql/SET-ROLE.html.md.erb | 19 +++++++++++++++++++
.../sql/SET-SESSION-AUTHORIZATION.html.md.erb | 19 +++++++++++++++++++
markdown/reference/sql/SET.html.md.erb | 19 +++++++++++++++++++
markdown/reference/sql/SHOW.html.md.erb | 19 +++++++++++++++++++
markdown/reference/sql/TRUNCATE.html.md.erb | 19 +++++++++++++++++++
markdown/reference/sql/VACUUM.html.md.erb | 19 +++++++++++++++++++
.../reference/toolkit/hawq_toolkit.html.md.erb | 19 +++++++++++++++++++
.../requirements/system-requirements.html.md.erb | 19 +++++++++++++++++++
.../ConfigureResourceManagement.html.md.erb | 19 +++++++++++++++++++
.../HAWQResourceManagement.html.md.erb | 19 +++++++++++++++++++
markdown/resourcemgmt/ResourceQueues.html.md.erb | 19 +++++++++++++++++++
.../resourcemgmt/YARNIntegration.html.md.erb | 19 +++++++++++++++++++
markdown/resourcemgmt/best-practices.html.md.erb | 19 +++++++++++++++++++
markdown/resourcemgmt/index.md.erb | 19 +++++++++++++++++++
.../troubleshooting/Troubleshooting.html.md.erb | 19 +++++++++++++++++++
356 files changed, 6764 insertions(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/admin/BackingUpandRestoringHAWQDatabases.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/admin/BackingUpandRestoringHAWQDatabases.html.md.erb b/markdown/admin/BackingUpandRestoringHAWQDatabases.html.md.erb
index 78b0dec..f1e651a 100644
--- a/markdown/admin/BackingUpandRestoringHAWQDatabases.html.md.erb
+++ b/markdown/admin/BackingUpandRestoringHAWQDatabases.html.md.erb
@@ -2,6 +2,25 @@
title: Backing Up and Restoring HAWQ
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
This chapter provides information on backing up and restoring databases in HAWQ system.
As an administrator, you will need to back up and restore your database. HAWQ provides three utilities to help you back up your data:
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/admin/ClusterExpansion.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/admin/ClusterExpansion.html.md.erb b/markdown/admin/ClusterExpansion.html.md.erb
index d3d921b..f282903 100644
--- a/markdown/admin/ClusterExpansion.html.md.erb
+++ b/markdown/admin/ClusterExpansion.html.md.erb
@@ -2,6 +2,25 @@
title: Expanding a Cluster
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Apache HAWQ supports dynamic node expansion. You can add segment nodes while HAWQ is running without having to suspend or terminate cluster operations.
**Note:** This topic describes how to expand a cluster using the command-line interface. If you are using Ambari to manage your HAWQ cluster, see [Expanding the HAWQ Cluster](../admin/ambari-admin.html#amb-expand) in [Managing HAWQ Using Ambari](../admin/ambari-admin.html)
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/admin/ClusterShrink.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/admin/ClusterShrink.html.md.erb b/markdown/admin/ClusterShrink.html.md.erb
index 33c5cc2..32c2938 100644
--- a/markdown/admin/ClusterShrink.html.md.erb
+++ b/markdown/admin/ClusterShrink.html.md.erb
@@ -2,6 +2,25 @@
title: Removing a Node
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
This topic outlines the proper procedure for removing a node from a HAWQ cluster.
In general, you should not need to remove nodes manually from running HAWQ clusters. HAWQ isolates any nodes that HAWQ detects as failing due to hardware or other types of errors.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/admin/FaultTolerance.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/admin/FaultTolerance.html.md.erb b/markdown/admin/FaultTolerance.html.md.erb
index fc9de93..2334b04 100644
--- a/markdown/admin/FaultTolerance.html.md.erb
+++ b/markdown/admin/FaultTolerance.html.md.erb
@@ -2,6 +2,25 @@
title: Understanding the Fault Tolerance Service
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The fault tolerance service (FTS) enables HAWQ to continue operating in the event that a segment node fails. The fault tolerance service runs automatically and requires no additional configuration requirements.
Each segment runs a resource manager process that periodically sends (by default, every 30 seconds) the segment\u2019s status to the master's resource manager process. This interval is controlled by the `hawq_rm_segment_heartbeat_interval` server configuration parameter.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/admin/HAWQFilespacesandHighAvailabilityEnabledHDFS.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/admin/HAWQFilespacesandHighAvailabilityEnabledHDFS.html.md.erb b/markdown/admin/HAWQFilespacesandHighAvailabilityEnabledHDFS.html.md.erb
index b4284be..6923494 100644
--- a/markdown/admin/HAWQFilespacesandHighAvailabilityEnabledHDFS.html.md.erb
+++ b/markdown/admin/HAWQFilespacesandHighAvailabilityEnabledHDFS.html.md.erb
@@ -2,6 +2,25 @@
title: HAWQ Filespaces and High Availability Enabled HDFS
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
If you initialized HAWQ without the HDFS High Availability \(HA\) feature, you can enable it by using the following procedure.
## <a id="enablingthehdfsnamenodehafeature"></a>Enabling the HDFS NameNode HA Feature
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/admin/HighAvailability.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/admin/HighAvailability.html.md.erb b/markdown/admin/HighAvailability.html.md.erb
index 0c2e32b..a6dac79 100644
--- a/markdown/admin/HighAvailability.html.md.erb
+++ b/markdown/admin/HighAvailability.html.md.erb
@@ -2,6 +2,25 @@
title: High Availability in HAWQ
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
A HAWQ cluster can be made highly available by providing fault-tolerant hardware, by enabling HAWQ or HDFS high-availability features, and by performing regular monitoring and maintenance procedures to ensure the health of all system components.
Hardware components eventually fail either due to normal wear or to unexpected circumstances. Loss of power can lead to temporarily unavailable components. You can make a system highly available by providing redundant standbys for components that can fail so services can continue uninterrupted when a failure does occur. In some cases, the cost of redundancy is higher than a user\u2019s tolerance for interruption in service. When this is the case, the goal is to ensure that full service is able to be restored, and can be restored within an expected timeframe.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/admin/MasterMirroring.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/admin/MasterMirroring.html.md.erb b/markdown/admin/MasterMirroring.html.md.erb
index b9352f0..80aa242 100644
--- a/markdown/admin/MasterMirroring.html.md.erb
+++ b/markdown/admin/MasterMirroring.html.md.erb
@@ -2,6 +2,25 @@
title: Using Master Mirroring
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
There are two masters in a HAWQ cluster-- a primary master and a standby master. Clients connect to the primary master and queries can be executed only on the primary master.
You deploy a backup or mirror of the master instance on a separate host machine from the primary master so that the cluster can tolerate a single host failure. A backup master or standby master serves as a warm standby if the primary master becomes non-operational. You create a standby master from the primary master while the primary is online.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/admin/RecommendedMonitoringTasks.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/admin/RecommendedMonitoringTasks.html.md.erb b/markdown/admin/RecommendedMonitoringTasks.html.md.erb
index 5083b44..1c2c6df 100644
--- a/markdown/admin/RecommendedMonitoringTasks.html.md.erb
+++ b/markdown/admin/RecommendedMonitoringTasks.html.md.erb
@@ -2,6 +2,25 @@
title: Recommended Monitoring and Maintenance Tasks
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
This section lists monitoring and maintenance activities recommended to ensure high availability and consistent performance of your HAWQ cluster.
The tables in the following sections suggest activities that a HAWQ System Administrator can perform periodically to ensure that all components of the system are operating optimally. Monitoring activities help you to detect and diagnose problems early. Maintenance activities help you to keep the system up-to-date and avoid deteriorating performance, for example, from bloated system tables or diminishing free disk space.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/admin/RunningHAWQ.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/admin/RunningHAWQ.html.md.erb b/markdown/admin/RunningHAWQ.html.md.erb
index c7de1d5..45164d9 100644
--- a/markdown/admin/RunningHAWQ.html.md.erb
+++ b/markdown/admin/RunningHAWQ.html.md.erb
@@ -2,6 +2,25 @@
title: Running a HAWQ Cluster
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
This section provides information for system administrators responsible for administering a HAWQ deployment.
You should have some knowledge of Linux/UNIX system administration, database management systems, database administration, and structured query language \(SQL\) to administer a HAWQ cluster. Because HAWQ is based on PostgreSQL, you should also have some familiarity with PostgreSQL. The HAWQ documentation calls out similarities between HAWQ and PostgreSQL features throughout.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/admin/logfiles.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/admin/logfiles.html.md.erb b/markdown/admin/logfiles.html.md.erb
index 3202ed3..799191d 100644
--- a/markdown/admin/logfiles.html.md.erb
+++ b/markdown/admin/logfiles.html.md.erb
@@ -2,6 +2,25 @@
title: HAWQ Administrative Log Files
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Log files are files that include messages and other information about your HAWQ deployment, including the database and utilities. HAWQ administrative log files reside in pre-defined or configured locations on the local file system of the HAWQ node. These log files are distinctly located, formatted, configured, and managed.
Every database instance in HAWQ \(master, standby, and segments\) runs a PostgreSQL database server with its own server log file. You generate log files when you invoke HAWQ management utilities directly, or indirectly via Ambari management operations. Additionally, other components in your HAWQ cluster (PXF, HDFS) generate log files of their own.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/admin/maintain.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/admin/maintain.html.md.erb b/markdown/admin/maintain.html.md.erb
index b3e8ab6..25e62db 100644
--- a/markdown/admin/maintain.html.md.erb
+++ b/markdown/admin/maintain.html.md.erb
@@ -2,6 +2,25 @@
title: Routine System Maintenance Tasks
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
## <a id="overview-topic"></a>Overview
To keep a HAWQ system running efficiently, the database must be regularly cleared of expired data and the table statistics must be updated so that the query optimizer has accurate information.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/admin/monitor.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/admin/monitor.html.md.erb b/markdown/admin/monitor.html.md.erb
index c6e96f1..bfd1ec0 100644
--- a/markdown/admin/monitor.html.md.erb
+++ b/markdown/admin/monitor.html.md.erb
@@ -2,6 +2,25 @@
title: Monitoring a HAWQ System
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
You can monitor a HAWQ system using a variety of tools included with the system or available as add-ons.
Observing the HAWQ system day-to-day performance helps administrators understand the system behavior, plan workflow, and troubleshoot problems. This chapter discusses tools for monitoring database performance and activity.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/admin/setuphawqopenv.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/admin/setuphawqopenv.html.md.erb b/markdown/admin/setuphawqopenv.html.md.erb
index 01e62ce..b7f69d8 100644
--- a/markdown/admin/setuphawqopenv.html.md.erb
+++ b/markdown/admin/setuphawqopenv.html.md.erb
@@ -2,6 +2,25 @@
title: Introducing the HAWQ Operating Environment
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Before invoking operations on a HAWQ cluster, you must set up your HAWQ environment. This set up is required for both administrative and non-administrative HAWQ users.
## <a id="hawq_setupenv"></a>Procedure: Setting Up Your HAWQ Operating Environment
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/admin/startstop.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/admin/startstop.html.md.erb b/markdown/admin/startstop.html.md.erb
index 7aac723..523e1de 100644
--- a/markdown/admin/startstop.html.md.erb
+++ b/markdown/admin/startstop.html.md.erb
@@ -2,6 +2,25 @@
title: Starting and Stopping HAWQ
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
In a HAWQ DBMS, the database server instances \(the master and all segments\) are started or stopped across all of the hosts in the system in such a way that they can work together as a unified DBMS.
Because a HAWQ system is distributed across many machines, the process for starting and stopping a HAWQ system is different than the process for starting and stopping a regular PostgreSQL DBMS.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/bestpractices/HAWQBestPracticesOverview.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/bestpractices/HAWQBestPracticesOverview.html.md.erb b/markdown/bestpractices/HAWQBestPracticesOverview.html.md.erb
index 13b4dca..5fa7769 100644
--- a/markdown/bestpractices/HAWQBestPracticesOverview.html.md.erb
+++ b/markdown/bestpractices/HAWQBestPracticesOverview.html.md.erb
@@ -2,6 +2,25 @@
title: Best Practices
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
This chapter provides best practices on using the components and features that are part of a HAWQ system.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/bestpractices/general_bestpractices.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/bestpractices/general_bestpractices.html.md.erb b/markdown/bestpractices/general_bestpractices.html.md.erb
index 503887b..9332d14 100644
--- a/markdown/bestpractices/general_bestpractices.html.md.erb
+++ b/markdown/bestpractices/general_bestpractices.html.md.erb
@@ -2,6 +2,25 @@
title: HAWQ Best Practices
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
This topic addresses general best practices for users who are new to HAWQ.
When using HAWQ, adhere to the following guidelines for best results:
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/bestpractices/managing_data_bestpractices.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/bestpractices/managing_data_bestpractices.html.md.erb b/markdown/bestpractices/managing_data_bestpractices.html.md.erb
index 11d6e02..bae0d2f 100644
--- a/markdown/bestpractices/managing_data_bestpractices.html.md.erb
+++ b/markdown/bestpractices/managing_data_bestpractices.html.md.erb
@@ -2,6 +2,25 @@
title: Best Practices for Managing Data
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
This topic describes best practices for creating databases, loading data, partioning data, and recovering data in HAWQ.
## <a id="topic_xhy_v2j_1v"></a>Best Practices for Loading Data
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/bestpractices/managing_resources_bestpractices.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/bestpractices/managing_resources_bestpractices.html.md.erb b/markdown/bestpractices/managing_resources_bestpractices.html.md.erb
index f770611..0dfe665 100644
--- a/markdown/bestpractices/managing_resources_bestpractices.html.md.erb
+++ b/markdown/bestpractices/managing_resources_bestpractices.html.md.erb
@@ -2,6 +2,25 @@
title: Best Practices for Managing Resources
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
This topic describes best practices for managing resources in HAWQ.
## <a id="topic_ikz_ndx_15"></a>Best Practices for Configuring Resource Management
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/bestpractices/operating_hawq_bestpractices.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/bestpractices/operating_hawq_bestpractices.html.md.erb b/markdown/bestpractices/operating_hawq_bestpractices.html.md.erb
index 9dc56e9..b19ba25 100644
--- a/markdown/bestpractices/operating_hawq_bestpractices.html.md.erb
+++ b/markdown/bestpractices/operating_hawq_bestpractices.html.md.erb
@@ -2,6 +2,25 @@
title: Best Practices for Operating HAWQ
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
This topic provides best practices for operating HAWQ, including recommendations for stopping, starting and monitoring HAWQ.
## <a id="best_practice_config"></a>Best Practices for Configuring HAWQ Parameters
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/bestpractices/querying_data_bestpractices.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/bestpractices/querying_data_bestpractices.html.md.erb b/markdown/bestpractices/querying_data_bestpractices.html.md.erb
index 3efe569..b3c9fe8 100644
--- a/markdown/bestpractices/querying_data_bestpractices.html.md.erb
+++ b/markdown/bestpractices/querying_data_bestpractices.html.md.erb
@@ -2,6 +2,25 @@
title: Best Practices for Querying Data
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
To obtain the best results when querying data in HAWQ, review the best practices described in this topic.
## <a id="virtual_seg_performance"></a>Factors Impacting Query Performance
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/bestpractices/secure_bestpractices.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/bestpractices/secure_bestpractices.html.md.erb b/markdown/bestpractices/secure_bestpractices.html.md.erb
index 04c5343..69f73b2 100644
--- a/markdown/bestpractices/secure_bestpractices.html.md.erb
+++ b/markdown/bestpractices/secure_bestpractices.html.md.erb
@@ -2,6 +2,25 @@
title: Best Practices for Securing HAWQ
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
To secure your HAWQ deployment, review the recommendations listed in this topic.
- Set up SSL to encrypt your client server communication channel. See [Encrypting Client/Server Connections](../clientaccess/client_auth.html#topic5).
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/clientaccess/client_auth.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/clientaccess/client_auth.html.md.erb b/markdown/clientaccess/client_auth.html.md.erb
index a13f4e1..33dc717 100644
--- a/markdown/clientaccess/client_auth.html.md.erb
+++ b/markdown/clientaccess/client_auth.html.md.erb
@@ -2,6 +2,25 @@
title: Configuring Client Authentication
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
When a HAWQ system is first initialized, the system contains one predefined *superuser* role. This role will have the same name as the operating system user who initialized the HAWQ system. This role is referred to as `gpadmin`. By default, the system is configured to only allow local connections to the database from the `gpadmin` role. To allow any other roles to connect, or to allow connections from remote hosts, you configure HAWQ to allow such connections.
## <a id="topic2"></a>Allowing Connections to HAWQ
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/clientaccess/disable-kerberos.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/clientaccess/disable-kerberos.html.md.erb b/markdown/clientaccess/disable-kerberos.html.md.erb
index 5646eec..fe379f4 100644
--- a/markdown/clientaccess/disable-kerberos.html.md.erb
+++ b/markdown/clientaccess/disable-kerberos.html.md.erb
@@ -2,6 +2,25 @@
title: Disabling Kerberos Security
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Follow these steps to disable Kerberos security for HAWQ and PXF for manual installations.
**Note:** If you install or manage your cluster using Ambari, then the HAWQ Ambari plug-in automatically disables security for HAWQ and PXF when you disable security for Hadoop. The following instructions are only necessary for manual installations, or when Hadoop security is disabled outside of Ambari.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/clientaccess/g-connecting-with-psql.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/clientaccess/g-connecting-with-psql.html.md.erb b/markdown/clientaccess/g-connecting-with-psql.html.md.erb
index 0fa501c..2ef3c9f 100644
--- a/markdown/clientaccess/g-connecting-with-psql.html.md.erb
+++ b/markdown/clientaccess/g-connecting-with-psql.html.md.erb
@@ -2,6 +2,25 @@
title: Connecting with psql
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Depending on the default values used or the environment variables you have set, the following examples show how to access a database via `psql`:
``` bash
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/clientaccess/g-database-application-interfaces.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/clientaccess/g-database-application-interfaces.html.md.erb b/markdown/clientaccess/g-database-application-interfaces.html.md.erb
index 29e22c5..db80d38 100644
--- a/markdown/clientaccess/g-database-application-interfaces.html.md.erb
+++ b/markdown/clientaccess/g-database-application-interfaces.html.md.erb
@@ -2,6 +2,25 @@
title: HAWQ Database Drivers and APIs
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
You may want to connect your existing Business Intelligence (BI) or Analytics applications with HAWQ. The database application programming interfaces most commonly used with HAWQ are the Postgres and ODBC and JDBC APIs.
HAWQ provides the following connectivity tools for connecting to the database:
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/clientaccess/g-establishing-a-database-session.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/clientaccess/g-establishing-a-database-session.html.md.erb b/markdown/clientaccess/g-establishing-a-database-session.html.md.erb
index a1c5f1c..454a6c9 100644
--- a/markdown/clientaccess/g-establishing-a-database-session.html.md.erb
+++ b/markdown/clientaccess/g-establishing-a-database-session.html.md.erb
@@ -2,6 +2,25 @@
title: Establishing a Database Session
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Users can connect to HAWQ using a PostgreSQL-compatible client program, such as `psql`. Users and administrators *always* connect to HAWQ through the *master*; the segments cannot accept client connections.
In order to establish a connection to the HAWQ master, you will need to know the following connection information and configure your client program accordingly.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/clientaccess/g-hawq-database-client-applications.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/clientaccess/g-hawq-database-client-applications.html.md.erb b/markdown/clientaccess/g-hawq-database-client-applications.html.md.erb
index a1e8ff3..70604da 100644
--- a/markdown/clientaccess/g-hawq-database-client-applications.html.md.erb
+++ b/markdown/clientaccess/g-hawq-database-client-applications.html.md.erb
@@ -2,6 +2,25 @@
title: HAWQ Client Applications
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
HAWQ comes installed with a number of client utility applications located in the `$GPHOME/bin` directory of your HAWQ master host installation. The following are the most commonly used client utility applications:
|Name|Usage|
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/clientaccess/g-supported-client-applications.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/clientaccess/g-supported-client-applications.html.md.erb b/markdown/clientaccess/g-supported-client-applications.html.md.erb
index 202f625..7569c36 100644
--- a/markdown/clientaccess/g-supported-client-applications.html.md.erb
+++ b/markdown/clientaccess/g-supported-client-applications.html.md.erb
@@ -2,6 +2,25 @@
title: Supported Client Applications
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Users can connect to HAWQ using various client applications:
- A number of [HAWQ Client Applications](g-hawq-database-client-applications.html) are provided with your HAWQ installation. The `psql` client application provides an interactive command-line interface to HAWQ.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/clientaccess/g-troubleshooting-connection-problems.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/clientaccess/g-troubleshooting-connection-problems.html.md.erb b/markdown/clientaccess/g-troubleshooting-connection-problems.html.md.erb
index 0328606..4f7caeb 100644
--- a/markdown/clientaccess/g-troubleshooting-connection-problems.html.md.erb
+++ b/markdown/clientaccess/g-troubleshooting-connection-problems.html.md.erb
@@ -2,6 +2,25 @@
title: Troubleshooting Connection Problems
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
A number of things can prevent a client application from successfully connecting to HAWQ. This topic explains some of the common causes of connection problems and how to correct them.
|Problem|Solution|
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/clientaccess/index.md.erb
----------------------------------------------------------------------
diff --git a/markdown/clientaccess/index.md.erb b/markdown/clientaccess/index.md.erb
index c88adeb..3da95e4 100644
--- a/markdown/clientaccess/index.md.erb
+++ b/markdown/clientaccess/index.md.erb
@@ -2,6 +2,25 @@
title: Managing Client Access
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
This section explains how to configure client connections and authentication for HAWQ:
* <a class="subnav" href="./client_auth.html">Configuring Client Authentication</a>
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/clientaccess/kerberos.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/clientaccess/kerberos.html.md.erb b/markdown/clientaccess/kerberos.html.md.erb
index 2e7cfe5..3a1729d 100644
--- a/markdown/clientaccess/kerberos.html.md.erb
+++ b/markdown/clientaccess/kerberos.html.md.erb
@@ -2,6 +2,25 @@
title: Using Kerberos Authentication
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
**Note:** The following steps for enabling Kerberos *are not required* if you install HAWQ using Ambari.
You can control access to HAWQ with a Kerberos authentication server.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/clientaccess/ldap.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/clientaccess/ldap.html.md.erb b/markdown/clientaccess/ldap.html.md.erb
index 27b204f..cb7da69 100644
--- a/markdown/clientaccess/ldap.html.md.erb
+++ b/markdown/clientaccess/ldap.html.md.erb
@@ -2,6 +2,25 @@
title: Using LDAP Authentication with TLS/SSL
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
You can control access to HAWQ with an LDAP server and, optionally, secure the connection with encryption by adding parameters to pg\_hba.conf file entries.
HAWQ supports LDAP authentication with the TLS/SSL protocol to encrypt communication with an LDAP server:
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/clientaccess/roles_privs.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/clientaccess/roles_privs.html.md.erb b/markdown/clientaccess/roles_privs.html.md.erb
index 4bdf3ee..6c496b3 100644
--- a/markdown/clientaccess/roles_privs.html.md.erb
+++ b/markdown/clientaccess/roles_privs.html.md.erb
@@ -2,6 +2,25 @@
title: Managing Roles and Privileges
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The HAWQ authorization mechanism stores roles and permissions to access database objects in the database and is administered using SQL statements or command-line utilities.
HAWQ manages database access permissions using *roles*. The concept of roles subsumes the concepts of *users* and *groups*. A role can be a database user, a group, or both. Roles can own database objects \(for example, tables\) and can assign privileges on those objects to other roles to control access to the objects. Roles can be members of other roles, thus a member role can inherit the object privileges of its parent role.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/datamgmt/BasicDataOperations.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/datamgmt/BasicDataOperations.html.md.erb b/markdown/datamgmt/BasicDataOperations.html.md.erb
index 66328c7..3118272 100644
--- a/markdown/datamgmt/BasicDataOperations.html.md.erb
+++ b/markdown/datamgmt/BasicDataOperations.html.md.erb
@@ -2,6 +2,25 @@
title: Basic Data Operations
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
This topic describes basic data operations that you perform in HAWQ.
## <a id="topic3"></a>Inserting Rows
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/datamgmt/ConcurrencyControl.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/datamgmt/ConcurrencyControl.html.md.erb b/markdown/datamgmt/ConcurrencyControl.html.md.erb
index 2ced135..805e69e 100644
--- a/markdown/datamgmt/ConcurrencyControl.html.md.erb
+++ b/markdown/datamgmt/ConcurrencyControl.html.md.erb
@@ -2,6 +2,25 @@
title: Concurrency Control
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
This topic discusses the mechanisms used in HAWQ to provide concurrency control.
HAWQ and PostgreSQL do not use locks for concurrency control. They maintain data consistency using a multiversion model, Multiversion Concurrency Control (MVCC). MVCC achieves transaction isolation for each database session, and each query transaction sees a snapshot of data. This ensures the transaction sees consistent data that is not affected by other concurrent transactions.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/datamgmt/HAWQInputFormatforMapReduce.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/datamgmt/HAWQInputFormatforMapReduce.html.md.erb b/markdown/datamgmt/HAWQInputFormatforMapReduce.html.md.erb
index a6fcca2..10c3751 100644
--- a/markdown/datamgmt/HAWQInputFormatforMapReduce.html.md.erb
+++ b/markdown/datamgmt/HAWQInputFormatforMapReduce.html.md.erb
@@ -2,6 +2,25 @@
title: HAWQ InputFormat for MapReduce
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
MapReduce is a programming model developed by Google for processing and generating large data sets on an array of commodity servers. You can use the HAWQ InputFormat class to enable MapReduce jobs to access HAWQ data stored in HDFS.
To use HAWQ InputFormat, you need only to provide the URL of the database to connect to, along with the table name you want to access. HAWQ InputFormat fetches only the metadata of the database and table of interest, which is much less data than the table data itself. After getting the metadata, HAWQ InputFormat determines where and how the table data is stored in HDFS. It reads and parses those HDFS files and processes the parsed table tuples directly inside a Map task.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/datamgmt/Transactions.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/datamgmt/Transactions.html.md.erb b/markdown/datamgmt/Transactions.html.md.erb
index dfc9a5e..f2dc408 100644
--- a/markdown/datamgmt/Transactions.html.md.erb
+++ b/markdown/datamgmt/Transactions.html.md.erb
@@ -2,6 +2,25 @@
title: Working with Transactions
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
This topic describes transaction support in HAWQ.
Transactions allow you to bundle multiple SQL statements in one all-or-nothing operation.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/datamgmt/about_statistics.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/datamgmt/about_statistics.html.md.erb b/markdown/datamgmt/about_statistics.html.md.erb
index 5e2184a..a2df3ba 100644
--- a/markdown/datamgmt/about_statistics.html.md.erb
+++ b/markdown/datamgmt/about_statistics.html.md.erb
@@ -2,6 +2,25 @@
title: About Database Statistics
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
## <a id="overview"></a>Overview
Statistics are metadata that describe the data stored in the database. The query optimizer needs up-to-date statistics to choose the best execution plan for a query. For example, if a query joins two tables and one of them must be broadcast to all segments, the optimizer can choose the smaller of the two tables to minimize network traffic.
[11/18] incubator-hawq-docs git commit: HAWQ-1304 - multiple doc
changes for PXF and Hive Plugin (closes #94)
Posted by yo...@apache.org.
HAWQ-1304 - multiple doc changes for PXF and Hive Plugin (closes #94)
Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/commit/dcfe1a47
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/tree/dcfe1a47
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/diff/dcfe1a47
Branch: refs/heads/release/2.1.0.0-incubating
Commit: dcfe1a47a71dc7eb501d0a74aa5605cf2deba3e5
Parents: 1102fdc
Author: Lisa Owen <lo...@pivotal.io>
Authored: Wed Feb 8 14:51:42 2017 -0800
Committer: David Yozie <yo...@apache.org>
Committed: Wed Feb 8 14:51:42 2017 -0800
----------------------------------------------------------------------
markdown/pxf/HivePXF.html.md.erb | 301 +++++++++----------
.../PXFExternalTableandAPIReference.html.md.erb | 206 ++++++++-----
markdown/pxf/ReadWritePXF.html.md.erb | 24 +-
markdown/pxf/TroubleshootingPXF.html.md.erb | 4 +
4 files changed, 301 insertions(+), 234 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/dcfe1a47/markdown/pxf/HivePXF.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/pxf/HivePXF.html.md.erb b/markdown/pxf/HivePXF.html.md.erb
index e0f6c79..51b317c 100644
--- a/markdown/pxf/HivePXF.html.md.erb
+++ b/markdown/pxf/HivePXF.html.md.erb
@@ -25,15 +25,17 @@ Apache Hive is a distributed data warehousing infrastructure. Hive facilitates
This section describes how to use PXF to access Hive data. Options for querying data stored in Hive include:
-- Creating an external table in PXF and querying that table
- Querying Hive tables via PXF's integration with HCatalog
+- Creating an external table in PXF and querying that table
## <a id="installingthepxfhiveplugin"></a>Prerequisites
Before accessing Hive data with HAWQ and PXF, ensure that:
-- The PXF HDFS plug-in is installed on all cluster nodes. See [Installing PXF Plug-ins](InstallPXFPlugins.html) for PXF plug-in installation information.
-- The PXF Hive plug-in is installed on all cluster nodes.
+- The PXF HDFS plug-in is installed on all HAWQ and HDFS cluster nodes (master, segment, NameNode, DataNode). See [Installing PXF Plug-ins](InstallPXFPlugins.html) for PXF plug-in installation information.
+- The PXF Hive plug-in is installed on all HAWQ and HDFS cluster nodes.
+- If you configured Hadoop with high availability, PXF must also be installed on all HDFS nodes running NameNode services.
+- The Hive client is installed on all PXF nodes.
- The Hive JAR files and conf directory�are installed on all cluster nodes.
- You have tested PXF on HDFS.
- You are running the Hive Metastore service on a machine in your cluster.�
@@ -80,7 +82,7 @@ The following table summarizes external mapping rules for Hive primitive types.
Hive supports complex data types including array, struct, map, and union. PXF maps each of these complex types to `text`. While HAWQ does not natively support these types, you can create HAWQ functions or application code to extract subcomponents of these complex data types.
-An example using complex data types is provided later in this topic.
+An example using complex data types with the `Hive` profile is provided later in this topic.
## <a id="hive_sampledataset"></a>Sample Data Set
@@ -162,9 +164,143 @@ Create a Hive table to expose our sample data set.
In examples later in this section, you will access the `sales_info` Hive table directly via PXF. You will also insert `sales_info` data into tables of other Hive file format types, and use PXF to access those directly as well.
+
+## <a id="hcatalog"></a>Using PXF and HCatalog to Query Hive
+
+You can query Hive tables directly through HCatalog integration with HAWQ and PXF, regardless of the underlying file storage format. This integration allows HAWQ to directly use table metadata stored in HCatalog.
+
+HCatalog is built on top of the Hive metastore and incorporates Hive's DDL. This provides several advantages:
+
+- You do not need to know the table schema of your Hive tables
+- You do not need to manually enter information about Hive table location or format
+- If Hive table metadata changes, HCatalog provides updated metadata. This is in contrast to the use of static external PXF tables to define Hive table metadata for HAWQ.
+
+The following diagram depicts how HAWQ integrates with HCatalog to query Hive tables:
+
+<img src="../images/hawq_hcatalog.png" id="hcatalog__image_ukw_h2v_c5" class="image" width="672" />
+
+1. HAWQ retrieves table metadata from HCatalog using PXF.
+2. HAWQ creates in-memory catalog tables from the retrieved metadata. If a table is referenced multiple times in a transaction, HAWQ uses its in-memory metadata to reduce external calls to HCatalog.
+3. PXF queries Hive using table metadata that is stored in the HAWQ in-memory catalog tables. Table metadata is dropped at the end of the transaction.
+
+
+### <a id="topic_j1l_enabling"></a>Enabling HCatalog Integration
+
+To enable HCatalog query integration in HAWQ, perform the following steps:
+
+1. Make sure your deployment meets the requirements listed in [Prerequisites](#installingthepxfhiveplugin).
+2. If necessary, set the `pxf_service_address` global configuration property to the hostname or IP address and port where you have installed the PXF Hive plug-in. By default, the value is set to `localhost:51200`.
+
+ ``` sql
+ postgres=# SET pxf_service_address TO <hivenode>:51200
+ ```
+
+3. HCatalog internally uses the `pxf` protocol to query. Grant this protocol privilege to all roles requiring access:
+
+ ``` sql
+ postgres=# GRANT ALL ON PROTOCOL pxf TO <role>;
+ ```
+
+4. It is not recommended to create a HAWQ table using the `WITH (OIDS)` clause. If any user tables were created using the `WITH (OIDS)` clause, additional operations are required to enable HCatalog integration. To access a Hive table via HCatalog when user tables were created using `WITH (OIDS)`, HAWQ users must have `SELECT` permission to query every user table within the same schema that was created using the `WITH (OIDS)` clause.
+
+ 1. Determine which user tables were created using the `WITH (OIDS)` clause:
+
+ ``` sql
+ postgres=# SELECT oid, relname FROM pg_class
+ WHERE relhasoids = true
+ AND relnamespace <> (SELECT oid FROM pg_namespace WHERE nspname = 'pg_catalog');
+ ```
+
+ 2. Grant `SELECT` privilege on all returned tables to all roles to which you chose to provide HCatalog query access. For example:
+
+ ``` sql
+ postgres=# GRANT SELECT ON <table-created-WITH-OIDS> TO <role>
+ ```
+
+### <a id="topic_j1l_y55_c5"></a>Usage
+
+To query a Hive table with HCatalog integration, query HCatalog directly from HAWQ. The query syntax is:
+
+``` sql
+postgres=# SELECT * FROM hcatalog.hive-db-name.hive-table-name;
+```
+
+For example:
+
+``` sql
+postgres=# SELECT * FROM hcatalog.default.sales_info;
+```
+
+To obtain a description of a Hive table with HCatalog integration, you can use the `psql` client interface.
+
+- Within HAWQ, use either the `\d hcatalog.hive-db-name.hive-table-name` or `\d+ hcatalog.hive-db-name.hive-table-name` commands to describe a single table. `\d` displays only HAWQ's interpretation of the underlying source (Hive in this case) data type, while `\d+` displays both the HAWQ interpreted and Hive source data types. For example, from the `psql` client interface:
+
+ ``` shell
+ $ psql -d postgres
+ ```
+
+ ``` sql
+ postgres=# \d+ hcatalog.default.sales_info_rcfile;
+ ```
+
+ ``` shell
+ PXF Hive Table "default.sales_info"
+ Column | Type | Source type
+ ------------------+--------+-------------
+ location | text | string
+ month | text | string
+ number_of_orders | int4 | int
+ total_sales | float8 | double
+ ```
+- Use `\d hcatalog.hive-db-name.*` to describe the whole database schema, i.e. all tables in `hive-db-name`.
+- Use `\d hcatalog.*.*` to describe the whole schema, i.e. all databases and tables.
+
+When using `\d` or `\d+` commands in the `psql` HAWQ client, `hcatalog` will not be listed as a database. If you use other `psql` compatible clients, `hcatalog` will be listed as a database with a size value of `-1` since `hcatalog` is not a real database in HAWQ.
+
+Alternatively, you can use the `pxf_get_item_fields` user-defined function (UDF) to obtain Hive table descriptions from other client interfaces or third-party applications. The UDF takes a PXF profile and a table pattern string as its input parameters. **Note:** The only supported input profile at this time is `'Hive'`.
+
+- The following statement returns a description of a specific table. The description includes path, itemname (table), fieldname, fieldtype (HAWQ type), and sourcefieldtype (Hive type).
+
+ ``` sql
+ postgres=# SELECT * FROM pxf_get_item_fields('Hive','default.sales_info');
+ ```
+
+ ``` pre
+ path | itemname | fieldname | fieldtype | sourcefieldtype
+ ---------+------------+------------------+-----------+-----------------
+ default | sales_info | location | text | string
+ default | sales_info | month | text | string
+ default | sales_info | number_of_orders | int4 | int
+ default | sales_info | total_sales | float8 | double
+ ```
+
+- The following statement returns table descriptions from the default database.
+
+ ``` sql
+ postgres=# SELECT * FROM pxf_get_item_fields('Hive','default.*');
+ ```
+
+- The following statement returns a description of the entire schema.
+
+ ``` sql
+ postgres=# SELECT * FROM pxf_get_item_fields('Hive', '*.*');
+ ```
+
+### <a id="topic_r5k_pst_25"></a>Limitations
+
+HCatalog integration has the following limitations:
+
+- HCatalog queries on Hive tables with complex type fields return those fields serialized as text.
+- Even for primitive types, HCatalog metadata descriptions produced by `\d` are HAWQ's interpretation of the underlying Hive data types. For example, the Hive type `tinyint` is converted to HAWQ type `int2`. (See [Data Type Mapping](#hive_primdatatypes).)
+- HAWQ reserves the database name `hcatalog` for system use. You cannot connect to or alter the system `hcatalog` database.
+
+
+
## <a id="topic_p2s_lvl_28"></a>Querying External Hive Data
-The PXF Hive plug-in supports several Hive-related profiles. These include `Hive`, `HiveText`, and `HiveRC`.
+In the previous section, you used HCatalog integration to query a Hive table. You can also create a PXF/HAWQ external table to access Hive table data. This Hive table access mechanism requires that you identify an appropriate Hive profile.
+
+The PXF Hive plug-in supports several Hive-related profiles. These include `Hive`, `HiveText`, and `HiveRC`. The `HiveText` and `HiveRC` profiles are specifically optimized for text and RC file formats, respectively. The `Hive` profile is optimized for all file storage types; use the `Hive` profile when the underlying Hive table is composed of multiple partitions with differing file formats.
Use the following syntax to create a HAWQ external table representing Hive data:
@@ -191,10 +327,7 @@ Hive-plug-in-specific keywords and values used in the [CREATE EXTERNAL TABLE](..
## <a id="profile_hive"></a>Hive Profile
-The `Hive` profile works with any Hive file format. It can access heterogenous format data in a single table where each partition may be stored as a different file format.
-
-While you can use the `Hive` profile to access any file format, the more specific profiles perform better for those single file format types.
-
+Use the `Hive` profile with any Hive file storage format. With the `Hive` profile, you can also access heterogenous format data in a single table where each partition may be stored in a different file format. In both cases, the `Hive` profile will use the optimal `Hive*` profile for the underlying file format type.
### <a id="profile_hive_using"></a>Example: Using the Hive Profile
@@ -226,7 +359,7 @@ Use the `Hive` profile to create a queryable HAWQ external table from the Hive `
## <a id="profile_hivetext"></a>HiveText Profile
-Use the `HiveText` profile to query text format files. The `HiveText` profile is more performant than the `Hive` profile.
+Use the `HiveText` profile to query text format files.
**Note**: When using the `HiveText` profile, you *must* specify a delimiter option in *both* the `LOCATION` and `FORMAT` clauses.
@@ -266,9 +399,11 @@ Use the PXF `HiveText` profile to create a queryable HAWQ external table from th
The RCFile Hive format is used for row columnar formatted data. The `HiveRC` profile provides access to RCFile data.
+**Note**: When using the `HiveRC` profile, you *must* specify a delimiter option in *both* the `LOCATION` and `FORMAT` clauses.
+
### <a id="profile_hiverc_rcfiletbl_using"></a>Example: Using the HiveRC Profile
-Use the `HiveRC` profile to query RCFile-formatted data in Hive tables. The `HiveRC` profile is more performant than the `Hive` profile for this file format type.
+Use the `HiveRC` profile to query RCFile-formatted data in Hive tables.
1. Create a Hive table with RCFile format:
@@ -347,14 +482,9 @@ postgres=# SELECT fname,lname FROM pxf_parquet_table;
```
-## <a id="profileperf"></a>Profile Performance Considerations
-
-The `HiveRC` and `HiveText` profiles are faster than the generic `Hive` profile.
+## <a id="complex_dt_example"></a>Hive Profile Complex Data Type Example
-
-## <a id="complex_dt_example"></a>Complex Data Type Example
-
-This example will employ the array and map complex types, specifically an array of integers and a string key/value pair map.
+This example employs the `Hive` profile and the array and map complex types, specifically an array of integers and a string key/value pair map.
The data schema for this example includes fields with the following names and data types:
@@ -458,138 +588,7 @@ When specifying an array field in a Hive table, you must identify the terminator
(10 rows)
```
- `intarray` and `propmap` are each text strings.
-
-## <a id="hcatalog"></a>Using PXF and HCatalog to Query Hive
-
-Hive tables can be queried directly through HCatalog integration with HAWQ and PXF, regardless of the underlying file storage format.
-
-In previous sections, you created an external table in PXF that described the target table's Hive metadata. Another option for querying Hive tables is to take advantage of HAWQ's integration with HCatalog. This integration allows HAWQ to directly use table metadata stored in HCatalog.
-
-HCatalog is built on top of the Hive metastore and incorporates Hive's DDL. This provides several advantages:
-
-- You do not need to know the table schema of your Hive tables
-- You do not need to manually enter information about Hive table location or format
-- If Hive table metadata changes, HCatalog provides updated metadata. This is in contrast to the use of static external PXF tables to define Hive table metadata for HAWQ.
-
-The following diagram depicts how HAWQ integrates with HCatalog to query Hive tables:
-
-<img src="../images/hawq_hcatalog.png" id="hcatalog__image_ukw_h2v_c5" class="image" width="672" />
-
-1. HAWQ retrieves table metadata from HCatalog using PXF.
-2. HAWQ creates in-memory catalog tables from the retrieved metadata. If a table is referenced multiple times in a transaction, HAWQ uses its in-memory metadata to reduce external calls to HCatalog.
-3. PXF queries Hive using table metadata that is stored in the HAWQ in-memory catalog tables. Table metadata is dropped at the end of the transaction.
-
-
-### <a id="topic_j1l_enabling"></a>Enabling HCatalog Integration
-
-To enable HCatalog query integration in HAWQ, perform the following steps:
-
-1. Make sure your deployment meets the requirements listed in [Prerequisites](#installingthepxfhiveplugin).
-2. If necessary, set the `pxf_service_address` global configuration property to the hostname or IP address and port where you have installed the PXF Hive plug-in. By default, the value is set to `localhost:51200`.
-
- ``` sql
- postgres=# SET pxf_service_address TO <hivenode>:51200
- ```
-
-3. HCatalog internally uses the `pxf` protocol to query. Grant this protocol privilege to all roles requiring access:
-
- ``` sql
- postgres=# GRANT ALL ON PROTOCOL pxf TO <role>;
- ```
-
-4. It is not recommended to create a HAWQ table using the `WITH (OIDS)` clause. If any user tables were created using the `WITH (OIDS)` clause, additional operations are required to enable HCatalog integration. To access a Hive table via HCatalog when user tables were created using `WITH (OIDS)`, HAWQ users must have `SELECT` permission to query every user table within the same schema that was created using the `WITH (OIDS)` clause.
-
- 1. Determine which user tables were created using the `WITH (OIDS)` clause:
-
- ``` sql
- postgres=# SELECT oid, relname FROM pg_class
- WHERE relhasoids = true
- AND relnamespace <> (SELECT oid FROM pg_namespace WHERE nspname = 'pg_catalog');
- ```
-
- 2. Grant `SELECT` privilege on all returned tables to all roles to which you chose to provide HCatalog query access. For example:
-
- ``` sql
- postgres=# GRANT SELECT ON <table-created-WITH-OIDS> TO <role>
- ```
-
-### <a id="topic_j1l_y55_c5"></a>Usage
-
-To query a Hive table with HCatalog integration, query HCatalog directly from HAWQ. The query syntax is:
-
-``` sql
-postgres=# SELECT * FROM hcatalog.hive-db-name.hive-table-name;
-```
-
-For example:
-
-``` sql
-postgres=# SELECT * FROM hcatalog.default.sales_info;
-```
-
-To obtain a description of a Hive table with HCatalog integration, you can use the `psql` client interface.
-
-- Within HAWQ, use either the `\d hcatalog.hive-db-name.hive-table-name` or `\d+ hcatalog.hive-db-name.hive-table-name` commands to describe a single table. `\d` displays only HAWQ's interpretation of the underlying source (Hive in this case) data type, while `\d+` displays both the HAWQ interpreted and Hive source data types. For example, from the `psql` client interface:
-
- ``` shell
- $ psql -d postgres
- ```
-
- ``` sql
- postgres=# \d+ hcatalog.default.sales_info_rcfile;
- ```
-
- ``` shell
- PXF Hive Table "default.sales_info_rcfile"
- Column | Type | Source type
- ------------------+--------+-------------
- location | text | string
- month | text | string
- number_of_orders | int4 | int
- total_sales | float8 | double
- ```
-- Use `\d hcatalog.hive-db-name.*` to describe the whole database schema, i.e. all tables in `hive-db-name`.
-- Use `\d hcatalog.*.*` to describe the whole schema, i.e. all databases and tables.
-
-When using `\d` or `\d+` commands in the `psql` HAWQ client, `hcatalog` will not be listed as a database. If you use other `psql` compatible clients, `hcatalog` will be listed as a database with a size value of `-1` since `hcatalog` is not a real database in HAWQ.
-
-Alternatively, you can use the `pxf_get_item_fields` user-defined function (UDF) to obtain Hive table descriptions from other client interfaces or third-party applications. The UDF takes a PXF profile and a table pattern string as its input parameters. **Note:** The only supported input profile at this time is `'Hive'`.
-
-- The following statement returns a description of a specific table. The description includes path, itemname (table), fieldname, and fieldtype.
-
- ``` sql
- postgres=# SELECT * FROM pxf_get_item_fields('Hive','default.sales_info_rcfile');
- ```
-
- ``` pre
- path | itemname | fieldname | fieldtype
- ---------+-------------------+------------------+-----------
- default | sales_info_rcfile | location | text
- default | sales_info_rcfile | month | text
- default | sales_info_rcfile | number_of_orders | int4
- default | sales_info_rcfile | total_sales | float8
- ```
-
-- The following statement returns table descriptions from the default database.
-
- ``` sql
- postgres=# SELECT * FROM pxf_get_item_fields('Hive','default.*');
- ```
-
-- The following statement returns a description of the entire schema.
-
- ``` sql
- postgres=# SELECT * FROM pxf_get_item_fields('Hive', '*.*');
- ```
-
-### <a id="topic_r5k_pst_25"></a>Limitations
-
-HCatalog integration has the following limitations:
-
-- HCatalog integration queries and describe commands do not support complex types; only primitive types are supported. Use PXF external tables to query complex types in Hive. (See [Complex Types Example](#complex_dt_example).)
-- Even for primitive types, HCatalog metadata descriptions produced by `\d` are HAWQ's interpretation of the underlying Hive data types. For example, the Hive type `tinyint` is converted to HAWQ type `int2`. (See [Data Type Mapping](#hive_primdatatypes).)
-- HAWQ reserves the database name `hcatalog` for system use. You cannot connect to or alter the system `hcatalog` database.
+ `intarray` and `propmap` are each serialized as text strings.
## <a id="partitionfiltering"></a>Partition Filtering
@@ -710,7 +709,7 @@ The insertion of row 4 creates a Hive default partition, because the partition c
In Hive, any query that filters on the partition column omits data in the default partition. For example, the following query returns no rows:
``` sql
-hive> SELECT * FROM sales WHERE xdate is null;
+hive> SELECT * FROM sales WHERE xdate IS null;
```
However, if you map this table as a PXF external table in HAWQ, all default partition values are translated into actual NULL values. In HAWQ, executing the same query against the PXF table returns row 4 as the result, because the filter matches the NULL value.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/dcfe1a47/markdown/pxf/PXFExternalTableandAPIReference.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/pxf/PXFExternalTableandAPIReference.html.md.erb b/markdown/pxf/PXFExternalTableandAPIReference.html.md.erb
index c947775..8a29d1d 100644
--- a/markdown/pxf/PXFExternalTableandAPIReference.html.md.erb
+++ b/markdown/pxf/PXFExternalTableandAPIReference.html.md.erb
@@ -23,56 +23,73 @@ under the License.
You can use the PXF API to create�your own connectors to access any other type of parallel data store or processing engine.
-The PXF Java API lets you extend PXF functionality and add new services and formats without changing HAWQ. The API includes three classes that are extended to allow HAWQ to access an external data source: Fragmenter, Accessor, and Resolver.
+The PXF Java API lets you extend PXF functionality and add new services and formats without changing HAWQ. The API includes three classes that are extended to allow HAWQ to access an external data source: `Fragmenter`, `Accessor`, and `Resolver`.
-The Fragmenter produces a list of data fragments that can be read in parallel from the data source. The Accessor produces a list of records from a single fragment, and the Resolver both deserializes and serializes records.
+The `Fragmenter` produces a list of data fragments that can be read in parallel from the data source. The `Accessor` produces a list of records from a single fragment, and the `Resolver` both deserializes and serializes records.
-Together, the Fragmenter, Accessor, and Resolver classes implement a connector. PXF includes plug-ins for tables in HDFS, HBase, and Hive.
+Together, the `Fragmenter`, `Accessor`, and `Resolver` classes implement a connector. PXF includes plug-ins for HDFS and JSON files and tables in HBase and Hive.
## <a id="creatinganexternaltable"></a>Creating an External Table
-The syntax for a readable `EXTERNAL TABLE` that uses the PXF protocol is as follows:
+The syntax for an `EXTERNAL TABLE` that uses the PXF protocol is as follows:
``` sql
-CREATE [READABLE|WRITABLE] EXTERNAL TABLE table_name
- ( column_name data_type [, ...] | LIKE other_table )
-LOCATION('pxf://host[:port]/path-to-data<pxf parameters>[&custom-option=value...]')
+CREATE [READABLE|WRITABLE] EXTERNAL TABLE <table_name>
+ ( <column_name> <data_type> [, ...] | LIKE <other_table> )
+LOCATION('pxf://<host>[:<port>]/<path-to-data>?<pxf-parameters>[&<custom-option>=<value>[...]]')
FORMAT 'custom' (formatter='pxfwritable_import|pxfwritable_export');
```
-�where *<pxf parameters>* is:
+�where \<pxf\-parameters\> is:
``` pre
- ?FRAGMENTER=fragmenter_class&ACCESSOR=accessor_class&RESOLVER=resolver_class]
- | ?PROFILE=profile-name
+ [FRAGMENTER=<fragmenter_class>&ACCESSOR=<accessor_class>
+ &RESOLVER=<resolver_class>] | ?PROFILE=profile-name
```
+
<caption><span class="tablecap">Table 1. Parameter values and description</span></caption>
<a id="creatinganexternaltable__table_pfy_htz_4p"></a>
| Parameter | Value and description |
|-------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| host | The current host of the PXF service. |
-| port� | Connection port for the PXF service. If the port is omitted, PXF assumes that High Availability (HA) is enabled and connects to the HA name service port, 51200 by default. The HA name service port can be changed by setting the `pxf_service_port` configuration parameter. |
-| *path\_to\_data* | A directory, file name, wildcard pattern, table name, etc. |
-| FRAGMENTER | The plug-in (Java class) to use for fragmenting data. Used for READABLE external tables only. |
-| ACCESSOR | The plug-in (Java class) to use for accessing the data. Used for READABLE and WRITABLE tables. |
-| RESOLVER | The�plug-in (Java class) to use for serializing and deserializing the data. Used for READABLE and WRITABLE tables. |
-| *custom-option*=*value* | Additional values to pass to the plug-in class. The parameters are passed at runtime to the plug-ins indicated above. The plug-ins can lookup custom options with `org.apache.hawq.pxf.api.utilities.InputData`.� |
+| host | The HDFS NameNode. |
+| port� | Connection port for the PXF service. If the port is omitted, PXF assumes that High Availability (HA) is enabled and connects to the HA name service port, 51200, by default. The HA name service port can be changed by setting the `pxf_service_port` configuration parameter. |
+| \<path\-to\-data\> | A directory, file name, wildcard pattern, table name, etc. |
+| PROFILE | The profile PXF uses to access the data. PXF supports multiple plug-ins that currently expose profiles named `HBase`, `Hive`, `HiveRC`, `HiveText`, `HiveORC`, `HdfsTextSimple`, `HdfsTextMulti`, `Avro`, `SequenceWritable`, and `Json`. |
+| FRAGMENTER | The Java class the plug-in uses for fragmenting data. Used for READABLE external tables only. |
+| ACCESSOR | The Java class the plug-in uses for accessing the data. Used for READABLE and WRITABLE tables. |
+| RESOLVER | The�Java class the plug-in uses for serializing and deserializing the data. Used for READABLE and WRITABLE tables. |
+| \<custom-option\> | Additional values to pass to the plug-in at runtime. A plug-in can parse custom options with the PXF helper class `org.apache.hawq.pxf.api.utilities.InputData`.� |
**Note:** When creating PXF external tables, you cannot use the `HEADER` option in your `FORMAT` specification.
-For more information about this example, see [About the Java Class Services and Formats](#aboutthejavaclassservicesandformats).
## <a id="aboutthejavaclassservicesandformats"></a>About the Java Class Services and Formats
-The `LOCATION` string in a PXF `CREATE EXTERNAL TABLE` statement is a URI that specifies the host and port of an external data source and the path to the data in the external data source. The query portion of the URI, introduced by the question mark (?), must include the required parameters `FRAGMENTER` (readable tables only), `ACCESSOR`, and `RESOLVER`, which specify Java class names that extend the base PXF API plug-in classes. Alternatively, the required parameters can be replaced with a `PROFILE` parameter with the name of a profile defined in the `/etc/conf/pxf-profiles.xml` that defines the required classes.
+The `LOCATION` string in a PXF `CREATE EXTERNAL TABLE` statement is a URI that specifies the host and port of an external data source and the path to the data in the external data source. The query portion of the URI, introduced by the question mark (?), must include the PXF profile name or the plug-in's `FRAGMENTER` (readable tables only), `ACCESSOR`, and `RESOLVER` class names.
+
+PXF profiles are defined in the `/etc/pxf/conf/pxf-profiles.xml` file. Profile definitions include plug-in class names. For example, the `HdfsTextSimple` profile definition is:
+
+``` xml
+<profile>
+ <name>HdfsTextSimple</name>
+ <description> This profile is suitable for use when reading delimited
+ single line records from plain text files on HDFS.
+ </description>
+ <plugins>
+ <fragmenter>org.apache.hawq.pxf.plugins.hdfs.HdfsDataFragmenter</fragmenter>
+ <accessor>org.apache.hawq.pxf.plugins.hdfs.LineBreakAccessor</accessor>
+ <resolver>org.apache.hawq.pxf.plugins.hdfs.StringPassResolver</resolver>
+ </plugins>
+</profile>
+```
-The parameters in the PXF URI are passed from HAWQ as headers to the PXF Java service. You can pass custom information to user-implemented PXF plug-ins by adding optional parameters to the LOCATION string.
+The parameters in the PXF URI are passed from HAWQ as headers to the PXF Java service. You can pass custom information to user-implemented PXF plug-ins by adding optional parameters to the `LOCATION` string.
The Java PXF service�retrieves the source data from the external data source and converts it to a HAWQ-readable table format.
-The Accessor, Resolver, and Fragmenter Java classes extend the `org.apache.hawq.pxf.api.utilities.Plugin` class:
+The `Accessor`, `Resolver`, and `Fragmenter` Java classes extend the `org.apache.hawq.pxf.api.utilities.Plugin` class:
``` java
package org.apache.hawq.pxf.api.utilities;
@@ -101,7 +118,7 @@ public class Plugin {
}
```
-The parameters in the `LOCATION` string are available to the plug-ins through methods in the `org.apache.hawq.pxf.api.utilities.InputData` class. Custom parameters added to the location string can be looked up with the `getUserProperty()` method.
+The parameters in the `LOCATION` string are available to the plug-ins through methods in the `org.apache.hawq.pxf.api.utilities.InputData` class. Plug-ins can look up the custom parameters added to the location string with the `getUserProperty()` method.
``` java
/**
@@ -232,25 +249,32 @@ public class InputData {
### <a id="fragmenter"></a>Fragmenter
-**Note:** The Fragmenter Plugin reads data into HAWQ readable external tables. The Fragmenter Plugin cannot write data out of HAWQ into writable external tables.
+**Note:** You use the `Fragmenter` class to read data into HAWQ. You cannot use this class to write data out of HAWQ.
-The Fragmenter is responsible for passing datasource metadata back to HAWQ. It also returns a list of data fragments to the Accessor or Resolver. Each data fragment describes some part of the requested data set. It contains the datasource name, such as the file or table name, including the hostname where it is located. For example, if the source is a HDFS file, the Fragmenter returns a list of data fragments containing a HDFS file block.�Each fragment includes the location of the block. If the source data is an HBase table, the Fragmenter returns information about table regions, including their locations.
+The `Fragmenter` is responsible for passing datasource metadata back to HAWQ. It also returns a list of data fragments to the `Accessor` or `Resolver`. Each data fragment describes some part of the requested data set. It contains the datasource name, such as the file or table name, including the hostname where it is located. For example, if the source is an HDFS file, the `Fragmenter` returns a list of data fragments containing an HDFS file block.�Each fragment includes the location of the block. If the source data is an HBase table, the `Fragmenter` returns information about table regions, including their locations.
The `ANALYZE` command now retrieves advanced statistics for PXF readable tables by estimating the number of tuples in a table, creating a sample table from the external table, and running advanced statistics queries on the sample table in the same way statistics are collected for native HAWQ tables.
The configuration parameter `pxf_enable_stat_collection` controls collection of advanced statistics. If `pxf_enable_stat_collection` is set to false, no analysis is performed on PXF tables. An additional parameter, `pxf_stat_max_fragments`, controls the number of fragments sampled to build a sample table. By default `pxf_stat_max_fragments` is set to 100, which means that even if there are more than 100 fragments, only this number of fragments will be used in `ANALYZE` to sample the data. Increasing this number will result in better sampling, but can also impact performance.
-When a PXF table is analyzed and `pxf_enable_stat_collection` is set to off, or an error occurs because the table is not defined correctly, the PXF service is down, or `getFragmentsStats` is not implemented, a warning message is shown and no statistics are gathered for that table. If `ANALYZE` is running over all tables in the database, the next table will be processed \u2013 a failure processing one table does not stop the command.
+When a PXF table is analyzed, any of the following conditions might result in a warning message with no statistics gathered for the table:
+
+- `pxf_enable_stat_collection` is set to off,
+- an error occurs because the table is not defined correctly,
+- the PXF service is down, or
+- `getFragmentsStats()` is not implemented
-For a�detailed explanation�about HAWQ statistical data gathering, see `ANALYZE` in the SQL Commands Reference.
+If `ANALYZE` is running over all tables in the database, the next table will be processed \u2013 a failure processing one table does not stop the command.
+
+For a�detailed explanation�about HAWQ statistical data gathering, refer to the [`ANALYZE`](../reference/sql/ANALYZE.html) SQL command reference.
**Note:**
-- Depending on external table size, the time required to complete an ANALYZE operation can be lengthy. The boolean parameter `pxf_enable_stat_collection` enables statistics collection for PXF. The default value is `on`. Turning this parameter off (disabling PXF statistics collection) can help decrease the time needed for the ANALYZE operation.
-- You can also use *pxf\_stat\_max\_fragments* to limit the number of fragments to be sampled by decreasing it from the default (100). However, if the number is too low, the sample might not be uniform and the statistics might be skewed.
-- You can also implement getFragmentsStats to return an error. This will cause ANALYZE on a table with this Fragmenter to fail immediately, and default statistics values will be used for that table.
+- Depending on external table size, the time required to complete an `ANALYZE` operation can be lengthy. The boolean parameter `pxf_enable_stat_collection` enables statistics collection for PXF. The default value is `on`. Turning this parameter off (disabling PXF statistics collection) can help decrease the time needed for the `ANALYZE` operation.
+- You can also use `pxf_stat_max_fragments` to limit the number of fragments to be sampled by decreasing it from the default (100). However, if the number is too low, the sample might not be uniform and the statistics might be skewed.
+- You can also implement `getFragmentsStats()` to return an error. This will cause `ANALYZE` on a table with this `Fragmenter` to fail immediately, and default statistics values will be used for that table.
-The following table lists the Fragmenter plug-in implementations included with the PXF API.
+The following table lists the `Fragmenter` plug-in implementations included with the PXF API.
<a id="fragmenter__table_cgs_svp_3s"></a>
@@ -262,31 +286,31 @@ The following table lists the Fragmenter plug-in implementations included with t
</colgroup>
<thead>
<tr class="header">
-<th><p><code class="ph codeph">Fragmenter class</code></p></th>
-<th><p><code class="ph codeph">Description</code></p></th>
+<th><p>Fragmenter class</p></th>
+<th><p>Description</p></th>
</tr>
</thead>
<tbody>
<tr class="odd">
<td>org.apache.hawq.pxf.plugins.hdfs.HdfsDataFragmenter</td>
-<td>Fragmenter for Hdfs files</td>
+<td>Fragmenter for HDFS, JSON files</td>
</tr>
<tr class="even">
-<td>org.apache.hawq.pxf.plugins.hbase.HBaseAtomicDataAccessor</td>
+<td>org.apache.hawq.pxf.plugins.hbase.HBaseDataFragmenter</td>
<td>Fragmenter for HBase tables</td>
</tr>
<tr class="odd">
-<td>org.apache.hawq.pxf.plugins.hive.HiveDataFragmenter</td>
+<td>org.apache.hawq.pxf.plugins.hive.HiveDataFragmenter</li>
<td>Fragmenter for Hive tables�</td>
</tr>
<tr class="even">
<td>org.apache.hawq.pxf.plugins.hdfs.HiveInputFormatFragmenter</td>
-<td>Fragmenter for Hive tables with RC or text files�</td>
+<td>Fragmenter for Hive tables with RC, ORC, or text file formats�</td>
</tr>
</tbody>
</table>
-A�Fragmenter class extends `org.apache.hawq.pxf.api.Fragmenter`:
+A�`Fragmenter` class extends `org.apache.hawq.pxf.api.Fragmenter`:
#### <a id="com.pivotal.pxf.api.fragmenter"></a>org.apache.hawq.pxf.api.Fragmenter
@@ -339,7 +363,7 @@ public abstract class Fragmenter extends Plugin {
#### <a id="classdescription"></a>Class Description
-The Fragmenter.getFragments()�method returns a�List<Fragment>;:
+The `Fragmenter.getFragments()`�method returns a�`List<Fragment>`:
``` java
package org.apache.hawq.pxf.api;
@@ -416,7 +440,7 @@ public class FragmentsStats {
### <a id="accessor"></a>Accessor
-The Accessor retrieves specific fragments and passes records back to the Resolver.�For example, the HDFS plug-ins create a `org.apache.hadoop.mapred.FileInputFormat` and a `org.apache.hadoop.mapred.RecordReader` for an HDFS file and sends this to the Resolver.�In the case of HBase or Hive files, the Accessor returns single rows from an HBase or Hive table. PXF 1.x or higher contains the following Accessor implementations:
+The `Accessor` retrieves specific fragments and passes records back to the Resolver.�For example, the HDFS plug-ins create a `org.apache.hadoop.mapred.FileInputFormat` and a `org.apache.hadoop.mapred.RecordReader` for an HDFS file and sends this to the `Resolver`.�In the case of HBase or Hive files, the `Accessor` returns single rows from an HBase or Hive table. PXF includes the following `Accessor` implementations:
<a id="accessor__table_ewm_ttz_4p"></a>
@@ -428,8 +452,8 @@ The Accessor retrieves specific fragments and passes records back to the Resolve
</colgroup>
<thead>
<tr class="header">
-<th><p><code class="ph codeph">Accessor class</code></p></th>
-<th><p><code class="ph codeph">Description</code></p></th>
+<th><p>Accessor class</p></th>
+<th><p>Description</p></th>
</tr>
</thead>
<tbody>
@@ -467,16 +491,26 @@ The Accessor retrieves specific fragments and passes records back to the Resolve
</tr>
<tr class="odd">
<td>org.apache.hawq.pxf.plugins.hive.HiveLineBreakAccessor</td>
-<td>Accessor for Hive tables with text files</td>
+<td>Accessor for Hive tables stored as text file format</td>
</tr>
<tr class="even">
<td>org.apache.hawq.pxf.plugins.hive.HiveRCFileAccessor</td>
-<td>Accessor for Hive tables with RC files</td>
+<td>Accessor for Hive tables stored as RC file format</td>
+</tr>
+</tr>
+<tr class="odd">
+<td>org.apache.hawq.pxf.plugins.hive.HiveORCAccessor</td>
+<td>Accessor for Hive tables stored as ORC format </td>
+</tr>
+</tr>
+<tr class="odd">
+<td>org.apache.hawq.pxf.plugins.json.JsonAccessor</td>
+<td>Accessor for JSON files</td>
</tr>
</tbody>
</table>
-The class must extend the `org.apache.hawq.pxf.Plugin`� class, and�implement one or both interfaces:
+The class must extend the `org.apache.hawq.pxf.Plugin`� class, and�implement one or both of the interfaces:
- `org.apache.hawq.pxf.api.ReadAccessor`
- `org.apache.hawq.pxf.api.WriteAccessor`
@@ -511,12 +545,12 @@ public interface WriteAccessor {
}
```
-The Accessor calls `openForRead()` to read existing data. After reading the data, it calls `closeForRead()`. `readNextObject()` returns one of the�following:
+The `Accessor` calls `openForRead()` to read existing data. After reading the data, it calls `closeForRead()`. `readNextObject()` returns one of the�following:
-- a single record, encapsulated in a OneRow object
+- a single record, encapsulated in a `OneRow` object
- null if it reaches `EOF`
-The Accessor calls `openForWrite()` to write data out. After writing the data, it�writes a `OneRow` object with `writeNextObject()`, and when done calls `closeForWrite()`. `OneRow` represents a key-value item.
+The `Accessor` calls `openForWrite()` to write data out. After writing the data, it�writes a `OneRow` object with `writeNextObject()`, and when done calls `closeForWrite()`. `OneRow` represents a key-value item.
#### <a id="com.pivotal.pxf.api.onerow"></a>org.apache.hawq.pxf.api.OneRow
@@ -568,7 +602,7 @@ public class OneRow {
### <a id="resolver"></a>Resolver
-The Resolver deserializes records in the `OneRow` format and serializes them to a list of `OneField` objects. PXF converts a `OneField` object to a HAWQ-readable�`GPDBWritable` format.�PXF 1.x or higher contains the following implementations:
+The `Resolver` deserializes records in the `OneRow` format and serializes them to a list of `OneField` objects. PXF converts a `OneField` object to a HAWQ-readable�`GPDBWritable` format.�PXF 1.x or higher contains the following implementations:
<a id="resolver__table_nbd_d5z_4p"></a>
@@ -580,18 +614,18 @@ The Resolver deserializes records in the `OneRow` format and serializes them to
</colgroup>
<thead>
<tr class="header">
-<th><p><code class="ph codeph">Resolver class</code></p></th>
-<th><p><code class="ph codeph">Description</code></p></th>
+<th><p>Resolver class</p></th>
+<th><p>Description</p></th>
</tr>
</thead>
<tbody>
<tr class="odd">
-<td><p><code class="ph codeph">org.apache.hawq.pxf.plugins.hdfs.StringPassResolver</code></p></td>
+<td><p>org.apache.hawq.pxf.plugins.hdfs.StringPassResolver</p></td>
<td><p><code class="ph codeph">StringPassResolver</code> replaced the deprecated <code class="ph codeph">TextResolver</code>. It passes whole records (composed of any data types) as strings without parsing them</p></td>
</tr>
<tr class="even">
-<td><p><code class="ph codeph">org.apache.hawq.pxf.plugins.hdfs.WritableResolver</code></p></td>
-<td><p>Resolver for custom Hadoop Writable implementations. Custom class can be specified with the schema in DATA-SCHEMA. Supports the following types:</p>
+<td><p>org.apache.hawq.pxf.plugins.hdfs.WritableResolver</p></td>
+<td><p>Resolver for custom Hadoop Writable implementations. Custom class can be specified with the schema in `DATA-SCHEMA`. Supports the following types:</p>
<pre class="pre codeblock"><code>DataType.BOOLEAN
DataType.INTEGER
DataType.BIGINT
@@ -601,11 +635,11 @@ DataType.VARCHAR
DataType.BYTEA</code></pre></td>
</tr>
<tr class="odd">
-<td><p><code class="ph codeph">org.apache.hawq.pxf.plugins.hdfs.AvroResolver</code></p></td>
+<td><p>org.apache.hawq.pxf.plugins.hdfs.AvroResolver</p></td>
<td><p>Supports the same field objects as <code class="ph codeph">WritableResolver</code>.�</p></td>
</tr>
<tr class="even">
-<td><p><code class="ph codeph">org.apache.hawq.pxf.plugins.hbase.HBaseResolver</code></p></td>
+<td><p>org.apache.hawq.pxf.plugins.hbase.HBaseResolver</p></td>
<td><p>Supports the same field objects as <code class="ph codeph">WritableResolver</code> and also supports the following:</p>
<pre class="pre codeblock"><code>DataType.SMALLINT
DataType.NUMERIC
@@ -614,20 +648,24 @@ DataType.BPCHAR
DataType.TIMESTAMP</code></pre></td>
</tr>
<tr class="odd">
-<td><p><code class="ph codeph">org.apache.hawq.pxf.plugins.hive.HiveResolver</code></p></td>
+<td><p>org.apache.hawq.pxf.plugins.hive.HiveResolver</p></td>
<td><p>Supports the same field objects as <code class="ph codeph">WritableResolver</code> and also supports the following:</p>
<pre class="pre codeblock"><code>DataType.SMALLINT
DataType.TEXT
DataType.TIMESTAMP</code></pre></td>
</tr>
<tr class="even">
-<td><p><code class="ph codeph">org.apache.hawq.pxf.plugins.hive.HiveStringPassResolver</code></p></td>
+<td><p>org.apache.hawq.pxf.plugins.hive.HiveStringPassResolver</p></td>
<td>Specialized <code class="ph codeph">HiveResolver</code> for a Hive table stored as Text files. Should be used together with <code class="ph codeph">HiveInputFormatFragmenter</code>/<code class="ph codeph">HiveLineBreakAccessor</code>.</td>
</tr>
<tr class="odd">
-<td><code class="ph codeph">org.apache.hawq.pxf.plugins.hive.HiveColumnarSerdeResolver</code></td>
+<td>org.apache.hawq.pxf.plugins.hive.HiveColumnarSerdeResolver</td>
<td>Specialized <code class="ph codeph">HiveResolver</code> for a Hive table stored as RC file. Should be used together with <code class="ph codeph">HiveInputFormatFragmenter</code>/<code class="ph codeph">HiveRCFileAccessor</code>.</td>
</tr>
+<tr class="odd">
+<td>org.apache.hawq.pxf.plugins.hive.HiveORCSerdeResolver</td>
+<td>Specialized <code class="ph codeph">HiveResolver</code> for a Hive table stored in ORC format. Should be used together with <code class="ph codeph">HiveInputFormatFragmenter</code>/<code class="ph codeph">HiveORCAccessor</code>.</td>
+</tr>
</tbody>
</table>
@@ -663,8 +701,8 @@ public interface WriteResolver {
**Note:**
-- getFields should return a List<OneField>, each OneField representing a single field.
-- `setFields�`should return a single�`OneRow�`object, given a List<OneField>.
+- `getFields()` should return a `List<OneField>`, with each `OneField` representing a single field.
+- `setFields()` should return a single�`OneRow` object, given a `List<OneField>`.
#### <a id="com.pivotal.pxf.api.onefield"></a>org.apache.hawq.pxf.api.OneField
@@ -687,7 +725,7 @@ public class OneField {
}
```
-The value of `type` should follow the org.apache.hawq.pxf.api.io.DataType�`enums`. `val` is the appropriate Java class. Supported types are as follows:
+The value of `type` should follow the `org.apache.hawq.pxf.api.io.DataType`�`enums`. `val` is the appropriate Java class. Supported types are:
<a id="com.pivotal.pxf.api.onefield__table_f4x_35z_4p"></a>
@@ -759,19 +797,13 @@ The value of `type` should follow the org.apache.hawq.pxf.api.io.DataType�`enum
</tbody>
</table>
-### <a id="analyzer"></a>Analyzer
-
-The Analyzer has been deprecated. A new function in the Fragmenter API (Fragmenter.getFragmentsStats) is used to gather initial statistics for the data source, and provides PXF statistical data for the HAWQ query optimizer. For a�detailed explanation�about HAWQ statistical data gathering, see `ANALYZE` in the SQL Command Reference.
-
-Using the Analyzer API will result in an error message. Use the Fragmenter and getFragmentsStats to gather advanced statistics.
-
## <a id="aboutcustomprofiles"></a>About Custom Profiles
-Administrators can add new profiles or edit the built-in profiles in�`/etc/conf/pxf-profiles.xml` file. See [Using Profiles to Read and Write Data](ReadWritePXF.html#readingandwritingdatawithpxf) for information on how to add custom profiles.
+Administrators can add new profiles or edit the built-in profiles in�`/etc/pxf/conf/pxf-profiles.xml`. See [Using Profiles to Read and Write Data](ReadWritePXF.html#readingandwritingdatawithpxf) for information on how to add custom profiles.
## <a id="aboutqueryfilterpush-down"></a>About Query Filter Push-Down
-If a query includes a number of WHERE clause filters, �HAWQ may push all or some queries to PXF. If pushed to PXF, the Accessor can use the filtering information when accessing the data source to fetch tuples. These filters�only return records that pass filter evaluation conditions.�This reduces data processing�and�reduces network traffic from the SQL engine.
+If a query includes a number of `WHERE` clause filters, �HAWQ may push all or some queries to PXF. If pushed to PXF, the `Accessor` can use the filtering information when accessing the data source to fetch tuples. These filters�only return records that pass filter evaluation conditions.�This reduces data processing�and�reduces network traffic from the SQL engine.
This topic includes the following information:
@@ -788,17 +820,17 @@ PXF�allows push-down filtering if the following rules are met:
- Uses only�single expressions or a group of AND'ed expressions - no OR'ed expressions.
- Uses only expressions of supported data types and operators.
-FilterParser�scans the pushed down filter list and uses the user's build() implementation to build the filter.
+`FilterParser`�scans the pushed down filter list and uses the user's `build()` implementation to build the filter.
-- For simple expressions (e.g, a >= 5), FilterParser places column objects on the left of the�expression and constants on the right.
-- For compound expressions (e.g <expression> AND <expression>) it handles three cases in the build() function:
+- For simple expressions (e.g, a >= 5), `FilterParser` places column objects on the left of the�expression and constants on the right.
+- For compound expressions (e.g <expression> AND <expression>) it handles three cases in the `build()` function:
1. Simple Expression: <Column Index> <Operation> <Constant>
2. Compound Expression: <Filter Object> AND <Filter Object>
3. Compound Expression: <List of Filter Objects> AND <Filter Object>
### <a id="creatingafilterbuilderclass"></a>Creating a Filter Builder Class
-To check�if a filter queried PXF, call the `InputData hasFilter()` function:
+To check�if a filter queried PXF, call the `InputData.hasFilter()` function:
``` java
/*
@@ -810,7 +842,7 @@ public boolean hasFilter()
}
```
-If `hasFilter()` returns `false`, there is no filter information. If it returns `true`,�PXF parses the serialized filter string into a meaningful filter object to use later. To do so, create a filter builder�class that implements the�`FilterParser.FilterBuilder�` interface:
+If `hasFilter()` returns `false`, there is no filter information. If it returns `true`,�PXF parses the serialized filter string into a meaningful filter object to use later. To do so, create a filter builder�class that implements the�`FilterParser.FilterBuilder` interface:
``` java
package org.apache.hawq.pxf.api;
@@ -827,7 +859,7 @@ interface FilterBuilder {
}
```
-While PXF parses the serialized filter string from the incoming HAWQ query, it calls the `build() interface` function. PXF�calls this function for each condition or filter pushed down to PXF. Implementing this function returns some Filter object or representation that the Fragmenter, Accessor, or Resolver uses in runtime to filter out records. The `build()` function accepts an Operation as input, and�left and right operands.
+While PXF parses the serialized filter string from the incoming HAWQ query, it calls the `build()` function. PXF�calls this function for each condition or filter pushed down to PXF. Implementing this function returns some Filter object or representation that the `Fragmenter`, `Accessor`, or `Resolver` uses in runtime to filter out records. The `build()` function accepts an Operation as input, and�left and right operands.
### <a id="filteroperations"></a>Filter Operations
@@ -843,8 +875,20 @@ public enum Operation
HDOP_GE, //greater than or equal
HDOP_EQ, //equal
HDOP_NE, //not equal
- HDOP_AND //AND'ed conditions
+ HDOP_LIKE,
+ HDOP_IS_NULL,
+ HDOP_IS_NOT_NULL,
+ HDOP_IN
};
+
+/**
+ * Logical operators
+ */
+public enum LogicalOperation {
+ HDOP_AND,
+ HDOP_OR,
+ HDOP_NOT
+}
```
#### <a id="filteroperands"></a>Filter Operands
@@ -885,7 +929,7 @@ public class Constant
#### <a id="filterobject"></a>Filter Object
-Filter Objects can be internal, such as those you define; or external, those that the remote system uses. For example, for HBase, you define the HBase�`Filter` class (`org.apache.hadoop.hbase.filter.Filter`), while�for Hive, you use an internal default representation created by the PXF framework, called�`BasicFilter`. You can decide�the filter object to use, including writing a new one. `BasicFilter` is the most common:
+Filter Objects can be internal - such as those you define - or external, those that the remote system uses. For example, for HBase you define the HBase�`Filter` class (`org.apache.hadoop.hbase.filter.Filter`), while�for Hive you use an internal default representation created by the PXF framework, called�`BasicFilter`. You can choose�the filter object to use, including writing a new one. `BasicFilter` is the most common:
``` java
/*
@@ -919,7 +963,7 @@ static public class BasicFilter
### <a id="sampleimplementation"></a>Sample Implementation
-Let's look at the following sample implementation of the filter builder class and its `build()` function that handles all 3 cases. Let's assume that BasicFilter was used to hold our filter operations.
+Let's look at the following sample implementation of the filter builder class and its `build()` function that handles all 3 cases. Let's assume that `BasicFilter` was used to hold our filter operations.
``` java
import java.util.LinkedList;
@@ -1003,7 +1047,7 @@ public class MyDemoFilterBuilder implements FilterParser.FilterBuilder
}
```
-Here is an example of creating a filter-builder class to implement the Filter interface, implement the `build()` function, and generate�the Filter object. To do this, use either the Accessor, Resolver, or both to�call the `getFilterObject` function:
+Here is an example of creating a filter-builder class to implement the Filter interface, implement the `build()` function, and generate�the Filter object. To do this, use either the `Accessor`, `Resolver`, or both to�call the `getFilterObject()` function:
``` java
if (inputData.hasFilter())
@@ -1034,7 +1078,7 @@ else
}
```
-Example of evaluating a single filter:
+Example showing evaluation of a single filter:
``` java
//Get our BasicFilter Object
@@ -1134,7 +1178,7 @@ FORMAT 'custom' (formatter='pxfwritable_export');
### <a id="pluginexamples"></a>Plug-in Examples
-This section contains sample dummy implementations of all three plug-ins. It also contains a usage example.
+This section contains sample dummy implementations of all three plug-ins. It also includes a usage example.
#### <a id="dummyfragmenter"></a>Dummy Fragmenter
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/dcfe1a47/markdown/pxf/ReadWritePXF.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/pxf/ReadWritePXF.html.md.erb b/markdown/pxf/ReadWritePXF.html.md.erb
index f424afd..5c29ae8 100644
--- a/markdown/pxf/ReadWritePXF.html.md.erb
+++ b/markdown/pxf/ReadWritePXF.html.md.erb
@@ -34,7 +34,7 @@ PXF comes with a number of built-in profiles that group together�a collection o
- HBase (Read only)
- JSON (Read only)
-You can specify a built-in profile when you want to read data that exists inside HDFS files, Hive tables, HBase tables, and JSON files and for writing data into HDFS files.
+You can specify a built-in profile when you want to read data that exists inside HDFS files, Hive tables, HBase tables, or JSON files, and when you want to write data into HDFS files.
<table>
<colgroup>
@@ -46,7 +46,7 @@ You can specify a built-in profile when you want to read data that exists inside
<tr class="header">
<th>Profile</th>
<th>Description</th>
-<th>Fragmenter/Accessor/Resolver</th>
+<th>Fragmenter/Accessor/Resolver/Metadata/OutputFormat</th>
</tr>
</thead>
<tbody>
@@ -75,6 +75,8 @@ You can specify a built-in profile when you want to read data that exists inside
<li>org.apache.hawq.pxf.plugins.hive.HiveDataFragmenter</li>
<li>org.apache.hawq.pxf.plugins.hive.HiveAccessor</li>
<li>org.apache.hawq.pxf.plugins.hive.HiveResolver</li>
+<li>org.apache.hawq.pxf.plugins.hive.HiveMetadataFetcher</li>
+<li>org.apache.hawq.pxf.service.io.GPDBWritable</li>
</ul></td>
</tr>
<tr class="even">
@@ -87,6 +89,20 @@ Note: The <code class="ph codeph">DELIMITER</code> parameter is mandatory.
<li>org.apache.hawq.pxf.plugins.hive.HiveInputFormatFragmenter</li>
<li>org.apache.hawq.pxf.plugins.hive.HiveRCFileAccessor</li>
<li>org.apache.hawq.pxf.plugins.hive.HiveColumnarSerdeResolver</li>
+<li>org.apache.hawq.pxf.plugins.hive.HiveMetadataFetcher</li>
+<li>org.apache.hawq.pxf.service.io.Text</li>
+</ul></td>
+</tr>
+<tr class="odd">
+<td>HiveORC</td>
+<td>Optimized read of a Hive table where each partition is stored as an ORC file.
+</td>
+<td><ul>
+<li>org.apache.hawq.pxf.plugins.hive.HiveInputFormatFragmenter</li>
+<li>org.apache.hawq.pxf.plugins.hive.HiveORCAccessor</li>
+<li>org.apache.hawq.pxf.plugins.hive.HiveORCSerdeResolver</li>
+<li>org.apache.hawq.pxf.plugins.hive.HiveMetadataFetcher</li>
+<li>org.apache.hawq.pxf.service.io.GPDBWritable</li>
</ul></td>
</tr>
<tr class="odd">
@@ -99,6 +115,8 @@ Note: The <code class="ph codeph">DELIMITER</code> parameter is mandatory.
<li>org.apache.hawq.pxf.plugins.hive.HiveInputFormatFragmenter</li>
<li>org.apache.hawq.pxf.plugins.hive.HiveLineBreakAccessor</li>
<li>org.apache.hawq.pxf.plugins.hive.HiveStringPassResolver</li>
+<li>org.apache.hawq.pxf.plugins.hive.HiveMetadataFetcher</li>
+<li>org.apache.hawq.pxf.service.io.Text</li>
</ul></td>
</tr>
<tr class="even">
@@ -131,6 +149,8 @@ Note: The <code class="ph codeph">DELIMITER</code> parameter is mandatory.
</tbody>
</table>
+**Notes**: Metadata identifies the Java class that provides field definitions in the relation. OutputFormat identifies the output serialization format (text or binary) for which a specific profile is optimized. While the built-in `Hive*` profiles provide Metadata and OutputFormat classes, other profiles may have no need to implement or specify these classes.
+
## <a id="addingandupdatingprofiles"></a>Adding and Updating Profiles
Each profile has a mandatory unique�name and an optional�description. In addition, each profile contains a set of plug-ins�that�are an�extensible set of metadata attributes. Administrators can add new profiles or edit the built-in profiles defined in�`/etc/pxf/conf/pxf-profiles.xml`.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/dcfe1a47/markdown/pxf/TroubleshootingPXF.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/pxf/TroubleshootingPXF.html.md.erb b/markdown/pxf/TroubleshootingPXF.html.md.erb
index c1bd43a..57fe9d5 100644
--- a/markdown/pxf/TroubleshootingPXF.html.md.erb
+++ b/markdown/pxf/TroubleshootingPXF.html.md.erb
@@ -195,6 +195,8 @@ Examine/collect the log messages from `pxf-service.log`.
### <a id="pxfdblogmsg"></a>Database-Level Logging
+Database-level logging may provide insight into internal PXF service operations. Additionally, when you access Hive tables using `hcatalog` or the `Hive*` profiles, log messages identify the underlying `Hive*` profile(s) employed to access the data.
+
Enable HAWQ and PXF debug message logging during operations on PXF external tables by setting the `client_min_messages` server configuration parameter to `DEBUG2` in your `psql` session.
``` shell
@@ -211,6 +213,8 @@ DEBUG2: churl http header: cell #21: X-GP-DATA-DIR: pxf_hive1
DEBUG2: churl http header: cell #22: X-GP-profile: Hive
DEBUG2: churl http header: cell #23: X-GP-URI: pxf://namenode:51200/pxf_hive1?profile=Hive
...
+DEBUG2: pxf: set_current_fragment_headers: using profile: Hive
+...
```
Examine/collect the log messages from `stdout`.
[07/18] incubator-hawq-docs git commit: HAWQ-1305 Add apache license
header to doc source
Posted by yo...@apache.org.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/datamgmt/load/g-troubleshooting-gpfdist.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/datamgmt/load/g-troubleshooting-gpfdist.html.md.erb b/markdown/datamgmt/load/g-troubleshooting-gpfdist.html.md.erb
index 2e6a450..1b88477 100644
--- a/markdown/datamgmt/load/g-troubleshooting-gpfdist.html.md.erb
+++ b/markdown/datamgmt/load/g-troubleshooting-gpfdist.html.md.erb
@@ -2,6 +2,25 @@
title: Troubleshooting gpfdist
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The segments access `gpfdist` at runtime. Ensure that the HAWQ segment hosts have network access to `gpfdist`. `gpfdist` is a web server: test connectivity by running the following command from each host in the HAWQ array (segments and master):
``` shell
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/datamgmt/load/g-unloading-data-from-hawq-database.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/datamgmt/load/g-unloading-data-from-hawq-database.html.md.erb b/markdown/datamgmt/load/g-unloading-data-from-hawq-database.html.md.erb
index e0690ad..5576750 100644
--- a/markdown/datamgmt/load/g-unloading-data-from-hawq-database.html.md.erb
+++ b/markdown/datamgmt/load/g-unloading-data-from-hawq-database.html.md.erb
@@ -2,6 +2,25 @@
title: Unloading Data from HAWQ
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
A writable external table allows you to select rows from other database tables and output the rows to files, named pipes, to applications, or as output targets for parallel MapReduce calculations. You can define file-based and web-based writable external tables.
This topic describes how to unload data from HAWQ using parallel unload (writable external tables) and non-parallel unload (`COPY`).
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/datamgmt/load/g-unloading-data-using-a-writable-external-table.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/datamgmt/load/g-unloading-data-using-a-writable-external-table.html.md.erb b/markdown/datamgmt/load/g-unloading-data-using-a-writable-external-table.html.md.erb
index 377f2d6..a20c9ff 100644
--- a/markdown/datamgmt/load/g-unloading-data-using-a-writable-external-table.html.md.erb
+++ b/markdown/datamgmt/load/g-unloading-data-using-a-writable-external-table.html.md.erb
@@ -2,6 +2,25 @@
title: Unloading Data Using a Writable External Table
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Writable external tables allow only `INSERT` operations. You must grant `INSERT` permission on a table to enable access to users who are not the table owner or a superuser. For example:
``` sql
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/datamgmt/load/g-unloading-data-using-copy.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/datamgmt/load/g-unloading-data-using-copy.html.md.erb b/markdown/datamgmt/load/g-unloading-data-using-copy.html.md.erb
index 816a2b5..7f30941 100644
--- a/markdown/datamgmt/load/g-unloading-data-using-copy.html.md.erb
+++ b/markdown/datamgmt/load/g-unloading-data-using-copy.html.md.erb
@@ -2,6 +2,25 @@
title: Unloading Data Using COPY
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
`COPY TO` copies data from a table to a file (or standard input) on the HAWQ master host using a single process on the HAWQ master instance. Use `COPY` to output a table's entire contents, or filter the output using a `SELECT` statement. For example:
``` sql
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/datamgmt/load/g-url-based-web-external-tables.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/datamgmt/load/g-url-based-web-external-tables.html.md.erb b/markdown/datamgmt/load/g-url-based-web-external-tables.html.md.erb
index a115972..7df7fc7 100644
--- a/markdown/datamgmt/load/g-url-based-web-external-tables.html.md.erb
+++ b/markdown/datamgmt/load/g-url-based-web-external-tables.html.md.erb
@@ -2,6 +2,25 @@
title: URL-based Web External Tables
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
A URL-based web table accesses data from a web server using the HTTP protocol. Web table data is dynamic; the data is not rescannable.
Specify the `LOCATION` of files on a web server using `http://`. The web data file(s) must reside on a web server that HAWQ segment hosts can access. The number of URLs specified corresponds to the minimum number of virtual segments that work in parallel to access the web table.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/datamgmt/load/g-using-a-custom-format.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/datamgmt/load/g-using-a-custom-format.html.md.erb b/markdown/datamgmt/load/g-using-a-custom-format.html.md.erb
index e83744a..8abe90d 100644
--- a/markdown/datamgmt/load/g-using-a-custom-format.html.md.erb
+++ b/markdown/datamgmt/load/g-using-a-custom-format.html.md.erb
@@ -2,6 +2,25 @@
title: Using a Custom Format
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
You specify a custom data format in the `FORMAT` clause of `CREATE EXTERNAL TABLE`.
```
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/datamgmt/load/g-using-the-hawq-file-server--gpfdist-.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/datamgmt/load/g-using-the-hawq-file-server--gpfdist-.html.md.erb b/markdown/datamgmt/load/g-using-the-hawq-file-server--gpfdist-.html.md.erb
index 0c68b2c..a0619ff 100644
--- a/markdown/datamgmt/load/g-using-the-hawq-file-server--gpfdist-.html.md.erb
+++ b/markdown/datamgmt/load/g-using-the-hawq-file-server--gpfdist-.html.md.erb
@@ -2,6 +2,25 @@
title: Using the HAWQ File Server (gpfdist)
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The `gpfdist` protocol provides the best performance and is the easiest to set up. `gpfdist` ensures optimum use of all segments in your HAWQ system for external table reads.
This topic describes the setup and management tasks for using `gpfdist` with external tables.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/datamgmt/load/g-working-with-file-based-ext-tables.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/datamgmt/load/g-working-with-file-based-ext-tables.html.md.erb b/markdown/datamgmt/load/g-working-with-file-based-ext-tables.html.md.erb
index e024a7d..ae23711 100644
--- a/markdown/datamgmt/load/g-working-with-file-based-ext-tables.html.md.erb
+++ b/markdown/datamgmt/load/g-working-with-file-based-ext-tables.html.md.erb
@@ -2,6 +2,25 @@
title: Working with File-Based External Tables
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
External tables provide access to data stored in data sources outside of HAWQ as if the data were stored in regular database tables. Data can be read from or written to external tables.
An external table is a HAWQ database table backed with data that resides outside of the database. An external table is either readable or writable. It can be used like a regular database table in SQL commands such as `SELECT` and `INSERT` and joined with other tables. External tables are most often used to load and unload database data.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/datamgmt/load/g-write-a-transform.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/datamgmt/load/g-write-a-transform.html.md.erb b/markdown/datamgmt/load/g-write-a-transform.html.md.erb
index 6b35ab2..5bb11dc 100644
--- a/markdown/datamgmt/load/g-write-a-transform.html.md.erb
+++ b/markdown/datamgmt/load/g-write-a-transform.html.md.erb
@@ -2,6 +2,25 @@
title: Write a Transform
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The transform specifies what to extract from the data.You can use any authoring environment and language appropriate for your project. For XML transformations, choose a technology such as XSLT, Joost (STX), Java, Python, or Perl, based on the goals and scope of the project.
In the price example, the next step is to transform the XML data into a simple two-column delimited format.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/datamgmt/load/g-write-the-gpfdist-configuration.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/datamgmt/load/g-write-the-gpfdist-configuration.html.md.erb b/markdown/datamgmt/load/g-write-the-gpfdist-configuration.html.md.erb
index 89733cd..a6a7e78 100644
--- a/markdown/datamgmt/load/g-write-the-gpfdist-configuration.html.md.erb
+++ b/markdown/datamgmt/load/g-write-the-gpfdist-configuration.html.md.erb
@@ -2,6 +2,25 @@
title: Write the gpfdist Configuration
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The `gpfdist` configuration is specified as a YAML 1.1 document. It specifies rules that `gpfdist` uses to select a Transform to apply when loading or extracting data.
This example `gpfdist` configuration contains the following items:
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/datamgmt/load/g-xml-transformation-examples.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/datamgmt/load/g-xml-transformation-examples.html.md.erb b/markdown/datamgmt/load/g-xml-transformation-examples.html.md.erb
index 12ad1d6..5d83224 100644
--- a/markdown/datamgmt/load/g-xml-transformation-examples.html.md.erb
+++ b/markdown/datamgmt/load/g-xml-transformation-examples.html.md.erb
@@ -2,6 +2,25 @@
title: XML Transformation Examples
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The following examples demonstrate the complete process for different types of XML data and STX transformations. Files and detailed instructions associated with these examples can be downloaded from the Apache site `gpfdist_transform` tools demo page. Read the README file before you run the examples.
- **[Command-based Web External Tables](../../datamgmt/load/g-example-1-dblp-database-publications-in-demo-directory.html)**
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/ddl/ddl-database.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/ddl/ddl-database.html.md.erb b/markdown/ddl/ddl-database.html.md.erb
index d643fae..ba68098 100644
--- a/markdown/ddl/ddl-database.html.md.erb
+++ b/markdown/ddl/ddl-database.html.md.erb
@@ -2,6 +2,25 @@
title: Creating and Managing Databases
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Your HAWQ deployment may have multiple databases. This is different from some database management systems \(such as Oracle\) where the database instance *is* the database. Although you can create many databases in a HAWQ system, client programs can connect to and access only one database at a time \u2014 you cannot cross-query between databases.
## <a id="topic3"></a>About Template Databases
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/ddl/ddl-partition.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/ddl/ddl-partition.html.md.erb b/markdown/ddl/ddl-partition.html.md.erb
index f790161..b0dc3f4 100644
--- a/markdown/ddl/ddl-partition.html.md.erb
+++ b/markdown/ddl/ddl-partition.html.md.erb
@@ -2,6 +2,25 @@
title: Partitioning Large Tables
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Table partitioning enables supporting very large tables, such as fact tables, by logically dividing them into smaller, more manageable pieces. Partitioned tables can improve query performance by allowing the HAWQ query optimizer to scan only the data needed to satisfy a given query instead of scanning all the contents of a large table.
Partitioning does not change the physical distribution of table data across the segments. Table distribution is physical: HAWQ physically divides partitioned tables and non-partitioned tables across segments to enable parallel query processing. Table *partitioning* is logical: HAWQ logically divides big tables to improve query performance and facilitate data warehouse maintenance tasks, such as rolling old data out of the data warehouse.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/ddl/ddl-schema.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/ddl/ddl-schema.html.md.erb b/markdown/ddl/ddl-schema.html.md.erb
index 7c361ba..6fb9b83 100644
--- a/markdown/ddl/ddl-schema.html.md.erb
+++ b/markdown/ddl/ddl-schema.html.md.erb
@@ -2,6 +2,25 @@
title: Creating and Managing Schemas
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Schemas logically organize objects and data in a database. Schemas allow you to have more than one object \(such as tables\) with the same name in the database without conflict if the objects are in different schemas.
## <a id="topic18"></a>The Default "Public" Schema
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/ddl/ddl-storage.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/ddl/ddl-storage.html.md.erb b/markdown/ddl/ddl-storage.html.md.erb
index 264e552..9d73711 100644
--- a/markdown/ddl/ddl-storage.html.md.erb
+++ b/markdown/ddl/ddl-storage.html.md.erb
@@ -2,6 +2,25 @@
title: Table Storage Model and Distribution Policy
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
HAWQ supports several storage models and a mix of storage models. When you create a table, you choose how to store its data. This topic explains the options for table storage and how to choose the best storage model for your workload.
**Note:** To simplify the creation of database tables, you can specify the default values for some table storage options with the HAWQ server configuration parameter `gp_default_storage_options`.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/ddl/ddl-table.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/ddl/ddl-table.html.md.erb b/markdown/ddl/ddl-table.html.md.erb
index bc4f0c4..4eb56af 100644
--- a/markdown/ddl/ddl-table.html.md.erb
+++ b/markdown/ddl/ddl-table.html.md.erb
@@ -2,6 +2,25 @@
title: Creating and Managing Tables
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
HAWQ Tables are similar to tables in any relational database, except that table rows are distributed across the different segments in the system. When you create a table, you specify the table's distribution policy.
## <a id="topic26"></a>Creating a Table
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/ddl/ddl-tablespace.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/ddl/ddl-tablespace.html.md.erb b/markdown/ddl/ddl-tablespace.html.md.erb
index 8720665..9b4bdf0 100644
--- a/markdown/ddl/ddl-tablespace.html.md.erb
+++ b/markdown/ddl/ddl-tablespace.html.md.erb
@@ -2,6 +2,25 @@
title: Creating and Managing Tablespaces
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Tablespaces allow database administrators to have multiple file systems per machine and decide how to best use physical storage to store database objects. They are named locations within a filespace in which you can create objects. Tablespaces allow you to assign different storage for frequently and infrequently used database objects or to control the I/O performance on certain database objects. For example, place frequently-used tables on file systems that use high performance solid-state drives \(SSD\), and place other tables on standard hard drives.
A tablespace requires a file system location to store its database files. In HAWQ, the master and each segment require a distinct storage location. The collection of file system locations for all components in a HAWQ system is a *filespace*. Filespaces can be used by one or more tablespaces.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/ddl/ddl-view.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/ddl/ddl-view.html.md.erb b/markdown/ddl/ddl-view.html.md.erb
index 35da41e..999b81b 100644
--- a/markdown/ddl/ddl-view.html.md.erb
+++ b/markdown/ddl/ddl-view.html.md.erb
@@ -2,6 +2,25 @@
title: Creating and Managing Views
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Views enable you to save frequently used or complex queries, then access them in a `SELECT` statement as if they were a table. A view is not physically materialized on disk: the query runs as a subquery when you access the view.
If a subquery is associated with a single query, consider using the `WITH` clause of the `SELECT` command instead of creating a seldom-used view.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/ddl/ddl.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/ddl/ddl.html.md.erb b/markdown/ddl/ddl.html.md.erb
index 7873fe7..8d4a044 100644
--- a/markdown/ddl/ddl.html.md.erb
+++ b/markdown/ddl/ddl.html.md.erb
@@ -2,6 +2,25 @@
title: Defining Database Objects
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
This section covers data definition language \(DDL\) in HAWQ and how to create and manage database objects.
Creating objects in a HAWQ includes making up-front choices about data distribution, storage options, data loading, and other HAWQ features that will affect the ongoing performance of your database system. Understanding the options that are available and how the database will be used will help you make the right decisions.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/install/aws-config.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/install/aws-config.html.md.erb b/markdown/install/aws-config.html.md.erb
index 1bb519e..8f3cf5e 100644
--- a/markdown/install/aws-config.html.md.erb
+++ b/markdown/install/aws-config.html.md.erb
@@ -2,6 +2,25 @@
title: Amazon EC2 Configuration
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Amazon Elastic Compute Cloud (EC2) is a service provided by Amazon Web Services (AWS). You can install and configure HAWQ on virtual servers provided by Amazon EC2. The following information describes some considerations when deploying a HAWQ cluster in an Amazon EC2 environment.
## <a id="topic_wqv_yfx_y5"></a>About Amazon EC2
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/install/select-hosts.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/install/select-hosts.html.md.erb b/markdown/install/select-hosts.html.md.erb
index ecbe0b5..c59a9e7 100644
--- a/markdown/install/select-hosts.html.md.erb
+++ b/markdown/install/select-hosts.html.md.erb
@@ -2,6 +2,25 @@
title: Select HAWQ Host Machines
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Before you begin to install HAWQ, follow these steps to select and prepare the host machines.
Complete this procedure for all HAWQ deployments:
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/overview/ElasticSegments.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/overview/ElasticSegments.html.md.erb b/markdown/overview/ElasticSegments.html.md.erb
index 383eab5..b925ae8 100755
--- a/markdown/overview/ElasticSegments.html.md.erb
+++ b/markdown/overview/ElasticSegments.html.md.erb
@@ -2,6 +2,25 @@
title: Elastic Query Execution Runtime
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
HAWQ uses dynamically allocated virtual segments to provide resources for query execution.
In HAWQ 1.x, the number of segments \(compute resource carrier\) used to run a query is fixed, no matter whether the underlying query is big query requiring many resources or a small query requiring little resources. This architecture is simple, however it uses resources inefficiently.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/overview/HAWQArchitecture.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/overview/HAWQArchitecture.html.md.erb b/markdown/overview/HAWQArchitecture.html.md.erb
index d42d241..7febdda 100755
--- a/markdown/overview/HAWQArchitecture.html.md.erb
+++ b/markdown/overview/HAWQArchitecture.html.md.erb
@@ -2,6 +2,25 @@
title: HAWQ Architecture
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
This topic presents HAWQ architecture and its main components.
In a typical HAWQ deployment, each slave node has one physical HAWQ segment, an HDFS DataNode and a NodeManager installed. Masters for HAWQ, HDFS and YARN are hosted on separate nodes.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/overview/HAWQOverview.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/overview/HAWQOverview.html.md.erb b/markdown/overview/HAWQOverview.html.md.erb
index c41f3d9..2ee1ba8 100755
--- a/markdown/overview/HAWQOverview.html.md.erb
+++ b/markdown/overview/HAWQOverview.html.md.erb
@@ -2,6 +2,25 @@
title: What is HAWQ?
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
HAWQ is a Hadoop native SQL query engine that combines the key technological advantages of MPP database with the scalability and convenience of Hadoop. HAWQ reads data from and writes data to HDFS natively.
HAWQ delivers industry-leading performance and linear scalability. It provides users the tools to confidently and successfully interact with petabyte range data sets. HAWQ provides users with a complete, standards compliant SQL interface. More specifically, HAWQ has the following features:
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/overview/HDFSCatalogCache.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/overview/HDFSCatalogCache.html.md.erb b/markdown/overview/HDFSCatalogCache.html.md.erb
index 8803dc4..a081143 100755
--- a/markdown/overview/HDFSCatalogCache.html.md.erb
+++ b/markdown/overview/HDFSCatalogCache.html.md.erb
@@ -2,6 +2,25 @@
title: HDFS Catalog Cache
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
HDFS catalog cache is a caching service used by HAWQ master to determine the distribution information of table data on HDFS.
HDFS is slow at RPC handling, especially when the number of concurrent requests is high. In order to decide which segments handle which part of data, HAWQ needs data location information from HDFS NameNodes. HDFS catalog cache is used to cache the data location information and accelerate HDFS RPCs.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/overview/ManagementTools.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/overview/ManagementTools.html.md.erb b/markdown/overview/ManagementTools.html.md.erb
index 0c7439d..9a5074b 100755
--- a/markdown/overview/ManagementTools.html.md.erb
+++ b/markdown/overview/ManagementTools.html.md.erb
@@ -2,6 +2,25 @@
title: HAWQ Management Tools
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
HAWQ management tools are consolidated into one `hawq` command.
The `hawq` command can init, start and stop each segment separately, and supports dynamic expansion of the cluster.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/overview/RedundancyFailover.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/overview/RedundancyFailover.html.md.erb b/markdown/overview/RedundancyFailover.html.md.erb
index 90eec63..5a48bd0 100755
--- a/markdown/overview/RedundancyFailover.html.md.erb
+++ b/markdown/overview/RedundancyFailover.html.md.erb
@@ -2,6 +2,25 @@
title: High Availability, Redundancy and Fault Tolerance
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
HAWQ ensures high availability for its clusters through system redundancy. HAWQ deployments utilize platform hardware redundancy, such as RAID for the master catalog, JBOD for segments and network redundancy for its interconnect layer. On the software level, HAWQ provides redundancy via master mirroring and dual cluster maintenance. In addition, HAWQ supports high availability NameNode configuration within HDFS.
To maintain cluster health, HAWQ uses a fault tolerance service based on heartbeats and on-demand probe protocols. It can identify newly added nodes dynamically and remove nodes from the cluster when it becomes unusable.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/overview/ResourceManagement.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/overview/ResourceManagement.html.md.erb b/markdown/overview/ResourceManagement.html.md.erb
index 8f7e2fd..7f99069 100755
--- a/markdown/overview/ResourceManagement.html.md.erb
+++ b/markdown/overview/ResourceManagement.html.md.erb
@@ -2,6 +2,25 @@
title: Resource Management
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
HAWQ provides several approaches to resource management and includes several user-configurable options, including integration with YARN's resource management.
HAWQ has the ability to manage resources by using the following mechanisms:
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/overview/TableDistributionStorage.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/overview/TableDistributionStorage.html.md.erb b/markdown/overview/TableDistributionStorage.html.md.erb
index ec1d8b5..678885b 100755
--- a/markdown/overview/TableDistributionStorage.html.md.erb
+++ b/markdown/overview/TableDistributionStorage.html.md.erb
@@ -2,6 +2,25 @@
title: Table Distribution and Storage
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
HAWQ stores all table data, except the system table, in HDFS. When a user creates a table, the metadata is stored on the master's local file system and the table content is stored in HDFS.
In order to simplify table data management, all the data of one relation are saved under one HDFS folder.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/overview/system-overview.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/overview/system-overview.html.md.erb b/markdown/overview/system-overview.html.md.erb
index 9fc1c53..455ea0a 100644
--- a/markdown/overview/system-overview.html.md.erb
+++ b/markdown/overview/system-overview.html.md.erb
@@ -1,6 +1,25 @@
---
title: Apache HAWQ (Incubating) System Overview
---
+
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
* <a href="./HAWQOverview.html" class="subnav">What is HAWQ?</a>
* <a href="./HAWQArchitecture.html" class="subnav">HAWQ Architecture</a>
* <a href="./TableDistributionStorage.html" class="subnav">Table Distribution and Storage</a>
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/plext/UsingProceduralLanguages.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/plext/UsingProceduralLanguages.html.md.erb b/markdown/plext/UsingProceduralLanguages.html.md.erb
index bef1b93..2b83b91 100644
--- a/markdown/plext/UsingProceduralLanguages.html.md.erb
+++ b/markdown/plext/UsingProceduralLanguages.html.md.erb
@@ -2,6 +2,25 @@
title: Using Languages and Extensions in HAWQ
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
HAWQ supports user-defined functions that are created with the SQL and C built-in languages, and also supports user-defined aliases for internal functions.
HAWQ also supports user-defined functions written in languages other than SQL and C. These other languages are generically called *procedural languages* (PLs) and are extensions to the core HAWQ functionality. HAWQ specifically supports the PL/Java, PL/Perl, PL/pgSQL, PL/Python, and PL/R procedural languages.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/plext/builtin_langs.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/plext/builtin_langs.html.md.erb b/markdown/plext/builtin_langs.html.md.erb
index 01891e8..882044a 100644
--- a/markdown/plext/builtin_langs.html.md.erb
+++ b/markdown/plext/builtin_langs.html.md.erb
@@ -2,6 +2,25 @@
title: Using HAWQ Built-In Languages
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
This section provides an introduction to using the HAWQ built-in languages.
HAWQ supports user-defined functions created with the SQL and C built-in languages. HAWQ also supports user-defined aliases for internal functions.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/plext/using_pgcrypto.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/plext/using_pgcrypto.html.md.erb b/markdown/plext/using_pgcrypto.html.md.erb
index e3e9225..1e06f0b 100644
--- a/markdown/plext/using_pgcrypto.html.md.erb
+++ b/markdown/plext/using_pgcrypto.html.md.erb
@@ -2,6 +2,25 @@
title: Enabling Cryptographic Functions for PostgreSQL (pgcrypto)
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
`pgcrypto` is a package extension included in your HAWQ distribution. You must explicitly enable the cryptographic functions to use this extension.
## <a id="pgcryptoprereq"></a>Prerequisites
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/plext/using_pljava.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/plext/using_pljava.html.md.erb b/markdown/plext/using_pljava.html.md.erb
index e463093..cb97f29 100644
--- a/markdown/plext/using_pljava.html.md.erb
+++ b/markdown/plext/using_pljava.html.md.erb
@@ -2,6 +2,25 @@
title: Using PL/Java
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
This section provides an overview of the HAWQ PL/Java procedural language extension.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/plext/using_plperl.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/plext/using_plperl.html.md.erb b/markdown/plext/using_plperl.html.md.erb
index d6ffa04..f642fea 100644
--- a/markdown/plext/using_plperl.html.md.erb
+++ b/markdown/plext/using_plperl.html.md.erb
@@ -2,6 +2,25 @@
title: Using PL/Perl
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
This section contains an overview of the HAWQ PL/Perl language extension.
## <a id="enableplperl"></a>Enabling PL/Perl
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/plext/using_plpgsql.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/plext/using_plpgsql.html.md.erb b/markdown/plext/using_plpgsql.html.md.erb
index 3661e9b..c778c70 100644
--- a/markdown/plext/using_plpgsql.html.md.erb
+++ b/markdown/plext/using_plpgsql.html.md.erb
@@ -2,6 +2,25 @@
title: Using PL/pgSQL in HAWQ
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
SQL is the language of most other relational databases use as query language. It is portable and easy to learn. But every SQL statement must be executed individually by the database server.
PL/pgSQL is a loadable procedural language. PL/SQL can do the following:
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/plext/using_plpython.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/plext/using_plpython.html.md.erb b/markdown/plext/using_plpython.html.md.erb
index 063509a..32a6e6d 100644
--- a/markdown/plext/using_plpython.html.md.erb
+++ b/markdown/plext/using_plpython.html.md.erb
@@ -2,6 +2,25 @@
title: Using PL/Python in HAWQ
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
This section provides an overview of the HAWQ PL/Python procedural language extension.
## <a id="abouthawqplpython"></a>About HAWQ PL/Python
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/plext/using_plr.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/plext/using_plr.html.md.erb b/markdown/plext/using_plr.html.md.erb
index 367a1d0..ca4def7 100644
--- a/markdown/plext/using_plr.html.md.erb
+++ b/markdown/plext/using_plr.html.md.erb
@@ -2,6 +2,25 @@
title: Using PL/R in HAWQ
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
PL/R is a procedural language. With the HAWQ PL/R extension, you can write database functions in the R programming language and use R packages that contain R functions and data sets.
**Note**: To use PL/R in HAWQ, R must be installed on each node in your HAWQ cluster. Additionally, you must install the PL/R package on an existing HAWQ deployment or have specified PL/R as a build option when compiling HAWQ.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/pxf/ConfigurePXF.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/pxf/ConfigurePXF.html.md.erb b/markdown/pxf/ConfigurePXF.html.md.erb
index fec6b27..c21866e 100644
--- a/markdown/pxf/ConfigurePXF.html.md.erb
+++ b/markdown/pxf/ConfigurePXF.html.md.erb
@@ -2,6 +2,25 @@
title: Configuring PXF
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
This topic describes how to configure the PXF service.
**Note:** After you make any changes to a PXF configuration file (such as `pxf-profiles.xml` for adding custom profiles), propagate the changes to all nodes with PXF installed, and then restart the PXF service on all nodes.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/pxf/HBasePXF.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/pxf/HBasePXF.html.md.erb b/markdown/pxf/HBasePXF.html.md.erb
index 8b89730..3be06d2 100644
--- a/markdown/pxf/HBasePXF.html.md.erb
+++ b/markdown/pxf/HBasePXF.html.md.erb
@@ -2,6 +2,25 @@
title: Accessing HBase Data
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
## <a id="installingthepxfhbaseplugin"></a>Prerequisites
Before trying to access HBase data with PXF, verify the following:
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/pxf/HDFSFileDataPXF.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/pxf/HDFSFileDataPXF.html.md.erb b/markdown/pxf/HDFSFileDataPXF.html.md.erb
index 2021565..6780650 100644
--- a/markdown/pxf/HDFSFileDataPXF.html.md.erb
+++ b/markdown/pxf/HDFSFileDataPXF.html.md.erb
@@ -2,6 +2,25 @@
title: Accessing HDFS File Data
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
HDFS is the primary distributed storage mechanism used by Apache Hadoop applications. The PXF HDFS plug-in reads file data stored in HDFS. The plug-in supports plain delimited and comma-separated-value format text files. The HDFS plug-in also supports the Avro binary format.
This section describes how to use PXF to access HDFS data, including how to create and query an external table from files in the HDFS data store.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/pxf/HDFSWritablePXF.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/pxf/HDFSWritablePXF.html.md.erb b/markdown/pxf/HDFSWritablePXF.html.md.erb
index b99b03e..021b6b9 100644
--- a/markdown/pxf/HDFSWritablePXF.html.md.erb
+++ b/markdown/pxf/HDFSWritablePXF.html.md.erb
@@ -2,6 +2,25 @@
title: Writing Data to HDFS
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The PXF HDFS plug-in supports writable external tables using the `HdfsTextSimple` and `SequenceWritable` profiles. You might create a writable table to export data from a HAWQ internal table to binary or text HDFS files.
Use the `HdfsTextSimple` profile when writing text data. Use the `SequenceWritable` profile when dealing with binary data.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/pxf/HawqExtensionFrameworkPXF.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/pxf/HawqExtensionFrameworkPXF.html.md.erb b/markdown/pxf/HawqExtensionFrameworkPXF.html.md.erb
index 72ca3b6..1a49576 100644
--- a/markdown/pxf/HawqExtensionFrameworkPXF.html.md.erb
+++ b/markdown/pxf/HawqExtensionFrameworkPXF.html.md.erb
@@ -2,6 +2,25 @@
title: Using PXF with Unmanaged Data
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
HAWQ Extension Framework (PXF) is an extensible framework that allows HAWQ to query external system data.�
PXF includes built-in connectors for accessing data inside HDFS files, Hive tables, and HBase tables. PXF also integrates with HCatalog to query Hive tables directly.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/pxf/HivePXF.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/pxf/HivePXF.html.md.erb b/markdown/pxf/HivePXF.html.md.erb
index 199c7a1..e0f6c79 100644
--- a/markdown/pxf/HivePXF.html.md.erb
+++ b/markdown/pxf/HivePXF.html.md.erb
@@ -2,6 +2,25 @@
title: Accessing Hive Data
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Apache Hive is a distributed data warehousing infrastructure. Hive facilitates managing large data sets supporting multiple data formats, including comma-separated value (.csv), RC, ORC, and parquet. The PXF Hive plug-in reads data stored in Hive, as well as HDFS or HBase.
This section describes how to use PXF to access Hive data. Options for querying data stored in Hive include:
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/pxf/InstallPXFPlugins.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/pxf/InstallPXFPlugins.html.md.erb b/markdown/pxf/InstallPXFPlugins.html.md.erb
index 4ae4101..4fbda74 100644
--- a/markdown/pxf/InstallPXFPlugins.html.md.erb
+++ b/markdown/pxf/InstallPXFPlugins.html.md.erb
@@ -2,6 +2,25 @@
title: Installing PXF Plug-ins
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
This topic describes how to install the built-in PXF service plug-ins that are required to connect PXF to HDFS, Hive, HBase, and JSON.
**Note:** PXF requires that you run Tomcat on the host machine. Tomcat reserves ports 8005, 8080, and 8009. If you have configured Oozie JXM reporting on a host that will run PXF, make sure that the reporting service uses a port other than 8005. This helps to prevent port conflict errors from occurring when you start the PXF service.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/pxf/JsonPXF.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/pxf/JsonPXF.html.md.erb b/markdown/pxf/JsonPXF.html.md.erb
index 97195ad..5f156c4 100644
--- a/markdown/pxf/JsonPXF.html.md.erb
+++ b/markdown/pxf/JsonPXF.html.md.erb
@@ -2,6 +2,25 @@
title: Accessing JSON File Data
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The PXF JSON plug-in reads native JSON stored in HDFS. The plug-in supports common data types, as well as basic (N-level) projection and arrays.
To access JSON file data with HAWQ, the data must be stored in HDFS and an external table created from the HDFS data store.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/pxf/PXFExternalTableandAPIReference.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/pxf/PXFExternalTableandAPIReference.html.md.erb b/markdown/pxf/PXFExternalTableandAPIReference.html.md.erb
index 292616b..c947775 100644
--- a/markdown/pxf/PXFExternalTableandAPIReference.html.md.erb
+++ b/markdown/pxf/PXFExternalTableandAPIReference.html.md.erb
@@ -2,6 +2,25 @@
title: PXF External Tables and API
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
You can use the PXF API to create�your own connectors to access any other type of parallel data store or processing engine.
The PXF Java API lets you extend PXF functionality and add new services and formats without changing HAWQ. The API includes three classes that are extended to allow HAWQ to access an external data source: Fragmenter, Accessor, and Resolver.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/pxf/ReadWritePXF.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/pxf/ReadWritePXF.html.md.erb b/markdown/pxf/ReadWritePXF.html.md.erb
index 18f655d..f424afd 100644
--- a/markdown/pxf/ReadWritePXF.html.md.erb
+++ b/markdown/pxf/ReadWritePXF.html.md.erb
@@ -2,6 +2,25 @@
title: Using Profiles to Read and Write Data
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
PXF profiles are collections of common metadata attributes that can be used to simplify the reading and writing of data. You can use any of the built-in profiles that come with PXF or you can create your own.
For example, if you are writing single line records to text files on HDFS, you could use the built-in HdfsTextSimple profile. You specify this profile when you create the PXF external table used to write the data to HDFS.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/pxf/TroubleshootingPXF.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/pxf/TroubleshootingPXF.html.md.erb b/markdown/pxf/TroubleshootingPXF.html.md.erb
index 9febe09..c1bd43a 100644
--- a/markdown/pxf/TroubleshootingPXF.html.md.erb
+++ b/markdown/pxf/TroubleshootingPXF.html.md.erb
@@ -2,6 +2,25 @@
title: Troubleshooting PXF
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
## <a id="pxerrortbl"></a>PXF Errors
The following table lists some common errors encountered while using PXF:
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/query/HAWQQueryProcessing.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/query/HAWQQueryProcessing.html.md.erb b/markdown/query/HAWQQueryProcessing.html.md.erb
index 1d221f4..0538592 100644
--- a/markdown/query/HAWQQueryProcessing.html.md.erb
+++ b/markdown/query/HAWQQueryProcessing.html.md.erb
@@ -2,6 +2,25 @@
title: About HAWQ Query Processing
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
This topic provides an overview of how HAWQ processes queries. Understanding this process can be useful when writing and tuning queries.
Users issue queries to HAWQ as they would to any database management system. They connect to the database instance on the HAWQ master host using a client application such as `psql` and submit SQL statements.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/query/defining-queries.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/query/defining-queries.html.md.erb b/markdown/query/defining-queries.html.md.erb
index b796511..360ebe2 100644
--- a/markdown/query/defining-queries.html.md.erb
+++ b/markdown/query/defining-queries.html.md.erb
@@ -2,6 +2,25 @@
title: Defining Queries
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
HAWQ is based on the PostgreSQL implementation of the SQL standard. SQL commands are typically entered using the standard PostgreSQL interactive terminal `psql`, but other programs that have similar functionality can be used as well.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/query/functions-operators.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/query/functions-operators.html.md.erb b/markdown/query/functions-operators.html.md.erb
index 8f14ee6..4b4b03a 100644
--- a/markdown/query/functions-operators.html.md.erb
+++ b/markdown/query/functions-operators.html.md.erb
@@ -2,6 +2,25 @@
title: Using Functions and Operators
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
HAWQ evaluates functions and operators used in SQL expressions.
## <a id="topic27"></a>Using Functions in HAWQ
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/query/gporca/query-gporca-changed.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/query/gporca/query-gporca-changed.html.md.erb b/markdown/query/gporca/query-gporca-changed.html.md.erb
index 041aa4b..4e7d586 100644
--- a/markdown/query/gporca/query-gporca-changed.html.md.erb
+++ b/markdown/query/gporca/query-gporca-changed.html.md.erb
@@ -2,6 +2,25 @@
title: Changed Behavior with GPORCA
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
<span class="shortdesc">When GPORCA is enabled, HAWQ's behavior changes. This topic describes these changes.</span>
- The command `CREATE TABLE AS` distributes table data randomly if the `DISTRIBUTED BY` clause is not specified and no primary or unique keys are specified.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/query/gporca/query-gporca-enable.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/query/gporca/query-gporca-enable.html.md.erb b/markdown/query/gporca/query-gporca-enable.html.md.erb
index e8cc93f..510b4e4 100644
--- a/markdown/query/gporca/query-gporca-enable.html.md.erb
+++ b/markdown/query/gporca/query-gporca-enable.html.md.erb
@@ -2,6 +2,25 @@
title: Enabling GPORCA
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
<span class="shortdesc">Precompiled versions of HAWQ that include the GPORCA query optimizer enable it by default, no additional configuration is required. To use the GPORCA query optimizer in a HAWQ built from source, your build must include GPORCA. You must also enable specific HAWQ server configuration parameters at or after install time: </span>
- [Set the <code class="ph codeph">optimizer\_analyze\_root\_partition</code> parameter to <code class="ph codeph">on</code>](#topic_r5d_hv1_kr) to enable statistics collection for the root partition of a partitioned table.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/query/gporca/query-gporca-fallback.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/query/gporca/query-gporca-fallback.html.md.erb b/markdown/query/gporca/query-gporca-fallback.html.md.erb
index 999e9a7..2c540a1 100644
--- a/markdown/query/gporca/query-gporca-fallback.html.md.erb
+++ b/markdown/query/gporca/query-gporca-fallback.html.md.erb
@@ -2,6 +2,25 @@
title: Determining The Query Optimizer In Use
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
<span class="shortdesc"> When GPORCA is enabled, you can determine if HAWQ is using GPORCA or is falling back to the legacy query optimizer. </span>
These are two ways to determine which query optimizer HAWQ used to execute the query:
[13/18] incubator-hawq-docs git commit: Removing erroneous targettype
parameter
Posted by yo...@apache.org.
Removing erroneous targettype parameter
Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/commit/10938ae6
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/tree/10938ae6
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/diff/10938ae6
Branch: refs/heads/release/2.1.0.0-incubating
Commit: 10938ae6a3ac1b71f22b02ebb0245c0eb4cd6bda
Parents: ab49891
Author: Jane Beckman <jb...@pivotal.io>
Authored: Thu Feb 9 08:50:56 2017 -0800
Committer: David Yozie <yo...@apache.org>
Committed: Thu Feb 9 08:50:56 2017 -0800
----------------------------------------------------------------------
markdown/reference/sql/DROP-CONVERSION.html.md.erb | 3 ---
1 file changed, 3 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/10938ae6/markdown/reference/sql/DROP-CONVERSION.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/sql/DROP-CONVERSION.html.md.erb b/markdown/reference/sql/DROP-CONVERSION.html.md.erb
index a7ac247..76d1d1f 100644
--- a/markdown/reference/sql/DROP-CONVERSION.html.md.erb
+++ b/markdown/reference/sql/DROP-CONVERSION.html.md.erb
@@ -41,9 +41,6 @@ DROP CONVERSION [IF EXISTS] <name> [CASCADE | RESTRICT]
<dt>\<name\> </dt>
<dd>The name of the conversion. The conversion name may be schema-qualified.</dd>
-<dt>\<targettype\> </dt>
-<dd>The name of the target data type of the cast to be removed.</dd>
-
<dt>CASCADE</dt>
<dt>RESTRICT</dt>
<dd>These keywords have no effect since there are no dependencies on conversions.</dd>
[18/18] incubator-hawq-docs git commit: adding /latest/index.html
redirect for incubator-hawq-site PR
https://github.com/apache/incubator-hawq-site/pull/10#issuecomment-281249380
Posted by yo...@apache.org.
adding /latest/index.html redirect for incubator-hawq-site PR https://github.com/apache/incubator-hawq-site/pull/10#issuecomment-281249380
Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/commit/67ca8cc9
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/tree/67ca8cc9
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/diff/67ca8cc9
Branch: refs/heads/release/2.1.0.0-incubating
Commit: 67ca8cc93b2b01728af87acfcd6ff6d61b7c9beb
Parents: 31fa585
Author: David Yozie <yo...@apache.org>
Authored: Wed Feb 22 14:08:46 2017 -0800
Committer: David Yozie <yo...@apache.org>
Committed: Wed Feb 22 14:08:46 2017 -0800
----------------------------------------------------------------------
.../source/docs/userguide/latest/index.html | 11 +++++++++++
1 file changed, 11 insertions(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/67ca8cc9/book/master_middleman/source/docs/userguide/latest/index.html
----------------------------------------------------------------------
diff --git a/book/master_middleman/source/docs/userguide/latest/index.html b/book/master_middleman/source/docs/userguide/latest/index.html
new file mode 100644
index 0000000..f9ff594
--- /dev/null
+++ b/book/master_middleman/source/docs/userguide/latest/index.html
@@ -0,0 +1,11 @@
+<html>
+
+<head>
+
+ <script type="text/javascript">
+ window.location = "/docs/userguide/2.1.0.0-incubating/overview/HAWQOverview.html";
+ </script>
+
+ <body></body>
+
+</html>
\ No newline at end of file
[03/18] incubator-hawq-docs git commit: HAWQ-1305 Add apache license
header to doc source
Posted by yo...@apache.org.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/sql/DROP-SCHEMA.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/sql/DROP-SCHEMA.html.md.erb b/markdown/reference/sql/DROP-SCHEMA.html.md.erb
index 8d7846f..993bddd 100644
--- a/markdown/reference/sql/DROP-SCHEMA.html.md.erb
+++ b/markdown/reference/sql/DROP-SCHEMA.html.md.erb
@@ -2,6 +2,25 @@
title: DROP SCHEMA
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Removes a schema.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/sql/DROP-SEQUENCE.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/sql/DROP-SEQUENCE.html.md.erb b/markdown/reference/sql/DROP-SEQUENCE.html.md.erb
index 59c0d85..4d787ae 100644
--- a/markdown/reference/sql/DROP-SEQUENCE.html.md.erb
+++ b/markdown/reference/sql/DROP-SEQUENCE.html.md.erb
@@ -2,6 +2,25 @@
title: DROP SEQUENCE
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Removes a sequence.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/sql/DROP-TABLE.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/sql/DROP-TABLE.html.md.erb b/markdown/reference/sql/DROP-TABLE.html.md.erb
index b277273..02a57c0 100644
--- a/markdown/reference/sql/DROP-TABLE.html.md.erb
+++ b/markdown/reference/sql/DROP-TABLE.html.md.erb
@@ -2,6 +2,25 @@
title: DROP TABLE
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Removes a table.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/sql/DROP-TABLESPACE.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/sql/DROP-TABLESPACE.html.md.erb b/markdown/reference/sql/DROP-TABLESPACE.html.md.erb
index 9ffdfef..2112ea9 100644
--- a/markdown/reference/sql/DROP-TABLESPACE.html.md.erb
+++ b/markdown/reference/sql/DROP-TABLESPACE.html.md.erb
@@ -2,6 +2,25 @@
title: DROP TABLESPACE
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Removes a tablespace.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/sql/DROP-TYPE.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/sql/DROP-TYPE.html.md.erb b/markdown/reference/sql/DROP-TYPE.html.md.erb
index 1ffd44a..f6e3476 100644
--- a/markdown/reference/sql/DROP-TYPE.html.md.erb
+++ b/markdown/reference/sql/DROP-TYPE.html.md.erb
@@ -2,6 +2,25 @@
title: DROP TYPE
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Removes a data type.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/sql/DROP-USER.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/sql/DROP-USER.html.md.erb b/markdown/reference/sql/DROP-USER.html.md.erb
index 6ab3992..268ec18 100644
--- a/markdown/reference/sql/DROP-USER.html.md.erb
+++ b/markdown/reference/sql/DROP-USER.html.md.erb
@@ -2,6 +2,25 @@
title: DROP USER
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Removes a database role.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/sql/DROP-VIEW.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/sql/DROP-VIEW.html.md.erb b/markdown/reference/sql/DROP-VIEW.html.md.erb
index b8b9968..83562d3 100644
--- a/markdown/reference/sql/DROP-VIEW.html.md.erb
+++ b/markdown/reference/sql/DROP-VIEW.html.md.erb
@@ -2,6 +2,25 @@
title: DROP VIEW
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Removes a view.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/sql/END.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/sql/END.html.md.erb b/markdown/reference/sql/END.html.md.erb
index 484afcf..41c7466 100644
--- a/markdown/reference/sql/END.html.md.erb
+++ b/markdown/reference/sql/END.html.md.erb
@@ -2,6 +2,25 @@
title: END
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Commits the current transaction.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/sql/EXECUTE.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/sql/EXECUTE.html.md.erb b/markdown/reference/sql/EXECUTE.html.md.erb
index ff57cc6..94972a6 100644
--- a/markdown/reference/sql/EXECUTE.html.md.erb
+++ b/markdown/reference/sql/EXECUTE.html.md.erb
@@ -2,6 +2,25 @@
title: EXECUTE
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Executes a prepared SQL statement.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/sql/EXPLAIN.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/sql/EXPLAIN.html.md.erb b/markdown/reference/sql/EXPLAIN.html.md.erb
index ca0e908..562fd39 100644
--- a/markdown/reference/sql/EXPLAIN.html.md.erb
+++ b/markdown/reference/sql/EXPLAIN.html.md.erb
@@ -2,6 +2,25 @@
title: EXPLAIN
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Shows the query plan of a statement.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/sql/FETCH.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/sql/FETCH.html.md.erb b/markdown/reference/sql/FETCH.html.md.erb
index bdd9292..eeffc44 100644
--- a/markdown/reference/sql/FETCH.html.md.erb
+++ b/markdown/reference/sql/FETCH.html.md.erb
@@ -2,6 +2,25 @@
title: FETCH
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Retrieves rows from a query using a cursor.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/sql/GRANT.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/sql/GRANT.html.md.erb b/markdown/reference/sql/GRANT.html.md.erb
index 1673df5..f87c274 100644
--- a/markdown/reference/sql/GRANT.html.md.erb
+++ b/markdown/reference/sql/GRANT.html.md.erb
@@ -2,6 +2,25 @@
title: GRANT
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Defines access privileges.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/sql/INSERT.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/sql/INSERT.html.md.erb b/markdown/reference/sql/INSERT.html.md.erb
index d23a2aa..ca533a4 100644
--- a/markdown/reference/sql/INSERT.html.md.erb
+++ b/markdown/reference/sql/INSERT.html.md.erb
@@ -2,6 +2,25 @@
title: INSERT
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Creates new rows in a table.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/sql/PREPARE.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/sql/PREPARE.html.md.erb b/markdown/reference/sql/PREPARE.html.md.erb
index c633f14..338bb02 100644
--- a/markdown/reference/sql/PREPARE.html.md.erb
+++ b/markdown/reference/sql/PREPARE.html.md.erb
@@ -2,6 +2,25 @@
title: PREPARE
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Prepare a statement for execution.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/sql/REASSIGN-OWNED.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/sql/REASSIGN-OWNED.html.md.erb b/markdown/reference/sql/REASSIGN-OWNED.html.md.erb
index c037bfe..053526e 100644
--- a/markdown/reference/sql/REASSIGN-OWNED.html.md.erb
+++ b/markdown/reference/sql/REASSIGN-OWNED.html.md.erb
@@ -2,6 +2,25 @@
title: REASSIGN OWNED
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Changes the ownership of database objects owned by a database role.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/sql/RELEASE-SAVEPOINT.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/sql/RELEASE-SAVEPOINT.html.md.erb b/markdown/reference/sql/RELEASE-SAVEPOINT.html.md.erb
index ca25d9e..43641b4 100644
--- a/markdown/reference/sql/RELEASE-SAVEPOINT.html.md.erb
+++ b/markdown/reference/sql/RELEASE-SAVEPOINT.html.md.erb
@@ -2,6 +2,25 @@
title: RELEASE SAVEPOINT
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Destroys a previously defined savepoint.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/sql/RESET.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/sql/RESET.html.md.erb b/markdown/reference/sql/RESET.html.md.erb
index cb04d32..25c882d 100644
--- a/markdown/reference/sql/RESET.html.md.erb
+++ b/markdown/reference/sql/RESET.html.md.erb
@@ -2,6 +2,25 @@
title: RESET
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Restores the value of a system configuration parameter to the default value.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/sql/REVOKE.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/sql/REVOKE.html.md.erb b/markdown/reference/sql/REVOKE.html.md.erb
index cad809a..8afcf00 100644
--- a/markdown/reference/sql/REVOKE.html.md.erb
+++ b/markdown/reference/sql/REVOKE.html.md.erb
@@ -2,6 +2,25 @@
title: REVOKE
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Removes access privileges.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/sql/ROLLBACK-TO-SAVEPOINT.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/sql/ROLLBACK-TO-SAVEPOINT.html.md.erb b/markdown/reference/sql/ROLLBACK-TO-SAVEPOINT.html.md.erb
index 33c771b..da09a00 100644
--- a/markdown/reference/sql/ROLLBACK-TO-SAVEPOINT.html.md.erb
+++ b/markdown/reference/sql/ROLLBACK-TO-SAVEPOINT.html.md.erb
@@ -2,6 +2,25 @@
title: ROLLBACK TO SAVEPOINT
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Rolls back the current transaction to a savepoint.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/sql/ROLLBACK.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/sql/ROLLBACK.html.md.erb b/markdown/reference/sql/ROLLBACK.html.md.erb
index eb1345d..576e3e3 100644
--- a/markdown/reference/sql/ROLLBACK.html.md.erb
+++ b/markdown/reference/sql/ROLLBACK.html.md.erb
@@ -2,6 +2,25 @@
title: ROLLBACK
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Aborts the current transaction.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/sql/SAVEPOINT.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/sql/SAVEPOINT.html.md.erb b/markdown/reference/sql/SAVEPOINT.html.md.erb
index c2f6917..34e2d0a 100644
--- a/markdown/reference/sql/SAVEPOINT.html.md.erb
+++ b/markdown/reference/sql/SAVEPOINT.html.md.erb
@@ -2,6 +2,25 @@
title: SAVEPOINT
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Defines a new savepoint within the current transaction.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/sql/SELECT-INTO.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/sql/SELECT-INTO.html.md.erb b/markdown/reference/sql/SELECT-INTO.html.md.erb
index 524a5f1..8461306 100644
--- a/markdown/reference/sql/SELECT-INTO.html.md.erb
+++ b/markdown/reference/sql/SELECT-INTO.html.md.erb
@@ -2,6 +2,25 @@
title: SELECT INTO
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Defines a new table from the results of a query.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/sql/SELECT.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/sql/SELECT.html.md.erb b/markdown/reference/sql/SELECT.html.md.erb
index 4649bad..32bb0a8 100644
--- a/markdown/reference/sql/SELECT.html.md.erb
+++ b/markdown/reference/sql/SELECT.html.md.erb
@@ -2,6 +2,25 @@
title: SELECT
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Retrieves rows from a table or view.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/sql/SET-ROLE.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/sql/SET-ROLE.html.md.erb b/markdown/reference/sql/SET-ROLE.html.md.erb
index 63a03f6..206568e 100644
--- a/markdown/reference/sql/SET-ROLE.html.md.erb
+++ b/markdown/reference/sql/SET-ROLE.html.md.erb
@@ -2,6 +2,25 @@
title: SET ROLE
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Sets the current role identifier of the current session.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/sql/SET-SESSION-AUTHORIZATION.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/sql/SET-SESSION-AUTHORIZATION.html.md.erb b/markdown/reference/sql/SET-SESSION-AUTHORIZATION.html.md.erb
index adea314..f94bea6 100644
--- a/markdown/reference/sql/SET-SESSION-AUTHORIZATION.html.md.erb
+++ b/markdown/reference/sql/SET-SESSION-AUTHORIZATION.html.md.erb
@@ -2,6 +2,25 @@
title: SET SESSION AUTHORIZATION
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Sets the session role identifier and the current role identifier of the current session.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/sql/SET.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/sql/SET.html.md.erb b/markdown/reference/sql/SET.html.md.erb
index 4f4ad24..206f8ce 100644
--- a/markdown/reference/sql/SET.html.md.erb
+++ b/markdown/reference/sql/SET.html.md.erb
@@ -2,6 +2,25 @@
title: SET
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Changes the value of a HAWQ configuration parameter.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/sql/SHOW.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/sql/SHOW.html.md.erb b/markdown/reference/sql/SHOW.html.md.erb
index 802761b..b631829 100644
--- a/markdown/reference/sql/SHOW.html.md.erb
+++ b/markdown/reference/sql/SHOW.html.md.erb
@@ -2,6 +2,25 @@
title: SHOW
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Shows the value of a system configuration parameter.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/sql/TRUNCATE.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/sql/TRUNCATE.html.md.erb b/markdown/reference/sql/TRUNCATE.html.md.erb
index c91ae84..bfd9aea 100644
--- a/markdown/reference/sql/TRUNCATE.html.md.erb
+++ b/markdown/reference/sql/TRUNCATE.html.md.erb
@@ -2,6 +2,25 @@
title: TRUNCATE
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Empties a table of all rows.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/sql/VACUUM.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/sql/VACUUM.html.md.erb b/markdown/reference/sql/VACUUM.html.md.erb
index 57ccc0e..59c5bdb 100644
--- a/markdown/reference/sql/VACUUM.html.md.erb
+++ b/markdown/reference/sql/VACUUM.html.md.erb
@@ -2,6 +2,25 @@
title: VACUUM
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Garbage-collects and optionally analyzes a database.
**Note**: HAWQ `VACUUM` support is provided only for system catalog tables. `VACUUM`ing a HAWQ user table has no effect.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/toolkit/hawq_toolkit.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/toolkit/hawq_toolkit.html.md.erb b/markdown/reference/toolkit/hawq_toolkit.html.md.erb
index b898061..2796241 100644
--- a/markdown/reference/toolkit/hawq_toolkit.html.md.erb
+++ b/markdown/reference/toolkit/hawq_toolkit.html.md.erb
@@ -2,6 +2,25 @@
title: The hawq_toolkit Administrative Schema
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
This section provides a reference on the `hawq_toolkit` administrative schema.
HAWQ provides an administrative schema called `hawq_toolkit` that you can use to query the system catalogs, log files, and operating environment for system status information. The `hawq_toolkit` schema contains a number of views that you can access using SQL commands. The `hawq_toolkit` schema is accessible to all database users, although some objects may require superuser permissions.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/requirements/system-requirements.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/requirements/system-requirements.html.md.erb b/markdown/requirements/system-requirements.html.md.erb
index 7f117dc..5a52f09 100644
--- a/markdown/requirements/system-requirements.html.md.erb
+++ b/markdown/requirements/system-requirements.html.md.erb
@@ -2,6 +2,25 @@
title: Apache HAWQ System Requirements
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Follow these guidelines to configure each host machine that will run an Apache HAWQ or PXF service.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/resourcemgmt/ConfigureResourceManagement.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/resourcemgmt/ConfigureResourceManagement.html.md.erb b/markdown/resourcemgmt/ConfigureResourceManagement.html.md.erb
index 1b66068..6e6d0e4 100644
--- a/markdown/resourcemgmt/ConfigureResourceManagement.html.md.erb
+++ b/markdown/resourcemgmt/ConfigureResourceManagement.html.md.erb
@@ -2,6 +2,25 @@
title: Configuring Resource Management
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
This topic provides configuration information for system administrators and database superusers responsible for managing resources in a HAWQ system.
To configure resource management in HAWQ, follow these high-level steps:
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/resourcemgmt/HAWQResourceManagement.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/resourcemgmt/HAWQResourceManagement.html.md.erb b/markdown/resourcemgmt/HAWQResourceManagement.html.md.erb
index cfa9942..f5412df 100644
--- a/markdown/resourcemgmt/HAWQResourceManagement.html.md.erb
+++ b/markdown/resourcemgmt/HAWQResourceManagement.html.md.erb
@@ -2,6 +2,25 @@
title: How HAWQ Manages Resources
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
HAWQ manages resources (CPU, memory, I/O and file handles) using a variety of mechanisms including global resource management, resource queues and the enforcement of limits on resource usage.
## <a id="global-env"></a>Globally Managed Environments
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/resourcemgmt/ResourceQueues.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/resourcemgmt/ResourceQueues.html.md.erb b/markdown/resourcemgmt/ResourceQueues.html.md.erb
index cd019c6..833cac8 100644
--- a/markdown/resourcemgmt/ResourceQueues.html.md.erb
+++ b/markdown/resourcemgmt/ResourceQueues.html.md.erb
@@ -2,6 +2,25 @@
title: Working with Hierarchical Resource Queues
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
This section describes how administrators can define and work with resource queues in order to allocate resource usage within HAWQ. By designing hierarchical resource queues, system administrators can balance system resources to queries as needed.
## <a id="resource_queues"></a>HAWQ Resource Queues
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/resourcemgmt/YARNIntegration.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/resourcemgmt/YARNIntegration.html.md.erb b/markdown/resourcemgmt/YARNIntegration.html.md.erb
index 6898f6c..be504a0 100644
--- a/markdown/resourcemgmt/YARNIntegration.html.md.erb
+++ b/markdown/resourcemgmt/YARNIntegration.html.md.erb
@@ -2,6 +2,25 @@
title: Integrating YARN with HAWQ
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
HAWQ supports integration with YARN for global resource management. In a YARN managed environment, HAWQ can request resources \(containers\) dynamically from YARN, and return resources when HAWQ's workload is not heavy. This feature makes HAWQ a native citizen of the whole Hadoop eco-system.
To integrate YARN with HAWQ, use the following high-level steps.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/resourcemgmt/best-practices.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/resourcemgmt/best-practices.html.md.erb b/markdown/resourcemgmt/best-practices.html.md.erb
index 74bd815..493838a 100644
--- a/markdown/resourcemgmt/best-practices.html.md.erb
+++ b/markdown/resourcemgmt/best-practices.html.md.erb
@@ -2,6 +2,25 @@
title: Best Practices for Configuring Resource Management
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
When configuring resource management, you can apply certain best practices to ensure that resources are managed both efficiently and for best system performance.
The following is a list of high-level best practices for optimal resource management:
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/resourcemgmt/index.md.erb
----------------------------------------------------------------------
diff --git a/markdown/resourcemgmt/index.md.erb b/markdown/resourcemgmt/index.md.erb
index 7efb756..424163e 100644
--- a/markdown/resourcemgmt/index.md.erb
+++ b/markdown/resourcemgmt/index.md.erb
@@ -2,6 +2,25 @@
title: Managing Resources
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
This section describes how to use HAWQ's resource management features:
* <a class="subnav" href="./HAWQResourceManagement.html">How HAWQ Manages Resources</a>
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/troubleshooting/Troubleshooting.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/troubleshooting/Troubleshooting.html.md.erb b/markdown/troubleshooting/Troubleshooting.html.md.erb
index 2b7414b..994fdfb 100644
--- a/markdown/troubleshooting/Troubleshooting.html.md.erb
+++ b/markdown/troubleshooting/Troubleshooting.html.md.erb
@@ -2,6 +2,25 @@
title: Troubleshooting
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
This chapter describes how to resolve common problems and errors that occur in a HAWQ system.
[08/18] incubator-hawq-docs git commit: HAWQ-1305 Add apache license
header to doc source
Posted by yo...@apache.org.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/datamgmt/dml.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/datamgmt/dml.html.md.erb b/markdown/datamgmt/dml.html.md.erb
index 681883a..3c5d7fa 100644
--- a/markdown/datamgmt/dml.html.md.erb
+++ b/markdown/datamgmt/dml.html.md.erb
@@ -2,6 +2,25 @@
title: Managing Data with HAWQ
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
This chapter provides information about manipulating data and concurrent access in HAWQ.
- **[Basic Data Operations](../datamgmt/BasicDataOperations.html)**
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/datamgmt/load/client-loadtools.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/datamgmt/load/client-loadtools.html.md.erb b/markdown/datamgmt/load/client-loadtools.html.md.erb
index fe291d0..6587a9f 100644
--- a/markdown/datamgmt/load/client-loadtools.html.md.erb
+++ b/markdown/datamgmt/load/client-loadtools.html.md.erb
@@ -1,6 +1,25 @@
---
title: Client-Based HAWQ Load Tools
---
+
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
HAWQ supports data loading from Red Hat Enterprise Linux 5, 6, and 7 and Windows XP client systems. HAWQ Load Tools include both a loader program and a parallel file distribution program.
This topic presents the instructions to install the HAWQ Load Tools on your client machine. It also includes the information necessary to configure HAWQ databases to accept remote client connections.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/datamgmt/load/creating-external-tables-examples.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/datamgmt/load/creating-external-tables-examples.html.md.erb b/markdown/datamgmt/load/creating-external-tables-examples.html.md.erb
index 7d38971..4001968 100644
--- a/markdown/datamgmt/load/creating-external-tables-examples.html.md.erb
+++ b/markdown/datamgmt/load/creating-external-tables-examples.html.md.erb
@@ -2,6 +2,25 @@
title: Creating External Tables - Examples
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The following examples show how to define external data with different protocols. Each `CREATE EXTERNAL TABLE` command can contain only one protocol.
**Note:** When using IPv6, always enclose the numeric IP addresses in square brackets.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/datamgmt/load/g-about-gpfdist-setup-and-performance.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/datamgmt/load/g-about-gpfdist-setup-and-performance.html.md.erb b/markdown/datamgmt/load/g-about-gpfdist-setup-and-performance.html.md.erb
index 28a0bfe..9868850 100644
--- a/markdown/datamgmt/load/g-about-gpfdist-setup-and-performance.html.md.erb
+++ b/markdown/datamgmt/load/g-about-gpfdist-setup-and-performance.html.md.erb
@@ -2,6 +2,25 @@
title: About gpfdist Setup and Performance
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Consider the following scenarios for optimizing your ETL network performance.
- Allow network traffic to use all ETL host Network Interface Cards (NICs) simultaneously. Run one instance of `gpfdist` on the ETL host, then declare the host name of each NIC in the `LOCATION` clause of your external table definition (see [Creating External Tables - Examples](creating-external-tables-examples.html#topic44)).
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/datamgmt/load/g-character-encoding.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/datamgmt/load/g-character-encoding.html.md.erb b/markdown/datamgmt/load/g-character-encoding.html.md.erb
index 9f3756d..ea52334 100644
--- a/markdown/datamgmt/load/g-character-encoding.html.md.erb
+++ b/markdown/datamgmt/load/g-character-encoding.html.md.erb
@@ -2,6 +2,25 @@
title: Character Encoding
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Character encoding systems consist of a code that pairs each character from a character set with something else, such as a sequence of numbers or octets, to facilitate data stransmission and storage. HAWQ supports a variety of character sets, including single-byte character sets such as the ISO 8859 series and multiple-byte character sets such as EUC (Extended UNIX Code), UTF-8, and Mule internal code. Clients can use all supported character sets transparently, but a few are not supported for use within the server as a server-side encoding.
Data files must be in a character encoding recognized by HAWQ. Data files that contain invalid or unsupported encoding sequences encounter errors when loading into HAWQ.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/datamgmt/load/g-command-based-web-external-tables.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/datamgmt/load/g-command-based-web-external-tables.html.md.erb b/markdown/datamgmt/load/g-command-based-web-external-tables.html.md.erb
index 7830cc3..4a6a5cb 100644
--- a/markdown/datamgmt/load/g-command-based-web-external-tables.html.md.erb
+++ b/markdown/datamgmt/load/g-command-based-web-external-tables.html.md.erb
@@ -2,6 +2,25 @@
title: Command-based Web External Tables
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The output of a shell command or script defines command-based web table data. Specify the command in the `EXECUTE` clause of `CREATE EXTERNAL WEB TABLE`. The data is current as of the time the command runs. The `EXECUTE` clause runs the shell command or script on the specified master or virtual segments. The virtual segments run the command in parallel. Scripts must be executable by the gpadmin user and reside in the same location on the master or the hosts of virtual segments.
The command that you specify in the external table definition executes from the database and cannot access environment variables from `.bashrc` or `.profile`. Set environment variables in the `EXECUTE` clause. The following external web table, for example, runs a command on the HAWQ master host:
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/datamgmt/load/g-configuration-file-format.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/datamgmt/load/g-configuration-file-format.html.md.erb b/markdown/datamgmt/load/g-configuration-file-format.html.md.erb
index 73f51a9..f375e5c 100644
--- a/markdown/datamgmt/load/g-configuration-file-format.html.md.erb
+++ b/markdown/datamgmt/load/g-configuration-file-format.html.md.erb
@@ -2,6 +2,25 @@
title: Configuration File Format
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The `gpfdist` configuration file uses the YAML 1.1 document format and implements a schema for defining the transformation parameters. The configuration file must be a valid YAML document.
The `gpfdist` program processes the document in order and uses indentation (spaces) to determine the document hierarchy and relationships of the sections to one another. The use of white space is significant. Do not use white space for formatting and do not use tabs.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/datamgmt/load/g-controlling-segment-parallelism.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/datamgmt/load/g-controlling-segment-parallelism.html.md.erb b/markdown/datamgmt/load/g-controlling-segment-parallelism.html.md.erb
index 4e0096c..ff89ffb 100644
--- a/markdown/datamgmt/load/g-controlling-segment-parallelism.html.md.erb
+++ b/markdown/datamgmt/load/g-controlling-segment-parallelism.html.md.erb
@@ -2,6 +2,25 @@
title: Controlling Segment Parallelism
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The `gp_external_max_segs` server configuration parameter controls the number of virtual segments that can simultaneously access a single `gpfdist` instance. The default is 64. You can set the number of segments such that some segments process external data files and some perform other database processing. Set this parameter in the `hawq-site.xml` file of your master instance.
The number of segments in the `gpfdist` location list specify the minimum number of virtual segments required to serve data to a `gpfdist` external table.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/datamgmt/load/g-create-an-error-table-and-declare-a-reject-limit.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/datamgmt/load/g-create-an-error-table-and-declare-a-reject-limit.html.md.erb b/markdown/datamgmt/load/g-create-an-error-table-and-declare-a-reject-limit.html.md.erb
index ade14ea..6f001ea 100644
--- a/markdown/datamgmt/load/g-create-an-error-table-and-declare-a-reject-limit.html.md.erb
+++ b/markdown/datamgmt/load/g-create-an-error-table-and-declare-a-reject-limit.html.md.erb
@@ -2,6 +2,25 @@
title: Capture Row Formatting Errors and Declare a Reject Limit
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The following SQL fragment captures formatting errors internally in HAWQ and declares a reject limit of 10 rows.
``` sql
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/datamgmt/load/g-creating-and-using-web-external-tables.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/datamgmt/load/g-creating-and-using-web-external-tables.html.md.erb b/markdown/datamgmt/load/g-creating-and-using-web-external-tables.html.md.erb
index 4ef6cab..bdbe808 100644
--- a/markdown/datamgmt/load/g-creating-and-using-web-external-tables.html.md.erb
+++ b/markdown/datamgmt/load/g-creating-and-using-web-external-tables.html.md.erb
@@ -2,6 +2,25 @@
title: Creating and Using Web External Tables
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
`CREATE EXTERNAL WEB TABLE` creates a web table definition. Web external tables allow HAWQ to treat dynamic data sources like regular database tables. Because web table data can change as a query runs, the data is not rescannable.
You can define command-based or URL-based web external tables. The definition forms are distinct: you cannot mix command-based and URL-based definitions.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/datamgmt/load/g-define-an-external-table-with-single-row-error-isolation.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/datamgmt/load/g-define-an-external-table-with-single-row-error-isolation.html.md.erb b/markdown/datamgmt/load/g-define-an-external-table-with-single-row-error-isolation.html.md.erb
index e0c3c17..54cde2a 100644
--- a/markdown/datamgmt/load/g-define-an-external-table-with-single-row-error-isolation.html.md.erb
+++ b/markdown/datamgmt/load/g-define-an-external-table-with-single-row-error-isolation.html.md.erb
@@ -2,6 +2,25 @@
title: Define an External Table with Single Row Error Isolation
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The following example logs errors internally in HAWQ and sets an error threshold of 10 errors.
``` sql
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/datamgmt/load/g-defining-a-command-based-writable-external-web-table.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/datamgmt/load/g-defining-a-command-based-writable-external-web-table.html.md.erb b/markdown/datamgmt/load/g-defining-a-command-based-writable-external-web-table.html.md.erb
index 8a24474..067fbfb 100644
--- a/markdown/datamgmt/load/g-defining-a-command-based-writable-external-web-table.html.md.erb
+++ b/markdown/datamgmt/load/g-defining-a-command-based-writable-external-web-table.html.md.erb
@@ -2,6 +2,25 @@
title: Defining a Command-Based Writable External Web Table
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
You can define writable external web tables to send output rows to an application or script. The application must accept an input stream, reside in the same location on all of the HAWQ segment hosts, and be executable by the `gpadmin` user. All segments in the HAWQ system run the application or script, whether or not a segment has output rows to process.
Use `CREATE WRITABLE EXTERNAL WEB TABLE` to define the external table and specify the application or script to run on the segment hosts. Commands execute from within the database and cannot access environment variables (such as `$PATH`). Set environment variables in the `EXECUTE` clause of your writable external table definition. For example:
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/datamgmt/load/g-defining-a-file-based-writable-external-table.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/datamgmt/load/g-defining-a-file-based-writable-external-table.html.md.erb b/markdown/datamgmt/load/g-defining-a-file-based-writable-external-table.html.md.erb
index fa1ddfa..5a7468b 100644
--- a/markdown/datamgmt/load/g-defining-a-file-based-writable-external-table.html.md.erb
+++ b/markdown/datamgmt/load/g-defining-a-file-based-writable-external-table.html.md.erb
@@ -2,6 +2,25 @@
title: Defining a File-Based Writable External Table
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Writable external tables that output data to files use the HAWQ parallel file server program, `gpfdist`, or HAWQ Extensions Framework (PXF).
Use the `CREATE WRITABLE EXTERNAL TABLE` command to define the external table and specify the location and format of the output files.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/datamgmt/load/g-determine-the-transformation-schema.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/datamgmt/load/g-determine-the-transformation-schema.html.md.erb b/markdown/datamgmt/load/g-determine-the-transformation-schema.html.md.erb
index 1a4eb9b..4376c67 100644
--- a/markdown/datamgmt/load/g-determine-the-transformation-schema.html.md.erb
+++ b/markdown/datamgmt/load/g-determine-the-transformation-schema.html.md.erb
@@ -2,6 +2,25 @@
title: Determine the Transformation Schema
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
To prepare for the transformation project:
1. <span class="ph">Determine the goal of the project, such as indexing data, analyzing data, combining data, and so on.</span>
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/datamgmt/load/g-disabling-execute-for-web-or-writable-external-tables.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/datamgmt/load/g-disabling-execute-for-web-or-writable-external-tables.html.md.erb b/markdown/datamgmt/load/g-disabling-execute-for-web-or-writable-external-tables.html.md.erb
index f0332b5..a462807 100644
--- a/markdown/datamgmt/load/g-disabling-execute-for-web-or-writable-external-tables.html.md.erb
+++ b/markdown/datamgmt/load/g-disabling-execute-for-web-or-writable-external-tables.html.md.erb
@@ -2,6 +2,25 @@
title: Disabling EXECUTE for Web or Writable External Tables
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
There is a security risk associated with allowing external tables to execute OS commands or scripts. To disable the use of `EXECUTE` in web and writable external table definitions, set the `gp_external_enable_exec server` configuration parameter to off in your master `hawq-site.xml` file:
``` pre
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/datamgmt/load/g-escaping-in-csv-formatted-files.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/datamgmt/load/g-escaping-in-csv-formatted-files.html.md.erb b/markdown/datamgmt/load/g-escaping-in-csv-formatted-files.html.md.erb
index d07b463..1ac32a2 100644
--- a/markdown/datamgmt/load/g-escaping-in-csv-formatted-files.html.md.erb
+++ b/markdown/datamgmt/load/g-escaping-in-csv-formatted-files.html.md.erb
@@ -2,6 +2,25 @@
title: Escaping in CSV Formatted Files
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
By default, the escape character is a `"` (double quote) for CSV-formatted files. If you want to use a different escape character, use the `ESCAPE` clause of `COPY`, `CREATE EXTERNAL TABLE` or the `hawq load` control file to declare a different escape character. In cases where your selected escape character is present in your data, you can use it to escape itself.
For example, suppose you have a table with three columns and you want to load the following three fields:
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/datamgmt/load/g-escaping-in-text-formatted-files.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/datamgmt/load/g-escaping-in-text-formatted-files.html.md.erb b/markdown/datamgmt/load/g-escaping-in-text-formatted-files.html.md.erb
index e24a2b7..110638e 100644
--- a/markdown/datamgmt/load/g-escaping-in-text-formatted-files.html.md.erb
+++ b/markdown/datamgmt/load/g-escaping-in-text-formatted-files.html.md.erb
@@ -2,6 +2,25 @@
title: Escaping in Text Formatted Files
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
By default, the escape character is a \\ (backslash) for text-formatted files. You can declare a different escape character in the `ESCAPE` clause of `COPY`, `CREATE EXTERNAL TABLE`, or the `hawq load` control file. If your escape character appears in your data, use it to escape itself.
For example, suppose you have a table with three columns and you want to load the following three fields:
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/datamgmt/load/g-escaping.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/datamgmt/load/g-escaping.html.md.erb b/markdown/datamgmt/load/g-escaping.html.md.erb
index 0a1e62a..d3a5aa7 100644
--- a/markdown/datamgmt/load/g-escaping.html.md.erb
+++ b/markdown/datamgmt/load/g-escaping.html.md.erb
@@ -2,6 +2,25 @@
title: Escaping
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
There are two reserved characters that have special meaning to HAWQ:
- The designated delimiter character separates columns or fields in the data file.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/datamgmt/load/g-example-1-dblp-database-publications-in-demo-directory.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/datamgmt/load/g-example-1-dblp-database-publications-in-demo-directory.html.md.erb b/markdown/datamgmt/load/g-example-1-dblp-database-publications-in-demo-directory.html.md.erb
index 4f61396..0c27cd7 100644
--- a/markdown/datamgmt/load/g-example-1-dblp-database-publications-in-demo-directory.html.md.erb
+++ b/markdown/datamgmt/load/g-example-1-dblp-database-publications-in-demo-directory.html.md.erb
@@ -2,6 +2,25 @@
title: Command-based Web External Tables
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The output of a shell command or script defines command-based web table data. Specify the command in the `EXECUTE` clause of `CREATE EXTERNAL WEB TABLE`. The data is current as of the time the command runs. The `EXECUTE` clause runs the shell command or script on the specified master, and/or segment host or hosts. The command or script must reside on the hosts corresponding to the host(s) defined in the `EXECUTE` clause.
By default, the command is run on segment hosts when active segments have output rows to process. For example, if each segment host runs four primary segment instances that have output rows to process, the command runs four times per segment host. You can optionally limit the number of segment instances that execute the web table command. All segments included in the web table definition in the `ON` clause run the command in parallel.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/datamgmt/load/g-example-hawq-file-server-gpfdist.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/datamgmt/load/g-example-hawq-file-server-gpfdist.html.md.erb b/markdown/datamgmt/load/g-example-hawq-file-server-gpfdist.html.md.erb
index a0bf669..af57c44 100644
--- a/markdown/datamgmt/load/g-example-hawq-file-server-gpfdist.html.md.erb
+++ b/markdown/datamgmt/load/g-example-hawq-file-server-gpfdist.html.md.erb
@@ -2,6 +2,25 @@
title: Example - HAWQ file server (gpfdist)
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
``` sql
=# CREATE WRITABLE EXTERNAL TABLE unload_expenses
( LIKE expenses )
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/datamgmt/load/g-example-irs-mef-xml-files-in-demo-directory.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/datamgmt/load/g-example-irs-mef-xml-files-in-demo-directory.html.md.erb b/markdown/datamgmt/load/g-example-irs-mef-xml-files-in-demo-directory.html.md.erb
index 6f5b9e3..df225b2 100644
--- a/markdown/datamgmt/load/g-example-irs-mef-xml-files-in-demo-directory.html.md.erb
+++ b/markdown/datamgmt/load/g-example-irs-mef-xml-files-in-demo-directory.html.md.erb
@@ -2,6 +2,25 @@
title: Example using IRS MeF XML Files (In demo Directory)
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
This example demonstrates loading a sample IRS Modernized eFile tax return using a Joost STX transformation. The data is in the form of a complex XML file.
The U.S. Internal Revenue Service (IRS) made a significant commitment to XML and specifies its use in its Modernized e-File (MeF) system. In MeF, each tax return is an XML document with a deep hierarchical structure that closely reflects the particular form of the underlying tax code.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/datamgmt/load/g-example-witsml-files-in-demo-directory.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/datamgmt/load/g-example-witsml-files-in-demo-directory.html.md.erb b/markdown/datamgmt/load/g-example-witsml-files-in-demo-directory.html.md.erb
index 0484523..e5de2b5 100644
--- a/markdown/datamgmt/load/g-example-witsml-files-in-demo-directory.html.md.erb
+++ b/markdown/datamgmt/load/g-example-witsml-files-in-demo-directory.html.md.erb
@@ -2,6 +2,25 @@
title: Example using WITSML\u2122 Files (In demo Directory)
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
This example demonstrates loading sample data describing an oil rig using a Joost STX transformation. The data is in the form of a complex XML file downloaded from energistics.org.
The Wellsite Information Transfer Standard Markup Language (WITSML\u2122) is an oil industry initiative to provide open, non-proprietary, standard interfaces for technology and software to share information among oil companies, service companies, drilling contractors, application vendors, and regulatory agencies. For more information about WITSML\u2122, see [http://www.witsml.org](http://www.witsml.org).
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/datamgmt/load/g-examples-read-fixed-width-data.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/datamgmt/load/g-examples-read-fixed-width-data.html.md.erb b/markdown/datamgmt/load/g-examples-read-fixed-width-data.html.md.erb
index 174529a..4303bad 100644
--- a/markdown/datamgmt/load/g-examples-read-fixed-width-data.html.md.erb
+++ b/markdown/datamgmt/load/g-examples-read-fixed-width-data.html.md.erb
@@ -2,6 +2,25 @@
title: Examples - Read Fixed-Width Data
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The following examples show how to read fixed-width data.
## Example 1 \u2013 Loading a table with PRESERVED\_BLANKS on
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/datamgmt/load/g-external-tables.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/datamgmt/load/g-external-tables.html.md.erb b/markdown/datamgmt/load/g-external-tables.html.md.erb
index 4142a07..e833756 100644
--- a/markdown/datamgmt/load/g-external-tables.html.md.erb
+++ b/markdown/datamgmt/load/g-external-tables.html.md.erb
@@ -2,6 +2,25 @@
title: Accessing File-Based External Tables
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
External tables enable accessing external files as if they are regular database tables. They are often used to move data into and out of a HAWQ database.
To create an external table definition, you specify the format of your input files and the location of your external data sources. For information input file formats, see [Formatting Data Files](g-formatting-data-files.html#topic95).
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/datamgmt/load/g-formatting-columns.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/datamgmt/load/g-formatting-columns.html.md.erb b/markdown/datamgmt/load/g-formatting-columns.html.md.erb
index b828212..6987fe2 100644
--- a/markdown/datamgmt/load/g-formatting-columns.html.md.erb
+++ b/markdown/datamgmt/load/g-formatting-columns.html.md.erb
@@ -2,6 +2,25 @@
title: Formatting Columns
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The default column or field delimiter is the horizontal `TAB` character (`0x09`) for text files and the comma character (`0x2C`) for CSV files. You can declare a single character delimiter using the `DELIMITER` clause of `COPY`, `CREATE EXTERNAL TABLE` or the `hawq load` configuration table when you define your data format. The delimiter character must appear between any two data value fields. Do not place a delimiter at the beginning or end of a row. For example, if the pipe character ( | ) is your delimiter:
``` pre
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/datamgmt/load/g-formatting-data-files.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/datamgmt/load/g-formatting-data-files.html.md.erb b/markdown/datamgmt/load/g-formatting-data-files.html.md.erb
index 6c929ad..28ffdfe 100644
--- a/markdown/datamgmt/load/g-formatting-data-files.html.md.erb
+++ b/markdown/datamgmt/load/g-formatting-data-files.html.md.erb
@@ -2,6 +2,25 @@
title: Formatting Data Files
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
When you use the HAWQ tools for loading and unloading data, you must specify how your data is formatted. `COPY`, `CREATE EXTERNAL TABLE, `and `hawq load` have clauses that allow you to specify how your data is formatted. Data can be delimited text (`TEXT`) or comma separated values (`CSV`) format. External data must be formatted correctly to be read by HAWQ. This topic explains the format of data files expected by HAWQ.
- **[Formatting Rows](../../datamgmt/load/g-formatting-rows.html)**
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/datamgmt/load/g-formatting-rows.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/datamgmt/load/g-formatting-rows.html.md.erb b/markdown/datamgmt/load/g-formatting-rows.html.md.erb
index ea9b416..0fac00c 100644
--- a/markdown/datamgmt/load/g-formatting-rows.html.md.erb
+++ b/markdown/datamgmt/load/g-formatting-rows.html.md.erb
@@ -2,6 +2,25 @@
title: Formatting Rows
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
HAWQ expects rows of data to be separated by the `LF` character (Line feed, `0x0A`), `CR` (Carriage return, `0x0D`), or `CR` followed by `LF` (`CR+LF`, `0x0D 0x0A`). `LF` is the standard newline representation on UNIX or UNIX-like operating systems. Operating systems such as Windows or Mac OS X use `CR` or `CR+LF`. All of these representations of a newline are supported by HAWQ as a row delimiter. For more information, see [Importing and Exporting Fixed Width Data](g-importing-and-exporting-fixed-width-data.html#topic37).
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/datamgmt/load/g-gpfdist-protocol.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/datamgmt/load/g-gpfdist-protocol.html.md.erb b/markdown/datamgmt/load/g-gpfdist-protocol.html.md.erb
index ee98609..0e8d8e2 100644
--- a/markdown/datamgmt/load/g-gpfdist-protocol.html.md.erb
+++ b/markdown/datamgmt/load/g-gpfdist-protocol.html.md.erb
@@ -2,6 +2,25 @@
title: gpfdist Protocol
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The `gpfdist://` protocol is used in a URI to reference a running `gpfdist` instance. The `gpfdist` utility serves external data files from a directory on a file host to all HAWQ segments in parallel.
`gpfdist` is located in the `$GPHOME/bin` directory on your HAWQ master host and on each segment host.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/datamgmt/load/g-gpfdists-protocol.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/datamgmt/load/g-gpfdists-protocol.html.md.erb b/markdown/datamgmt/load/g-gpfdists-protocol.html.md.erb
index 2f5641d..17f1e00 100644
--- a/markdown/datamgmt/load/g-gpfdists-protocol.html.md.erb
+++ b/markdown/datamgmt/load/g-gpfdists-protocol.html.md.erb
@@ -2,6 +2,25 @@
title: gpfdists Protocol
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The `gpfdists://` protocol is a secure version of the `gpfdist:// protocol`. To use it, you run the `gpfdist` utility with the `--ssl` option. When specified in a URI, the `gpfdists://` protocol enables encrypted communication and secure identification of the file server and the HAWQ to protect against attacks such as eavesdropping and man-in-the-middle attacks.
`gpfdists` implements SSL security in a client/server scheme with the following attributes and limitations:
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/datamgmt/load/g-handling-errors-ext-table-data.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/datamgmt/load/g-handling-errors-ext-table-data.html.md.erb b/markdown/datamgmt/load/g-handling-errors-ext-table-data.html.md.erb
index 2b8dc78..32e786f 100644
--- a/markdown/datamgmt/load/g-handling-errors-ext-table-data.html.md.erb
+++ b/markdown/datamgmt/load/g-handling-errors-ext-table-data.html.md.erb
@@ -2,6 +2,25 @@
title: Handling Errors in External Table Data
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
By default, if external table data contains an error, the command fails and no data loads into the target database table. Define the external table with single row error handling to enable loading correctly formatted rows and to isolate data errors in external table data. See [Handling Load Errors](g-handling-load-errors.html#topic55).
The `gpfdist` file server uses the `HTTP` protocol. External table queries that use `LIMIT` end the connection after retrieving the rows, causing an HTTP socket error. If you use `LIMIT` in queries of external tables that use the `gpfdist://` or `http:// protocols`, ignore these errors \u2013 data is returned to the database as expected.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/datamgmt/load/g-handling-load-errors.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/datamgmt/load/g-handling-load-errors.html.md.erb b/markdown/datamgmt/load/g-handling-load-errors.html.md.erb
index 6faf7a5..3068889 100644
--- a/markdown/datamgmt/load/g-handling-load-errors.html.md.erb
+++ b/markdown/datamgmt/load/g-handling-load-errors.html.md.erb
@@ -2,6 +2,25 @@
title: Handling Load Errors
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Readable external tables are most commonly used to select data to load into regular database tables. You use the `CREATE TABLE AS SELECT` or `INSERT INTO `commands to query the external table data. By default, if the data contains an error, the entire command fails and the data is not loaded into the target database table.
The `SEGMENT REJECT LIMIT` clause allows you to isolate format errors in external table data and to continue loading correctly formatted rows. Use `SEGMENT REJECT LIMIT `to set an error threshold, specifying the reject limit `count` as number of `ROWS` (the default) or as a `PERCENT` of total rows (1-100).
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/datamgmt/load/g-identifying-invalid-csv-files-in-error-table-data.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/datamgmt/load/g-identifying-invalid-csv-files-in-error-table-data.html.md.erb b/markdown/datamgmt/load/g-identifying-invalid-csv-files-in-error-table-data.html.md.erb
index 534d530..f13451f 100644
--- a/markdown/datamgmt/load/g-identifying-invalid-csv-files-in-error-table-data.html.md.erb
+++ b/markdown/datamgmt/load/g-identifying-invalid-csv-files-in-error-table-data.html.md.erb
@@ -2,6 +2,25 @@
title: Identifying Invalid CSV Files in Error Table Data
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
If a CSV file contains invalid formatting, the *rawdata* field in the error table can contain several combined rows. For example, if a closing quote for a specific field is missing, all the following newlines are treated as embedded newlines. When this happens, HAWQ stops parsing a row when it reaches 64K, puts that 64K of data into the error table as a single row, resets the quote flag, and continues. If this happens three times during load processing, the load file is considered invalid and the entire load fails with the message "`rejected ` `N` ` or more rows`". See [Escaping in CSV Formatted Files](g-escaping-in-csv-formatted-files.html#topic101) for more information on the correct use of quotes in CSV files.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/datamgmt/load/g-importing-and-exporting-fixed-width-data.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/datamgmt/load/g-importing-and-exporting-fixed-width-data.html.md.erb b/markdown/datamgmt/load/g-importing-and-exporting-fixed-width-data.html.md.erb
index f49cae0..8ac2797 100644
--- a/markdown/datamgmt/load/g-importing-and-exporting-fixed-width-data.html.md.erb
+++ b/markdown/datamgmt/load/g-importing-and-exporting-fixed-width-data.html.md.erb
@@ -2,6 +2,25 @@
title: Importing and Exporting Fixed Width Data
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Specify custom formats for fixed-width data with the HAWQ functions `fixedwith_in` and `fixedwidth_out`. These functions already exist in the file `$GPHOME/share/postgresql/cdb_external_extensions.sql`. The following example declares a custom format, then calls the `fixedwidth_in` function to format the data.
``` sql
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/datamgmt/load/g-installing-gpfdist.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/datamgmt/load/g-installing-gpfdist.html.md.erb b/markdown/datamgmt/load/g-installing-gpfdist.html.md.erb
index 85549df..704b392 100644
--- a/markdown/datamgmt/load/g-installing-gpfdist.html.md.erb
+++ b/markdown/datamgmt/load/g-installing-gpfdist.html.md.erb
@@ -2,6 +2,25 @@
title: Installing gpfdist
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
You may choose to run `gpfdist` from a machine other than the HAWQ master, such as on a machine devoted to ETL processing. To install `gpfdist` on your ETL server, refer to [Client-Based HAWQ Load Tools](client-loadtools.html) for information related to Linux and Windows load tools installation and configuration.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/datamgmt/load/g-load-the-data.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/datamgmt/load/g-load-the-data.html.md.erb b/markdown/datamgmt/load/g-load-the-data.html.md.erb
index 4c88c9f..51c7184 100644
--- a/markdown/datamgmt/load/g-load-the-data.html.md.erb
+++ b/markdown/datamgmt/load/g-load-the-data.html.md.erb
@@ -2,6 +2,25 @@
title: Load the Data
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Create the tables with SQL statements based on the appropriate schema.
There are no special requirements for the HAWQ tables that hold loaded data. In the prices example, the following command creates the appropriate table.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/datamgmt/load/g-loading-and-unloading-data.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/datamgmt/load/g-loading-and-unloading-data.html.md.erb b/markdown/datamgmt/load/g-loading-and-unloading-data.html.md.erb
index 8ea43d5..e687137 100644
--- a/markdown/datamgmt/load/g-loading-and-unloading-data.html.md.erb
+++ b/markdown/datamgmt/load/g-loading-and-unloading-data.html.md.erb
@@ -2,6 +2,25 @@
title: Loading and Unloading Data
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The topics in this section describe methods for loading and writing data into and out of HAWQ, and how to format data files. It also covers registering HDFS files and folders directly into HAWQ internal tables.
HAWQ supports high-performance parallel data loading and unloading, and for smaller amounts of data, single file, non-parallel data import and export.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/datamgmt/load/g-loading-and-writing-non-hdfs-custom-data.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/datamgmt/load/g-loading-and-writing-non-hdfs-custom-data.html.md.erb b/markdown/datamgmt/load/g-loading-and-writing-non-hdfs-custom-data.html.md.erb
index e826963..d21ad32 100644
--- a/markdown/datamgmt/load/g-loading-and-writing-non-hdfs-custom-data.html.md.erb
+++ b/markdown/datamgmt/load/g-loading-and-writing-non-hdfs-custom-data.html.md.erb
@@ -2,6 +2,25 @@
title: Loading and Writing Non-HDFS Custom Data
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
HAWQ supports `TEXT` and `CSV` formats for importing and exporting data. You can load and write the data in other formats by defining and using a custom format or custom protocol.
- **[Using a Custom Format](../../datamgmt/load/g-using-a-custom-format.html)**
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/datamgmt/load/g-loading-data-using-an-external-table.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/datamgmt/load/g-loading-data-using-an-external-table.html.md.erb b/markdown/datamgmt/load/g-loading-data-using-an-external-table.html.md.erb
index 32a741a..bd80880 100644
--- a/markdown/datamgmt/load/g-loading-data-using-an-external-table.html.md.erb
+++ b/markdown/datamgmt/load/g-loading-data-using-an-external-table.html.md.erb
@@ -2,6 +2,25 @@
title: Loading Data Using an External Table
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Use SQL commands such as `INSERT` and `SELECT` to query a readable external table, the same way that you query a regular database table. For example, to load travel expense data from an external table, `ext_expenses`, into a database table,` expenses_travel`:
``` sql
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/datamgmt/load/g-loading-data-with-copy.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/datamgmt/load/g-loading-data-with-copy.html.md.erb b/markdown/datamgmt/load/g-loading-data-with-copy.html.md.erb
index 72e5ac6..e47017b 100644
--- a/markdown/datamgmt/load/g-loading-data-with-copy.html.md.erb
+++ b/markdown/datamgmt/load/g-loading-data-with-copy.html.md.erb
@@ -2,6 +2,25 @@
title: Loading Data with COPY
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
`COPY FROM` copies data from a file or standard input into a table and appends the data to the table contents. `COPY` is non-parallel: data is loaded in a single process using the HAWQ master instance. Using `COPY` is only recommended for very small data files.
The `COPY` source file must be accessible to the master host. Specify the `COPY` source file name relative to the master host location.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/datamgmt/load/g-loading-data-with-hawqload.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/datamgmt/load/g-loading-data-with-hawqload.html.md.erb b/markdown/datamgmt/load/g-loading-data-with-hawqload.html.md.erb
index 68e4459..9fa4f1c 100644
--- a/markdown/datamgmt/load/g-loading-data-with-hawqload.html.md.erb
+++ b/markdown/datamgmt/load/g-loading-data-with-hawqload.html.md.erb
@@ -2,6 +2,25 @@
title: Loading Data with hawq load
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The HAWQ `hawq load` utility loads data using readable external tables and the HAWQ parallel file server ( `gpfdist` or `gpfdists`). It handles parallel file-based external table setup and allows users to configure their data format, external table definition, and `gpfdist` or `gpfdists` setup in a single configuration file.
## <a id="topic62__du168147"></a>To use hawq load
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/datamgmt/load/g-moving-data-between-tables.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/datamgmt/load/g-moving-data-between-tables.html.md.erb b/markdown/datamgmt/load/g-moving-data-between-tables.html.md.erb
index 2603ae4..bba28cc 100644
--- a/markdown/datamgmt/load/g-moving-data-between-tables.html.md.erb
+++ b/markdown/datamgmt/load/g-moving-data-between-tables.html.md.erb
@@ -2,6 +2,25 @@
title: Moving Data between Tables
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
You can use `CREATE TABLE AS` or `INSERT...SELECT` to load external and web external table data into another (non-external) database table, and the data will be loaded in parallel according to the external or web external table definition.
If an external table file or web external table data source has an error, one of the following will happen, depending on the isolation mode used:
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/datamgmt/load/g-optimizing-data-load-and-query-performance.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/datamgmt/load/g-optimizing-data-load-and-query-performance.html.md.erb b/markdown/datamgmt/load/g-optimizing-data-load-and-query-performance.html.md.erb
index ff1c230..96083a5 100644
--- a/markdown/datamgmt/load/g-optimizing-data-load-and-query-performance.html.md.erb
+++ b/markdown/datamgmt/load/g-optimizing-data-load-and-query-performance.html.md.erb
@@ -2,6 +2,25 @@
title: Optimizing Data Load and Query Performance
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Use the following tip to help optimize your data load and subsequent query performance.
- Run `ANALYZE` after loading data. If you significantly altered the data in a table, run `ANALYZE` or `VACUUM ANALYZE` (system catalog tables only) to update table statistics for the query optimizer. Current statistics ensure that the optimizer makes the best decisions during query planning and avoids poor performance due to inaccurate or nonexistent statistics.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/datamgmt/load/g-register_files.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/datamgmt/load/g-register_files.html.md.erb b/markdown/datamgmt/load/g-register_files.html.md.erb
index 25c24ca..c3a70be 100644
--- a/markdown/datamgmt/load/g-register_files.html.md.erb
+++ b/markdown/datamgmt/load/g-register_files.html.md.erb
@@ -2,6 +2,25 @@
title: Registering Files into HAWQ Internal Tables
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The `hawq register` utility loads and registers HDFS data files or folders into HAWQ internal tables. Files can be read directly, rather than having to be copied or loaded, resulting in higher performance and more efficient transaction processing.
Data from the file or directory specified by \<hdfsfilepath\> is loaded into the appropriate HAWQ table directory in HDFS and the utility updates the corresponding HAWQ metadata for the files. Either AO or Parquet-formatted tables in HDFS can be loaded into a corresponding table in HAWQ.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/datamgmt/load/g-representing-null-values.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/datamgmt/load/g-representing-null-values.html.md.erb b/markdown/datamgmt/load/g-representing-null-values.html.md.erb
index 4d4ffdd..a6ca89d 100644
--- a/markdown/datamgmt/load/g-representing-null-values.html.md.erb
+++ b/markdown/datamgmt/load/g-representing-null-values.html.md.erb
@@ -2,6 +2,25 @@
title: Representing NULL Values
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
`NULL` represents an unknown piece of data in a column or field. Within your data files you can designate a string to represent null values. The default string is `\N` (backslash-N) in `TEXT` mode, or an empty value with no quotations in `CSV` mode. You can also declare a different string using the `NULL` clause of `COPY`, `CREATE EXTERNAL TABLE `or the `hawq load` control file when defining your data format. For example, you can use an empty string if you do not want to distinguish nulls from empty strings. When using the HAWQ loading tools, any data item that matches the designated null string is considered a null value.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/datamgmt/load/g-running-copy-in-single-row-error-isolation-mode.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/datamgmt/load/g-running-copy-in-single-row-error-isolation-mode.html.md.erb b/markdown/datamgmt/load/g-running-copy-in-single-row-error-isolation-mode.html.md.erb
index ba0603c..d08231f 100644
--- a/markdown/datamgmt/load/g-running-copy-in-single-row-error-isolation-mode.html.md.erb
+++ b/markdown/datamgmt/load/g-running-copy-in-single-row-error-isolation-mode.html.md.erb
@@ -2,6 +2,25 @@
title: Running COPY in Single Row Error Isolation Mode
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
By default, `COPY` stops an operation at the first error: if the data contains an error, the operation fails and no data loads. If you run `COPY FROM` in *single row error isolation mode*, HAWQ skips rows that contain format errors and loads properly formatted rows. Single row error isolation mode applies only to rows in the input file that contain format errors. If the data contains a contraint error such as violation of a `NOT NULL` or `CHECK` constraint, the operation fails and no data loads.
Specifying `SEGMENT REJECT LIMIT` runs the `COPY` operation in single row error isolation mode. Specify the acceptable number of error rows on each segment, after which the entire `COPY FROM` operation fails and no rows load. The error row count is for each HAWQ segment, not for the entire load operation.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/datamgmt/load/g-starting-and-stopping-gpfdist.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/datamgmt/load/g-starting-and-stopping-gpfdist.html.md.erb b/markdown/datamgmt/load/g-starting-and-stopping-gpfdist.html.md.erb
index 7e2cca9..7d068fa 100644
--- a/markdown/datamgmt/load/g-starting-and-stopping-gpfdist.html.md.erb
+++ b/markdown/datamgmt/load/g-starting-and-stopping-gpfdist.html.md.erb
@@ -2,6 +2,25 @@
title: Starting and Stopping gpfdist
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
You can start `gpfdist` in your current directory location or in any directory that you specify. The default port is `8080`.
From your current directory, type:
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/datamgmt/load/g-transfer-and-store-the-data.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/datamgmt/load/g-transfer-and-store-the-data.html.md.erb b/markdown/datamgmt/load/g-transfer-and-store-the-data.html.md.erb
index 8a6d7ab..3155839 100644
--- a/markdown/datamgmt/load/g-transfer-and-store-the-data.html.md.erb
+++ b/markdown/datamgmt/load/g-transfer-and-store-the-data.html.md.erb
@@ -2,6 +2,25 @@
title: Transfer and Store the Data
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Use one of the following approaches to transform the data with `gpfdist`.
- `GPLOAD` supports only input transformations, but is easier to implement in many cases.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/datamgmt/load/g-transforming-with-gpload.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/datamgmt/load/g-transforming-with-gpload.html.md.erb b/markdown/datamgmt/load/g-transforming-with-gpload.html.md.erb
index 438fedb..4c2e146 100644
--- a/markdown/datamgmt/load/g-transforming-with-gpload.html.md.erb
+++ b/markdown/datamgmt/load/g-transforming-with-gpload.html.md.erb
@@ -2,6 +2,25 @@
title: Transforming with GPLOAD
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
To transform data using the `GPLOAD ` control file, you must specify both the file name for the `TRANSFORM_CONFIG` file and the name of the `TRANSFORM` operation in the `INPUT` section of the `GPLOAD` control file.
- `TRANSFORM_CONFIG `specifies the name of the `gpfdist` configuration file.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/datamgmt/load/g-transforming-with-insert-into-select-from.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/datamgmt/load/g-transforming-with-insert-into-select-from.html.md.erb b/markdown/datamgmt/load/g-transforming-with-insert-into-select-from.html.md.erb
index d91cc93..342c907 100644
--- a/markdown/datamgmt/load/g-transforming-with-insert-into-select-from.html.md.erb
+++ b/markdown/datamgmt/load/g-transforming-with-insert-into-select-from.html.md.erb
@@ -2,6 +2,25 @@
title: Transforming with INSERT INTO SELECT FROM
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Specify the transformation in the `CREATE EXTERNAL TABLE` definition's `LOCATION` clause. For example, the transform is shown in bold in the following command. (Run `gpfdist` first, using the command `gpfdist -c config.yaml`).
``` sql
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/datamgmt/load/g-transforming-xml-data.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/datamgmt/load/g-transforming-xml-data.html.md.erb b/markdown/datamgmt/load/g-transforming-xml-data.html.md.erb
index f9520bb..ca183ae 100644
--- a/markdown/datamgmt/load/g-transforming-xml-data.html.md.erb
+++ b/markdown/datamgmt/load/g-transforming-xml-data.html.md.erb
@@ -2,6 +2,25 @@
title: Transforming XML Data
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The HAWQ data loader *gpfdist* provides transformation features to load XML data into a table and to write data from the HAWQ to XML files. The following diagram shows *gpfdist* performing an XML transform.
<a id="topic75__du185408"></a>
[04/18] incubator-hawq-docs git commit: HAWQ-1305 Add apache license
header to doc source
Posted by yo...@apache.org.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/cli/client_utilities/pg_dump.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/cli/client_utilities/pg_dump.html.md.erb b/markdown/reference/cli/client_utilities/pg_dump.html.md.erb
index d4a3186..43cab84 100644
--- a/markdown/reference/cli/client_utilities/pg_dump.html.md.erb
+++ b/markdown/reference/cli/client_utilities/pg_dump.html.md.erb
@@ -2,6 +2,25 @@
title: pg_dump
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Extracts a database into a single script file or other archive file.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/cli/client_utilities/pg_dumpall.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/cli/client_utilities/pg_dumpall.html.md.erb b/markdown/reference/cli/client_utilities/pg_dumpall.html.md.erb
index 255b459..87f39ac 100644
--- a/markdown/reference/cli/client_utilities/pg_dumpall.html.md.erb
+++ b/markdown/reference/cli/client_utilities/pg_dumpall.html.md.erb
@@ -2,6 +2,25 @@
title: pg_dumpall
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Extracts all databases in a HAWQ system to a single script file or other archive file.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/cli/client_utilities/pg_restore.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/cli/client_utilities/pg_restore.html.md.erb b/markdown/reference/cli/client_utilities/pg_restore.html.md.erb
index e612282..c4a8495 100644
--- a/markdown/reference/cli/client_utilities/pg_restore.html.md.erb
+++ b/markdown/reference/cli/client_utilities/pg_restore.html.md.erb
@@ -2,6 +2,25 @@
title: pg_restore
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Restores a database from an archive file created by `pg_dump`.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/cli/client_utilities/psql.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/cli/client_utilities/psql.html.md.erb b/markdown/reference/cli/client_utilities/psql.html.md.erb
index ee245e6..28a1834 100644
--- a/markdown/reference/cli/client_utilities/psql.html.md.erb
+++ b/markdown/reference/cli/client_utilities/psql.html.md.erb
@@ -2,6 +2,25 @@
title: psql
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Interactive command-line interface for HAWQ.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/cli/client_utilities/vacuumdb.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/cli/client_utilities/vacuumdb.html.md.erb b/markdown/reference/cli/client_utilities/vacuumdb.html.md.erb
index cbc37f3..870613e 100644
--- a/markdown/reference/cli/client_utilities/vacuumdb.html.md.erb
+++ b/markdown/reference/cli/client_utilities/vacuumdb.html.md.erb
@@ -2,6 +2,25 @@
title: vacuumdb
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Garbage-collects and analyzes a database.
`vacuumdb` is typically run on system catalog tables. It has no effect when run on HAWQ user tables.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/cli/management_tools.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/cli/management_tools.html.md.erb b/markdown/reference/cli/management_tools.html.md.erb
index bbc4e3e..183c7f4 100644
--- a/markdown/reference/cli/management_tools.html.md.erb
+++ b/markdown/reference/cli/management_tools.html.md.erb
@@ -2,6 +2,25 @@
title: HAWQ Management Tools Reference
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Reference information for command-line utilities available in HAWQ.
- **[analyzedb](../../reference/cli/admin_utilities/analyzedb.html)**
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/guc/guc_category-list.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/guc/guc_category-list.html.md.erb b/markdown/reference/guc/guc_category-list.html.md.erb
index 896df99..e9f870a 100644
--- a/markdown/reference/guc/guc_category-list.html.md.erb
+++ b/markdown/reference/guc/guc_category-list.html.md.erb
@@ -2,6 +2,25 @@
title: Configuration Parameter Categories
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Configuration parameters affect categories of server behaviors, such as resource consumption, query tuning, and authentication. The following sections describe HAWQ configuration parameter categories.
## <a id="topic_hfd_1tl_zp"></a>Append-Only Table Parameters
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/guc/guc_config.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/guc/guc_config.html.md.erb b/markdown/reference/guc/guc_config.html.md.erb
index 56a2456..c128e23 100644
--- a/markdown/reference/guc/guc_config.html.md.erb
+++ b/markdown/reference/guc/guc_config.html.md.erb
@@ -2,6 +2,25 @@
title: About Server Configuration Parameters
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
There are many HAWQ server configuration parameters that affect the behavior of the HAWQ system. Many of these configuration parameters have the same names, settings, and behaviors as in a regular PostgreSQL database system.
- [Parameter Types and Values](#topic_vsn_22l_z4) describes the parameter data types and values.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/guc/parameter_definitions.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/guc/parameter_definitions.html.md.erb b/markdown/reference/guc/parameter_definitions.html.md.erb
index f3f5757..6d88075 100644
--- a/markdown/reference/guc/parameter_definitions.html.md.erb
+++ b/markdown/reference/guc/parameter_definitions.html.md.erb
@@ -2,6 +2,25 @@
title: Configuration Parameters
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Descriptions of the HAWQ server configuration parameters listed alphabetically.
- **[add\_missing\_from](../../reference/guc/parameter_definitions.html#add_missing_from)**
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/hawq-reference.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/hawq-reference.html.md.erb b/markdown/reference/hawq-reference.html.md.erb
index f5abd2a..ef9e0a2 100644
--- a/markdown/reference/hawq-reference.html.md.erb
+++ b/markdown/reference/hawq-reference.html.md.erb
@@ -2,6 +2,25 @@
title: HAWQ Reference
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
This section provides a complete reference to HAWQ SQL commands, management utilities, configuration parameters, environment variables, and database objects.
- **[Server Configuration Parameter Reference](../reference/HAWQSiteConfig.html)**
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/sql/ABORT.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/sql/ABORT.html.md.erb b/markdown/reference/sql/ABORT.html.md.erb
index ab053d8..3fa0d2c 100644
--- a/markdown/reference/sql/ABORT.html.md.erb
+++ b/markdown/reference/sql/ABORT.html.md.erb
@@ -2,6 +2,25 @@
title: ABORT
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Aborts the current transaction.
## <a id="synop"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/sql/ALTER-AGGREGATE.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/sql/ALTER-AGGREGATE.html.md.erb b/markdown/reference/sql/ALTER-AGGREGATE.html.md.erb
index b1131ef..af9dc61 100644
--- a/markdown/reference/sql/ALTER-AGGREGATE.html.md.erb
+++ b/markdown/reference/sql/ALTER-AGGREGATE.html.md.erb
@@ -2,6 +2,25 @@
title: ALTER AGGREGATE
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Changes the definition of an aggregate function.
## <a id="synop"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/sql/ALTER-CONVERSION.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/sql/ALTER-CONVERSION.html.md.erb b/markdown/reference/sql/ALTER-CONVERSION.html.md.erb
index 5b02ffc..13459f3 100644
--- a/markdown/reference/sql/ALTER-CONVERSION.html.md.erb
+++ b/markdown/reference/sql/ALTER-CONVERSION.html.md.erb
@@ -2,6 +2,25 @@
title: ALTER CONVERSION
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Changes the definition of a CONVERSION.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/sql/ALTER-DATABASE.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/sql/ALTER-DATABASE.html.md.erb b/markdown/reference/sql/ALTER-DATABASE.html.md.erb
index 782daf5..2760397 100644
--- a/markdown/reference/sql/ALTER-DATABASE.html.md.erb
+++ b/markdown/reference/sql/ALTER-DATABASE.html.md.erb
@@ -2,6 +2,25 @@
title: ALTER DATABASE
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Changes the attributes of a database.
## <a id="alterrole__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/sql/ALTER-FUNCTION.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/sql/ALTER-FUNCTION.html.md.erb b/markdown/reference/sql/ALTER-FUNCTION.html.md.erb
index f21a808..e9a7b5b 100644
--- a/markdown/reference/sql/ALTER-FUNCTION.html.md.erb
+++ b/markdown/reference/sql/ALTER-FUNCTION.html.md.erb
@@ -2,6 +2,25 @@
title: ALTER FUNCTION
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Changes the definition of a function.
## <a id="alterfunction__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/sql/ALTER-OPERATOR-CLASS.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/sql/ALTER-OPERATOR-CLASS.html.md.erb b/markdown/reference/sql/ALTER-OPERATOR-CLASS.html.md.erb
index 1d2878e..37d7d51 100644
--- a/markdown/reference/sql/ALTER-OPERATOR-CLASS.html.md.erb
+++ b/markdown/reference/sql/ALTER-OPERATOR-CLASS.html.md.erb
@@ -2,6 +2,25 @@
title: ALTER OPERATOR CLASS
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Changes the definition of an operator class.
## <a id="synop"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/sql/ALTER-OPERATOR.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/sql/ALTER-OPERATOR.html.md.erb b/markdown/reference/sql/ALTER-OPERATOR.html.md.erb
index a63d838..bd34eac 100644
--- a/markdown/reference/sql/ALTER-OPERATOR.html.md.erb
+++ b/markdown/reference/sql/ALTER-OPERATOR.html.md.erb
@@ -2,6 +2,25 @@
title: ALTER OPERATOR
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Changes the definition of an operator.
## <a id="synop"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/sql/ALTER-RESOURCE-QUEUE.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/sql/ALTER-RESOURCE-QUEUE.html.md.erb b/markdown/reference/sql/ALTER-RESOURCE-QUEUE.html.md.erb
index ec051e8..c8b650d 100644
--- a/markdown/reference/sql/ALTER-RESOURCE-QUEUE.html.md.erb
+++ b/markdown/reference/sql/ALTER-RESOURCE-QUEUE.html.md.erb
@@ -2,6 +2,25 @@
title: ALTER RESOURCE QUEUE
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Modify an existing resource queue.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/sql/ALTER-ROLE.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/sql/ALTER-ROLE.html.md.erb b/markdown/reference/sql/ALTER-ROLE.html.md.erb
index ccc2c28..ab933b4 100644
--- a/markdown/reference/sql/ALTER-ROLE.html.md.erb
+++ b/markdown/reference/sql/ALTER-ROLE.html.md.erb
@@ -2,6 +2,25 @@
title: ALTER ROLE
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Changes a database role (user or group).
## <a id="alterrole__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/sql/ALTER-TABLE.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/sql/ALTER-TABLE.html.md.erb b/markdown/reference/sql/ALTER-TABLE.html.md.erb
index 4303f0c..2e320c2 100644
--- a/markdown/reference/sql/ALTER-TABLE.html.md.erb
+++ b/markdown/reference/sql/ALTER-TABLE.html.md.erb
@@ -2,6 +2,25 @@
title: ALTER TABLE
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Changes the definition of a table.
## <a id="altertable__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/sql/ALTER-TABLESPACE.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/sql/ALTER-TABLESPACE.html.md.erb b/markdown/reference/sql/ALTER-TABLESPACE.html.md.erb
index e539177..e805268 100644
--- a/markdown/reference/sql/ALTER-TABLESPACE.html.md.erb
+++ b/markdown/reference/sql/ALTER-TABLESPACE.html.md.erb
@@ -2,6 +2,25 @@
title: ALTER TABLESPACE
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Changes the definition of a tablespace.
## <a id="synopsis"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/sql/ALTER-TYPE.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/sql/ALTER-TYPE.html.md.erb b/markdown/reference/sql/ALTER-TYPE.html.md.erb
index da50e80..2350b81 100644
--- a/markdown/reference/sql/ALTER-TYPE.html.md.erb
+++ b/markdown/reference/sql/ALTER-TYPE.html.md.erb
@@ -2,6 +2,25 @@
title: ALTER TYPE
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Changes the definition of a data type.
## <a id="synopsis"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/sql/ALTER-USER.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/sql/ALTER-USER.html.md.erb b/markdown/reference/sql/ALTER-USER.html.md.erb
index f53e788..500c5f3 100644
--- a/markdown/reference/sql/ALTER-USER.html.md.erb
+++ b/markdown/reference/sql/ALTER-USER.html.md.erb
@@ -2,6 +2,25 @@
title: ALTER USER
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Changes the definition of a database role (user).
## <a id="alteruser__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/sql/ANALYZE.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/sql/ANALYZE.html.md.erb b/markdown/reference/sql/ANALYZE.html.md.erb
index 983696a..eb0af77 100644
--- a/markdown/reference/sql/ANALYZE.html.md.erb
+++ b/markdown/reference/sql/ANALYZE.html.md.erb
@@ -2,6 +2,25 @@
title: ANALYZE
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Collects statistics about a database.
## <a id="synopsis"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/sql/BEGIN.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/sql/BEGIN.html.md.erb b/markdown/reference/sql/BEGIN.html.md.erb
index 265e66e..944a87c 100644
--- a/markdown/reference/sql/BEGIN.html.md.erb
+++ b/markdown/reference/sql/BEGIN.html.md.erb
@@ -2,6 +2,25 @@
title: BEGIN
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Starts a transaction block.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/sql/CHECKPOINT.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/sql/CHECKPOINT.html.md.erb b/markdown/reference/sql/CHECKPOINT.html.md.erb
index d699013..cd40ebd 100644
--- a/markdown/reference/sql/CHECKPOINT.html.md.erb
+++ b/markdown/reference/sql/CHECKPOINT.html.md.erb
@@ -2,6 +2,25 @@
title: CHECKPOINT
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Forces a transaction log checkpoint.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/sql/CLOSE.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/sql/CLOSE.html.md.erb b/markdown/reference/sql/CLOSE.html.md.erb
index ae9c958..4f13a01 100644
--- a/markdown/reference/sql/CLOSE.html.md.erb
+++ b/markdown/reference/sql/CLOSE.html.md.erb
@@ -2,6 +2,25 @@
title: CLOSE
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Closes a cursor.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/sql/COMMIT.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/sql/COMMIT.html.md.erb b/markdown/reference/sql/COMMIT.html.md.erb
index dd91969..55be66d 100644
--- a/markdown/reference/sql/COMMIT.html.md.erb
+++ b/markdown/reference/sql/COMMIT.html.md.erb
@@ -2,6 +2,25 @@
title: COMMIT
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Commits the current transaction.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/sql/COPY.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/sql/COPY.html.md.erb b/markdown/reference/sql/COPY.html.md.erb
index aaa2270..de34dfb 100644
--- a/markdown/reference/sql/COPY.html.md.erb
+++ b/markdown/reference/sql/COPY.html.md.erb
@@ -2,6 +2,25 @@
title: COPY
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Copies data between a file and a table.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/sql/CREATE-AGGREGATE.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/sql/CREATE-AGGREGATE.html.md.erb b/markdown/reference/sql/CREATE-AGGREGATE.html.md.erb
index a195224..e8f5d24 100644
--- a/markdown/reference/sql/CREATE-AGGREGATE.html.md.erb
+++ b/markdown/reference/sql/CREATE-AGGREGATE.html.md.erb
@@ -2,6 +2,25 @@
title: CREATE AGGREGATE
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Defines a new aggregate function.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/sql/CREATE-CAST.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/sql/CREATE-CAST.html.md.erb b/markdown/reference/sql/CREATE-CAST.html.md.erb
index af35173..ec2ba04 100644
--- a/markdown/reference/sql/CREATE-CAST.html.md.erb
+++ b/markdown/reference/sql/CREATE-CAST.html.md.erb
@@ -2,6 +2,25 @@
title: CREATE CAST
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Defines a new cast.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/sql/CREATE-CONVERSION.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/sql/CREATE-CONVERSION.html.md.erb b/markdown/reference/sql/CREATE-CONVERSION.html.md.erb
index 5bc7910..92e1dda 100644
--- a/markdown/reference/sql/CREATE-CONVERSION.html.md.erb
+++ b/markdown/reference/sql/CREATE-CONVERSION.html.md.erb
@@ -2,6 +2,25 @@
title: CREATE CONVERSION
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Defines a new encoding conversion.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/sql/CREATE-DATABASE.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/sql/CREATE-DATABASE.html.md.erb b/markdown/reference/sql/CREATE-DATABASE.html.md.erb
index 1f5467e..3a8612a 100644
--- a/markdown/reference/sql/CREATE-DATABASE.html.md.erb
+++ b/markdown/reference/sql/CREATE-DATABASE.html.md.erb
@@ -2,6 +2,25 @@
title: CREATE DATABASE
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Creates a new database.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/sql/CREATE-EXTERNAL-TABLE.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/sql/CREATE-EXTERNAL-TABLE.html.md.erb b/markdown/reference/sql/CREATE-EXTERNAL-TABLE.html.md.erb
index 3479e3e..c46870c 100644
--- a/markdown/reference/sql/CREATE-EXTERNAL-TABLE.html.md.erb
+++ b/markdown/reference/sql/CREATE-EXTERNAL-TABLE.html.md.erb
@@ -2,6 +2,25 @@
title: CREATE EXTERNAL TABLE
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Defines a new external table.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/sql/CREATE-FUNCTION.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/sql/CREATE-FUNCTION.html.md.erb b/markdown/reference/sql/CREATE-FUNCTION.html.md.erb
index 6675752..467ad69 100644
--- a/markdown/reference/sql/CREATE-FUNCTION.html.md.erb
+++ b/markdown/reference/sql/CREATE-FUNCTION.html.md.erb
@@ -2,6 +2,25 @@
title: CREATE FUNCTION
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Defines a new function.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/sql/CREATE-GROUP.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/sql/CREATE-GROUP.html.md.erb b/markdown/reference/sql/CREATE-GROUP.html.md.erb
index 79cc6aa..81d09a3 100644
--- a/markdown/reference/sql/CREATE-GROUP.html.md.erb
+++ b/markdown/reference/sql/CREATE-GROUP.html.md.erb
@@ -2,6 +2,25 @@
title: CREATE GROUP
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Defines a new database role.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/sql/CREATE-LANGUAGE.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/sql/CREATE-LANGUAGE.html.md.erb b/markdown/reference/sql/CREATE-LANGUAGE.html.md.erb
index 6643fef..e12b05c 100644
--- a/markdown/reference/sql/CREATE-LANGUAGE.html.md.erb
+++ b/markdown/reference/sql/CREATE-LANGUAGE.html.md.erb
@@ -2,6 +2,25 @@
title: CREATE LANGUAGE
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Defines a new procedural language.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/sql/CREATE-OPERATOR-CLASS.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/sql/CREATE-OPERATOR-CLASS.html.md.erb b/markdown/reference/sql/CREATE-OPERATOR-CLASS.html.md.erb
index 9c093c1..fd4e59c 100644
--- a/markdown/reference/sql/CREATE-OPERATOR-CLASS.html.md.erb
+++ b/markdown/reference/sql/CREATE-OPERATOR-CLASS.html.md.erb
@@ -2,6 +2,25 @@
title: CREATE OPERATOR CLASS
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Defines a new operator class.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/sql/CREATE-OPERATOR.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/sql/CREATE-OPERATOR.html.md.erb b/markdown/reference/sql/CREATE-OPERATOR.html.md.erb
index 570d226..0622abb 100644
--- a/markdown/reference/sql/CREATE-OPERATOR.html.md.erb
+++ b/markdown/reference/sql/CREATE-OPERATOR.html.md.erb
@@ -2,6 +2,25 @@
title: CREATE OPERATOR
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Defines a new operator.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/sql/CREATE-RESOURCE-QUEUE.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/sql/CREATE-RESOURCE-QUEUE.html.md.erb b/markdown/reference/sql/CREATE-RESOURCE-QUEUE.html.md.erb
index 8f9fe93..4ebaf7a 100644
--- a/markdown/reference/sql/CREATE-RESOURCE-QUEUE.html.md.erb
+++ b/markdown/reference/sql/CREATE-RESOURCE-QUEUE.html.md.erb
@@ -2,6 +2,25 @@
title: CREATE RESOURCE QUEUE
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Defines a new resource queue.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/sql/CREATE-ROLE.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/sql/CREATE-ROLE.html.md.erb b/markdown/reference/sql/CREATE-ROLE.html.md.erb
index ec7ac7c..9125bd5 100644
--- a/markdown/reference/sql/CREATE-ROLE.html.md.erb
+++ b/markdown/reference/sql/CREATE-ROLE.html.md.erb
@@ -2,6 +2,25 @@
title: CREATE ROLE
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Defines a new database role (user or group).
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/sql/CREATE-SCHEMA.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/sql/CREATE-SCHEMA.html.md.erb b/markdown/reference/sql/CREATE-SCHEMA.html.md.erb
index f24e0cc..dbc8bfa 100644
--- a/markdown/reference/sql/CREATE-SCHEMA.html.md.erb
+++ b/markdown/reference/sql/CREATE-SCHEMA.html.md.erb
@@ -2,6 +2,25 @@
title: CREATE SCHEMA
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Defines a new schema.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/sql/CREATE-SEQUENCE.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/sql/CREATE-SEQUENCE.html.md.erb b/markdown/reference/sql/CREATE-SEQUENCE.html.md.erb
index b2557c6..01b81d3 100644
--- a/markdown/reference/sql/CREATE-SEQUENCE.html.md.erb
+++ b/markdown/reference/sql/CREATE-SEQUENCE.html.md.erb
@@ -2,6 +2,25 @@
title: CREATE SEQUENCE
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Defines a new sequence generator.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/sql/CREATE-TABLE-AS.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/sql/CREATE-TABLE-AS.html.md.erb b/markdown/reference/sql/CREATE-TABLE-AS.html.md.erb
index 1979af4..54a6ccd 100644
--- a/markdown/reference/sql/CREATE-TABLE-AS.html.md.erb
+++ b/markdown/reference/sql/CREATE-TABLE-AS.html.md.erb
@@ -2,6 +2,25 @@
title: CREATE TABLE AS
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Defines a new table from the results of a query.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/sql/CREATE-TABLE.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/sql/CREATE-TABLE.html.md.erb b/markdown/reference/sql/CREATE-TABLE.html.md.erb
index 162a438..825fa7e 100644
--- a/markdown/reference/sql/CREATE-TABLE.html.md.erb
+++ b/markdown/reference/sql/CREATE-TABLE.html.md.erb
@@ -2,6 +2,25 @@
title: CREATE TABLE
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Defines a new table.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/sql/CREATE-TABLESPACE.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/sql/CREATE-TABLESPACE.html.md.erb b/markdown/reference/sql/CREATE-TABLESPACE.html.md.erb
index 2d20107..9f35e47 100644
--- a/markdown/reference/sql/CREATE-TABLESPACE.html.md.erb
+++ b/markdown/reference/sql/CREATE-TABLESPACE.html.md.erb
@@ -2,6 +2,25 @@
title: CREATE TABLESPACE
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Defines a new tablespace.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/sql/CREATE-TYPE.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/sql/CREATE-TYPE.html.md.erb b/markdown/reference/sql/CREATE-TYPE.html.md.erb
index 9e7b59f..8062198 100644
--- a/markdown/reference/sql/CREATE-TYPE.html.md.erb
+++ b/markdown/reference/sql/CREATE-TYPE.html.md.erb
@@ -2,6 +2,25 @@
title: CREATE TYPE
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Defines a new data type.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/sql/CREATE-USER.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/sql/CREATE-USER.html.md.erb b/markdown/reference/sql/CREATE-USER.html.md.erb
index 738c645..abc1a04 100644
--- a/markdown/reference/sql/CREATE-USER.html.md.erb
+++ b/markdown/reference/sql/CREATE-USER.html.md.erb
@@ -2,6 +2,25 @@
title: CREATE USER
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Defines a new database role with the `LOGIN` privilege by default.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/sql/CREATE-VIEW.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/sql/CREATE-VIEW.html.md.erb b/markdown/reference/sql/CREATE-VIEW.html.md.erb
index e39d8d3..aec3d7c 100644
--- a/markdown/reference/sql/CREATE-VIEW.html.md.erb
+++ b/markdown/reference/sql/CREATE-VIEW.html.md.erb
@@ -2,6 +2,25 @@
title: CREATE VIEW
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Defines a new view.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/sql/DEALLOCATE.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/sql/DEALLOCATE.html.md.erb b/markdown/reference/sql/DEALLOCATE.html.md.erb
index 846f282..5a1f78b 100644
--- a/markdown/reference/sql/DEALLOCATE.html.md.erb
+++ b/markdown/reference/sql/DEALLOCATE.html.md.erb
@@ -2,6 +2,25 @@
title: DEALLOCATE
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Deallocates a prepared statement.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/sql/DECLARE.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/sql/DECLARE.html.md.erb b/markdown/reference/sql/DECLARE.html.md.erb
index d6fed83..9e3a246 100644
--- a/markdown/reference/sql/DECLARE.html.md.erb
+++ b/markdown/reference/sql/DECLARE.html.md.erb
@@ -2,6 +2,25 @@
title: DECLARE
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Defines a cursor.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/sql/DROP-AGGREGATE.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/sql/DROP-AGGREGATE.html.md.erb b/markdown/reference/sql/DROP-AGGREGATE.html.md.erb
index f40ca5f..7ec57d6 100644
--- a/markdown/reference/sql/DROP-AGGREGATE.html.md.erb
+++ b/markdown/reference/sql/DROP-AGGREGATE.html.md.erb
@@ -2,6 +2,25 @@
title: DROP AGGREGATE
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Removes an aggregate function.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/sql/DROP-CAST.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/sql/DROP-CAST.html.md.erb b/markdown/reference/sql/DROP-CAST.html.md.erb
index dd8e928..6623575 100644
--- a/markdown/reference/sql/DROP-CAST.html.md.erb
+++ b/markdown/reference/sql/DROP-CAST.html.md.erb
@@ -2,6 +2,25 @@
title: DROP CAST
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Removes a cast.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/sql/DROP-CONVERSION.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/sql/DROP-CONVERSION.html.md.erb b/markdown/reference/sql/DROP-CONVERSION.html.md.erb
index d435417..a7ac247 100644
--- a/markdown/reference/sql/DROP-CONVERSION.html.md.erb
+++ b/markdown/reference/sql/DROP-CONVERSION.html.md.erb
@@ -2,6 +2,25 @@
title: DROP CONVERSION
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Removes a CONVERSION.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/sql/DROP-DATABASE.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/sql/DROP-DATABASE.html.md.erb b/markdown/reference/sql/DROP-DATABASE.html.md.erb
index d8ae296..ead5e41 100644
--- a/markdown/reference/sql/DROP-DATABASE.html.md.erb
+++ b/markdown/reference/sql/DROP-DATABASE.html.md.erb
@@ -2,6 +2,25 @@
title: DROP DATABASE
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Removes a database.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/sql/DROP-EXTERNAL-TABLE.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/sql/DROP-EXTERNAL-TABLE.html.md.erb b/markdown/reference/sql/DROP-EXTERNAL-TABLE.html.md.erb
index 01d0fb1..4168073 100644
--- a/markdown/reference/sql/DROP-EXTERNAL-TABLE.html.md.erb
+++ b/markdown/reference/sql/DROP-EXTERNAL-TABLE.html.md.erb
@@ -2,6 +2,25 @@
title: DROP EXTERNAL TABLE
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Removes an external table definition.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/sql/DROP-FILESPACE.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/sql/DROP-FILESPACE.html.md.erb b/markdown/reference/sql/DROP-FILESPACE.html.md.erb
index afae3fe..b2246f3 100644
--- a/markdown/reference/sql/DROP-FILESPACE.html.md.erb
+++ b/markdown/reference/sql/DROP-FILESPACE.html.md.erb
@@ -2,6 +2,25 @@
title: DROP FILESPACE
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Removes a filespace.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/sql/DROP-FUNCTION.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/sql/DROP-FUNCTION.html.md.erb b/markdown/reference/sql/DROP-FUNCTION.html.md.erb
index 5ebd4e5..cbc1cb9 100644
--- a/markdown/reference/sql/DROP-FUNCTION.html.md.erb
+++ b/markdown/reference/sql/DROP-FUNCTION.html.md.erb
@@ -2,6 +2,25 @@
title: DROP FUNCTION
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Removes a function.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/sql/DROP-GROUP.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/sql/DROP-GROUP.html.md.erb b/markdown/reference/sql/DROP-GROUP.html.md.erb
index 5fce3ae..55929d3 100644
--- a/markdown/reference/sql/DROP-GROUP.html.md.erb
+++ b/markdown/reference/sql/DROP-GROUP.html.md.erb
@@ -2,6 +2,25 @@
title: DROP GROUP
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Removes a database role.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/sql/DROP-LANGUAGE.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/sql/DROP-LANGUAGE.html.md.erb b/markdown/reference/sql/DROP-LANGUAGE.html.md.erb
index efb95f8..0230ad8 100644
--- a/markdown/reference/sql/DROP-LANGUAGE.html.md.erb
+++ b/markdown/reference/sql/DROP-LANGUAGE.html.md.erb
@@ -2,6 +2,25 @@
title: DROP LANGUAGE
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Removes a procedural language.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/sql/DROP-OPERATOR-CLASS.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/sql/DROP-OPERATOR-CLASS.html.md.erb b/markdown/reference/sql/DROP-OPERATOR-CLASS.html.md.erb
index da22425..9cb9362 100644
--- a/markdown/reference/sql/DROP-OPERATOR-CLASS.html.md.erb
+++ b/markdown/reference/sql/DROP-OPERATOR-CLASS.html.md.erb
@@ -2,6 +2,25 @@
title: DROP OPERATOR CLASS
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Removes an operator class.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/sql/DROP-OPERATOR.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/sql/DROP-OPERATOR.html.md.erb b/markdown/reference/sql/DROP-OPERATOR.html.md.erb
index b59fde4..0bcff62 100644
--- a/markdown/reference/sql/DROP-OPERATOR.html.md.erb
+++ b/markdown/reference/sql/DROP-OPERATOR.html.md.erb
@@ -2,6 +2,25 @@
title: DROP OPERATOR
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Removes an operator.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/sql/DROP-OWNED.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/sql/DROP-OWNED.html.md.erb b/markdown/reference/sql/DROP-OWNED.html.md.erb
index 50c5272..0fde376 100644
--- a/markdown/reference/sql/DROP-OWNED.html.md.erb
+++ b/markdown/reference/sql/DROP-OWNED.html.md.erb
@@ -2,6 +2,25 @@
title: DROP OWNED
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Removes database objects owned by a database role.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/sql/DROP-RESOURCE-QUEUE.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/sql/DROP-RESOURCE-QUEUE.html.md.erb b/markdown/reference/sql/DROP-RESOURCE-QUEUE.html.md.erb
index 473923f..4bf8ed4 100644
--- a/markdown/reference/sql/DROP-RESOURCE-QUEUE.html.md.erb
+++ b/markdown/reference/sql/DROP-RESOURCE-QUEUE.html.md.erb
@@ -2,6 +2,25 @@
title: DROP RESOURCE QUEUE
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Removes a resource queue.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/sql/DROP-ROLE.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/sql/DROP-ROLE.html.md.erb b/markdown/reference/sql/DROP-ROLE.html.md.erb
index b1d305b..6ea5f76 100644
--- a/markdown/reference/sql/DROP-ROLE.html.md.erb
+++ b/markdown/reference/sql/DROP-ROLE.html.md.erb
@@ -2,6 +2,25 @@
title: DROP ROLE
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Removes a database role.
## <a id="topic1__section2"></a>Synopsis
[15/18] incubator-hawq-docs git commit: HAWQ-1289 - add hive profile
example with partitioning and mixed data types (closes #90)
Posted by yo...@apache.org.
HAWQ-1289 - add hive profile example with partitioning and mixed data types (closes #90)
Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/commit/d0745d7e
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/tree/d0745d7e
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/diff/d0745d7e
Branch: refs/heads/release/2.1.0.0-incubating
Commit: d0745d7e7924287a7ef57a30b83adcc60f1a6655
Parents: 7a7a87a
Author: Lisa Owen <lo...@pivotal.io>
Authored: Tue Feb 21 16:39:19 2017 -0800
Committer: David Yozie <yo...@apache.org>
Committed: Tue Feb 21 16:39:19 2017 -0800
----------------------------------------------------------------------
markdown/pxf/HivePXF.html.md.erb | 278 ++++++++++++++++++++++++++--------
1 file changed, 216 insertions(+), 62 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/d0745d7e/markdown/pxf/HivePXF.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/pxf/HivePXF.html.md.erb b/markdown/pxf/HivePXF.html.md.erb
index 51b317c..6101016 100644
--- a/markdown/pxf/HivePXF.html.md.erb
+++ b/markdown/pxf/HivePXF.html.md.erb
@@ -155,6 +155,8 @@ Create a Hive table to expose our sample data set.
hive> LOAD DATA LOCAL INPATH '/tmp/pxf_hive_datafile.txt'
INTO TABLE sales_info;
```
+
+ In examples later in this section, you will access the `sales_info` Hive table directly via PXF. You will also insert `sales_info` data into tables of other Hive file format types, and use PXF to access those directly as well.
3. Perform a query on `sales_info` to verify that the data was loaded successfully:
@@ -162,7 +164,21 @@ Create a Hive table to expose our sample data set.
hive> SELECT * FROM sales_info;
```
-In examples later in this section, you will access the `sales_info` Hive table directly via PXF. You will also insert `sales_info` data into tables of other Hive file format types, and use PXF to access those directly as well.
+### <a id="get_hdfs_file_location"></a>Determine the HDFS location of a Hive Table
+
+Should you need to identify the HDFS file location of a Hive managed table, reference it using its HDFS file path. You can determine a Hive table's location in HDFS using the `DESCRIBE` command, for example:
+
+``` sql
+hive> DESCRIBE EXTENDED sales_info;
+Detailed Table Information
+...
+location:hdfs://<namenode>:<port>/apps/hive/warehouse/sales_info
+...
+```
+
+The `location` value identifies the HDFS file path of the table.
+
+
## <a id="hcatalog"></a>Using PXF and HCatalog to Query Hive
@@ -240,7 +256,7 @@ To obtain a description of a Hive table with HCatalog integration, you can use t
```
``` sql
- postgres=# \d+ hcatalog.default.sales_info_rcfile;
+ postgres=# \d+ hcatalog.default.sales_info;
```
``` shell
@@ -327,7 +343,7 @@ Hive-plug-in-specific keywords and values used in the [CREATE EXTERNAL TABLE](..
## <a id="profile_hive"></a>Hive Profile
-Use the `Hive` profile with any Hive file storage format. With the `Hive` profile, you can also access heterogenous format data in a single table where each partition may be stored in a different file format. In both cases, the `Hive` profile will use the optimal `Hive*` profile for the underlying file format type.
+Use the `Hive` profile with any Hive file storage format. With the `Hive` profile, you can also access heterogenous format data in a single table where each partition may be stored in a different file format. In both cases, the `Hive` profile will use the optimal `Hive*` profile for the underlying file storage type. Refer to the [Partition Filtering](#partitionfiltering) discussion later in this topic for additional information on partitioning and the `Hive` profile.
### <a id="profile_hive_using"></a>Example: Using the Hive Profile
@@ -361,7 +377,7 @@ Use the `Hive` profile to create a queryable HAWQ external table from the Hive `
Use the `HiveText` profile to query text format files.
-**Note**: When using the `HiveText` profile, you *must* specify a delimiter option in *both* the `LOCATION` and `FORMAT` clauses.
+**Note**: When using the `HiveText` profile, you **must** specify a delimiter option in both the `LOCATION` and `FORMAT` clauses.
### <a id="profile_hivetext_using"></a>Example: Using the HiveText Profile
@@ -399,7 +415,7 @@ Use the PXF `HiveText` profile to create a queryable HAWQ external table from th
The RCFile Hive format is used for row columnar formatted data. The `HiveRC` profile provides access to RCFile data.
-**Note**: When using the `HiveRC` profile, you *must* specify a delimiter option in *both* the `LOCATION` and `FORMAT` clauses.
+**Note**: When using the `HiveRC` profile, you **must** specify a delimiter option in both the `LOCATION` and `FORMAT` clauses.
### <a id="profile_hiverc_rcfiletbl_using"></a>Example: Using the HiveRC Profile
@@ -432,7 +448,7 @@ Use the `HiveRC` profile to query RCFile-formatted data in Hive tables.
hive> SELECT * FROM sales_info_rcfile;
```
-4. Use the PXF `HiveRC` profile to create a queryable HAWQ external table from the Hive `sales_info_rcfile` table created in the previous step. When using the `HiveRC` profile, you **must** specify a delimiter option in *both* the `LOCATION` and `FORMAT` clauses.:
+4. Use the PXF `HiveRC` profile to create a queryable HAWQ external table from the Hive `sales_info_rcfile` table created in the previous step. You *must* specify a delimiter option in both the `LOCATION` and `FORMAT` clauses.:
``` sql
postgres=# CREATE EXTERNAL TABLE salesinfo_hivercprofile(location text, month text, num_orders int, total_sales float8)
@@ -611,76 +627,214 @@ postgres=# SHOW pxf_enable_filter_pushdown;
postgres=# SET pxf_enable_filter_pushdown=off;
```
-### <a id="example2"></a>Create Partitioned Hive Table
+### <a id="example_hive_part"></a>Example: Using the Hive Profile to Access Partitioned Homogenous Data
-Create a�Hive table `sales_part`�with two partition columns, `delivery_state` and `delivery_city:`
+In this example, you will use the `Hive` profile to query a Hive table named `sales_part` you partition on `delivery_state` and `delivery_city` fields. You will then create a HAWQ external table to query `sales_part`, including specific examples illustrating filter pushdown.
-``` sql
-hive> CREATE TABLE sales_part (name string, type string, supplier_key int, price double)
- PARTITIONED BY (delivery_state string, delivery_city string)
- ROW FORMAT DELIMITED FIELDS TERMINATED BY ',';
-```
+1. Create a�Hive table named `sales_part`�with two partition columns, `delivery_state` and `delivery_city:`
-Load data into this Hive table and�add some partitions:
+ ``` sql
+ hive> CREATE TABLE sales_part (name string, type string, supplier_key int, price double)
+ PARTITIONED BY (delivery_state string, delivery_city string)
+ ROW FORMAT DELIMITED FIELDS TERMINATED BY ',';
+ ```
-``` sql
-hive> INSERT INTO TABLE sales_part
- PARTITION(delivery_state = 'CALIFORNIA', delivery_city = 'Fresno')
- VALUES ('block', 'widget', 33, 15.17);
-hive> INSERT INTO TABLE sales_part
- PARTITION(delivery_state = 'CALIFORNIA', delivery_city = 'Sacramento')
- VALUES ('cube', 'widget', 11, 1.17);
-hive> INSERT INTO TABLE sales_part
- PARTITION(delivery_state = 'NEVADA', delivery_city = 'Reno')
- VALUES ('dowel', 'widget', 51, 31.82);
-hive> INSERT INTO TABLE sales_part
- PARTITION(delivery_state = 'NEVADA', delivery_city = 'Las Vegas')
- VALUES ('px49', 'pipe', 52, 99.82);
-```
+2. Load data into this Hive table and�add some partitions:
-The Hive storage directory structure for the `sales_part` table appears as follows:
+ ``` sql
+ hive> INSERT INTO TABLE sales_part
+ PARTITION(delivery_state = 'CALIFORNIA', delivery_city = 'Fresno')
+ VALUES ('block', 'widget', 33, 15.17);
+ hive> INSERT INTO TABLE sales_part
+ PARTITION(delivery_state = 'CALIFORNIA', delivery_city = 'Sacramento')
+ VALUES ('cube', 'widget', 11, 1.17);
+ hive> INSERT INTO TABLE sales_part
+ PARTITION(delivery_state = 'NEVADA', delivery_city = 'Reno')
+ VALUES ('dowel', 'widget', 51, 31.82);
+ hive> INSERT INTO TABLE sales_part
+ PARTITION(delivery_state = 'NEVADA', delivery_city = 'Las Vegas')
+ VALUES ('px49', 'pipe', 52, 99.82);
+ ```
+
+3. Query the `sales_part` table:
-``` pre
-$ sudo -u hdfs hdfs dfs -ls -R /apps/hive/warehouse/sales_part
-/apps/hive/warehouse/sales_part/delivery_state=CALIFORNIA/delivery_city=\u2019Fresno\u2019/
-/apps/hive/warehouse/sales_part/delivery_state=CALIFORNIA/delivery_city=Sacramento/
-/apps/hive/warehouse/sales_part/delivery_state=NEVADA/delivery_city=Reno/
-/apps/hive/warehouse/sales_part/delivery_state=NEVADA/delivery_city=\u2019Las Vegas\u2019/
-```
+ ``` sql
+ hive> SELECT * FROM sales_part;
+ ```
+
+ A�`SELECT *`�statement on a Hive partitioned table shows the partition fields at the end of the record.
-To define a HAWQ PXF table that will read this Hive table�and�take advantage of partition filter push-down, define the fields corresponding to the Hive partition fields at the end of the `CREATE EXTERNAL TABLE` attribute list.�In HiveQL,�a�`SELECT *`�statement on a partitioned table shows the partition fields at the end of the record.
+3. Examine the Hive/HDFS directory structure for the `sales_part` table:
-``` sql
-postgres=# CREATE EXTERNAL TABLE pxf_sales_part(
- item_name TEXT,
- item_type TEXT,
- supplier_key INTEGER,
- item_price DOUBLE PRECISION,
- delivery_state TEXT,
- delivery_city TEXT
-)
-LOCATION ('pxf://namenode:51200/sales_part?Profile=Hive')
-FORMAT 'CUSTOM' (FORMATTER='pxfwritable_import');
-
-postgres=# SELECT * FROM pxf_sales_part;
-```
+ ``` shell
+ $ sudo -u hdfs hdfs dfs -ls -R /apps/hive/warehouse/sales_part
+ /apps/hive/warehouse/sales_part/delivery_state=CALIFORNIA/delivery_city=Fresno/
+ /apps/hive/warehouse/sales_part/delivery_state=CALIFORNIA/delivery_city=Sacramento/
+ /apps/hive/warehouse/sales_part/delivery_state=NEVADA/delivery_city=Reno/
+ /apps/hive/warehouse/sales_part/delivery_state=NEVADA/delivery_city=Las Vegas/
+ ```
-### <a id="example3"></a>Query Without Pushdown
+4. Create a PXF external table to read the partitioned `sales_part` Hive table. To take advantage of partition filter push-down, define fields corresponding to the Hive partition fields at the end of the `CREATE EXTERNAL TABLE` attribute list.
-In the following example, the HAWQ query filters the `delivery_city` partition `Sacramento`. The filter on� `item_name` is not pushed down, since it is not a partition column. It is performed on the HAWQ side after all the data on `Sacramento` is transferred for processing.
+ ``` shell
+ $ psql -d postgres
+ ```
-``` sql
-postgres=# SELECT * FROM pxf_sales_part WHERE delivery_city = 'Sacramento' AND item_name = 'cube';
-```
+ ``` sql
+ postgres=# CREATE EXTERNAL TABLE pxf_sales_part(
+ item_name TEXT, item_type TEXT,
+ supplier_key INTEGER, item_price DOUBLE PRECISION,
+ delivery_state TEXT, delivery_city TEXT)
+ LOCATION ('pxf://namenode:51200/sales_part?Profile=Hive')
+ FORMAT 'CUSTOM' (FORMATTER='pxfwritable_import');
+ ```
+
+5. Query the table:
+
+ ``` sql
+ postgres=# SELECT * FROM pxf_sales_part;
+ ```
-### <a id="example4"></a>Query With Pushdown
+6. Perform another query (no pushdown) on `pxf_sales_part` to return records where the `delivery_city` is `Sacramento` and� `item_name` is `cube`
-The following HAWQ query reads all the data under�`delivery_state` partition `CALIFORNIA`, regardless of the city.
+ ``` sql
+ postgres=# SELECT * FROM pxf_sales_part WHERE delivery_city = 'Sacramento' AND item_name = 'cube';
+ ```
-``` sql
-postgres=# SET pxf_enable_filter_pushdown=on;
-postgres=# SELECT * FROM pxf_sales_part WHERE delivery_state = 'CALIFORNIA';
-```
+ The query filters the `delivery_city` partition `Sacramento`. The filter on� `item_name` is not pushed down, since it is not a partition column. It is performed on the HAWQ side after all the data in the `Sacramento` partition is transferred for processing.
+
+7. Query (with pushdown) for all records where�`delivery_state` is `CALIFORNIA`:
+
+ ``` sql
+ postgres=# SET pxf_enable_filter_pushdown=on;
+ postgres=# SELECT * FROM pxf_sales_part WHERE delivery_state = 'CALIFORNIA';
+ ```
+
+ This query reads all of the data in the�`CALIFORNIA` `delivery_state` partition, regardless of the city.
+
+### <a id="example_hive_part_multi"></a>Example: Using the Hive Profile to Access Partitioned Heterogenous Data
+
+The `Hive` profile supports multiple data format types. This support enables you to query a partitioned Hive table that may be composed of data of different formats.
+
+In this example, you will use the `Hive` profile both directly and indirectly via PXF HCatalog integration to query a partitioned Hive external table. The table is composed of the HDFS data files associated with the `sales_info` (text format) and `sales_info_rcfile` (RC format) Hive tables you created in previous exercises. You will partition the data by year, assigning the data from `sales_info` to the year 2013, and the data from `sales_info_rcfile` to the year 2016. (Ignore at the moment the fact that the tables contain the same data.)
+
+1. Create a Hive external table named `hive_multiformpart` that is partitioned by a string field named `year`:
+
+ ``` shell
+ $ HADOOP_USER_NAME=hdfs hive
+ ```
+
+ ``` sql
+ hive> CREATE EXTERNAL TABLE hive_multiformpart( location string, month string, number_of_orders int, total_sales double)
+ PARTITIONED BY( year string )
+ ROW FORMAT DELIMITED FIELDS TERMINATED BY ',';
+ ```
+
+2. Describe the `sales_info` and `sales_info_rcfile` tables, making note of the HDFS file `location`s:
+
+ ``` sql
+ hive> DESCRIBE EXTENDED sales_info;
+ hive> DESCRIBE EXTENDED sales_info_rcfile;
+ ```
+
+3. Create partitions in the `hive_multiformpart` table for the HDFS locations associated with each of the `sales_info` and `sales_info_rcfile` tables:
+
+ ``` sql
+ hive> ALTER TABLE hive_multiformpart ADD PARTITION (year = '2013') LOCATION 'hdfs://namenode:8020/apps/hive/warehouse/sales_info';
+ hive> ALTER TABLE hive_multiformpart ADD PARTITION (year = '2016') LOCATION 'hdfs://namenode:8020/apps/hive/warehouse/sales_info_rcfile';
+ ```
+
+4. Explicitly identify the file format of the partition associated with the `sales_info_rcfile` table:
+
+ ``` sql
+ hive> ALTER TABLE hive_multiformpart PARTITION (year='2016') SET FILEFORMAT RCFILE;
+ ```
+
+ You need not specify the file format of the partition associated with the `sales_info` table, as `TEXTFILE` format is the default.
+
+5. Query the `hive_multiformpart` table:
+
+ ``` sql
+ hive> SELECT * from hive_multiformpart;
+ ...
+ Bangalore Jul 271 8320.55 2016
+ Beijing Dec 100 4248.41 2016
+ Prague Jan 101 4875.33 2013
+ Rome Mar 87 1557.39 2013
+ ...
+ hive> SELECT * from hive_multiformpart WHERE year='2013';
+ hive> SELECT * from hive_multiformpart WHERE year='2016';
+ ```
+
+6. Show the partitions defined for the `hive_multiformpart` table and exit `hive`:
+
+ ``` sql
+ hive> SHOW PARTITIONS hive_multiformpart;
+ year=2013
+ year=2016
+ hive> quit;
+ ```
+
+7. Start the `psql` subsystem:
+
+ ``` shell
+ $ psql -d postgres
+ ```
+
+7. Use PXF HCatalog integration to query the Hive `hive_multiformpart` external table you created in the previous steps:
+
+ ``` sql
+ postgres=# SELECT * FROM hcatalog.default.hive_multiformpart;
+ ```
+
+ ``` shell
+ location | month | number_of_orders | total_sales | year
+ ---------------+-------+------------------+-------------+------
+ ...
+ Prague | Dec | 333 | 9894.77 | 2013
+ Bangalore | Jul | 271 | 8320.55 | 2013
+ Beijing | Dec | 100 | 4248.41 | 2013
+ Prague | Jan | 101 | 4875.33 | 2016
+ Rome | Mar | 87 | 1557.39 | 2016
+ Bangalore | May | 317 | 8936.99 | 2016
+ ...
+ ```
+
+7. Use the PXF `Hive` profile to create a readable HAWQ external table derived from the Hive `hive_multiformpart` external table you created in the previous steps:
+
+ ``` sql
+ postgres=# CREATE EXTERNAL TABLE pxf_multiformpart(location text, month text, num_orders int, total_sales float8, year text)
+ LOCATION ('pxf://namenode:51200/default.hive_multiformpart?PROFILE=Hive')
+ FORMAT 'CUSTOM' (formatter='pxfwritable_import');
+ ```
+
+5. Query the PXF external table:
+
+ ``` sql
+ postgres=# SELECT * FROM pxf_multiformpart;
+ ```
+
+ ``` shell
+ location | month | num_orders | total_sales | year
+ ---------------+-------+------------+-------------+--------
+ ....
+ Prague | Dec | 333 | 9894.77 | 2013
+ Bangalore | Jul | 271 | 8320.55 | 2013
+ Beijing | Dec | 100 | 4248.41 | 2013
+ Prague | Jan | 101 | 4875.33 | 2016
+ Rome | Mar | 87 | 1557.39 | 2016
+ Bangalore | May | 317 | 8936.99 | 2016
+ ....
+ ```
+
+6. Perform a second query to calculate the total number of orders for the year 2013:
+
+ ``` sql
+ postgres=# SELECT sum(num_orders) FROM pxf_multiformpart WHERE month='Dec' AND year='2013';
+ sum
+ -----
+ 433
+ ```
## <a id="topic_fdm_zrh_1s"></a>Using PXF with Hive Default Partitions
[10/18] incubator-hawq-docs git commit: empty commit to synchronize
mirror
Posted by yo...@apache.org.
empty commit to synchronize mirror
Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/commit/1102fdc0
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/tree/1102fdc0
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/diff/1102fdc0
Branch: refs/heads/release/2.1.0.0-incubating
Commit: 1102fdc049229853f7fcf79f30f3032215ff9276
Parents: a42c05a
Author: David Yozie <yo...@apache.org>
Authored: Wed Feb 1 09:51:13 2017 -0800
Committer: David Yozie <yo...@apache.org>
Committed: Wed Feb 1 09:51:13 2017 -0800
----------------------------------------------------------------------
----------------------------------------------------------------------
[17/18] incubator-hawq-docs git commit: fix typeo
Posted by yo...@apache.org.
fix typeo
Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/commit/31fa5854
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/tree/31fa5854
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/diff/31fa5854
Branch: refs/heads/release/2.1.0.0-incubating
Commit: 31fa5854202f2883c804c8ca295ecb10ab42581a
Parents: c277b27
Author: Lisa Owen <lo...@pivotal.io>
Authored: Wed Feb 22 09:44:15 2017 -0800
Committer: Lisa Owen <lo...@pivotal.io>
Committed: Wed Feb 22 09:47:07 2017 -0800
----------------------------------------------------------------------
markdown/reference/cli/admin_utilities/hawqregister.html.md.erb | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/31fa5854/markdown/reference/cli/admin_utilities/hawqregister.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/cli/admin_utilities/hawqregister.html.md.erb b/markdown/reference/cli/admin_utilities/hawqregister.html.md.erb
index a89d2e9..b5711f3 100644
--- a/markdown/reference/cli/admin_utilities/hawqregister.html.md.erb
+++ b/markdown/reference/cli/admin_utilities/hawqregister.html.md.erb
@@ -58,7 +58,7 @@ The client machine where `hawq register` is executed must meet the following con
## <a id="topic1__section4"></a>Description
-`hawq register` is a utility that loads and registers existing data files or folders in HDFS into HAWQ internal tables, allowing HAWQ to directly read the data and use internal table processing for operations such as transactions and high perforance, without needing to load or copy it. Data from the file or directory specified by \<hdfsfilepath\> is loaded into the appropriate HAWQ table directory in HDFS and the utility updates the corresponding HAWQ metadata for the files.
+`hawq register` is a utility that loads and registers existing data files or folders in HDFS into HAWQ internal tables, allowing HAWQ to directly read the data and use internal table processing for operations such as transactions and high performance, without needing to load or copy it. Data from the file or directory specified by \<hdfsfilepath\> is loaded into the appropriate HAWQ table directory in HDFS and the utility updates the corresponding HAWQ metadata for the files.
You can use `hawq register` to:
[14/18] incubator-hawq-docs git commit: HAWQ-1307 Configuring HAWQ
Best Practices page (closes #93)
Posted by yo...@apache.org.
HAWQ-1307 Configuring HAWQ Best Practices page (closes #93)
Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/commit/7a7a87a6
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/tree/7a7a87a6
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/diff/7a7a87a6
Branch: refs/heads/release/2.1.0.0-incubating
Commit: 7a7a87a6584588a36955361bab6432f9901af762
Parents: 10938ae
Author: Jane Beckman <jb...@pivotal.io>
Authored: Thu Feb 9 08:53:27 2017 -0800
Committer: David Yozie <yo...@apache.org>
Committed: Thu Feb 9 08:53:27 2017 -0800
----------------------------------------------------------------------
.../source/subnavs/apache-hawq-nav-210.erb | 1 +
.../HAWQBestPracticesOverview.html.md.erb | 3 ++
.../config_hawq_bestpractices.html.md.erb | 32 +++++++++++++++++
.../operating_hawq_bestpractices.html.md.erb | 36 +++++++++-----------
4 files changed, 52 insertions(+), 20 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/7a7a87a6/book/master_middleman/source/subnavs/apache-hawq-nav-210.erb
----------------------------------------------------------------------
diff --git a/book/master_middleman/source/subnavs/apache-hawq-nav-210.erb b/book/master_middleman/source/subnavs/apache-hawq-nav-210.erb
index bede889..3316cba 100644
--- a/book/master_middleman/source/subnavs/apache-hawq-nav-210.erb
+++ b/book/master_middleman/source/subnavs/apache-hawq-nav-210.erb
@@ -396,6 +396,7 @@
</li>
<li class="has_submenu"><a href="/docs/userguide/2.1.0.0-incubating/bestpractices/HAWQBestPracticesOverview.html">Best Practices</a>
<ul>
+ <li><a href="/docs/userguide/2.1.0.0-incubating/bestpractices/config_hawq_bestpractices.html">Configuring HAWQ</a></li>
<li><a href="/docs/userguide/2.1.0.0-incubating/bestpractices/operating_hawq_bestpractices.html">Operating HAWQ</a></li>
<li><a href="/docs/userguide/2.1.0.0-incubating/bestpractices/secure_bestpractices.html">Securing HAWQ</a></li>
<li><a href="/docs/userguide/2.1.0.0-incubating/bestpractices/managing_resources_bestpractices.html">Managing Resources</a></li>
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/7a7a87a6/markdown/bestpractices/HAWQBestPracticesOverview.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/bestpractices/HAWQBestPracticesOverview.html.md.erb b/markdown/bestpractices/HAWQBestPracticesOverview.html.md.erb
index 5fa7769..ddca235 100644
--- a/markdown/bestpractices/HAWQBestPracticesOverview.html.md.erb
+++ b/markdown/bestpractices/HAWQBestPracticesOverview.html.md.erb
@@ -23,7 +23,10 @@ under the License.
This chapter provides best practices on using the components and features that are part of a HAWQ system.
+- **[Best Practices for Configuring HAWQ Parameters](../bestpractices/config_hawq_bestpractices.html)**
+ This topic provides best practices for configuring the parameters in `hawq-site.xml`.
+
- **[Best Practices for Operating HAWQ](../bestpractices/operating_hawq_bestpractices.html)**
This topic provides best practices for operating HAWQ, including recommendations for stopping, starting and monitoring HAWQ.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/7a7a87a6/markdown/bestpractices/config_hawq_bestpractices.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/bestpractices/config_hawq_bestpractices.html.md.erb b/markdown/bestpractices/config_hawq_bestpractices.html.md.erb
new file mode 100644
index 0000000..e4b27b0
--- /dev/null
+++ b/markdown/bestpractices/config_hawq_bestpractices.html.md.erb
@@ -0,0 +1,32 @@
+---
+title: Best Practices for Configuring HAWQ Parameters
+---
+
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+
+The configuration parameters for HAWQ are maintained in the `$GPHOME/etc/hawq-site.xml` file. This file resides on all HAWQ instances and can be modified either via Ambari or by using the HAWQ command line interface. Use a consistent strategy (Ambari or command line interface) to maintain the contents of `hawq-site.xml`.
+
+For Ambari-managed clusters, always use Ambari for configuring `hawq-site.xml` parameters; restarting or reconfiguring HAWQ via Ambari will overwrite any configuration changes to `hawq-site.xml` made outside the Ambari interface.
+
+If you manage your cluster using command line tools instead of Ambari, use a consistent `hawq-site.xml` file to configure your entire cluster.
+
+**Note:** Do not configure HAWQ parameters through the `postgresql.conf` file. Use only `hawq-site.xml` to configure your HAWQ cluster. Any parameters defined in `hawq-site.xml` will overwrite parameters in `postgresql.conf`.
+
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/7a7a87a6/markdown/bestpractices/operating_hawq_bestpractices.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/bestpractices/operating_hawq_bestpractices.html.md.erb b/markdown/bestpractices/operating_hawq_bestpractices.html.md.erb
index b19ba25..61687df 100644
--- a/markdown/bestpractices/operating_hawq_bestpractices.html.md.erb
+++ b/markdown/bestpractices/operating_hawq_bestpractices.html.md.erb
@@ -23,21 +23,14 @@ under the License.
This topic provides best practices for operating HAWQ, including recommendations for stopping, starting and monitoring HAWQ.
-## <a id="best_practice_config"></a>Best Practices for Configuring HAWQ Parameters
+## <a id="task_qgk_bz3_1v"></a>Best Practices Using the Command Line to Start/Stop HAWQ Cluster Members
-The HAWQ configuration guc/parameters are located in `$GPHOME/etc/hawq-site.xml`. This configuration file resides on all HAWQ instances and can be modified either by the Ambari interface or the command line.
+The following best practices are recommended when using `hawq start` and `hawq stop` to manage your HAWQ cluster.
-If you install and manage HAWQ using Ambari, use the Ambari interface for all configuration changes. Do not use command line utilities such as `hawq config` to set or change HAWQ configuration properties for Ambari-managed clusters. Configuration changes to `hawq-site.xml` made outside the Ambari interface will be overwritten when you restart or reconfigure HAWQ using Ambari.
-
-If you manage your cluster using command line tools instead of Ambari, use a consistent `hawq-site.xml` file to configure your entire cluster.
-
-**Note:** While `postgresql.conf` still exists in HAWQ, any parameters defined in `hawq-site.xml` will overwrite configurations in `postgresql.conf`. For this reason, we recommend that you only use `hawq-site.xml` to configure your HAWQ cluster. For Ambari clusters, always use Ambari for configuring `hawq-site.xml` parameters.
-
-## <a id="task_qgk_bz3_1v"></a>Best Practices to Start/Stop HAWQ Cluster Members
-
-For best results in using `hawq start` and `hawq stop` to manage your HAWQ system, the following best practices are recommended.
-
-- Issue the `CHECKPOINT` command to update and flush all data files to disk and update the log file before stopping the cluster. A checkpoint ensures that, in the event of a crash, files can be restored from the checkpoint snapshot.
+- Always use HAWQ management commands to start and stop HAWQ, instead of Postgres equivalents.
+- Use `hawq start cluster` or `hawq restart cluster` to start the entire cluster, rather than starting the master and individual segments.
+- If you do decide to use `hawq start standby|master|segment` to start nodes individually, always start the standby before the active master. Otherwise, the standby can become unsynchronized with the active master.
+- When stopping a cluster, issue the `CHECKPOINT` command to update and flush all data files to disk and update the log file before stopping the cluster. A checkpoint ensures that, in the event of a crash, files can be restored from the checkpoint snapshot.
- Stop the entire HAWQ system by stopping the cluster on the master host:
```shell
$ hawq stop cluster
@@ -70,8 +63,6 @@ For best results in using `hawq start` and `hawq stop` to manage your HAWQ syste
$ hawq stop allsegments
```
-- Typically you should always use `hawq start cluster` or `hawq restart cluster` to start the cluster. If you do end up using `hawq start standby|master|segment` to start nodes individually, make sure you always start the standby before the active master. Otherwise, the standby can become unsynchronized with the active master.
-
## <a id="id_trr_m1j_1v"></a>Guidelines for Cluster Expansion
This topic provides some guidelines around expanding your HAWQ cluster.
@@ -84,7 +75,12 @@ There are several recommendations to keep in mind when modifying the size of you
- Note that for hash distributed tables, expanding the cluster will not immediately improve performance since hash distributed tables use a fixed number of virtual segments. In order to obtain better performance with hash distributed tables, you must redistribute the table to the updated cluster by either the [ALTER TABLE](../reference/sql/ALTER-TABLE.html) or [CREATE TABLE AS](../reference/sql/CREATE-TABLE-AS.html#topic1) command.
- If you are using hash tables, consider updating the `default_hash_table_bucket_number` server configuration parameter to a larger value after expanding the cluster but before redistributing the hash tables.
-## <a id="id_o5n_p1j_1v"></a>Database State Monitoring Activities
+## <a id="best_pract_monitoring"></a>Best Practices for Monitoring a HAWQ System
+The sections below provide recommendations for common monitoring tasks.
+
+**Note:** If your HAWQ cluster is managed through the Ambari interface, use Ambari to perform the suggested monitoring tasks.
+
+### <a id="id_o5n_p1j_1v"></a>Database State Monitoring Activities
<a id="id_o5n_p1j_1v__d112e31"></a>
@@ -120,7 +116,7 @@ WHERE status <> 'u';</code></pre></td>
</table>
-## <a id="id_d3w_p1j_1v"></a>Hardware and Operating System Monitoring
+### <a id="id_d3w_p1j_1v"></a>Hardware and Operating System Monitoring
<a id="id_d3w_p1j_1v__d112e111"></a>
@@ -195,7 +191,7 @@ If transfer rates are lower than expected, consult with your data architect rega
</table>
-## <a id="id_khd_q1j_1v"></a>Data Maintenance
+### <a id="id_khd_q1j_1v"></a>Data Maintenance
<a id="id_khd_q1j_1v__d112e279"></a>
@@ -224,7 +220,7 @@ If transfer rates are lower than expected, consult with your data architect rega
</table>
-## <a id="id_lx4_q1j_1v"></a>Database Maintenance
+### <a id="id_lx4_q1j_1v"></a>Database Maintenance
<a id="id_lx4_q1j_1v__d112e343"></a>
@@ -277,7 +273,7 @@ If transfer rates are lower than expected, consult with your data architect rega
</table>
-## <a id="id_blv_q1j_1v"></a>Patching and Upgrading
+### <a id="id_blv_q1j_1v"></a>Patching and Upgrading
<a id="id_blv_q1j_1v__d112e472"></a>
[16/18] incubator-hawq-docs git commit: empty commit to sync mirror
Posted by yo...@apache.org.
empty commit to sync mirror
Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/commit/c277b272
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/tree/c277b272
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/diff/c277b272
Branch: refs/heads/release/2.1.0.0-incubating
Commit: c277b272de4ca008776f4d0cfadb265cfc68cf18
Parents: d0745d7
Author: David Yozie <yo...@apache.org>
Authored: Wed Feb 22 09:09:26 2017 -0800
Committer: David Yozie <yo...@apache.org>
Committed: Wed Feb 22 09:09:26 2017 -0800
----------------------------------------------------------------------
----------------------------------------------------------------------
[05/18] incubator-hawq-docs git commit: HAWQ-1305 Add apache license
header to doc source
Posted by yo...@apache.org.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/catalog/pg_language.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/catalog/pg_language.html.md.erb b/markdown/reference/catalog/pg_language.html.md.erb
index 9b626f9..14143f8 100644
--- a/markdown/reference/catalog/pg_language.html.md.erb
+++ b/markdown/reference/catalog/pg_language.html.md.erb
@@ -2,6 +2,25 @@
title: pg_language
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The `pg_language` system catalog table registers languages in which you can write functions or stored procedures. It is populated by `CREATE LANGUAGE`.
<a id="topic1__gs143898"></a>
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/catalog/pg_largeobject.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/catalog/pg_largeobject.html.md.erb b/markdown/reference/catalog/pg_largeobject.html.md.erb
index 59d2c6d..42fdfc0 100644
--- a/markdown/reference/catalog/pg_largeobject.html.md.erb
+++ b/markdown/reference/catalog/pg_largeobject.html.md.erb
@@ -2,6 +2,25 @@
title: pg_largeobject
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The `pg_largeobject` system catalog table holds the data making up 'large objects'. A large object is identified by an OID assigned when it is created. Each large object is broken into segments or 'pages' small enough to be conveniently stored as rows in `pg_largeobject`. The amount of data per page is defined to be `LOBLKSIZE` (which is currently `BLCKSZ`/4, or typically 8K).
Each row of `pg_largeobject` holds data for one page of a large object, beginning at byte offset (*pageno*` * LOBLKSIZE`) within the object. The implementation allows sparse storage: pages may be missing, and may be shorter than `LOBLKSIZE` bytes even if they are not the last page of the object. Missing regions within a large object read as zeroes.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/catalog/pg_listener.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/catalog/pg_listener.html.md.erb b/markdown/reference/catalog/pg_listener.html.md.erb
index 4df5fda..dac736d 100644
--- a/markdown/reference/catalog/pg_listener.html.md.erb
+++ b/markdown/reference/catalog/pg_listener.html.md.erb
@@ -2,6 +2,25 @@
title: pg_listener
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The `pg_listener` system catalog table supports the `LISTEN` and `NOTIFY` commands. A listener creates an entry in `pg_listener` for each notification name it is listening for. A notifier scans and updates each matching entry to show that a notification has occurred. The notifier also sends a signal (using the PID recorded in the table) to awaken the listener from sleep.
This table is not currently used in HAWQ.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/catalog/pg_locks.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/catalog/pg_locks.html.md.erb b/markdown/reference/catalog/pg_locks.html.md.erb
index a20a4d7..f7eaada 100644
--- a/markdown/reference/catalog/pg_locks.html.md.erb
+++ b/markdown/reference/catalog/pg_locks.html.md.erb
@@ -2,6 +2,25 @@
title: pg_locks
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The `pg_locks` view provides access to information about the locks held by open transactions within HAWQ.
`pg_locks` contains one row per active lockable object, requested lock mode, and relevant transaction. Thus, the same lockable object may appear many times, if multiple transactions are holding or waiting for locks on it. However, an object that currently has no locks on it will not appear at all.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/catalog/pg_namespace.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/catalog/pg_namespace.html.md.erb b/markdown/reference/catalog/pg_namespace.html.md.erb
index b307ecb..caf1661 100644
--- a/markdown/reference/catalog/pg_namespace.html.md.erb
+++ b/markdown/reference/catalog/pg_namespace.html.md.erb
@@ -2,6 +2,25 @@
title: pg_namespace
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The `pg_namespace` system catalog table stores namespaces. A namespace is the structure underlying SQL schemas: each namespace can have a separate collection of relations, types, etc. without name conflicts.
<a id="topic1__gx143898"></a>
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/catalog/pg_opclass.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/catalog/pg_opclass.html.md.erb b/markdown/reference/catalog/pg_opclass.html.md.erb
index d03315c..cfbc781 100644
--- a/markdown/reference/catalog/pg_opclass.html.md.erb
+++ b/markdown/reference/catalog/pg_opclass.html.md.erb
@@ -2,6 +2,25 @@
title: pg_opclass
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The `pg_opclass` system catalog table defines index access method operator classes. Each operator class defines semantics for index columns of a particular data type and a particular index access method. Note that there can be multiple operator classes for a given data type/access method combination, thus supporting multiple behaviors. The majority of the information defining an operator class is actually not in its `pg_opclass` row, but in the associated rows in [pg\_amop](pg_amop.html#topic1) and [pg\_amproc](pg_amproc.html#topic1). Those rows are considered to be part of the operator class definition \u2014 this is not unlike the way that a relation is defined by a single [pg\_class](pg_class.html#topic1) row plus associated rows in [pg\_attribute](pg_attribute.html#topic1) and other tables.
<a id="topic1__gw141982"></a>
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/catalog/pg_operator.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/catalog/pg_operator.html.md.erb b/markdown/reference/catalog/pg_operator.html.md.erb
index 71c632d..86a028f 100644
--- a/markdown/reference/catalog/pg_operator.html.md.erb
+++ b/markdown/reference/catalog/pg_operator.html.md.erb
@@ -2,6 +2,25 @@
title: pg_operator
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The `pg_operator` system catalog table stores information about operators, both built-in and those defined by `CREATE OPERATOR`. Unused column contain zeroes. For example, `oprleft` is zero for a prefix operator.
<a id="topic1__gy150092"></a>
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/catalog/pg_partition.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/catalog/pg_partition.html.md.erb b/markdown/reference/catalog/pg_partition.html.md.erb
index 6795930..271cead 100644
--- a/markdown/reference/catalog/pg_partition.html.md.erb
+++ b/markdown/reference/catalog/pg_partition.html.md.erb
@@ -2,6 +2,25 @@
title: pg_partition
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The `pg_partition` system catalog table is used to track partitioned tables and their inheritance level relationships. Each row of `pg_partition` represents either the level of a partitioned table in the partition hierarchy, or a subpartition template description. The value of the attribute `paristemplate` determines what a particular row represents.
<a id="topic1__gz143898"></a>
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/catalog/pg_partition_columns.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/catalog/pg_partition_columns.html.md.erb b/markdown/reference/catalog/pg_partition_columns.html.md.erb
index 2205a24..6acdbfe 100644
--- a/markdown/reference/catalog/pg_partition_columns.html.md.erb
+++ b/markdown/reference/catalog/pg_partition_columns.html.md.erb
@@ -2,6 +2,25 @@
title: pg_partition_columns
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The `pg_partition_columns` system view is used to show the partition key columns of a partitioned table.
<a id="topic1__ha179967"></a>
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/catalog/pg_partition_encoding.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/catalog/pg_partition_encoding.html.md.erb b/markdown/reference/catalog/pg_partition_encoding.html.md.erb
index e1dbabb..2fa1485 100644
--- a/markdown/reference/catalog/pg_partition_encoding.html.md.erb
+++ b/markdown/reference/catalog/pg_partition_encoding.html.md.erb
@@ -2,6 +2,25 @@
title: pg_partition_encoding
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The `pg_partition_encoding` system catalog table describes the available column compression options for a partition template.
<a id="topic1__hb177831"></a>
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/catalog/pg_partition_rule.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/catalog/pg_partition_rule.html.md.erb b/markdown/reference/catalog/pg_partition_rule.html.md.erb
index 9648132..da671bf 100644
--- a/markdown/reference/catalog/pg_partition_rule.html.md.erb
+++ b/markdown/reference/catalog/pg_partition_rule.html.md.erb
@@ -2,6 +2,25 @@
title: pg_partition_rule
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The `pg_partition_rule` system catalog table is used to track partitioned tables, their check constraints, and data containment rules. Each row of `pg_partition_rule` represents either a leaf partition (the bottom level partitions that contain data), or a branch partition (a top or mid-level partition that is used to define the partition hierarchy, but does not contain any data).
<a id="topic1__hc179425"></a>
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/catalog/pg_partition_templates.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/catalog/pg_partition_templates.html.md.erb b/markdown/reference/catalog/pg_partition_templates.html.md.erb
index ff397fb..ead1241 100644
--- a/markdown/reference/catalog/pg_partition_templates.html.md.erb
+++ b/markdown/reference/catalog/pg_partition_templates.html.md.erb
@@ -2,6 +2,25 @@
title: pg_partition_templates
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The `pg_partition_templates` system view is used to show the subpartitions that were created using a subpartition template.
<a id="topic1__hd179967"></a>
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/catalog/pg_partitions.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/catalog/pg_partitions.html.md.erb b/markdown/reference/catalog/pg_partitions.html.md.erb
index 2c0b26a..70a39b9 100644
--- a/markdown/reference/catalog/pg_partitions.html.md.erb
+++ b/markdown/reference/catalog/pg_partitions.html.md.erb
@@ -2,6 +2,25 @@
title: pg_partitions
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The `pg_partitions` system view is used to show the structure of a partitioned table.
<a id="topic1__he143898"></a>
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/catalog/pg_pltemplate.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/catalog/pg_pltemplate.html.md.erb b/markdown/reference/catalog/pg_pltemplate.html.md.erb
index 0aee00a..182aa3a 100644
--- a/markdown/reference/catalog/pg_pltemplate.html.md.erb
+++ b/markdown/reference/catalog/pg_pltemplate.html.md.erb
@@ -2,6 +2,25 @@
title: pg_pltemplate
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The `pg_pltemplate` system catalog table stores template information for procedural languages. A template for a language allows the language to be created in a particular database by a simple `CREATE LANGUAGE` command, with no need to specify implementation details. Unlike most system catalogs, `pg_pltemplate` is shared across all databases of HAWQ system: there is only one copy of `pg_pltemplate` per system, not one per database. This allows the information to be accessible in each database as it is needed.
There are not currently any commands that manipulate procedural language templates; to change the built-in information, a superuser must modify the table using ordinary `INSERT`, `DELETE`, or `UPDATE` commands.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/catalog/pg_proc.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/catalog/pg_proc.html.md.erb b/markdown/reference/catalog/pg_proc.html.md.erb
index 4d1d194..5fdcf84 100644
--- a/markdown/reference/catalog/pg_proc.html.md.erb
+++ b/markdown/reference/catalog/pg_proc.html.md.erb
@@ -2,6 +2,25 @@
title: pg_proc
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The `pg_proc` system catalog table stores information about functions (or procedures), both built-in functions and those defined by `CREATE FUNCTION`. The table contains data for aggregate and window functions as well as plain functions. If `proisagg` is true, there should be a matching row in `pg_aggregate`. If `proiswin` is true, there should be a matching row in `pg_window`.
For compiled functions, both built-in and dynamically loaded, `prosrc` contains the function's C-language name (link symbol). For all other currently-known language types, `prosrc` contains the function's source text. `probin` is unused except for dynamically-loaded C functions, for which it gives the name of the shared library file containing the function.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/catalog/pg_resqueue.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/catalog/pg_resqueue.html.md.erb b/markdown/reference/catalog/pg_resqueue.html.md.erb
index 0b8d414..651f1fd 100644
--- a/markdown/reference/catalog/pg_resqueue.html.md.erb
+++ b/markdown/reference/catalog/pg_resqueue.html.md.erb
@@ -2,6 +2,25 @@
title: pg_resqueue
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The `pg_resqueue` system catalog table contains information about HAWQ resource queues, which are used for managing resources. This table is populated only on the master. This table is defined in the `pg_global` tablespace, meaning it is globally shared across all databases in the system.
<a id="topic1__hi141982"></a>
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/catalog/pg_resqueue_status.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/catalog/pg_resqueue_status.html.md.erb b/markdown/reference/catalog/pg_resqueue_status.html.md.erb
index 7c841c2..e835cfe 100644
--- a/markdown/reference/catalog/pg_resqueue_status.html.md.erb
+++ b/markdown/reference/catalog/pg_resqueue_status.html.md.erb
@@ -2,6 +2,25 @@
title: pg_resqueue_status
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The `pg_resqueue_status` view allows administrators to see status and activity for a workload management resource queue. It shows how many queries are waiting to run and how many queries are currently active in the system from a particular resource queue.
<a id="topic1__fp141982"></a>
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/catalog/pg_rewrite.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/catalog/pg_rewrite.html.md.erb b/markdown/reference/catalog/pg_rewrite.html.md.erb
index 9b2a76b..b30089a 100644
--- a/markdown/reference/catalog/pg_rewrite.html.md.erb
+++ b/markdown/reference/catalog/pg_rewrite.html.md.erb
@@ -2,6 +2,25 @@
title: pg_rewrite
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The `pg_rewrite` system catalog table stores rewrite rules for tables and views. `pg_class.relhasrules` must be true if a table has any rules in this catalog.
<a id="topic1__hm149830"></a>
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/catalog/pg_roles.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/catalog/pg_roles.html.md.erb b/markdown/reference/catalog/pg_roles.html.md.erb
index 9e70f46..f068bca 100644
--- a/markdown/reference/catalog/pg_roles.html.md.erb
+++ b/markdown/reference/catalog/pg_roles.html.md.erb
@@ -2,6 +2,25 @@
title: pg_roles
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The view `pg_roles` provides access to information about database roles. This is simply a publicly readable view of [pg\_authid](pg_authid.html#topic1) that blanks out the password field. This view explicitly exposes the OID column of the underlying table, since that is needed to do joins to other catalogs.
<a id="topic1__hn141982"></a>
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/catalog/pg_shdepend.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/catalog/pg_shdepend.html.md.erb b/markdown/reference/catalog/pg_shdepend.html.md.erb
index b966155..92bc4f4 100644
--- a/markdown/reference/catalog/pg_shdepend.html.md.erb
+++ b/markdown/reference/catalog/pg_shdepend.html.md.erb
@@ -2,6 +2,25 @@
title: pg_shdepend
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The `pg_shdepend` system catalog table records the dependency relationships between database objects and shared objects, such as roles. This information allows HAWQ to ensure that those objects are unreferenced before attempting to delete them. See also [pg\_depend](pg_depend.html#topic1), which performs a similar function for dependencies involving objects within a single database. Unlike most system catalogs, `pg_shdepend` is shared across all databases of HAWQ system: there is only one copy of `pg_shdepend` per system, not one per database.
In all cases, a `pg_shdepend` entry indicates that the referenced object may not be dropped without also dropping the dependent object. However, there are several subflavors identified by `deptype`:
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/catalog/pg_shdescription.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/catalog/pg_shdescription.html.md.erb b/markdown/reference/catalog/pg_shdescription.html.md.erb
index 133e326..f384604 100644
--- a/markdown/reference/catalog/pg_shdescription.html.md.erb
+++ b/markdown/reference/catalog/pg_shdescription.html.md.erb
@@ -2,6 +2,25 @@
title: pg_shdescription
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The `pg_shdescription` system catalog table stores optional descriptions (comments) for shared database objects. Descriptions can be manipulated with the `COMMENT` command and viewed with `psql`'s `\d` meta-commands. See also [pg\_description](pg_description.html#topic1), which performs a similar function for descriptions involving objects within a single database. Unlike most system catalogs, `pg_shdescription` is shared across all databases of a HAWQ system: there is only one copy of `pg_shdescription` per system, not one per database.
<a id="topic1__hp143898"></a>
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/catalog/pg_stat_activity.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/catalog/pg_stat_activity.html.md.erb b/markdown/reference/catalog/pg_stat_activity.html.md.erb
index 008ae8b..6484822 100644
--- a/markdown/reference/catalog/pg_stat_activity.html.md.erb
+++ b/markdown/reference/catalog/pg_stat_activity.html.md.erb
@@ -2,6 +2,25 @@
title: pg_stat_activity
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The view `pg_stat_activity` shows one row per server process and details about it associated user session and query. The columns that report data on the current query are available unless the parameter `stats_command_string` has been turned off. Furthermore, these columns are only visible if the user examining the view is a superuser or the same as the user owning the process being reported on.
<a id="topic1__hq141982"></a>
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/catalog/pg_stat_last_operation.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/catalog/pg_stat_last_operation.html.md.erb b/markdown/reference/catalog/pg_stat_last_operation.html.md.erb
index b7f812b..d2d7a5f 100644
--- a/markdown/reference/catalog/pg_stat_last_operation.html.md.erb
+++ b/markdown/reference/catalog/pg_stat_last_operation.html.md.erb
@@ -2,6 +2,25 @@
title: pg_stat_last_operation
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The `pg_stat_last_operation` table contains metadata tracking information about database objects (tables, views, etc.).
<a id="topic1__hr138428"></a>
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/catalog/pg_stat_last_shoperation.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/catalog/pg_stat_last_shoperation.html.md.erb b/markdown/reference/catalog/pg_stat_last_shoperation.html.md.erb
index 0dc5a03..599d83c 100644
--- a/markdown/reference/catalog/pg_stat_last_shoperation.html.md.erb
+++ b/markdown/reference/catalog/pg_stat_last_shoperation.html.md.erb
@@ -2,6 +2,25 @@
title: pg_stat_last_shoperation
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The `pg_stat_last_shoperation` table contains metadata tracking information about global objects (roles, tablespaces, etc.).
<a id="topic1__hs138428"></a>
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/catalog/pg_stat_operations.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/catalog/pg_stat_operations.html.md.erb b/markdown/reference/catalog/pg_stat_operations.html.md.erb
index 65833f8..3f585f4 100644
--- a/markdown/reference/catalog/pg_stat_operations.html.md.erb
+++ b/markdown/reference/catalog/pg_stat_operations.html.md.erb
@@ -2,6 +2,25 @@
title: pg_stat_operations
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The view `pg_stat_operations` shows details about the last operation performed on a database object (such as a table, index, view or database) or a global object (such as a role).
<a id="topic1__ht141982"></a>
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/catalog/pg_stat_partition_operations.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/catalog/pg_stat_partition_operations.html.md.erb b/markdown/reference/catalog/pg_stat_partition_operations.html.md.erb
index 2d2fb17..8bde95e 100644
--- a/markdown/reference/catalog/pg_stat_partition_operations.html.md.erb
+++ b/markdown/reference/catalog/pg_stat_partition_operations.html.md.erb
@@ -2,6 +2,25 @@
title: pg_stat_partition_operations
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The `pg_stat_partition_operations` view shows details about the last operation performed on a partitioned table.
<a id="topic1__hu141982"></a>
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/catalog/pg_statistic.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/catalog/pg_statistic.html.md.erb b/markdown/reference/catalog/pg_statistic.html.md.erb
index b784da1..3dc7c33 100644
--- a/markdown/reference/catalog/pg_statistic.html.md.erb
+++ b/markdown/reference/catalog/pg_statistic.html.md.erb
@@ -2,6 +2,25 @@
title: pg_statistic
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The `pg_statistic` system catalog table stores statistical data about the contents of the database. Entries are created by `ANALYZE` and subsequently used by the query optimizer. There is one entry for each table column that has been analyzed. Note that all the statistical data is inherently approximate, even assuming that it is up-to-date.
`pg_statistic` also stores statistical data about the values of index expressions. These are described as if they were actual data columns; in particular, `starelid` references the index. No entry is made for an ordinary non-expression index column, however, since it would be redundant with the entry for the underlying table column.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/catalog/pg_stats.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/catalog/pg_stats.html.md.erb b/markdown/reference/catalog/pg_stats.html.md.erb
index f7cb0f4..555dea0 100644
--- a/markdown/reference/catalog/pg_stats.html.md.erb
+++ b/markdown/reference/catalog/pg_stats.html.md.erb
@@ -2,6 +2,25 @@
title: pg_stats
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The `pg_stats` is a publicly readable view on `pg_statistic` that only exposes information about those tables that are readable by the current user. The `pg_stats` view presents the contents of `pg_statistic` in a friendlier format.
All the statistical data is inherently approximate, even assuming that it is up-to-date. The `pg_stats` schema must be extended whenever new slot types are defined.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/catalog/pg_tablespace.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/catalog/pg_tablespace.html.md.erb b/markdown/reference/catalog/pg_tablespace.html.md.erb
index 493c6b0..0683a44 100644
--- a/markdown/reference/catalog/pg_tablespace.html.md.erb
+++ b/markdown/reference/catalog/pg_tablespace.html.md.erb
@@ -2,6 +2,25 @@
title: pg_tablespace
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The `pg_tablespace` system catalog table stores information about the available tablespaces. Tables can be placed in particular tablespaces to aid administration of disk layout. Unlike most system catalogs, `pg_tablespace` is shared across all databases of a HAWQ system: there is only one copy of `pg_tablespace` per system, not one per database.
<a id="topic1__hx156260"></a>
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/catalog/pg_trigger.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/catalog/pg_trigger.html.md.erb b/markdown/reference/catalog/pg_trigger.html.md.erb
index 3074e46..8e0fc87 100644
--- a/markdown/reference/catalog/pg_trigger.html.md.erb
+++ b/markdown/reference/catalog/pg_trigger.html.md.erb
@@ -2,6 +2,25 @@
title: pg_trigger
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The `pg_trigger` system catalog table stores triggers on tables.
**Note:** HAWQ does not support triggers.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/catalog/pg_type.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/catalog/pg_type.html.md.erb b/markdown/reference/catalog/pg_type.html.md.erb
index e2ea28a..7c8a9c1 100644
--- a/markdown/reference/catalog/pg_type.html.md.erb
+++ b/markdown/reference/catalog/pg_type.html.md.erb
@@ -2,6 +2,25 @@
title: pg_type
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The `pg_type` system catalog table stores information about data types. Base types (scalar types) are created with `CREATE TYPE`, and domains with `CREATE DOMAIN`. A composite type is automatically created for each table in the database, to represent the row structure of the table. It is also possible to create composite types with `CREATE TYPE AS`.
<a id="topic1__hz156260"></a>
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/catalog/pg_type_encoding.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/catalog/pg_type_encoding.html.md.erb b/markdown/reference/catalog/pg_type_encoding.html.md.erb
index b38ff10..0c2c237 100644
--- a/markdown/reference/catalog/pg_type_encoding.html.md.erb
+++ b/markdown/reference/catalog/pg_type_encoding.html.md.erb
@@ -2,6 +2,25 @@
title: pg_type_encoding
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The `pg_type_encoding` system catalog table contains the column storage type information.
<a id="topic1__ia177831"></a>
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/catalog/pg_window.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/catalog/pg_window.html.md.erb b/markdown/reference/catalog/pg_window.html.md.erb
index afe4c0d..b61b566 100644
--- a/markdown/reference/catalog/pg_window.html.md.erb
+++ b/markdown/reference/catalog/pg_window.html.md.erb
@@ -2,6 +2,25 @@
title: pg_window
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The `pg_window` table stores information about window functions. Window functions are often used to compose complex OLAP (online analytical processing) queries. Window functions are applied to partitioned result sets within the scope of a single query expression. A window partition is a subset of rows returned by a query, as defined in a special `OVER()` clause. Typical window functions are `rank`, `dense_rank`, and `row_number`. Each entry in `pg_window` is an extension of an entry in [pg\_proc](pg_proc.html#topic1). The [pg\_proc](pg_proc.html#topic1) entry carries the window function's name, input and output data types, and other information that is similar to ordinary functions.
<a id="topic1__ic143898"></a>
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/cli/admin_utilities/analyzedb.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/cli/admin_utilities/analyzedb.html.md.erb b/markdown/reference/cli/admin_utilities/analyzedb.html.md.erb
index 0384c34..72e7425 100644
--- a/markdown/reference/cli/admin_utilities/analyzedb.html.md.erb
+++ b/markdown/reference/cli/admin_utilities/analyzedb.html.md.erb
@@ -2,6 +2,25 @@
title: analyzedb
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
A utility that performs `ANALYZE` operations on tables incrementally and concurrently.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/cli/admin_utilities/gpfdist.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/cli/admin_utilities/gpfdist.html.md.erb b/markdown/reference/cli/admin_utilities/gpfdist.html.md.erb
index 1683ddb..2520aee 100644
--- a/markdown/reference/cli/admin_utilities/gpfdist.html.md.erb
+++ b/markdown/reference/cli/admin_utilities/gpfdist.html.md.erb
@@ -2,6 +2,25 @@
title: gpfdist
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Serves data files to or writes data files out from HAWQ segments.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/cli/admin_utilities/gplogfilter.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/cli/admin_utilities/gplogfilter.html.md.erb b/markdown/reference/cli/admin_utilities/gplogfilter.html.md.erb
index 44e73c5..d741a9f 100644
--- a/markdown/reference/cli/admin_utilities/gplogfilter.html.md.erb
+++ b/markdown/reference/cli/admin_utilities/gplogfilter.html.md.erb
@@ -2,6 +2,25 @@
title: gplogfilter
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Searches through HAWQ log files for specified entries.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/cli/admin_utilities/hawqactivate.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/cli/admin_utilities/hawqactivate.html.md.erb b/markdown/reference/cli/admin_utilities/hawqactivate.html.md.erb
index 7afd6c7..6bbcdd4 100644
--- a/markdown/reference/cli/admin_utilities/hawqactivate.html.md.erb
+++ b/markdown/reference/cli/admin_utilities/hawqactivate.html.md.erb
@@ -2,6 +2,25 @@
title: hawq activate
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Activates a standby master host and makes it the active master for the HAWQ system.
**Note:** If HAWQ was installed using Ambari, do not use `hawq activate` to activate a standby master host. The system catalogs could become unsynchronized if you mix Ambari and command line functions. For Ambari-managed HAWQ clusters, always use the Ambari administration interface to activate a standby master. For more information, see [Manging HAWQ Using Ambari](../../../admin/ambari-admin.html#topic1).
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/cli/admin_utilities/hawqcheck.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/cli/admin_utilities/hawqcheck.html.md.erb b/markdown/reference/cli/admin_utilities/hawqcheck.html.md.erb
index 23a496d..a36eb1e 100644
--- a/markdown/reference/cli/admin_utilities/hawqcheck.html.md.erb
+++ b/markdown/reference/cli/admin_utilities/hawqcheck.html.md.erb
@@ -2,6 +2,25 @@
title: hawq check
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Verifies and validates HAWQ platform settings.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/cli/admin_utilities/hawqcheckperf.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/cli/admin_utilities/hawqcheckperf.html.md.erb b/markdown/reference/cli/admin_utilities/hawqcheckperf.html.md.erb
index f5c7e2c..984ff16 100644
--- a/markdown/reference/cli/admin_utilities/hawqcheckperf.html.md.erb
+++ b/markdown/reference/cli/admin_utilities/hawqcheckperf.html.md.erb
@@ -2,6 +2,25 @@
title: hawq checkperf
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Verifies the baseline hardware performance of the specified hosts.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/cli/admin_utilities/hawqconfig.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/cli/admin_utilities/hawqconfig.html.md.erb b/markdown/reference/cli/admin_utilities/hawqconfig.html.md.erb
index 9f5e840..b99439a 100644
--- a/markdown/reference/cli/admin_utilities/hawqconfig.html.md.erb
+++ b/markdown/reference/cli/admin_utilities/hawqconfig.html.md.erb
@@ -2,6 +2,25 @@
title: hawq config
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Sets server configuration parameters on all nodes (master and segments) for HAWQ systems that are managed using command-line utilities.
**Note:** If you install and manage HAWQ using Ambari, do not use `hawq config` to configure HAWQ properties. Ambari will overwrite any changes that were made by `hawq config` when it restarts the cluster. For Ambari-managed HAWQ clusters, always use the Ambari administration interface to set or change HAWQ configuration properties.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/cli/admin_utilities/hawqextract.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/cli/admin_utilities/hawqextract.html.md.erb b/markdown/reference/cli/admin_utilities/hawqextract.html.md.erb
index b338523..45355e9 100644
--- a/markdown/reference/cli/admin_utilities/hawqextract.html.md.erb
+++ b/markdown/reference/cli/admin_utilities/hawqextract.html.md.erb
@@ -2,6 +2,25 @@
title: hawq extract
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Extracts the metadata of a specified table into a YAML file.
## Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/cli/admin_utilities/hawqfilespace.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/cli/admin_utilities/hawqfilespace.html.md.erb b/markdown/reference/cli/admin_utilities/hawqfilespace.html.md.erb
index eeb7b39..10cd022 100644
--- a/markdown/reference/cli/admin_utilities/hawqfilespace.html.md.erb
+++ b/markdown/reference/cli/admin_utilities/hawqfilespace.html.md.erb
@@ -2,6 +2,25 @@
title: hawq filespace
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Creates a filespace using a configuration file that defines a file system location. Filespaces describe the physical file system resources to be used by a tablespace.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/cli/admin_utilities/hawqinit.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/cli/admin_utilities/hawqinit.html.md.erb b/markdown/reference/cli/admin_utilities/hawqinit.html.md.erb
index de45ef3..7fdf36b 100644
--- a/markdown/reference/cli/admin_utilities/hawqinit.html.md.erb
+++ b/markdown/reference/cli/admin_utilities/hawqinit.html.md.erb
@@ -2,6 +2,25 @@
title: hawq init
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The `hawq init cluster` command initializes a HAWQ system and starts it.
Use the `hawq init master` and `hawq init segment` commands to individually initialize the master or segment nodes, respectively. Specify any format options at this time. The `hawq init standby` command initializes a standby master host for a HAWQ system.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/cli/admin_utilities/hawqload.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/cli/admin_utilities/hawqload.html.md.erb b/markdown/reference/cli/admin_utilities/hawqload.html.md.erb
index b9fe441..45aa182 100644
--- a/markdown/reference/cli/admin_utilities/hawqload.html.md.erb
+++ b/markdown/reference/cli/admin_utilities/hawqload.html.md.erb
@@ -2,6 +2,25 @@
title: hawq load
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Acts as an interface to the external table parallel loading feature. Executes a load specification defined in a YAML-formatted control file to invoke the HAWQ parallel file server (`gpfdist`).
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/cli/admin_utilities/hawqregister.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/cli/admin_utilities/hawqregister.html.md.erb b/markdown/reference/cli/admin_utilities/hawqregister.html.md.erb
index c230d6d..a89d2e9 100644
--- a/markdown/reference/cli/admin_utilities/hawqregister.html.md.erb
+++ b/markdown/reference/cli/admin_utilities/hawqregister.html.md.erb
@@ -2,6 +2,25 @@
title: hawq register
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Loads and registers AO or Parquet-formatted tables in HDFS into a corresponding table in HAWQ.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/cli/admin_utilities/hawqrestart.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/cli/admin_utilities/hawqrestart.html.md.erb b/markdown/reference/cli/admin_utilities/hawqrestart.html.md.erb
index 6d80e90..0560d74 100644
--- a/markdown/reference/cli/admin_utilities/hawqrestart.html.md.erb
+++ b/markdown/reference/cli/admin_utilities/hawqrestart.html.md.erb
@@ -2,6 +2,25 @@
title: hawq restart
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Shuts down and then restarts a HAWQ system after shutdown is complete.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/cli/admin_utilities/hawqscp.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/cli/admin_utilities/hawqscp.html.md.erb b/markdown/reference/cli/admin_utilities/hawqscp.html.md.erb
index 77f64a8..8b64cfe 100644
--- a/markdown/reference/cli/admin_utilities/hawqscp.html.md.erb
+++ b/markdown/reference/cli/admin_utilities/hawqscp.html.md.erb
@@ -2,6 +2,25 @@
title: hawq scp
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Copies files between multiple hosts at once.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/cli/admin_utilities/hawqssh-exkeys.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/cli/admin_utilities/hawqssh-exkeys.html.md.erb b/markdown/reference/cli/admin_utilities/hawqssh-exkeys.html.md.erb
index 2567faf..cf06e31 100644
--- a/markdown/reference/cli/admin_utilities/hawqssh-exkeys.html.md.erb
+++ b/markdown/reference/cli/admin_utilities/hawqssh-exkeys.html.md.erb
@@ -2,6 +2,25 @@
title: hawq ssh-exkeys
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Exchanges SSH public keys between hosts.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/cli/admin_utilities/hawqssh.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/cli/admin_utilities/hawqssh.html.md.erb b/markdown/reference/cli/admin_utilities/hawqssh.html.md.erb
index ee31308..fdaa760 100644
--- a/markdown/reference/cli/admin_utilities/hawqssh.html.md.erb
+++ b/markdown/reference/cli/admin_utilities/hawqssh.html.md.erb
@@ -2,6 +2,25 @@
title: hawq ssh
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Provides SSH access to multiple hosts at once.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/cli/admin_utilities/hawqstart.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/cli/admin_utilities/hawqstart.html.md.erb b/markdown/reference/cli/admin_utilities/hawqstart.html.md.erb
index ff7b427..6944dcc 100644
--- a/markdown/reference/cli/admin_utilities/hawqstart.html.md.erb
+++ b/markdown/reference/cli/admin_utilities/hawqstart.html.md.erb
@@ -2,6 +2,25 @@
title: hawq start
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Starts a HAWQ system.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/cli/admin_utilities/hawqstate.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/cli/admin_utilities/hawqstate.html.md.erb b/markdown/reference/cli/admin_utilities/hawqstate.html.md.erb
index 3927442..e56ecd1 100644
--- a/markdown/reference/cli/admin_utilities/hawqstate.html.md.erb
+++ b/markdown/reference/cli/admin_utilities/hawqstate.html.md.erb
@@ -2,6 +2,25 @@
title: hawq state
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Shows the status of a running HAWQ system.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/cli/admin_utilities/hawqstop.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/cli/admin_utilities/hawqstop.html.md.erb b/markdown/reference/cli/admin_utilities/hawqstop.html.md.erb
index dd54156..3e30a27 100644
--- a/markdown/reference/cli/admin_utilities/hawqstop.html.md.erb
+++ b/markdown/reference/cli/admin_utilities/hawqstop.html.md.erb
@@ -2,6 +2,25 @@
title: hawq stop
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Stops or restarts a HAWQ system.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/cli/client_utilities/createdb.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/cli/client_utilities/createdb.html.md.erb b/markdown/reference/cli/client_utilities/createdb.html.md.erb
index 31b0c80..2970d3d 100644
--- a/markdown/reference/cli/client_utilities/createdb.html.md.erb
+++ b/markdown/reference/cli/client_utilities/createdb.html.md.erb
@@ -2,6 +2,25 @@
title: createdb
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Creates a new database.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/cli/client_utilities/createuser.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/cli/client_utilities/createuser.html.md.erb b/markdown/reference/cli/client_utilities/createuser.html.md.erb
index 15c189b..838ab37 100644
--- a/markdown/reference/cli/client_utilities/createuser.html.md.erb
+++ b/markdown/reference/cli/client_utilities/createuser.html.md.erb
@@ -2,6 +2,25 @@
title: createuser
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Creates a new database role.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/cli/client_utilities/dropdb.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/cli/client_utilities/dropdb.html.md.erb b/markdown/reference/cli/client_utilities/dropdb.html.md.erb
index 13df828..0b731f7 100644
--- a/markdown/reference/cli/client_utilities/dropdb.html.md.erb
+++ b/markdown/reference/cli/client_utilities/dropdb.html.md.erb
@@ -2,6 +2,25 @@
title: dropdb
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Removes a database.
## <a id="topic1__section2"></a>Synopsis
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/cli/client_utilities/dropuser.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/cli/client_utilities/dropuser.html.md.erb b/markdown/reference/cli/client_utilities/dropuser.html.md.erb
index 9d888ae..27da912 100644
--- a/markdown/reference/cli/client_utilities/dropuser.html.md.erb
+++ b/markdown/reference/cli/client_utilities/dropuser.html.md.erb
@@ -2,6 +2,25 @@
title: dropuser
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
Removes a database role.
## <a id="topic1__section2"></a>Synopsis
[02/18] incubator-hawq-docs git commit: HAWQ-1280 Add documentation
for gp_interconnect* parameters (closes #89)
Posted by yo...@apache.org.
HAWQ-1280 Add documentation for gp_interconnect* parameters (closes #89)
Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/commit/18e72d6a
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/tree/18e72d6a
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/diff/18e72d6a
Branch: refs/heads/release/2.1.0.0-incubating
Commit: 18e72d6a22fffc3e1554825a06259a3ccb2a0810
Parents: 2351d28
Author: Jane Beckman <jb...@pivotal.io>
Authored: Wed Jan 25 14:05:09 2017 -0800
Committer: David Yozie <yo...@apache.org>
Committed: Wed Jan 25 14:05:09 2017 -0800
----------------------------------------------------------------------
.../source/subnavs/apache-hawq-nav-210.erb | 6 ++
.../reference/guc/guc_category-list.html.md.erb | 12 ++-
.../guc/parameter_definitions.html.md.erb | 99 ++++++++++++++++++++
3 files changed, 116 insertions(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/18e72d6a/book/master_middleman/source/subnavs/apache-hawq-nav-210.erb
----------------------------------------------------------------------
diff --git a/book/master_middleman/source/subnavs/apache-hawq-nav-210.erb b/book/master_middleman/source/subnavs/apache-hawq-nav-210.erb
index 1b997b8..bede889 100644
--- a/book/master_middleman/source/subnavs/apache-hawq-nav-210.erb
+++ b/book/master_middleman/source/subnavs/apache-hawq-nav-210.erb
@@ -518,11 +518,17 @@
<li><a href="/docs/userguide/2.1.0.0-incubating/reference/guc/parameter_definitions.html#gp_filerep_tcp_keepalives_interval">gp_filerep_tcp_keepalives_interval</a></li>
<li><a href="/docs/userguide/2.1.0.0-incubating/reference/guc/parameter_definitions.html#gp_hashjoin_tuples_per_bucket">gp_hashjoin_tuples_per_bucket</a></li>
<li><a href="/docs/userguide/2.1.0.0-incubating/reference/guc/parameter_definitions.html#gp_idf_deduplicate">gp_idf_deduplicate</a></li>
+ <li><a href="/docs/userguide/2.1.0.0-incubating/reference/guc/parameter_definitions.html#gp_interconnect_cache_future_packets">gp_interconnect_cache_future_packets</a></li>
+ <li><a href="/docs/userguide/2.1.0.0-incubating/reference/guc/parameter_definitions.html#gp_interconnect_default_rtt">gp_interconnect_default_rtt</a></li>
<li><a href="/docs/userguide/2.1.0.0-incubating/reference/guc/parameter_definitions.html#gp_interconnect_fc_method">gp_interconnect_fc_method</a></li>
<li><a href="/docs/userguide/2.1.0.0-incubating/reference/guc/parameter_definitions.html#gp_interconnect_hash_multiplier">gp_interconnect_hash_multiplier</a></li>
+ <li><a href="/docs/userguide/2.1.0.0-incubating/reference/guc/parameter_definitions.html#gp_interconnect_min_retries_before_timeout">gp_interconnect_min_retries_before_timeout</a></li>
+ <li><a href="/docs/userguide/2.1.0.0-incubating/reference/guc/parameter_definitions.html#gp_interconnect_min_rto">gp_interconnect_min_rto</a></li>
<li><a href="/docs/userguide/2.1.0.0-incubating/reference/guc/parameter_definitions.html#gp_interconnect_queue_depth">gp_interconnect_queue_depth</a></li>
<li><a href="/docs/userguide/2.1.0.0-incubating/reference/guc/parameter_definitions.html#gp_interconnect_setup_timeout">gp_interconnect_setup_timeout</a></li>
<li><a href="/docs/userguide/2.1.0.0-incubating/reference/guc/parameter_definitions.html#gp_interconnect_snd_queue_depth">gp_interconnect_snd_queue_depth</a></li>
+ <li><a href="/docs/userguide/2.1.0.0-incubating/reference/guc/parameter_definitions.html#gp_interconnect_timer_checking_period">gp_interconnect_timer_checking_period</a></li>
+ <li><a href="/docs/userguide/2.1.0.0-incubating/reference/guc/parameter_definitions.html#gp_interconnect_timer_period">gp_interconnect_timer_period</a></li>
<li><a href="/docs/userguide/2.1.0.0-incubating/reference/guc/parameter_definitions.html#gp_interconnect_type">gp_interconnect_type</a></li>
<li><a href="/docs/userguide/2.1.0.0-incubating/reference/guc/parameter_definitions.html#gp_log_format">gp_log_format</a></li>
<li><a href="/docs/userguide/2.1.0.0-incubating/reference/guc/parameter_definitions.html#gp_max_csv_line_length">gp_max_csv_line_length</a></li>
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/18e72d6a/markdown/reference/guc/guc_category-list.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/guc/guc_category-list.html.md.erb b/markdown/reference/guc/guc_category-list.html.md.erb
index c42bba9..896df99 100644
--- a/markdown/reference/guc/guc_category-list.html.md.erb
+++ b/markdown/reference/guc/guc_category-list.html.md.erb
@@ -154,11 +154,21 @@ The parameters in this topic control the configuration of the HAWQ array and its
### <a id="topic50"></a>Interconnect Configuration Parameters
+- [gp\_interconnect\_cache\_future\_packets](../../reference/guc/parameter_definitions.html#gp_interconnect_cache_future_packets)
+- [gp\_interconnect\_default\_rtt](../../reference/guc/parameter_definitions.html#gp_interconnect_default_rtt)
- [gp\_interconnect\_fc\_method](parameter_definitions.html#gp_interconnect_fc_method)
- [gp\_interconnect\_hash\_multiplier](parameter_definitions.html#gp_interconnect_hash_multiplier)
+- [gp\_interconnect\_min\_retries\_before\_timeout](../../reference/guc/parameter_definitions.html#gp_interconnect_min_retries_before_timeout)
+- [gp\_interconnect\_min\_rto](../../reference/guc/parameter_definitions.html#gp_interconnect_min_rto)
- [gp\_interconnect\_queue\_depth](parameter_definitions.html#gp_interconnect_queue_depth)
-- [gp\_interconnect\_snd\_queue\_depth](parameter_definitions.html#gp_interconnect_snd_queue_depth)
- [gp\_interconnect\_setup\_timeout](parameter_definitions.html#gp_interconnect_setup_timeout)
+- [gp\_interconnect\_snd\_queue\_depth](parameter_definitions.html#gp_interconnect_snd_queue_depth)
+- [gp\_interconnect\_timer\_checking\_period
+](../../reference/guc/parameter_definitions.html#gp_interconnect_timer_checking_period
+)
+- [gp\_interconnect\_timer\_period
+](../../reference/guc/parameter_definitions.html#gp_interconnect_timer_period
+)
- [gp\_interconnect\_type](parameter_definitions.html#gp_interconnect_type)
- [gp\_max\_packet\_size](parameter_definitions.html#gp_max_packet_size)
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/18e72d6a/markdown/reference/guc/parameter_definitions.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/guc/parameter_definitions.html.md.erb b/markdown/reference/guc/parameter_definitions.html.md.erb
index 1420793..f3f5757 100644
--- a/markdown/reference/guc/parameter_definitions.html.md.erb
+++ b/markdown/reference/guc/parameter_definitions.html.md.erb
@@ -167,17 +167,32 @@ Descriptions of the HAWQ server configuration parameters listed alphabetically.
- **[gp\_hashjoin\_tuples\_per\_bucket](../../reference/guc/parameter_definitions.html#gp_hashjoin_tuples_per_bucket)**
- **[gp\_idf\_deduplicate](../../reference/guc/parameter_definitions.html#gp_idf_deduplicate)**
+
+- **[gp\_interconnect\_cache\_future\_packets](../../reference/guc/parameter_definitions.html#gp_interconnect_cache_future_packets)**
+
+- **[gp\_interconnect\_default\_rtt](../../reference/guc/parameter_definitions.html#gp_interconnect_default_rtt)**
- **[gp\_interconnect\_fc\_method](../../reference/guc/parameter_definitions.html#gp_interconnect_fc_method)**
- **[gp\_interconnect\_hash\_multiplier](../../reference/guc/parameter_definitions.html#gp_interconnect_hash_multiplier)**
+- **[gp\_interconnect\_min\_retries\_before\_timeout](../../reference/guc/parameter_definitions.html#gp_interconnect_min_retries_before_timeout)**
+
+- **[gp\_interconnect\_min\_rto](../../reference/guc/parameter_definitions.html#gp_interconnect_min_rto)**
- **[gp\_interconnect\_queue\_depth](../../reference/guc/parameter_definitions.html#gp_interconnect_queue_depth)**
- **[gp\_interconnect\_setup\_timeout](../../reference/guc/parameter_definitions.html#gp_interconnect_setup_timeout)**
- **[gp\_interconnect\_snd\_queue\_depth](../../reference/guc/parameter_definitions.html#gp_interconnect_snd_queue_depth)**
+- **[gp\_interconnect\_timer\_period
+](../../reference/guc/parameter_definitions.html#gp_interconnect_timer_period
+)**
+
+- **[gp\_interconnect\_timer\_checking\_period
+](../../reference/guc/parameter_definitions.html#gp_interconnect_timer_checking_period
+)**
+
- **[gp\_interconnect\_type](../../reference/guc/parameter_definitions.html#gp_interconnect_type)**
- **[gp\_log\_format](../../reference/guc/parameter_definitions.html#gp_log_format)**
@@ -1371,6 +1386,41 @@ Changes the strategy to compute and process MEDIAN, and PERCENTILE\_DISC.
</tbody>
</table>
+## <a name="gp_interconnect_cache_future_packets"></a>gp\_interconnect\_cache\_future\_packets
+
+Controls whether future packets are cached.
+
+| Value Range | Default | Set Classifications |
+|-------------|-----------|-------------------------|
+| Boolean | true (on) | master, session, reload |
+
+
+## <a name="gp_interconnect_default_rtt"></a>gp\_interconnect\_default\_rtt
+
+Sets the default rtt (in ms) for UDP interconnect.
+
+<table>
+<colgroup>
+<col width="33%" />
+<col width="33%" />
+<col width="33%" />
+</colgroup>
+<thead>
+<tr class="header">
+<th>Value Range</th>
+<th>Default</th>
+<th>Set Classifications</th>
+</tr>
+</thead>
+<tbody>
+<tr class="odd">
+<td>1-1000ms</td>
+<td>20ms</td>
+<td>master, session, reload</td>
+</tr>
+</tbody>
+</table>
+
## <a name="gp_interconnect_fc_method"></a>gp\_interconnect\_fc\_method
Specifies the flow control method used for UDP interconnect when the value of [gp\_interconnect\_type](#gp_interconnect_type) is UDPIFC.
@@ -1410,6 +1460,25 @@ Sets the size of the hash table used by the UDP interconnect to track connection
|-------------|---------|-------------------------|
| 2-25 | 2 | master, session, reload |
+## <a name="gp_interconnect_min_retries_before\_timeout"></a>gp\_interconnect\_min\_retries\_before\_timeout
+
+Sets the minimum number of retries before reporting a transmit timeout in the interconnect.
+
+| Value Range | Default | Set Classifications |
+|-------------|---------|-------------------------|
+| 1-4096 | 100 | master, session, reload |
+
+
+## <a name="gp_interconnect_min_rto"></a>gp\_interconnect\_min\_rto
+
+Sets the minimum rto (in ms) for UDP interconnect.
+
+| Value Range | Default | Set Classifications |
+|-------------|---------|-------------------------|
+|1-1000ms | 20ms | master, session, reload |
+
+
+
## <a name="gp_interconnect_queue_depth"></a>gp\_interconnect\_queue\_depth
Sets the amount of data per-peer to be queued by the UDP interconnect on receivers (when data is received but no space is available to receive it the data will be dropped, and the transmitter will need to resend it). Increasing the depth from its default value will cause the system to use more memory; but may increase performance. It is reasonable for this to be set between 1 and 10. Queries with data skew potentially perform better when this is increased. Increasing this may radically increase the amount of memory used by the system.
@@ -1438,6 +1507,36 @@ This parameter is used only when [gp\_interconnect\_type](#gp_interconnect_type)
|-------------|---------|-------------------------|
| 1 - 4096 | 2 | master, session, reload |
+## <a name="gp_interconnect_timer_period"></a>gp\_interconnect\_timer\_period
+
+Sets the timer period (in ms) for UDP interconnect.
+
+| Value Range | Default | Set Classifications |
+|-------------|---------|-------------------------|
+| 1-100ms | 5ms | master, session, reload |
+
+
+
+## <a name="gp_interconnect_timer_checking_period"></a>gp\_interconnect\_timer\_checking\_period
+
+Sets the timer checking period (in ms) for UDP interconnect.
+
+| Value Range | Default | Set Classifications |
+|-------------|---------|-------------------------|
+| 1-100ms | 20ms | master, session, reload |
+
+
+
+## <a name="gp_interconnect_transmit_timeout"></a>gp\_interconnect\_transmit\_timeout
+
+Timeout (in seconds) on interconnect to transmit a packet.
+
+| Value Range | Default | Set Classifications |
+|-------------|---------|-------------------------|
+| 1-7200s | 3600s | master, session, reload |
+
+
+
## <a name="gp_interconnect_type"></a>gp\_interconnect\_type
Sets the networking protocol used for Interconnect traffic. With the TCP protocol, HAWQ has an upper limit of 1000 segment instances - less than that if the query workload involves complex, multi-slice queries.
[12/18] incubator-hawq-docs git commit: HAWQ-1293 - identify/tag
deprecated gucs (closes #91)
Posted by yo...@apache.org.
HAWQ-1293 - identify/tag deprecated gucs (closes #91)
Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/commit/ab49891f
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/tree/ab49891f
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/diff/ab49891f
Branch: refs/heads/release/2.1.0.0-incubating
Commit: ab49891f412c631ba350b9632b73c49ef69aa8d2
Parents: dcfe1a4
Author: Lisa Owen <lo...@pivotal.io>
Authored: Thu Feb 9 08:48:32 2017 -0800
Committer: David Yozie <yo...@apache.org>
Committed: Thu Feb 9 08:48:37 2017 -0800
----------------------------------------------------------------------
markdown/query/query-profiling.html.md.erb | 5 ++---
markdown/reference/guc/parameter_definitions.html.md.erb | 4 +++-
2 files changed, 5 insertions(+), 4 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/ab49891f/markdown/query/query-profiling.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/query/query-profiling.html.md.erb b/markdown/query/query-profiling.html.md.erb
index fccef18..00fffe6 100644
--- a/markdown/query/query-profiling.html.md.erb
+++ b/markdown/query/query-profiling.html.md.erb
@@ -107,8 +107,7 @@ The estimated startup cost for this plan is `00.00` (no cost) and a total cost o
Work_mem wanted: 90K bytes avg, 90K byes max (seg0) to lessen
workfile I/O affecting 2 workers.
```
-**Note**
-The *work\_mem* property is not configurable. Use resource queues to manage memory use. For more information on resource queues, see [Configuring Resource Management](../resourcemgmt/ConfigureResourceManagement.html) and [Working with Hierarchical Resource Queues](../resourcemgmt/ResourceQueues.html).
+**Warning**: The `work_mem` server configuration parameter is deprecated and will be removed in a future HAWQ release. Use resource queues to manage memory use. For more information on resource queues, see [Configuring Resource Management](../resourcemgmt/ConfigureResourceManagement.html) and [Working with Hierarchical Resource Queues](../resourcemgmt/ResourceQueues.html).
- The time (in milliseconds) in which the segment that produced the most rows retrieved the first row, and the time taken for that segment to retrieve all rows. The result may omit *<time> to first row* if it is the same as the *<time> to end*.
@@ -162,7 +161,7 @@ If a query performs poorly, examine its query plan and ask the following questio
`Work_mem used: 23430K bytes avg, 23430K bytes max (seg0). Work_mem wanted: 33649K bytes avg, 33649K bytes max (seg0) to lessen workfile I/O affecting 2 workers.`
- The "bytes wanted" message from `EXPLAIN ANALYZE` is based on the amount of data written to work files and is not exact. The minimum `work_mem` needed can differ from the suggested value.
+ The "bytes wanted" message from `EXPLAIN ANALYZE` is based on the amount of data written to work files and is not exact. The minimum `work_mem` needed can differ from the suggested value.
## <a id="explainplan_plpgsql"></a>Generating EXPLAIN Plan from a PL/pgSQL Function
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/ab49891f/markdown/reference/guc/parameter_definitions.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/guc/parameter_definitions.html.md.erb b/markdown/reference/guc/parameter_definitions.html.md.erb
index 6d88075..a389f51 100644
--- a/markdown/reference/guc/parameter_definitions.html.md.erb
+++ b/markdown/reference/guc/parameter_definitions.html.md.erb
@@ -1508,10 +1508,12 @@ Sets the amount of data per-peer to be queued by the UDP interconnect on receive
## <a name="gp_interconnect_setup_timeout"></a>gp\_interconnect\_setup\_timeout
-When the interconnect type is UDP, the time to wait for the Interconnect to complete setup before it times out.
+When the interconnect type is UDP, the time to wait for the Interconnect to complete setup before it times out.
This parameter is used only when [gp\_interconnect\_type](#gp_interconnect_type) is set to UDP.
+**Warning**: The `gp_interconnect_setup_timeout` server configuration parameter is deprecated and will be removed in a future HAWQ release.
+
| Value Range | Default | Set Classifications |
|---------------------------------------------|---------|-------------------------|
| Any valid time expression (number and unit) | 2 hours | master, session, reload |
[06/18] incubator-hawq-docs git commit: HAWQ-1305 Add apache license
header to doc source
Posted by yo...@apache.org.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/query/gporca/query-gporca-features.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/query/gporca/query-gporca-features.html.md.erb b/markdown/query/gporca/query-gporca-features.html.md.erb
index 4941866..6693425 100644
--- a/markdown/query/gporca/query-gporca-features.html.md.erb
+++ b/markdown/query/gporca/query-gporca-features.html.md.erb
@@ -2,6 +2,25 @@
title: GPORCA Features and Enhancements
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
GPORCA includes enhancements for specific types of queries and operations. GPORCA also includes these optimization enhancements:
- Improved join ordering
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/query/gporca/query-gporca-limitations.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/query/gporca/query-gporca-limitations.html.md.erb b/markdown/query/gporca/query-gporca-limitations.html.md.erb
index b63f0d2..2e7aa6d 100644
--- a/markdown/query/gporca/query-gporca-limitations.html.md.erb
+++ b/markdown/query/gporca/query-gporca-limitations.html.md.erb
@@ -2,6 +2,25 @@
title: GPORCA Limitations
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
<span class="shortdesc">There are limitations in HAWQ when GPORCA is enabled. GPORCA and the legacy query optimizer currently coexist in HAWQ because GPORCA does not support all HAWQ features. </span>
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/query/gporca/query-gporca-notes.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/query/gporca/query-gporca-notes.html.md.erb b/markdown/query/gporca/query-gporca-notes.html.md.erb
index ed943e4..3747c80 100644
--- a/markdown/query/gporca/query-gporca-notes.html.md.erb
+++ b/markdown/query/gporca/query-gporca-notes.html.md.erb
@@ -2,6 +2,25 @@
title: Considerations when Using GPORCA
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
<span class="shortdesc"> To execute queries optimally with GPORCA, consider certain criteria for the query. </span>
Ensure the following criteria are met:
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/query/gporca/query-gporca-optimizer.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/query/gporca/query-gporca-optimizer.html.md.erb b/markdown/query/gporca/query-gporca-optimizer.html.md.erb
index 11814f8..b297b96 100644
--- a/markdown/query/gporca/query-gporca-optimizer.html.md.erb
+++ b/markdown/query/gporca/query-gporca-optimizer.html.md.erb
@@ -2,6 +2,25 @@
title: About GPORCA
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
In HAWQ, you can use GPORCA or the legacy query optimizer.
**Note:** To use the GPORCA query optimizer, you must be running a version of HAWQ built with GPORCA, and GPORCA must be enabled in your HAWQ deployment.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/query/gporca/query-gporca-overview.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/query/gporca/query-gporca-overview.html.md.erb b/markdown/query/gporca/query-gporca-overview.html.md.erb
index 56f97eb..c18b4c2 100644
--- a/markdown/query/gporca/query-gporca-overview.html.md.erb
+++ b/markdown/query/gporca/query-gporca-overview.html.md.erb
@@ -2,6 +2,25 @@
title: Overview of GPORCA
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
<span class="shortdesc">GPORCA extends the planning and optimization capabilities of the HAWQ legacy optimizer. </span> GPORCA is extensible and achieves better optimization in multi-core architecture environments. When GPORCA is available in your HAWQ installation and enabled, HAWQ uses GPORCA to generate an execution plan for a query when possible.
GPORCA also enhances HAWQ query performance tuning in the following areas:
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/query/query-performance.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/query/query-performance.html.md.erb b/markdown/query/query-performance.html.md.erb
index 981d77b..045a978 100644
--- a/markdown/query/query-performance.html.md.erb
+++ b/markdown/query/query-performance.html.md.erb
@@ -2,6 +2,25 @@
title: Query Performance
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
<span class="shortdesc">HAWQ dynamically allocates resources to queries. Query performance depends on several factors such as data locality, number of virtual segments used for the query and general cluster health.</span>
- Dynamic Partition Elimination
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/query/query-profiling.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/query/query-profiling.html.md.erb b/markdown/query/query-profiling.html.md.erb
index ea20e0a..fccef18 100644
--- a/markdown/query/query-profiling.html.md.erb
+++ b/markdown/query/query-profiling.html.md.erb
@@ -2,6 +2,25 @@
title: Query Profiling
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
<span class="shortdesc">Examine the query plans of poorly performing queries to identify possible performance tuning opportunities.</span>
HAWQ devises a *query plan* for each query. Choosing the right query plan to match the query and data structure is necessary for good performance. A query plan defines how HAWQ will run the query in the parallel execution environment.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/query/query.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/query/query.html.md.erb b/markdown/query/query.html.md.erb
index 9c218c7..7df1a14 100644
--- a/markdown/query/query.html.md.erb
+++ b/markdown/query/query.html.md.erb
@@ -2,6 +2,25 @@
title: Querying Data
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
This topic provides information about using SQL in HAWQ databases.
You enter SQL statements called queries to view and analyze data in a database using the `psql` interactive SQL client and other client tools.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/CharacterSetSupportReference.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/CharacterSetSupportReference.html.md.erb b/markdown/reference/CharacterSetSupportReference.html.md.erb
index 8a12471..d978ecf 100644
--- a/markdown/reference/CharacterSetSupportReference.html.md.erb
+++ b/markdown/reference/CharacterSetSupportReference.html.md.erb
@@ -2,6 +2,25 @@
title: Character Set Support Reference
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
This topic provides a referene of the character sets supported in HAWQ.
The character set support in HAWQ allows you to store text in a variety of character sets, including single-byte character sets such as the ISO 8859 series and multiple-byte character sets such as EUC (Extended Unix Code), UTF-8, and Mule internal code. All supported character sets can be used transparently by clients, but a few are not supported for use within the server (that is, as a server-side encoding). The default character set is selected while initializing your HAWQ using `hawq init.` It can be overridden when you create a database, so you can have multiple databases each with a different character set.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/HAWQDataTypes.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/HAWQDataTypes.html.md.erb b/markdown/reference/HAWQDataTypes.html.md.erb
index fe5cff7..fe31458 100644
--- a/markdown/reference/HAWQDataTypes.html.md.erb
+++ b/markdown/reference/HAWQDataTypes.html.md.erb
@@ -2,6 +2,25 @@
title: Data Types
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
This topic provides a reference of the data types supported in HAWQ.
HAWQ has a rich set of native data types available to users. Users may also define new data types using the `CREATE TYPE` command. This reference shows all of the built-in data types. In addition to the types listed here, there are also some internally�used data types, such as **oid**�(object identifier), but those are not documented in this guide.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/HAWQEnvironmentVariables.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/HAWQEnvironmentVariables.html.md.erb b/markdown/reference/HAWQEnvironmentVariables.html.md.erb
index ce21798..2cf10ea 100644
--- a/markdown/reference/HAWQEnvironmentVariables.html.md.erb
+++ b/markdown/reference/HAWQEnvironmentVariables.html.md.erb
@@ -2,6 +2,25 @@
title: Environment Variables
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
This topic contains a reference of the environment variables that you set for HAWQ.
Set these in your user\u2019s startup shell profile (such as `~/.bashrc` or `~/.bash_profile`), or in `/etc/profile`, if you want to set them for all users.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/HAWQSampleSiteConfig.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/HAWQSampleSiteConfig.html.md.erb b/markdown/reference/HAWQSampleSiteConfig.html.md.erb
index d4cae5a..1161087 100644
--- a/markdown/reference/HAWQSampleSiteConfig.html.md.erb
+++ b/markdown/reference/HAWQSampleSiteConfig.html.md.erb
@@ -2,6 +2,25 @@
title: Sample hawq-site.xml Configuration File
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
```xml
<configuration>
<property>
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/HAWQSiteConfig.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/HAWQSiteConfig.html.md.erb b/markdown/reference/HAWQSiteConfig.html.md.erb
index 3d20297..578a883 100644
--- a/markdown/reference/HAWQSiteConfig.html.md.erb
+++ b/markdown/reference/HAWQSiteConfig.html.md.erb
@@ -2,6 +2,25 @@
title: Server Configuration Parameter Reference
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
This section describes all server configuration guc/parameters that are available in HAWQ.
Configuration guc/parameters are located in `$GPHOME/etc/hawq-site.xml`. This configuration file resides on all HAWQ instances and is managed either by Ambari or by using the `hawq config` utility. On HAWQ clusters installed and managed by Ambari, always use the Ambari administration interface, and not `hawq config`, to configure HAWQ properties. Ambari will overwrite any changes made using `hawq config`.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/HDFSConfigurationParameterReference.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/HDFSConfigurationParameterReference.html.md.erb b/markdown/reference/HDFSConfigurationParameterReference.html.md.erb
index aef4ed2..28b863a 100644
--- a/markdown/reference/HDFSConfigurationParameterReference.html.md.erb
+++ b/markdown/reference/HDFSConfigurationParameterReference.html.md.erb
@@ -2,6 +2,25 @@
title: HDFS Configuration Reference
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
This reference page describes HDFS configuration values that are configured for HAWQ either within `hdfs-site.xml`, `core-site.xml`, or `hdfs-client.xml`.
## <a id="topic_ixj_xw1_1w"></a>HDFS Site Configuration (hdfs-site.xml and core-site.xml)
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/SQLCommandReference.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/SQLCommandReference.html.md.erb b/markdown/reference/SQLCommandReference.html.md.erb
index 7bbe625..44938ee 100644
--- a/markdown/reference/SQLCommandReference.html.md.erb
+++ b/markdown/reference/SQLCommandReference.html.md.erb
@@ -2,6 +2,25 @@
title: SQL Commands
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
This�section contains a description and the syntax�of�the SQL commands supported by HAWQ.
- **[ABORT](../reference/sql/ABORT.html)**
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/catalog/catalog_ref-html.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/catalog/catalog_ref-html.html.md.erb b/markdown/reference/catalog/catalog_ref-html.html.md.erb
index 864f106..bf73d47 100644
--- a/markdown/reference/catalog/catalog_ref-html.html.md.erb
+++ b/markdown/reference/catalog/catalog_ref-html.html.md.erb
@@ -2,6 +2,25 @@
title: System Catalogs Definitions
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
System catalog table and view definitions in alphabetical order.
- **[gp\_configuration\_history](../../reference/catalog/gp_configuration_history.html)**
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/catalog/catalog_ref-tables.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/catalog/catalog_ref-tables.html.md.erb b/markdown/reference/catalog/catalog_ref-tables.html.md.erb
index 61aa936..4e1c745 100644
--- a/markdown/reference/catalog/catalog_ref-tables.html.md.erb
+++ b/markdown/reference/catalog/catalog_ref-tables.html.md.erb
@@ -2,6 +2,25 @@
title: System Tables
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
This topic lists the system tables included in HAWQ.
- gp\_configuration (Deprecated. See [gp\_segment\_configuration](gp_segment_configuration.html#topic1).)
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/catalog/catalog_ref-views.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/catalog/catalog_ref-views.html.md.erb b/markdown/reference/catalog/catalog_ref-views.html.md.erb
index 4a7ad74..94b1c39 100644
--- a/markdown/reference/catalog/catalog_ref-views.html.md.erb
+++ b/markdown/reference/catalog/catalog_ref-views.html.md.erb
@@ -2,6 +2,25 @@
title: System Views
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
HAWQ provides the following system views not available in PostgreSQL.
- pg\_max\_external\_files (shows number of external table files allowed per segment host when using the file:// protocol)
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/catalog/catalog_ref.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/catalog/catalog_ref.html.md.erb b/markdown/reference/catalog/catalog_ref.html.md.erb
index 143091f..142e3c3 100644
--- a/markdown/reference/catalog/catalog_ref.html.md.erb
+++ b/markdown/reference/catalog/catalog_ref.html.md.erb
@@ -2,6 +2,25 @@
title: System Catalog Reference
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
This reference describes the HAWQ system catalog tables and views.
System tables prefixed with '`gp_`' relate to the parallel features of HAWQ. Tables prefixed with '`pg_`' are either standard PostgreSQL system catalog tables supported in HAWQ, or are related to features HAWQ that provides to enhance PostgreSQL for data warehousing workloads. Note that the global system catalog for HAWQ resides on the master instance.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/catalog/gp_configuration_history.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/catalog/gp_configuration_history.html.md.erb b/markdown/reference/catalog/gp_configuration_history.html.md.erb
index e501d55..d89b0e2 100644
--- a/markdown/reference/catalog/gp_configuration_history.html.md.erb
+++ b/markdown/reference/catalog/gp_configuration_history.html.md.erb
@@ -2,6 +2,25 @@
title: gp_configuration_history
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The `gp_configuration_history` table contains information about system changes related to fault detection and recovery operations. The HAWQ [fault tolerance service](../../admin/FaultTolerance.html) logs data to this table, as do certain related management utilities such as `hawq init`. For example, when you add a new segment to the system, records for these events are logged to `gp_configuration_history`. If a segment is marked as down by the fault tolerance service in the [gp\_segment\_configuration](gp_segment_configuration.html) catalog table, then the reason for being marked as down is recorded in this table.
The event descriptions stored in this table may be helpful for troubleshooting serious system issues in collaboration with HAWQ support technicians.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/catalog/gp_distribution_policy.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/catalog/gp_distribution_policy.html.md.erb b/markdown/reference/catalog/gp_distribution_policy.html.md.erb
index 6b227e6..4b205f1 100644
--- a/markdown/reference/catalog/gp_distribution_policy.html.md.erb
+++ b/markdown/reference/catalog/gp_distribution_policy.html.md.erb
@@ -2,6 +2,25 @@
title: gp_distribution_policy
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The `gp_distribution_policy` table contains information about HAWQ tables and their policy for distributing table data across the segments. This table is populated only on the master. This table is not globally shared, meaning each database has its own copy of this table.
<a id="topic1__ey142842"></a>
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/catalog/gp_global_sequence.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/catalog/gp_global_sequence.html.md.erb b/markdown/reference/catalog/gp_global_sequence.html.md.erb
index ac2ff2f..7774e09 100644
--- a/markdown/reference/catalog/gp_global_sequence.html.md.erb
+++ b/markdown/reference/catalog/gp_global_sequence.html.md.erb
@@ -2,6 +2,25 @@
title: gp_global_sequence
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The `gp_global_sequence` table contains the log sequence number position in the transaction log. This table is used by persistent tables.
<a id="topic1__fe138428"></a>
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/catalog/gp_master_mirroring.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/catalog/gp_master_mirroring.html.md.erb b/markdown/reference/catalog/gp_master_mirroring.html.md.erb
index fa7ea18..fbb5fce 100644
--- a/markdown/reference/catalog/gp_master_mirroring.html.md.erb
+++ b/markdown/reference/catalog/gp_master_mirroring.html.md.erb
@@ -2,6 +2,25 @@
title: gp_master_mirroring
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The `gp_master_mirroring` table contains state information about the standby master host and its associated write-ahead log (WAL) replication process. If this synchronization process (`gpsyncagent`) fails on the standby master, it may not always be noticeable to users of the system. This catalog is a place where HAWQ administrators can check to see if the standby master is current and fully synchronized.
<a id="topic1__op164584"></a>
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/catalog/gp_persistent_database_node.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/catalog/gp_persistent_database_node.html.md.erb b/markdown/reference/catalog/gp_persistent_database_node.html.md.erb
index 2a3537a..3f154a0 100644
--- a/markdown/reference/catalog/gp_persistent_database_node.html.md.erb
+++ b/markdown/reference/catalog/gp_persistent_database_node.html.md.erb
@@ -2,6 +2,25 @@
title: gp_persistent_database_node
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The `gp_persistent_database_node` table keeps track of the status of file system objects in relation to the transaction status of database objects. This information is used to make sure the state of the system catalogs and the file system files persisted to disk are synchronized.
<a id="topic1__fi138428"></a>
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/catalog/gp_persistent_filespace_node.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/catalog/gp_persistent_filespace_node.html.md.erb b/markdown/reference/catalog/gp_persistent_filespace_node.html.md.erb
index b682a88..6033aff 100644
--- a/markdown/reference/catalog/gp_persistent_filespace_node.html.md.erb
+++ b/markdown/reference/catalog/gp_persistent_filespace_node.html.md.erb
@@ -2,6 +2,25 @@
title: gp_persistent_filespace_node
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The `gp_persistent_filespace_node` table keeps track of the status of file system objects in relation to the transaction status of filespace objects. This information is used to make sure the state of the system catalogs and the file system files persisted to disk are synchronized.
<a id="topic1__fj138428"></a>
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/catalog/gp_persistent_relation_node.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/catalog/gp_persistent_relation_node.html.md.erb b/markdown/reference/catalog/gp_persistent_relation_node.html.md.erb
index f141cf7..a7c5041 100644
--- a/markdown/reference/catalog/gp_persistent_relation_node.html.md.erb
+++ b/markdown/reference/catalog/gp_persistent_relation_node.html.md.erb
@@ -2,6 +2,25 @@
title: gp_persistent_relation_node
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The `gp_persistent_relation_node` table table keeps track of the status of file system objects in relation to the transaction status of relation objects (tables, view, indexes, and so on). This information is used to make sure the state of the system catalogs and the file system files persisted to disk are synchronized.
<a id="topic1__fk138428"></a>
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/catalog/gp_persistent_relfile_node.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/catalog/gp_persistent_relfile_node.html.md.erb b/markdown/reference/catalog/gp_persistent_relfile_node.html.md.erb
index 6d24a41..90c0e26 100644
--- a/markdown/reference/catalog/gp_persistent_relfile_node.html.md.erb
+++ b/markdown/reference/catalog/gp_persistent_relfile_node.html.md.erb
@@ -2,6 +2,25 @@
title: gp_persistent_relfile_node
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The `gp_persistent_relfile_node` table keeps track of the status of file system objects in relation to the transaction status of database objects. This information is used to make sure the state of the system catalogs and the file system files persisted to disk are synchronized.
<a id="topic1__fk138428"></a>
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/catalog/gp_persistent_tablespace_node.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/catalog/gp_persistent_tablespace_node.html.md.erb b/markdown/reference/catalog/gp_persistent_tablespace_node.html.md.erb
index 55c853e..1385b67 100644
--- a/markdown/reference/catalog/gp_persistent_tablespace_node.html.md.erb
+++ b/markdown/reference/catalog/gp_persistent_tablespace_node.html.md.erb
@@ -2,6 +2,25 @@
title: gp_persistent_tablespace_node
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The `gp_persistent_tablespace_node` table keeps track of the status of file system objects in relation to the transaction status of tablespace objects. This information is used to make sure the state of the system catalogs and the file system files persisted to disk are synchronized.
<a id="topic1__fm138428"></a>
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/catalog/gp_relfile_node.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/catalog/gp_relfile_node.html.md.erb b/markdown/reference/catalog/gp_relfile_node.html.md.erb
index 971ff16..2958fc0 100644
--- a/markdown/reference/catalog/gp_relfile_node.html.md.erb
+++ b/markdown/reference/catalog/gp_relfile_node.html.md.erb
@@ -2,6 +2,25 @@
title: gp_relfile_node
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The `gp_relfile_node` table contains information about the file system objects for a relation (table, view, index, and so on).
<a id="topic1__fo138428"></a>
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/catalog/gp_segment_configuration.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/catalog/gp_segment_configuration.html.md.erb b/markdown/reference/catalog/gp_segment_configuration.html.md.erb
index 9fcf0cb..020510c 100644
--- a/markdown/reference/catalog/gp_segment_configuration.html.md.erb
+++ b/markdown/reference/catalog/gp_segment_configuration.html.md.erb
@@ -2,6 +2,25 @@
title: gp_segment_configuration
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The `gp_segment_configuration` table contains information about master, standby and segment configuration.
The HAWQ fault tolerance service (FTS) automatically detects the status of individual segments and marks the status of each segment in this table. If a segment is marked as DOWN, the corresponding reason is recorded in the [gp\_configuration\_history](gp_configuration_history.html) table. See [Understanding the Fault Tolerance Service](../../admin/FaultTolerance.html) for a description of the fault tolerance service.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/catalog/gp_version_at_initdb.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/catalog/gp_version_at_initdb.html.md.erb b/markdown/reference/catalog/gp_version_at_initdb.html.md.erb
index 9e922e1..e24ab6b 100644
--- a/markdown/reference/catalog/gp_version_at_initdb.html.md.erb
+++ b/markdown/reference/catalog/gp_version_at_initdb.html.md.erb
@@ -2,6 +2,25 @@
title: gp_version_at_initdb
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The `gp_version_at_initdb` table is populated on the master and each segment in the HAWQ system. It identifies the version of HAWQ used when the system was first initialized. This table is defined in the `pg_global` tablespace, meaning it is globally shared across all databases in the system.
<a id="topic1__ft142845"></a>
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/catalog/pg_aggregate.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/catalog/pg_aggregate.html.md.erb b/markdown/reference/catalog/pg_aggregate.html.md.erb
index 1a02e89..6341c7b 100644
--- a/markdown/reference/catalog/pg_aggregate.html.md.erb
+++ b/markdown/reference/catalog/pg_aggregate.html.md.erb
@@ -2,6 +2,25 @@
title: pg_aggregate
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The `pg_aggregate` table stores information about aggregate functions. An aggregate function is a function that operates on a set of values (typically one column from each row that matches a query condition) and returns a single value computed from all these values. Typical aggregate functions are `sum`, `count`, and `max`. Each entry in `pg_aggregate` is an extension of an entry in `pg_proc`. The `pg_proc` entry carries the aggregate's name, input and output data types, and other information that is similar to ordinary functions.
<a id="topic1__fu141982"></a>
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/catalog/pg_am.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/catalog/pg_am.html.md.erb b/markdown/reference/catalog/pg_am.html.md.erb
index 96ba56b..8a72f35 100644
--- a/markdown/reference/catalog/pg_am.html.md.erb
+++ b/markdown/reference/catalog/pg_am.html.md.erb
@@ -2,6 +2,25 @@
title: pg_am
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The `pg_am` table stores information about index access methods. There is one row for each index access method supported by the system.
<a id="topic1__fv141982"></a>
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/catalog/pg_amop.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/catalog/pg_amop.html.md.erb b/markdown/reference/catalog/pg_amop.html.md.erb
index c07bcd1..431cecf 100644
--- a/markdown/reference/catalog/pg_amop.html.md.erb
+++ b/markdown/reference/catalog/pg_amop.html.md.erb
@@ -2,6 +2,25 @@
title: pg_amop
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The `pg_amop` table stores information about operators associated with index access method operator classes. There is one row for each operator that is a member of an operator class.
<a id="topic1__fw143542"></a>
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/catalog/pg_amproc.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/catalog/pg_amproc.html.md.erb b/markdown/reference/catalog/pg_amproc.html.md.erb
index 7668bac..8f8b27c 100644
--- a/markdown/reference/catalog/pg_amproc.html.md.erb
+++ b/markdown/reference/catalog/pg_amproc.html.md.erb
@@ -2,6 +2,25 @@
title: pg_amproc
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The `pg_amproc` table stores information about support procedures associated with index access method operator classes. There is one row for each support procedure belonging to an operator class.
<a id="topic1__fx143898"></a>
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/catalog/pg_appendonly.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/catalog/pg_appendonly.html.md.erb b/markdown/reference/catalog/pg_appendonly.html.md.erb
index b0a56d6..6b031f9 100644
--- a/markdown/reference/catalog/pg_appendonly.html.md.erb
+++ b/markdown/reference/catalog/pg_appendonly.html.md.erb
@@ -2,6 +2,25 @@
title: pg_appendonly
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The `pg_appendonly` table contains information about the storage options and other characteristics of append-only tables.
<a id="topic1__fy138428"></a>
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/catalog/pg_attrdef.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/catalog/pg_attrdef.html.md.erb b/markdown/reference/catalog/pg_attrdef.html.md.erb
index f4af77d..27e9671 100644
--- a/markdown/reference/catalog/pg_attrdef.html.md.erb
+++ b/markdown/reference/catalog/pg_attrdef.html.md.erb
@@ -2,6 +2,25 @@
title: pg_attrdef
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The `pg_attrdef` table stores column default values. The main information about columns is stored in [pg\_attribute](pg_attribute.html#topic1). Only columns that explicitly specify a default value (when the table is created or the column is added) will have an entry here.
<a id="topic1__fz143898"></a>
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/catalog/pg_attribute.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/catalog/pg_attribute.html.md.erb b/markdown/reference/catalog/pg_attribute.html.md.erb
index 53db267..9b6c1a8 100644
--- a/markdown/reference/catalog/pg_attribute.html.md.erb
+++ b/markdown/reference/catalog/pg_attribute.html.md.erb
@@ -2,6 +2,25 @@
title: pg_attribute
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The `pg_attribute` table stores information about table columns. There will be exactly one `pg_attribute` row for every column in every table in the database. (There will also be attribute entries for indexes, and all objects that have `pg_class` entries.) The term attribute is equivalent to column.
<a id="topic1__ga143898"></a>
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/catalog/pg_attribute_encoding.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/catalog/pg_attribute_encoding.html.md.erb b/markdown/reference/catalog/pg_attribute_encoding.html.md.erb
index 3067a93..48517e4 100644
--- a/markdown/reference/catalog/pg_attribute_encoding.html.md.erb
+++ b/markdown/reference/catalog/pg_attribute_encoding.html.md.erb
@@ -2,6 +2,25 @@
title: pg_attribute_encoding
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The `pg_attribute_encoding` system catalog table contains column storage information.
<a id="topic1__gb177839"></a>
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/catalog/pg_auth_members.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/catalog/pg_auth_members.html.md.erb b/markdown/reference/catalog/pg_auth_members.html.md.erb
index 7e770e0..ee8cf3c 100644
--- a/markdown/reference/catalog/pg_auth_members.html.md.erb
+++ b/markdown/reference/catalog/pg_auth_members.html.md.erb
@@ -2,6 +2,25 @@
title: pg_auth_members
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The `pg_auth_members` system catalog table shows the membership relations between roles. Any non-circular set of relationships is allowed. Because roles are system-wide, `pg_auth_members` is shared across all databases of a HAWQ system.
<a id="topic1__gc143898"></a>
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/catalog/pg_authid.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/catalog/pg_authid.html.md.erb b/markdown/reference/catalog/pg_authid.html.md.erb
index ebae67c..2f1e8a8 100644
--- a/markdown/reference/catalog/pg_authid.html.md.erb
+++ b/markdown/reference/catalog/pg_authid.html.md.erb
@@ -2,6 +2,25 @@
title: pg_authid
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The `pg_authid` table contains information about database authorization identifiers (roles). A role subsumes the concepts of users and groups. A user is a role with the `rolcanlogin` flag set. Any role (with or without `rolcanlogin`) may have other roles as members. See [pg\_auth\_members](pg_auth_members.html#topic1).
Since this catalog contains passwords, it must not be publicly readable. [pg\_roles](pg_roles.html#topic1) is a publicly readable view on `pg_authid` that blanks out the password field.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/catalog/pg_cast.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/catalog/pg_cast.html.md.erb b/markdown/reference/catalog/pg_cast.html.md.erb
index 513c7a3..d5adf9d 100644
--- a/markdown/reference/catalog/pg_cast.html.md.erb
+++ b/markdown/reference/catalog/pg_cast.html.md.erb
@@ -2,6 +2,25 @@
title: pg_cast
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The `pg_cast` table stores data type conversion paths, both built-in paths and those defined with `CREATE CAST`. The cast functions listed in `pg_cast` must always take the cast source type as their first argument type, and return the cast destination type as their result type. A cast function can have up to three arguments. The second argument, if present, must be type `integer`; it receives the type modifier associated with the destination type, or `-1` if there is none. The third argument, if present, must be type `boolean`; it receives `true` if the cast is an explicit cast, `false` otherwise.
It is legitimate to create a `pg_cast` entry in which the source and target types are the same, if the associated function takes more than one argument. Such entries represent 'length coercion functions' that coerce values of the type to be legal for a particular type modifier value. Note however that at present there is no support for associating non-default type modifiers with user-created data types, and so this facility is only of use for the small number of built-in types that have type modifier syntax built into the grammar.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/catalog/pg_class.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/catalog/pg_class.html.md.erb b/markdown/reference/catalog/pg_class.html.md.erb
index 112375e..23a7dc8 100644
--- a/markdown/reference/catalog/pg_class.html.md.erb
+++ b/markdown/reference/catalog/pg_class.html.md.erb
@@ -2,6 +2,25 @@
title: pg_class
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The system catalog table `pg_class` catalogs tables and most everything else that has columns or is otherwise similar to a table (also known as *relations*). This includes indexes (see also [pg\_index](pg_index.html#topic1)), sequences, views, composite types, and TOAST tables. Not all columns are meaningful for all relation types.
<a id="topic1__gf143898"></a>
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/catalog/pg_compression.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/catalog/pg_compression.html.md.erb b/markdown/reference/catalog/pg_compression.html.md.erb
index 3524af0..f03ba41 100644
--- a/markdown/reference/catalog/pg_compression.html.md.erb
+++ b/markdown/reference/catalog/pg_compression.html.md.erb
@@ -2,6 +2,25 @@
title: pg_compression
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The `pg_compression` system catalog table describes the compression methods available..
<a id="topic1__gg143898"></a>
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/catalog/pg_constraint.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/catalog/pg_constraint.html.md.erb b/markdown/reference/catalog/pg_constraint.html.md.erb
index 0f591fd..8db725b 100644
--- a/markdown/reference/catalog/pg_constraint.html.md.erb
+++ b/markdown/reference/catalog/pg_constraint.html.md.erb
@@ -2,6 +2,25 @@
title: pg_constraint
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The `pg_constraint` system catalog table stores check and foreign key constraints on tables. Column constraints are not treated specially. Every column constraint is equivalent to some table constraint. Not-null constraints are represented in the [pg\_attribute](pg_attribute.html#topic1) catalog table. Check constraints on domains are stored here, too.
<a id="topic1__gh143898"></a>
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/catalog/pg_conversion.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/catalog/pg_conversion.html.md.erb b/markdown/reference/catalog/pg_conversion.html.md.erb
index 43763bc..6e26377 100644
--- a/markdown/reference/catalog/pg_conversion.html.md.erb
+++ b/markdown/reference/catalog/pg_conversion.html.md.erb
@@ -2,6 +2,25 @@
title: pg_conversion
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The `pg_conversion` system catalog table describes the available encoding conversion procedures as defined by `CREATE CONVERSION`.
<a id="topic1__gi143898"></a>
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/catalog/pg_database.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/catalog/pg_database.html.md.erb b/markdown/reference/catalog/pg_database.html.md.erb
index b02a532..280af6a 100644
--- a/markdown/reference/catalog/pg_database.html.md.erb
+++ b/markdown/reference/catalog/pg_database.html.md.erb
@@ -2,6 +2,25 @@
title: pg_database
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The `pg_database` system catalog table stores information about the available databases. Databases are created with the `CREATE DATABASE` SQL command. Unlike most system catalogs, `pg_database` is shared across all databases in the system. There is only one copy of `pg_database` per system, not one per database.
<a id="topic1__gj143898"></a>
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/catalog/pg_depend.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/catalog/pg_depend.html.md.erb b/markdown/reference/catalog/pg_depend.html.md.erb
index 85a1835..c398c37 100644
--- a/markdown/reference/catalog/pg_depend.html.md.erb
+++ b/markdown/reference/catalog/pg_depend.html.md.erb
@@ -2,6 +2,25 @@
title: pg_depend
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The `pg_depend` system catalog table records the dependency relationships between database objects. This information allows `DROP` commands to find which other objects must be dropped by `DROP CASCADE` or prevent dropping in the `DROP RESTRICT` case. See also [pg\_shdepend](pg_shdepend.html#topic1), which performs a similar function for dependencies involving objects that are shared across a HAWQ system.
In all cases, a `pg_depend` entry indicates that the referenced object may not be dropped without also dropping the dependent object. However, there are several subflavors identified by `deptype`:
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/catalog/pg_description.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/catalog/pg_description.html.md.erb b/markdown/reference/catalog/pg_description.html.md.erb
index bad9627..d426f0b 100644
--- a/markdown/reference/catalog/pg_description.html.md.erb
+++ b/markdown/reference/catalog/pg_description.html.md.erb
@@ -2,6 +2,25 @@
title: pg_description
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The `pg_description` system catalog table stores optional descriptions (comments) for each database object. Descriptions can be manipulated with the `COMMENT` command and viewed with `psql`'s `\d` meta-commands. Descriptions of many built-in system objects are provided in the initial contents of `pg_description`. See also [pg\_shdescription](pg_shdescription.html#topic1), which performs a similar function for descriptions involving objects that are shared across a HAWQ system.
<a id="topic1__gm143898"></a>
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/catalog/pg_exttable.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/catalog/pg_exttable.html.md.erb b/markdown/reference/catalog/pg_exttable.html.md.erb
index ca5fc88..3d7245d 100644
--- a/markdown/reference/catalog/pg_exttable.html.md.erb
+++ b/markdown/reference/catalog/pg_exttable.html.md.erb
@@ -2,6 +2,25 @@
title: pg_exttable
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The `pg_exttable` system catalog table is used to track external tables and web tables created by the `CREATE EXTERNAL TABLE` command.
<a id="topic1__gn143898"></a>
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/catalog/pg_filespace.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/catalog/pg_filespace.html.md.erb b/markdown/reference/catalog/pg_filespace.html.md.erb
index e0b810e..9f700fd 100644
--- a/markdown/reference/catalog/pg_filespace.html.md.erb
+++ b/markdown/reference/catalog/pg_filespace.html.md.erb
@@ -2,6 +2,25 @@
title: pg_filespace
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The `pg_filespace` table contains information about the filespaces created in a HAWQ system. Every system contains a default filespace, `pg_system`, which is a collection of all the data directory locations created at system initialization time.
A tablespace requires a file system location to store its database files. In HAWQ, the master and each segment needs its own distinct storage location. This collection of file system locations for all components in a HAWQ system is referred to as a filespace.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/catalog/pg_filespace_entry.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/catalog/pg_filespace_entry.html.md.erb b/markdown/reference/catalog/pg_filespace_entry.html.md.erb
index 5a45113..d9ea302 100644
--- a/markdown/reference/catalog/pg_filespace_entry.html.md.erb
+++ b/markdown/reference/catalog/pg_filespace_entry.html.md.erb
@@ -2,6 +2,25 @@
title: pg_filespace_entry
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
A tablespace requires a file system location to store its database files. In HAWQ, the master and each segment needs its own distinct storage location. This collection of file system locations for all components in a HAWQ system is referred to as a *filespace*. The `pg_filespace_entry` table contains information about the collection of file system locations across a HAWQ system that comprise a HAWQ filespace.
<a id="topic1__gp138428"></a>
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/catalog/pg_index.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/catalog/pg_index.html.md.erb b/markdown/reference/catalog/pg_index.html.md.erb
index e93bd86..d94fe64 100644
--- a/markdown/reference/catalog/pg_index.html.md.erb
+++ b/markdown/reference/catalog/pg_index.html.md.erb
@@ -2,6 +2,25 @@
title: pg_index
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The `pg_index` system catalog table contains part of the information about indexes. The rest is mostly in [pg\_class](pg_class.html#topic1).
<a id="topic1__gq143898"></a>
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/a42c05a9/markdown/reference/catalog/pg_inherits.html.md.erb
----------------------------------------------------------------------
diff --git a/markdown/reference/catalog/pg_inherits.html.md.erb b/markdown/reference/catalog/pg_inherits.html.md.erb
index 9868602..ec4c992 100644
--- a/markdown/reference/catalog/pg_inherits.html.md.erb
+++ b/markdown/reference/catalog/pg_inherits.html.md.erb
@@ -2,6 +2,25 @@
title: pg_inherits
---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
The `pg_inherits` system catalog table records information about table inheritance hierarchies. There is one entry for each direct child table in the database. (Indirect inheritance can be determined by following chains of entries.) In HAWQ, inheritance relationships are created by both the `INHERITS` clause (standalone inheritance) and the `PARTITION BY` clause (partitioned child table inheritance) of `CREATE TABLE`.
<a id="topic1__gr143898"></a>