You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hawq.apache.org by yo...@apache.org on 2016/09/30 23:47:53 UTC
[2/6] incubator-hawq-docs git commit: Change yml terms, xref,
syntax corrections
Change yml terms, xref, syntax corrections
Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/commit/cbc83e12
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/tree/cbc83e12
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/diff/cbc83e12
Branch: refs/heads/develop
Commit: cbc83e1237aa7f6143c16eed269ac4b2fae6871b
Parents: deb1c4b
Author: Jane Beckman <jb...@pivotal.io>
Authored: Thu Sep 29 11:52:49 2016 -0700
Committer: Jane Beckman <jb...@pivotal.io>
Committed: Thu Sep 29 11:52:49 2016 -0700
----------------------------------------------------------------------
datamgmt/load/g-register_files.html.md.erb | 23 +++++++++----------
reference/.DS_Store | Bin 6148 -> 0 bytes
.../cli/admin_utilities/hawqextract.html.md.erb | 4 ++--
.../admin_utilities/hawqregister.html.md.erb | 11 ++++-----
4 files changed, 18 insertions(+), 20 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/cbc83e12/datamgmt/load/g-register_files.html.md.erb
----------------------------------------------------------------------
diff --git a/datamgmt/load/g-register_files.html.md.erb b/datamgmt/load/g-register_files.html.md.erb
index 9abcbe2..94d0140 100644
--- a/datamgmt/load/g-register_files.html.md.erb
+++ b/datamgmt/load/g-register_files.html.md.erb
@@ -43,7 +43,7 @@ In this example, the location of the database is `hdfs://localhost:8020/hawq_def
Enter:
``` pre
-$ hawq register postgres -f hdfs://localhost:8020/temp/hive.paq parquet_table
+$ hawq register -d postgres -f hdfs://localhost:8020/temp/hive.paq parquet_table
```
After running the `hawq register` command for the file location `hdfs://localhost:8020/temp/hive.paq`, the corresponding new location of the file in HDFS is: `hdfs://localhost:8020/hawq_default/16385/16387/77160/8`.
@@ -64,29 +64,28 @@ then running
select relname from pg_class where oid = segrelid
```
-##Registering Data Using Information from a .yml Configuration File<a id="topic1__section3"></a>
+##Registering Data Using Information from a YAML Configuration File<a id="topic1__section3"></a>
-The `hawq register` command can register HDFS files by using metadata loaded from a .yml configuration file by using the `--config <yml_config\>` option. Both AO and Parquet tables can be registered. Tables need not exist in HAWQ before being registered. This function can be useful in disaster recovery, allowing information created by the `hawq extract` command to be used to re-create HAWQ tables.
+The `hawq register` command can register HDFS files by using metadata loaded from a YAML configuration file by using the `--config <yaml_config\>` option. Both AO and Parquet tables can be registered. Tables need not exist in HAWQ before being registered. This function can be useful in disaster recovery, allowing information created by the `hawq extract` command to be used to re-create HAWQ tables.
-You can also use a .yml confguration file to append HDFS files to an existing HAWQ table or create a table and register it into HAWQ.
+You can also use a YAML confguration file to append HDFS files to an existing HAWQ table or create a table and register it into HAWQ.
-For disaster recovery, tables can be re-registered using the HDFS files and a .yml file. The clusters are assumed to have data periodically imported from Cluster A to Cluster B.
+For disaster recovery, tables can be re-registered using the HDFS files and a YAML file. The clusters are assumed to have data periodically imported from Cluster A to Cluster B.
Data is registered according to the following conditions:
- Existing tables have files appended to the existing HAWQ table.
-- If a table does not exist, it is created and registered into HAWQ. The catalog table will be updated with the file size specified by the .yml file.
+- If a table does not exist, it is created and registered into HAWQ. The catalog table will be updated with the file size specified by the YAML file.
- If the --force option is used, the data in existing catalog tables is erased and re-registered. All HDFS-related catalog contents in `pg_aoseg.pg_paqseg_$relid ` are cleared. The original files on HDFS are retained.
-- If the --repair option is used, data is rolled back to a previous state, as specified in the .yml file. Any files generated after the checkpoint specified in the .yml file will be erased. Both the file on HDFS and its metadata are erased.
-Tables using random distribution are preferred for registering into HAWQ. If hash tables are to be registered, the distribution policy in the .yml file must match that of the table being registered into.
+Tables using random distribution are preferred for registering into HAWQ. If hash tables are to be registered, the distribution policy in the YAML file must match that of the table being registered into.
-In registering hash tables, the size of the registered file should be identical to or a multiple of the hash table bucket number. When registering hash distributed tables using a .yml file, the order of the files in the .yml file should reflect the hash distribution.
+In registering hash tables, the size of the registered file should be identical to or a multiple of the hash table bucket number. When registering hash distributed tables using a YAML file, the order of the files in the YAML file should reflect the hash distribution.
-###Example: Registration using a .yml Configuration File
+###Example: Registration using a YAML Configuration File
-This example shows how to use hawq register to register HDFS data using a .yml configuration file generated by hawq extract.
+This example shows how to use hawq register to register HDFS data using a YAML configuration file generated by hawq extract.
First, create a table in SQL and insert some data into it.
@@ -106,7 +105,7 @@ Go into the hawq administration utilities, and extract the table metadata by usi
hawq extract -o paq1.yml paq1
```
-Register the data into new table paq2, using the --config option to identify the .yml file.
+Register the data into new table paq2, using the --config option to identify the YAML file.
```
hawq register --config paq1.yml paq2
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/cbc83e12/reference/.DS_Store
----------------------------------------------------------------------
diff --git a/reference/.DS_Store b/reference/.DS_Store
deleted file mode 100644
index dcca349..0000000
Binary files a/reference/.DS_Store and /dev/null differ
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/cbc83e12/reference/cli/admin_utilities/hawqextract.html.md.erb
----------------------------------------------------------------------
diff --git a/reference/cli/admin_utilities/hawqextract.html.md.erb b/reference/cli/admin_utilities/hawqextract.html.md.erb
index d638fbf..5b10818 100644
--- a/reference/cli/admin_utilities/hawqextract.html.md.erb
+++ b/reference/cli/admin_utilities/hawqextract.html.md.erb
@@ -26,7 +26,7 @@ where:
## Description
-`hawq extract` is a utility that extracts a table's metadata into a YAML formatted file. HAWQ's InputFormat uses this YAML-formatted file to read a HAWQ file stored on HDFS directly into the MapReduce program.
+`hawq extract` is a utility that extracts a table's metadata into a YAML formatted file. HAWQ's InputFormat uses this YAML-formatted file to read a HAWQ file stored on HDFS directly into the MapReduce program. The YAML configuration file can also be used provide the metadata for registering files into HAWQ with the `hawq register` command.
**Note:**
`hawq extract` is bound by the following rules:
@@ -288,6 +288,6 @@ Parquet_FileLocations:
## See Also
-[hawq load](hawqload.html#topic1)
+[hawq load](hawqload.html#topic1), [hawq register](hawqregister.html#topic1)
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/cbc83e12/reference/cli/admin_utilities/hawqregister.html.md.erb
----------------------------------------------------------------------
diff --git a/reference/cli/admin_utilities/hawqregister.html.md.erb b/reference/cli/admin_utilities/hawqregister.html.md.erb
index 38b88aa..de64a11 100644
--- a/reference/cli/admin_utilities/hawqregister.html.md.erb
+++ b/reference/cli/admin_utilities/hawqregister.html.md.erb
@@ -24,7 +24,6 @@ Misc. Options:
[-f <filepath>]
[-e <eof>]
[--force]
- [--repair]
[-c <yml_config>]
hawq register help | -?
hawq register --version
@@ -60,11 +59,11 @@ Metadata for the Parquet file(s) and the destination table must be consistent. D
Only HAWQ or Hive-generated Parquet tables are supported.
Hash tables and artitioned tables are not supported in this use model.
-###Usage Model 2: Use information from a .yml configuration file to register data
+###Usage Model 2: Use information from a YAML configuration file to register data
`hawq register [-h hostname] [-p port] [-U username] [-d databasename] [-c configfile] [--force] <tablename>`
-Files generated by the `hawq extract` command are registered through use of metadata in a .yml configuration file. Both AO and Parquet tables can be registered. Tables need not exist in HAWQ before being registered.
+Files generated by the `hawq extract` command are registered through use of metadata in a YAML configuration file. Both AO and Parquet tables can be registered. Tables need not exist in HAWQ before being registered.
The register process behaves differently, according to different conditions.
@@ -198,7 +197,7 @@ group {
<dd>The path of the file or directory in HDFS containing the files to be registered.</dd>
<dt>\<tablename\> </dt>
-<dd>The HAWQ table that will store the data to be registered. If the --config option is not supplied, the table cannot use hash distribution. Random table distribution is strongly preferred. If hash distribution must be used, make sure that the distribution policy for the data files described in the .yml file is consistent with the table being registered into.</dd>
+<dd>The HAWQ table that will store the data to be registered. If the --config option is not supplied, the table cannot use hash distribution. Random table distribution is strongly preferred. If hash distribution must be used, make sure that the distribution policy for the data files described in the YAML file is consistent with the table being registered into.</dd>
####Miscellaneous Options
@@ -216,7 +215,7 @@ The following options are used with specific use models.
## <a id="topic1__section6"></a>Example: Usage Model 2
-This example shows how to register files using a .yml configuration file. This file is usually generated by the `hawq extract` command.
+This example shows how to register files using a YAML configuration file. This file is usually generated by the `hawq extract` command.
In SQL, create a table and insert data into the table:
@@ -231,7 +230,7 @@ In HAWQ, extract the table's metadata.
hawq extract -o paq1.yml paq1
```
-In HAWQ, use the .yml file to register the new table paq2:
+In HAWQ, use the YAML file to register the new table paq2:
```
hawq register --config paq1.yml paq2