[jira] [Commented] (HAWQ-1044) Verify the correctness of hawq register

["Lili Ma (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HAWQ-1044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15486031#comment-15486031 ] 

Lili Ma commented on HAWQ-1044:
-------------------------------

We need design our test cases to verify hawq register from following aspects:
1. partition table/non-partition table,  
2. format: row-oriented/parquet
3. randomly distributed/hash distributed
4. partition policy, range partition or list partition.

> Verify the correctness of hawq register
> ---------------------------------------
>
>                 Key: HAWQ-1044
>                 URL: https://issues.apache.org/jira/browse/HAWQ-1044
>             Project: Apache HAWQ
>          Issue Type: Sub-task
>          Components: Command Line Tools
>            Reporter: Lili Ma
>            Assignee: hongwu
>             Fix For: backlog
>
>
> Verify the correctness of hawq register, summary all the use scenarios and design corresponding test cases for it.
> I think following test cases should be added for the HAWQ register.
> 1. Use Case 1: Register file/folder into HAWQ by specifying file/folder name
> a) hawq register -d postgres -f a.file tableA
> b) hawq register -d postgres -f a.file -e eof tableA
> c) hawq register -d postgres -f folderA tableA
> d) register file to existing table. normal path
> e) register file to existing table. error path: to-be-registered files under the file folder for the existing table on HDFS. Should throw error out.
> f) verify wrong input file. The file format not parquet format.
> 2. Use case 2: Register into HAWQ table using .yml configuration file to a non-existing table
> a) Verify normal input:
> create table a(a int, b int);
> insert into a values(generate_series(1,100), 25);
> hawq extract -d postgres -o a.yml a
> hawq register -d postgres -c a.yml b
> b) Modify the fileSize in .yml file to a value which is different from actual data size of data file
> 3. Use Case 2: Regsiter into HAWQ table using .yml configuration file to an existing table
> a) Verify normal path:
> Call multiple times of hawq register, to verify whether can succeed. Each time the to-be-registered files are not under the table directory.
> b) Error path: to-be-registered files under the file folder for the existing table on HDFS
> Should throw error out: not support!
> 4. Use Case 2: Register into HAWQ table using .yml configuration file by specifying --force option
> a) The table not exist: should create a new table, and do the register
> b) The table already exist, but no data there: can directly call hawq register
> c) Table already exist, and already data there -- normal path: .yml configuration file includes the data files under table directory, and 
> just include those data files.
> d) Table already exist, and already data there -- normal path: .yml configuration file includes the data files under table directory, and 
> also includes data files not under table directory.
> e) Table already exist, and already data there -- error path: .yml configuration file doesn't include the data files under that table directory. 
> Should throw error out, "there are already existing files under the table, but not included in .yml configuration file"
> 5. Use Case 2: Register into HAWQ table using .yml configuration file by specifying --repair option
> a) Normal Path 1: (Append to new file)
> create a tableA
> insert some data into tableA
> call hawq extract the metadata to a.yml file
> insert new data into tableA
> call hawq register --repair option to rollback to the state
> b) Normal Path 2: (New files generated)
> Same as Normal Path 1, but during the second insert, use multiple inserts concurrenly aiming at producing new files. Then call hawq register --repair,
> the new files should be discarded.
> c) Error Path: restributed
> Create a table with hash-distributed, distributed by column A
> insert some data into tableA
> call hawq extract the metadata to a.yml file
> alter table redistributed by column B
> insert new data into tableA
> call hawq register --repair option to rollback to the state  
> --> should throw error "the table is redistributed"
> d) Error Path: table being truncated
> Create a table with hash-distributed, distributed by column A
> insert some data into tableA
> call hawq extract the metadata to a.yml file
> truncate tableA
> call hawq register --repair option to rollback to the state  
> --> should throw error "the table becomes smaller than the .yml config file specified."
> e) Error Path: files specified in .yml configuration not under data directory of table A
> --> should throw error "the files should all under the table directory when --repair option specified for hawq register"
> 6. hawq register partition table support
> a) Normal Path: create a 1-level partition table, calling hawq extract and then hawq register, can work
> b) Error Path: create a 2-level partition table, calling hawq extract and then hawq register, 
> --> should throw error "only supports 1-level partition table"



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)