You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hawq.apache.org by "Lili Ma (JIRA)" <ji...@apache.org> on 2016/11/03 08:59:59 UTC
[jira] [Created] (HAWQ-1144) Register into a 2-level partition
table, hawq register didn't throw error, and indicates that hawq register
succeed, but no data can be selected out.
Lili Ma created HAWQ-1144:
-----------------------------
Summary: Register into a 2-level partition table, hawq register didn't throw error, and indicates that hawq register succeed, but no data can be selected out.
Key: HAWQ-1144
URL: https://issues.apache.org/jira/browse/HAWQ-1144
Project: Apache HAWQ
Issue Type: Bug
Components: Command Line Tools
Reporter: Lili Ma
Assignee: Lei Chang
Fix For: 2.0.1.0-incubating
Register into a 2-level partition table, hawq register didn't throw error, and indicates that hawq register succeed, but no data can be selected out.
Reproduce Steps:
1. Create a one-level partition table
```
create table parquet_wt (id SERIAL,a1 int,a2 char(5),a3 numeric,a4 boolean DEFAULT false ,a5 char DEFAULT 'd',a6 text,a7 timestamp,a8 character varying(705),a9 bigint,a10 date,a11 varchar(600),a12 text,a13 decimal,a14 real,a15 bigint,a16 int4 ,a17 bytea,a18 timestamp with time zone,a19 timetz,a20 path,a21 box,a22 macaddr,a23 interval,a24 character varying(800),a25 lseg,a26 point,a27 double precision,a28 circle,a29 int4,a30 numeric(8),a31 polygon,a32 date,a33 real,a34 money,a35 cidr,a36 inet,a37 time,a38 text,a39 bit,a40 bit varying(5),a41 smallint,a42 int ) WITH (appendonly=true, orientation=parquet) distributed randomly Partition by range(a1) (start(1) end(5000) every(1000) );
```
2. insert some data into this table
```
insert into parquet_wt (a1,a2,a3,a4,a5,a6,a7,a8,a9,a10,a11,a12,a13,a14,a15,a16,a17,a18,a19,a20,a21,a22,a23,a24,a25,a26,a27,a28,a29,a30,a31,a32,a33,a34,a35,a36,a37,a38,a39,a40,a41,a42) values(generate_series(1,20),'M',2011,'t','a','This is news of today: Deadlock between Republicans and Democrats over how best to reduce the U.S. deficit, and over what period, has blocked an agreement to allow the raising of the $14.3 trillion debt ceiling','2001-12-24 02:26:11','U.S. House of Representatives Speaker John Boehner, the top Republican in Congress who has put forward a deficit reduction plan to be voted on later on Thursday said he had no control over whether his bill would avert a credit downgrade.',generate_series(2490,2505),'2011-10-11','The Republican-controlled House is tentatively scheduled to vote on Boehner proposal this afternoon at around 6 p.m. EDT (2200 GMT). The main Republican vote counter in the House, Kevin McCarthy, would not say if there were enough votes to pass the bill.','WASHINGTON:House Speaker John Boehner says his plan mixing spending cuts in exchange for raising the nations $14.3 trillion debt limit is not perfect but is as large a step that a divided government can take that is doable and signable by President Barack Obama.The Ohio Republican says the measure is an honest and sincere attempt at compromise and was negotiated with Democrats last weekend and that passing it would end the ongoing debt crisis. The plan blends $900 billion-plus in spending cuts with a companion increase in the nations borrowing cap.','1234.56',323453,generate_series(3452,3462),7845,'0011','2005-07-16 01:51:15+1359','2001-12-13 01:51:15','((1,2),(0,3),(2,1))','((2,3)(4,5))','08:00:2b:01:02:03','1-2','Republicans had been working throughout the day Thursday to lock down support for their plan to raise the nations debt ceiling, even as Senate Democrats vowed to swiftly kill it if passed.','((2,3)(4,5))','(6,7)',11.222,'((4,5),7)',32,3214,'(1,0,2,3)','2010-02-21',43564,'$1,000.00','192.168.1','126.1.3.4','12:30:45','Johnson & Johnsons McNeil Consumer Healthcare announced the voluntary dosage reduction today. Labels will carry new dosing instructions this fall.The company says it will cut the maximum dosage of Regular Strength Tylenol and other acetaminophen-containing products in 2012.Acetaminophen is safe when used as directed, says Edwin Kuffner, MD, McNeil vice president of over-the-counter medical affairs. But, when too much is taken, it can cause liver damage.The action is intended to cut the risk of such accidental overdoses, the company says in a news release.','1','0',12,23);
```
3. extract the metadata out for the table
```
hawq extract -d postgres -o ~/parquet.yaml parquet_wt
```
4. create a two-level partition table
```
CREATE TABLE parquet_wt_subpartgzip2 (id SERIAL,a1 int,a2 char(5),a3 numeric,a4 boolean DEFAULT false ,a5 char DEFAULT 'd',a6 text,a7 timestamp,a8 character varying(705),a9 bigint,a10 date,a11 varchar(600),a12 text,a13 decimal,a14 real,a15 bigint,a16 int4 ,a17 bytea,a18 timestamp with time zone,a19 timetz,a20 path,a21 box,a22 macaddr,a23 interval,a24 character varying(800),a25 lseg,a26 point,a27 double precision,a28 circle,a29 int4,a30 numeric(8),a31 polygon,a32 date,a33 real,a34 money,a35 cidr,a36 inet,a37 time,a38 text,a39 bit,a40 bit varying(5),a41 smallint,a42 int ) WITH (appendonly=true, orientation=parquet) distributed randomly Partition by range(a1) Subpartition by list(a2) subpartition template ( default subpartition df_sp, subpartition sp1 values('M') , subpartition sp2 values('F') WITH (appendonly=true, orientation=parquet,compresstype=gzip,compresslevel=2)) (start(1) end(5000) every(1000) );
```
5. call register
```
hawq register -d postgres -c ~/parquet.yaml parquet_wt_subpartgzip2
```
6. It reflects register succeed.
```
malilis-MacBook-Pro:tpch malili$ hawq register -d postgres -c ~/parquet.yaml parquet_wt_subpartgzip2
20161103:15:58:10:083605 hawqregister:malilis-MacBook-Pro:malili-[INFO]:-try to connect database localhost:5432 postgres
20161103:15:58:10:083605 hawqregister:malilis-MacBook-Pro:malili-[INFO]:-Files check...
20161103:15:58:11:083605 hawqregister:malilis-MacBook-Pro:malili-[INFO]:-Files check done...
20161103:15:58:11:083605 hawqregister:malilis-MacBook-Pro:malili-[INFO]:-Files check...
20161103:15:58:13:083605 hawqregister:malilis-MacBook-Pro:malili-[INFO]:-Files check done...
20161103:15:58:13:083605 hawqregister:malilis-MacBook-Pro:malili-[INFO]:-Files check...
20161103:15:58:14:083605 hawqregister:malilis-MacBook-Pro:malili-[INFO]:-Files check done...
20161103:15:58:14:083605 hawqregister:malilis-MacBook-Pro:malili-[INFO]:-Files check...
20161103:15:58:16:083605 hawqregister:malilis-MacBook-Pro:malili-[INFO]:-Files check done...
20161103:15:58:16:083605 hawqregister:malilis-MacBook-Pro:malili-[INFO]:-Files check...
20161103:15:58:17:083605 hawqregister:malilis-MacBook-Pro:malili-[INFO]:-Files check done...
20161103:15:58:20:083605 hawqregister:malilis-MacBook-Pro:malili-[INFO]:-New file(s) to be registered: ['hdfs://localhost:8020/hawq_default/16385/16387/17065/1']
20161103:15:58:29:083605 hawqregister:malilis-MacBook-Pro:malili-[INFO]:-New file(s) to be registered: ['hdfs://localhost:8020/hawq_default/16385/16387/17074/1']
20161103:15:58:35:083605 hawqregister:malilis-MacBook-Pro:malili-[INFO]:-New file(s) to be registered: ['hdfs://localhost:8020/hawq_default/16385/16387/17083/1']
20161103:15:58:41:083605 hawqregister:malilis-MacBook-Pro:malili-[INFO]:-New file(s) to be registered: ['hdfs://localhost:8020/hawq_default/16385/16387/17092/1']
20161103:15:58:47:083605 hawqregister:malilis-MacBook-Pro:malili-[INFO]:-New file(s) to be registered: ['hdfs://localhost:8020/hawq_default/16385/16387/17101/1']
hdfscmd: "hadoop fs -mv hdfs://localhost:8020/hawq_default/16385/16387/17065/1 hdfs://localhost:8020/hawq_default/16385/16387/16784/1"
hdfscmd: "hadoop fs -mv hdfs://localhost:8020/hawq_default/16385/16387/17074/1 hdfs://localhost:8020/hawq_default/16385/16387/16822/1"
hdfscmd: "hadoop fs -mv hdfs://localhost:8020/hawq_default/16385/16387/17083/1 hdfs://localhost:8020/hawq_default/16385/16387/16860/1"
hdfscmd: "hadoop fs -mv hdfs://localhost:8020/hawq_default/16385/16387/17092/1 hdfs://localhost:8020/hawq_default/16385/16387/16898/1"
hdfscmd: "hadoop fs -mv hdfs://localhost:8020/hawq_default/16385/16387/17101/1 hdfs://localhost:8020/hawq_default/16385/16387/16936/1"
20161103:15:58:58:083605 hawqregister:malilis-MacBook-Pro:malili-[INFO]:-Hawq Register Succeed.
```
7. But when we select the table, no data can be selected out.
```
postgres=# select count(*) from parquet_wt_subpartgzip2;
count
-------
0
(1 row)
```
Actually we should throw error if hawq register wants to register into a 2 or more -level partition table.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)