You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by Thiruvel Thirumoolan <th...@yahoo-inc.com> on 2010/08/06 09:29:48 UTC

Shud partition order be same in create and insert?

Hello,

When the order of partitioning columns is different in create table and insert, I am not able to query for any data. However if the order is the same its possible to see data.

Should partitioning order be maintained through all inserts? As you see below, kv2.txt is still on HDFS and 2 different partition orders are created. Running off hive trunk.

hive> CREATE TABLE invites (foo INT, bar STRING) PARTITIONED BY (ds STRING, country STRING);                          
hive> LOAD DATA LOCAL INPATH '/tmp/kv2.txt' OVERWRITE INTO TABLE invites PARTITION (country='india', ds='2008-08-15');
hive> select * from invites;
hive> 

[thiruvel@hive]$ hadoop fs -lsr /user/hive/warehouse/invites
drwxr-xr-x   - thiruvel supergroup          0 2010-08-06 12:52 /user/hive/warehouse/invites/country=india
drwxr-xr-x   - thiruvel supergroup          0 2010-08-06 12:52 /user/hive/warehouse/invites/country=india/ds=2008-08-15
-rw-r--r--   1 thiruvel supergroup       5791 2010-08-06 12:52 /user/hive/warehouse/invites/country=india/ds=2008-08-15/kv2.txt
drwxr-xr-x   - thiruvel supergroup          0 2010-08-06 12:52 /user/hive/warehouse/invites/ds=2008-08-15
drwxr-xr-x   - thiruvel supergroup          0 2010-08-06 12:52 /user/hive/warehouse/invites/ds=2008-08-15/country=india
[thiruvel@ hive]$ 

Thanks,
Thiruvel

RE: Shud partition order be same in create and insert?

Posted by Namit Jain <nj...@facebook.com>.
The order should be the same. 

Can you file a jira for this issue ? We should throw an error.


Thanks,
-namit

________________________________________
From: Thiruvel Thirumoolan [thiruvel@yahoo-inc.com]
Sent: Friday, August 06, 2010 12:29 AM
To: hive-user@hadoop.apache.org
Subject: Shud partition order be same in create and insert?

Hello,

When the order of partitioning columns is different in create table and insert, I am not able to query for any data. However if the order is the same its possible to see data.

Should partitioning order be maintained through all inserts? As you see below, kv2.txt is still on HDFS and 2 different partition orders are created. Running off hive trunk.

hive> CREATE TABLE invites (foo INT, bar STRING) PARTITIONED BY (ds STRING, country STRING);
hive> LOAD DATA LOCAL INPATH '/tmp/kv2.txt' OVERWRITE INTO TABLE invites PARTITION (country='india', ds='2008-08-15');
hive> select * from invites;
hive>

[thiruvel@hive]$ hadoop fs -lsr /user/hive/warehouse/invites
drwxr-xr-x   - thiruvel supergroup          0 2010-08-06 12:52 /user/hive/warehouse/invites/country=india
drwxr-xr-x   - thiruvel supergroup          0 2010-08-06 12:52 /user/hive/warehouse/invites/country=india/ds=2008-08-15
-rw-r--r--   1 thiruvel supergroup       5791 2010-08-06 12:52 /user/hive/warehouse/invites/country=india/ds=2008-08-15/kv2.txt
drwxr-xr-x   - thiruvel supergroup          0 2010-08-06 12:52 /user/hive/warehouse/invites/ds=2008-08-15
drwxr-xr-x   - thiruvel supergroup          0 2010-08-06 12:52 /user/hive/warehouse/invites/ds=2008-08-15/country=india
[thiruvel@ hive]$

Thanks,
Thiruvel