You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Saravanan (JIRA)" <ji...@apache.org> on 2010/11/03 12:51:25 UTC
[jira] Created: (HIVE-1766) Dynamic partition is not working as
expected.
Dynamic partition is not working as expected.
----------------------------------------------
Key: HIVE-1766
URL: https://issues.apache.org/jira/browse/HIVE-1766
Project: Hive
Issue Type: Bug
Components: CLI
Affects Versions: 0.6.0, 0.7.0
Environment: Linux, Got the latest code from hive trunk and also tested in 0.6. hadoop version 0.20
Reporter: Saravanan
Fix For: 0.7.0, 0.6.0
Create source table
------------------
CREATE EXTERNAL TABLE testmove (
a string,
b string
)
PARTITIONED BY (cust string, dt string);
Data has been kept in /usr/hive/warehouse/testmove/cust=a/dt=20100102/a.txt
a.txt has 1 row the value is "a", "b"
Create Destination table
-----------------------
CREATE EXTERNAL TABLE testmove1 (
a string,
b string
)
PARTITIONED BY (cust string, dt string)
Run the query for dynamic partion insert
---------------------------------------
set hive.exec.dynamic.partition=true;
set hive.exec.dynamic.partition.mode=nonstrict;
FROM testmove t
INSERT OVERWRITE TABLE testmove1 PARTITION (cust, dt)
SELECT t.a, t.b, 'a', '20100102';
output
-------
otal MapReduce jobs = 2
Launching Job 1 out of 2
Number of reduce tasks is set to 0 since there's no reduce operator
Execution log at: /tmp/root/root_20101103170404_9e869676-7bb5-4655-b027-5bcb4b7fa2cb.log
Job running in-process (local Hadoop)
2010-11-03 17:04:06,818 null map = 100%, reduce = 0%
Ended Job = job_local_0001
Ended Job = -645725555, job is filtered out (removed at runtime).
Moving data to: file:/tmp/hive-root/hive_2010-11-03_17-03-59_979_5901061386316364507/-ext-10000
Loading data to table testmove1 partition (cust=null, dt=null)
[Warning] could not update stats.
OK
If i run as static partion is the data is inserted in to destination table.
FROM testmove t
INSERT OVERWRITE TABLE testmove1 PARTITION (cust='a', dt='20100102')
SELECT t.a, t.b;
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1766) Dynamic partition is not working as
expected.
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Carl Steinbach updated HIVE-1766:
---------------------------------
Affects Version/s: (was: 0.7.0)
(was: 0.6.0)
Fix Version/s: (was: 0.7.0)
(was: 0.6.0)
> Dynamic partition is not working as expected.
> ----------------------------------------------
>
> Key: HIVE-1766
> URL: https://issues.apache.org/jira/browse/HIVE-1766
> Project: Hive
> Issue Type: Bug
> Components: CLI
> Environment: Linux, Got the latest code from hive trunk and also tested in 0.6. hadoop version 0.20
> Reporter: Saravanan
>
> Create source table
> ------------------
> CREATE EXTERNAL TABLE testmove (
> a string,
> b string
> )
> PARTITIONED BY (cust string, dt string);
> Data has been kept in /usr/hive/warehouse/testmove/cust=a/dt=20100102/a.txt
> a.txt has 1 row the value is "a", "b"
> Create Destination table
> -----------------------
> CREATE EXTERNAL TABLE testmove1 (
> a string,
> b string
> )
> PARTITIONED BY (cust string, dt string)
> Run the query for dynamic partion insert
> ---------------------------------------
> set hive.exec.dynamic.partition=true;
> set hive.exec.dynamic.partition.mode=nonstrict;
> FROM testmove t
> INSERT OVERWRITE TABLE testmove1 PARTITION (cust, dt)
> SELECT t.a, t.b, 'a', '20100102';
> output
> -------
> otal MapReduce jobs = 2
> Launching Job 1 out of 2
> Number of reduce tasks is set to 0 since there's no reduce operator
> Execution log at: /tmp/root/root_20101103170404_9e869676-7bb5-4655-b027-5bcb4b7fa2cb.log
> Job running in-process (local Hadoop)
> 2010-11-03 17:04:06,818 null map = 100%, reduce = 0%
> Ended Job = job_local_0001
> Ended Job = -645725555, job is filtered out (removed at runtime).
> Moving data to: file:/tmp/hive-root/hive_2010-11-03_17-03-59_979_5901061386316364507/-ext-10000
> Loading data to table testmove1 partition (cust=null, dt=null)
> [Warning] could not update stats.
> OK
> If i run as static partion is the data is inserted in to destination table.
> FROM testmove t
> INSERT OVERWRITE TABLE testmove1 PARTITION (cust='a', dt='20100102')
> SELECT t.a, t.b;
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HIVE-1766) Dynamic partition is not working as
expected.
Posted by "Saravanan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Saravanan resolved HIVE-1766.
-----------------------------
Resolution: Not A Problem
Release Note: The data format was wrong. I have used CSV format with quotes. Which was not supported in hive.
The data format was wrong. I have used CSV format with quotes. Which was not supported in hive.
> Dynamic partition is not working as expected.
> ----------------------------------------------
>
> Key: HIVE-1766
> URL: https://issues.apache.org/jira/browse/HIVE-1766
> Project: Hive
> Issue Type: Bug
> Components: CLI
> Affects Versions: 0.6.0, 0.7.0
> Environment: Linux, Got the latest code from hive trunk and also tested in 0.6. hadoop version 0.20
> Reporter: Saravanan
> Fix For: 0.7.0, 0.6.0
>
>
> Create source table
> ------------------
> CREATE EXTERNAL TABLE testmove (
> a string,
> b string
> )
> PARTITIONED BY (cust string, dt string);
> Data has been kept in /usr/hive/warehouse/testmove/cust=a/dt=20100102/a.txt
> a.txt has 1 row the value is "a", "b"
> Create Destination table
> -----------------------
> CREATE EXTERNAL TABLE testmove1 (
> a string,
> b string
> )
> PARTITIONED BY (cust string, dt string)
> Run the query for dynamic partion insert
> ---------------------------------------
> set hive.exec.dynamic.partition=true;
> set hive.exec.dynamic.partition.mode=nonstrict;
> FROM testmove t
> INSERT OVERWRITE TABLE testmove1 PARTITION (cust, dt)
> SELECT t.a, t.b, 'a', '20100102';
> output
> -------
> otal MapReduce jobs = 2
> Launching Job 1 out of 2
> Number of reduce tasks is set to 0 since there's no reduce operator
> Execution log at: /tmp/root/root_20101103170404_9e869676-7bb5-4655-b027-5bcb4b7fa2cb.log
> Job running in-process (local Hadoop)
> 2010-11-03 17:04:06,818 null map = 100%, reduce = 0%
> Ended Job = job_local_0001
> Ended Job = -645725555, job is filtered out (removed at runtime).
> Moving data to: file:/tmp/hive-root/hive_2010-11-03_17-03-59_979_5901061386316364507/-ext-10000
> Loading data to table testmove1 partition (cust=null, dt=null)
> [Warning] could not update stats.
> OK
> If i run as static partion is the data is inserted in to destination table.
> FROM testmove t
> INSERT OVERWRITE TABLE testmove1 PARTITION (cust='a', dt='20100102')
> SELECT t.a, t.b;
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.