You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by Deepak Jaiswal <dj...@hortonworks.com> on 2018/04/25 22:31:52 UTC
Review Request 66805: HIVE-19311 : Partition and bucketing support for “load data” statement
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66805/
-----------------------------------------------------------
Review request for hive, Ashutosh Chauhan, Eugene Koifman, Jesús Camacho Rodríguez, and Vineet Garg.
Bugs: HIVE-19311
https://issues.apache.org/jira/browse/HIVE-19311
Repository: hive-git
Description
-------
Currently, "load data" statement is very limited. It errors out if any of the information is missing such as partitioning info if table is partitioned or appropriate names when table is bucketed.
It should be able to launch an insert job to load the data instead.
Diffs (updated)
-----
data/files/load_data_job/bucketing.txt PRE-CREATION
data/files/load_data_job/partitions/load_data_1_partition.txt PRE-CREATION
data/files/load_data_job/partitions/load_data_2_partitions.txt PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/Context.java 70846ac3ce
ql/src/java/org/apache/hadoop/hive/ql/ErrorMsg.java 7d33fa3892
ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java a51fdd322f
ql/src/java/org/apache/hadoop/hive/ql/parse/LoadSemanticAnalyzer.java c07991d434
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 1dccf969ff
ql/src/java/org/apache/hadoop/hive/ql/parse/UpdateDeleteSemanticAnalyzer.java 2f3b07f4af
ql/src/test/queries/clientpositive/load_data_using_job.q PRE-CREATION
ql/src/test/results/clientpositive/llap/load_data_using_job.q.out PRE-CREATION
Diff: https://reviews.apache.org/r/66805/diff/1/
Testing
-------
Added a unit test.
Thanks,
Deepak Jaiswal
Re: Review Request 66805: HIVE-19311 : Partition and bucketing support for “load data” statement
Posted by Deepak Jaiswal <dj...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66805/
-----------------------------------------------------------
(Updated April 30, 2018, 8:38 p.m.)
Review request for hive, Ashutosh Chauhan, Eugene Koifman, Jesús Camacho Rodríguez, and Vineet Garg.
Changes
-------
The result was updated due to use of murmur hash.
Bugs: HIVE-19311
https://issues.apache.org/jira/browse/HIVE-19311
Repository: hive-git
Description
-------
Currently, "load data" statement is very limited. It errors out if any of the information is missing such as partitioning info if table is partitioned or appropriate names when table is bucketed.
It should be able to launch an insert job to load the data instead.
Diffs (updated)
-----
data/files/load_data_job/bucketing.txt PRE-CREATION
data/files/load_data_job/load_data_1_partition.txt PRE-CREATION
data/files/load_data_job/partitions/load_data_1_partition.txt PRE-CREATION
data/files/load_data_job/partitions/load_data_2_partitions.txt PRE-CREATION
itests/src/test/resources/testconfiguration.properties 2ca7b5f63b
ql/src/java/org/apache/hadoop/hive/ql/Context.java 0fedf0e76e
ql/src/java/org/apache/hadoop/hive/ql/ErrorMsg.java 7d33fa3892
ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java abd678bb54
ql/src/java/org/apache/hadoop/hive/ql/parse/LoadSemanticAnalyzer.java c07991d434
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 020565014b
ql/src/java/org/apache/hadoop/hive/ql/parse/UpdateDeleteSemanticAnalyzer.java 2f3b07f4af
ql/src/test/org/apache/hadoop/hive/ql/TestTxnLoadData.java ec8c1507ec
ql/src/test/queries/clientnegative/load_part_nospec.q 81517991b2
ql/src/test/queries/clientnegative/nopart_load.q 966982fd5c
ql/src/test/queries/clientpositive/load_data_using_job.q PRE-CREATION
ql/src/test/results/clientnegative/load_part_nospec.q.out bebaf92311
ql/src/test/results/clientnegative/nopart_load.q.out 881514640c
ql/src/test/results/clientpositive/llap/load_data_using_job.q.out PRE-CREATION
Diff: https://reviews.apache.org/r/66805/diff/5/
Changes: https://reviews.apache.org/r/66805/diff/4-5/
Testing
-------
Added a unit test.
Thanks,
Deepak Jaiswal
Re: Review Request 66805: HIVE-19311 : Partition and bucketing support for “load data” statement
Posted by Deepak Jaiswal <dj...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66805/
-----------------------------------------------------------
(Updated April 28, 2018, 10:26 p.m.)
Review request for hive, Ashutosh Chauhan, Eugene Koifman, Jesús Camacho Rodríguez, and Vineet Garg.
Changes
-------
Removed some tests and updated one.
Bugs: HIVE-19311
https://issues.apache.org/jira/browse/HIVE-19311
Repository: hive-git
Description
-------
Currently, "load data" statement is very limited. It errors out if any of the information is missing such as partitioning info if table is partitioned or appropriate names when table is bucketed.
It should be able to launch an insert job to load the data instead.
Diffs (updated)
-----
data/files/load_data_job/bucketing.txt PRE-CREATION
data/files/load_data_job/load_data_1_partition.txt PRE-CREATION
data/files/load_data_job/partitions/load_data_1_partition.txt PRE-CREATION
data/files/load_data_job/partitions/load_data_2_partitions.txt PRE-CREATION
itests/src/test/resources/testconfiguration.properties 1a346593fd
ql/src/java/org/apache/hadoop/hive/ql/Context.java 0fedf0e76e
ql/src/java/org/apache/hadoop/hive/ql/ErrorMsg.java 7d33fa3892
ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java a51fdd322f
ql/src/java/org/apache/hadoop/hive/ql/parse/LoadSemanticAnalyzer.java c07991d434
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 1dccf969ff
ql/src/java/org/apache/hadoop/hive/ql/parse/UpdateDeleteSemanticAnalyzer.java 2f3b07f4af
ql/src/test/org/apache/hadoop/hive/ql/TestTxnLoadData.java ec8c1507ec
ql/src/test/queries/clientnegative/load_part_nospec.q 81517991b2
ql/src/test/queries/clientnegative/nopart_load.q 966982fd5c
ql/src/test/queries/clientpositive/load_data_using_job.q PRE-CREATION
ql/src/test/results/clientnegative/load_part_nospec.q.out bebaf92311
ql/src/test/results/clientnegative/nopart_load.q.out 881514640c
ql/src/test/results/clientpositive/llap/load_data_using_job.q.out PRE-CREATION
Diff: https://reviews.apache.org/r/66805/diff/4/
Changes: https://reviews.apache.org/r/66805/diff/3-4/
Testing
-------
Added a unit test.
Thanks,
Deepak Jaiswal
Re: Review Request 66805: HIVE-19311 : Partition and bucketing support for “load data” statement
Posted by Deepak Jaiswal <dj...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66805/
-----------------------------------------------------------
(Updated April 28, 2018, 6:26 a.m.)
Review request for hive, Ashutosh Chauhan, Eugene Koifman, Jesús Camacho Rodríguez, and Vineet Garg.
Changes
-------
Modified the tests to use correct data files.
Bugs: HIVE-19311
https://issues.apache.org/jira/browse/HIVE-19311
Repository: hive-git
Description
-------
Currently, "load data" statement is very limited. It errors out if any of the information is missing such as partitioning info if table is partitioned or appropriate names when table is bucketed.
It should be able to launch an insert job to load the data instead.
Diffs (updated)
-----
data/files/load_data_job/bucketing.txt PRE-CREATION
data/files/load_data_job/load_data_1_partition.txt PRE-CREATION
data/files/load_data_job/partitions/load_data_1_partition.txt PRE-CREATION
data/files/load_data_job/partitions/load_data_2_partitions.txt PRE-CREATION
itests/src/test/resources/testconfiguration.properties 1a346593fd
ql/src/java/org/apache/hadoop/hive/ql/Context.java 0fedf0e76e
ql/src/java/org/apache/hadoop/hive/ql/ErrorMsg.java 7d33fa3892
ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java a51fdd322f
ql/src/java/org/apache/hadoop/hive/ql/parse/LoadSemanticAnalyzer.java c07991d434
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 1dccf969ff
ql/src/java/org/apache/hadoop/hive/ql/parse/UpdateDeleteSemanticAnalyzer.java 2f3b07f4af
ql/src/test/queries/clientpositive/load_data_using_job.q PRE-CREATION
ql/src/test/results/clientpositive/llap/load_data_using_job.q.out PRE-CREATION
ql/src/test/results/clientpositive/spark/load_data_using_job.q.out PRE-CREATION
Diff: https://reviews.apache.org/r/66805/diff/3/
Changes: https://reviews.apache.org/r/66805/diff/2-3/
Testing
-------
Added a unit test.
Thanks,
Deepak Jaiswal
Re: Review Request 66805: HIVE-19311 : Partition and bucketing support for “load data” statement
Posted by Deepak Jaiswal <dj...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66805/
-----------------------------------------------------------
(Updated April 27, 2018, 7:44 a.m.)
Review request for hive, Ashutosh Chauhan, Eugene Koifman, Jesús Camacho Rodríguez, and Vineet Garg.
Changes
-------
fixed a bug causing output entities missed from POSTHOOKs.
Bugs: HIVE-19311
https://issues.apache.org/jira/browse/HIVE-19311
Repository: hive-git
Description
-------
Currently, "load data" statement is very limited. It errors out if any of the information is missing such as partitioning info if table is partitioned or appropriate names when table is bucketed.
It should be able to launch an insert job to load the data instead.
Diffs (updated)
-----
data/files/load_data_job/bucketing.txt PRE-CREATION
data/files/load_data_job/partitions/load_data_1_partition.txt PRE-CREATION
data/files/load_data_job/partitions/load_data_2_partitions.txt PRE-CREATION
itests/src/test/resources/testconfiguration.properties 1a346593fd
ql/src/java/org/apache/hadoop/hive/ql/Context.java 70846ac3ce
ql/src/java/org/apache/hadoop/hive/ql/ErrorMsg.java 7d33fa3892
ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java a51fdd322f
ql/src/java/org/apache/hadoop/hive/ql/parse/LoadSemanticAnalyzer.java c07991d434
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 1dccf969ff
ql/src/java/org/apache/hadoop/hive/ql/parse/UpdateDeleteSemanticAnalyzer.java 2f3b07f4af
ql/src/test/queries/clientpositive/load_data_using_job.q PRE-CREATION
ql/src/test/results/clientpositive/llap/load_data_using_job.q.out PRE-CREATION
ql/src/test/results/clientpositive/spark/load_data_using_job.q.out PRE-CREATION
Diff: https://reviews.apache.org/r/66805/diff/2/
Changes: https://reviews.apache.org/r/66805/diff/1-2/
Testing
-------
Added a unit test.
Thanks,
Deepak Jaiswal