You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by Carl Steinbach <ca...@cloudera.com> on 2011/02/05 02:13:17 UTC
Re: Review Request: HIVE-1918: Add export/import facilities to the hive
system
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/339/
-----------------------------------------------------------
(Updated 2011-02-04 17:13:17.569108)
Review request for hive.
Changes
-------
Updating diff with https://issues.apache.org/jira/secure/attachment/12470063/HIVE-1918.patch.3.txt
Summary
-------
Review for HIVE-1918.
This addresses bug HIVE-1918.
https://issues.apache.org/jira/browse/HIVE-1918
Diffs (updated)
-----
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 7e5e19f
conf/hive-default.xml 46156c0
ql/src/java/org/apache/hadoop/hive/ql/exec/CopyTask.java 30ea670
ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 6fea990
ql/src/java/org/apache/hadoop/hive/ql/exec/MoveTask.java e47992a
ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java 5f78082
ql/src/java/org/apache/hadoop/hive/ql/metadata/HiveUtils.java b7c51ae
ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java d8442b2
ql/src/java/org/apache/hadoop/hive/ql/parse/ErrorMsg.java 01eef69
ql/src/java/org/apache/hadoop/hive/ql/parse/EximUtil.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/parse/ExportSemanticAnalyzer.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g c5574b0
ql/src/java/org/apache/hadoop/hive/ql/parse/ImportSemanticAnalyzer.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 15e7a13
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzerFactory.java 7655154
ql/src/java/org/apache/hadoop/hive/ql/plan/AddPartitionDesc.java e7be269
ql/src/java/org/apache/hadoop/hive/ql/plan/CopyWork.java 7a62ec7
ql/src/java/org/apache/hadoop/hive/ql/plan/HiveOperation.java e484fe2
ql/src/java/org/apache/hadoop/hive/ql/plan/LoadTableDesc.java d5bccae
ql/src/test/org/apache/hadoop/hive/ql/metadata/TestHiveUtils.java PRE-CREATION
ql/src/test/queries/clientnegative/exim_00_unsupported_schema.q PRE-CREATION
ql/src/test/queries/clientnegative/exim_01_nonpart_over_loaded.q PRE-CREATION
ql/src/test/queries/clientnegative/exim_02_all_part_over_overlap.q PRE-CREATION
ql/src/test/queries/clientnegative/exim_03_nonpart_noncompat_colschema.q PRE-CREATION
ql/src/test/queries/clientnegative/exim_04_nonpart_noncompat_colnumber.q PRE-CREATION
ql/src/test/queries/clientnegative/exim_05_nonpart_noncompat_coltype.q PRE-CREATION
ql/src/test/queries/clientnegative/exim_06_nonpart_noncompat_storage.q PRE-CREATION
ql/src/test/queries/clientnegative/exim_07_nonpart_noncompat_ifof.q PRE-CREATION
ql/src/test/queries/clientnegative/exim_08_nonpart_noncompat_serde.q PRE-CREATION
ql/src/test/queries/clientnegative/exim_09_nonpart_noncompat_serdeparam.q PRE-CREATION
ql/src/test/queries/clientnegative/exim_10_nonpart_noncompat_bucketing.q PRE-CREATION
ql/src/test/queries/clientnegative/exim_11_nonpart_noncompat_sorting.q PRE-CREATION
ql/src/test/queries/clientnegative/exim_12_nonnative_export.q PRE-CREATION
ql/src/test/queries/clientnegative/exim_13_nonnative_import.q PRE-CREATION
ql/src/test/queries/clientnegative/exim_14_nonpart_part.q PRE-CREATION
ql/src/test/queries/clientnegative/exim_15_part_nonpart.q PRE-CREATION
ql/src/test/queries/clientnegative/exim_16_part_noncompat_schema.q PRE-CREATION
ql/src/test/queries/clientnegative/exim_17_part_spec_underspec.q PRE-CREATION
ql/src/test/queries/clientnegative/exim_18_part_spec_missing.q PRE-CREATION
ql/src/test/queries/clientnegative/exim_19_external_over_existing.q PRE-CREATION
ql/src/test/queries/clientnegative/exim_20_managed_location_over_existing.q PRE-CREATION
ql/src/test/queries/clientnegative/exim_21_part_managed_external.q PRE-CREATION
ql/src/test/queries/clientpositive/exim_00_nonpart_empty.q PRE-CREATION
ql/src/test/queries/clientpositive/exim_01_nonpart.q PRE-CREATION
ql/src/test/queries/clientpositive/exim_02_00_part_empty.q PRE-CREATION
ql/src/test/queries/clientpositive/exim_02_part.q PRE-CREATION
ql/src/test/queries/clientpositive/exim_03_nonpart_over_compat.q PRE-CREATION
ql/src/test/queries/clientpositive/exim_04_all_part.q PRE-CREATION
ql/src/test/queries/clientpositive/exim_04_evolved_parts.q PRE-CREATION
ql/src/test/queries/clientpositive/exim_05_some_part.q PRE-CREATION
ql/src/test/queries/clientpositive/exim_06_one_part.q PRE-CREATION
ql/src/test/queries/clientpositive/exim_07_all_part_over_nonoverlap.q PRE-CREATION
ql/src/test/queries/clientpositive/exim_08_nonpart_rename.q PRE-CREATION
ql/src/test/queries/clientpositive/exim_09_part_spec_nonoverlap.q PRE-CREATION
ql/src/test/queries/clientpositive/exim_10_external_managed.q PRE-CREATION
ql/src/test/queries/clientpositive/exim_11_managed_external.q PRE-CREATION
ql/src/test/queries/clientpositive/exim_12_external_location.q PRE-CREATION
ql/src/test/queries/clientpositive/exim_13_managed_location.q PRE-CREATION
ql/src/test/queries/clientpositive/exim_14_managed_location_over_existing.q PRE-CREATION
ql/src/test/queries/clientpositive/exim_15_external_part.q PRE-CREATION
ql/src/test/queries/clientpositive/exim_16_part_external.q PRE-CREATION
ql/src/test/queries/clientpositive/exim_17_part_managed.q PRE-CREATION
ql/src/test/queries/clientpositive/exim_18_part_external.q PRE-CREATION
ql/src/test/queries/clientpositive/exim_19_part_external_location.q PRE-CREATION
ql/src/test/queries/clientpositive/exim_20_part_managed_location.q PRE-CREATION
ql/src/test/results/clientnegative/exim_00_unsupported_schema.q.out PRE-CREATION
ql/src/test/results/clientnegative/exim_01_nonpart_over_loaded.q.out PRE-CREATION
ql/src/test/results/clientnegative/exim_02_all_part_over_overlap.q.out PRE-CREATION
ql/src/test/results/clientnegative/exim_03_nonpart_noncompat_colschema.q.out PRE-CREATION
ql/src/test/results/clientnegative/exim_04_nonpart_noncompat_colnumber.q.out PRE-CREATION
ql/src/test/results/clientnegative/exim_05_nonpart_noncompat_coltype.q.out PRE-CREATION
ql/src/test/results/clientnegative/exim_06_nonpart_noncompat_storage.q.out PRE-CREATION
ql/src/test/results/clientnegative/exim_07_nonpart_noncompat_ifof.q.out PRE-CREATION
ql/src/test/results/clientnegative/exim_08_nonpart_noncompat_serde.q.out PRE-CREATION
ql/src/test/results/clientnegative/exim_09_nonpart_noncompat_serdeparam.q.out PRE-CREATION
ql/src/test/results/clientnegative/exim_10_nonpart_noncompat_bucketing.q.out PRE-CREATION
ql/src/test/results/clientnegative/exim_11_nonpart_noncompat_sorting.q.out PRE-CREATION
ql/src/test/results/clientnegative/exim_12_nonnative_export.q.out PRE-CREATION
ql/src/test/results/clientnegative/exim_13_nonnative_import.q.out PRE-CREATION
ql/src/test/results/clientnegative/exim_14_nonpart_part.q.out PRE-CREATION
ql/src/test/results/clientnegative/exim_15_part_nonpart.q.out PRE-CREATION
ql/src/test/results/clientnegative/exim_16_part_noncompat_schema.q.out PRE-CREATION
ql/src/test/results/clientnegative/exim_17_part_spec_underspec.q.out PRE-CREATION
ql/src/test/results/clientnegative/exim_18_part_spec_missing.q.out PRE-CREATION
ql/src/test/results/clientnegative/exim_19_external_over_existing.q.out PRE-CREATION
ql/src/test/results/clientnegative/exim_20_managed_location_over_existing.q.out PRE-CREATION
ql/src/test/results/clientnegative/exim_21_part_managed_external.q.out PRE-CREATION
ql/src/test/results/clientpositive/exim_00_nonpart_empty.q.out PRE-CREATION
ql/src/test/results/clientpositive/exim_01_nonpart.q.out PRE-CREATION
ql/src/test/results/clientpositive/exim_02_00_part_empty.q.out PRE-CREATION
ql/src/test/results/clientpositive/exim_02_part.q.out PRE-CREATION
ql/src/test/results/clientpositive/exim_03_nonpart_over_compat.q.out PRE-CREATION
ql/src/test/results/clientpositive/exim_04_all_part.q.out PRE-CREATION
ql/src/test/results/clientpositive/exim_04_evolved_parts.q.out PRE-CREATION
ql/src/test/results/clientpositive/exim_05_some_part.q.out PRE-CREATION
ql/src/test/results/clientpositive/exim_06_one_part.q.out PRE-CREATION
ql/src/test/results/clientpositive/exim_07_all_part_over_nonoverlap.q.out PRE-CREATION
ql/src/test/results/clientpositive/exim_08_nonpart_rename.q.out PRE-CREATION
ql/src/test/results/clientpositive/exim_09_part_spec_nonoverlap.q.out PRE-CREATION
ql/src/test/results/clientpositive/exim_10_external_managed.q.out PRE-CREATION
ql/src/test/results/clientpositive/exim_11_managed_external.q.out PRE-CREATION
ql/src/test/results/clientpositive/exim_12_external_location.q.out PRE-CREATION
ql/src/test/results/clientpositive/exim_13_managed_location.q.out PRE-CREATION
ql/src/test/results/clientpositive/exim_14_managed_location_over_existing.q.out PRE-CREATION
ql/src/test/results/clientpositive/exim_15_external_part.q.out PRE-CREATION
ql/src/test/results/clientpositive/exim_16_part_external.q.out PRE-CREATION
ql/src/test/results/clientpositive/exim_17_part_managed.q.out PRE-CREATION
ql/src/test/results/clientpositive/exim_18_part_external.q.out PRE-CREATION
ql/src/test/results/clientpositive/exim_19_part_external_location.q.out PRE-CREATION
ql/src/test/results/clientpositive/exim_20_part_managed_location.q.out PRE-CREATION
Diff: https://reviews.apache.org/r/339/diff
Testing
-------
Thanks,
Carl
Re: Review Request: HIVE-1918: Add export/import facilities to the hive
system
Posted by Paul Yang <py...@fb.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/339/#review255
-----------------------------------------------------------
ql/src/java/org/apache/hadoop/hive/ql/parse/ExportSemanticAnalyzer.java
<https://reviews.apache.org/r/339/#comment500>
Can we avoid nesting the ternary operator? It makes the code a little confusing. There are several instances of this in the diff, but I've just highlighted the first one.
ql/src/java/org/apache/hadoop/hive/ql/parse/ExportSemanticAnalyzer.java
<https://reviews.apache.org/r/339/#comment501>
If we go with the route of having auto-generated code, then we might want to look into having JDO to handle this for us. Datanucleus/JDO has an option to persist to an XML file that might be applicable for this use case.
I would agree that some discretion is required in picking fields to serialize for import/export, but the fear is that adding a field now will require many changes.
- Paul
On 2011-02-04 17:13:17, Carl Steinbach wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/339/
> -----------------------------------------------------------
>
> (Updated 2011-02-04 17:13:17)
>
>
> Review request for hive.
>
>
> Summary
> -------
>
> Review for HIVE-1918.
>
>
> This addresses bug HIVE-1918.
> https://issues.apache.org/jira/browse/HIVE-1918
>
>
> Diffs
> -----
>
> common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 7e5e19f
> conf/hive-default.xml 46156c0
> ql/src/java/org/apache/hadoop/hive/ql/exec/CopyTask.java 30ea670
> ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 6fea990
> ql/src/java/org/apache/hadoop/hive/ql/exec/MoveTask.java e47992a
> ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java 5f78082
> ql/src/java/org/apache/hadoop/hive/ql/metadata/HiveUtils.java b7c51ae
> ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java d8442b2
> ql/src/java/org/apache/hadoop/hive/ql/parse/ErrorMsg.java 01eef69
> ql/src/java/org/apache/hadoop/hive/ql/parse/EximUtil.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/parse/ExportSemanticAnalyzer.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g c5574b0
> ql/src/java/org/apache/hadoop/hive/ql/parse/ImportSemanticAnalyzer.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 15e7a13
> ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzerFactory.java 7655154
> ql/src/java/org/apache/hadoop/hive/ql/plan/AddPartitionDesc.java e7be269
> ql/src/java/org/apache/hadoop/hive/ql/plan/CopyWork.java 7a62ec7
> ql/src/java/org/apache/hadoop/hive/ql/plan/HiveOperation.java e484fe2
> ql/src/java/org/apache/hadoop/hive/ql/plan/LoadTableDesc.java d5bccae
> ql/src/test/org/apache/hadoop/hive/ql/metadata/TestHiveUtils.java PRE-CREATION
> ql/src/test/queries/clientnegative/exim_00_unsupported_schema.q PRE-CREATION
> ql/src/test/queries/clientnegative/exim_01_nonpart_over_loaded.q PRE-CREATION
> ql/src/test/queries/clientnegative/exim_02_all_part_over_overlap.q PRE-CREATION
> ql/src/test/queries/clientnegative/exim_03_nonpart_noncompat_colschema.q PRE-CREATION
> ql/src/test/queries/clientnegative/exim_04_nonpart_noncompat_colnumber.q PRE-CREATION
> ql/src/test/queries/clientnegative/exim_05_nonpart_noncompat_coltype.q PRE-CREATION
> ql/src/test/queries/clientnegative/exim_06_nonpart_noncompat_storage.q PRE-CREATION
> ql/src/test/queries/clientnegative/exim_07_nonpart_noncompat_ifof.q PRE-CREATION
> ql/src/test/queries/clientnegative/exim_08_nonpart_noncompat_serde.q PRE-CREATION
> ql/src/test/queries/clientnegative/exim_09_nonpart_noncompat_serdeparam.q PRE-CREATION
> ql/src/test/queries/clientnegative/exim_10_nonpart_noncompat_bucketing.q PRE-CREATION
> ql/src/test/queries/clientnegative/exim_11_nonpart_noncompat_sorting.q PRE-CREATION
> ql/src/test/queries/clientnegative/exim_12_nonnative_export.q PRE-CREATION
> ql/src/test/queries/clientnegative/exim_13_nonnative_import.q PRE-CREATION
> ql/src/test/queries/clientnegative/exim_14_nonpart_part.q PRE-CREATION
> ql/src/test/queries/clientnegative/exim_15_part_nonpart.q PRE-CREATION
> ql/src/test/queries/clientnegative/exim_16_part_noncompat_schema.q PRE-CREATION
> ql/src/test/queries/clientnegative/exim_17_part_spec_underspec.q PRE-CREATION
> ql/src/test/queries/clientnegative/exim_18_part_spec_missing.q PRE-CREATION
> ql/src/test/queries/clientnegative/exim_19_external_over_existing.q PRE-CREATION
> ql/src/test/queries/clientnegative/exim_20_managed_location_over_existing.q PRE-CREATION
> ql/src/test/queries/clientnegative/exim_21_part_managed_external.q PRE-CREATION
> ql/src/test/queries/clientpositive/exim_00_nonpart_empty.q PRE-CREATION
> ql/src/test/queries/clientpositive/exim_01_nonpart.q PRE-CREATION
> ql/src/test/queries/clientpositive/exim_02_00_part_empty.q PRE-CREATION
> ql/src/test/queries/clientpositive/exim_02_part.q PRE-CREATION
> ql/src/test/queries/clientpositive/exim_03_nonpart_over_compat.q PRE-CREATION
> ql/src/test/queries/clientpositive/exim_04_all_part.q PRE-CREATION
> ql/src/test/queries/clientpositive/exim_04_evolved_parts.q PRE-CREATION
> ql/src/test/queries/clientpositive/exim_05_some_part.q PRE-CREATION
> ql/src/test/queries/clientpositive/exim_06_one_part.q PRE-CREATION
> ql/src/test/queries/clientpositive/exim_07_all_part_over_nonoverlap.q PRE-CREATION
> ql/src/test/queries/clientpositive/exim_08_nonpart_rename.q PRE-CREATION
> ql/src/test/queries/clientpositive/exim_09_part_spec_nonoverlap.q PRE-CREATION
> ql/src/test/queries/clientpositive/exim_10_external_managed.q PRE-CREATION
> ql/src/test/queries/clientpositive/exim_11_managed_external.q PRE-CREATION
> ql/src/test/queries/clientpositive/exim_12_external_location.q PRE-CREATION
> ql/src/test/queries/clientpositive/exim_13_managed_location.q PRE-CREATION
> ql/src/test/queries/clientpositive/exim_14_managed_location_over_existing.q PRE-CREATION
> ql/src/test/queries/clientpositive/exim_15_external_part.q PRE-CREATION
> ql/src/test/queries/clientpositive/exim_16_part_external.q PRE-CREATION
> ql/src/test/queries/clientpositive/exim_17_part_managed.q PRE-CREATION
> ql/src/test/queries/clientpositive/exim_18_part_external.q PRE-CREATION
> ql/src/test/queries/clientpositive/exim_19_part_external_location.q PRE-CREATION
> ql/src/test/queries/clientpositive/exim_20_part_managed_location.q PRE-CREATION
> ql/src/test/results/clientnegative/exim_00_unsupported_schema.q.out PRE-CREATION
> ql/src/test/results/clientnegative/exim_01_nonpart_over_loaded.q.out PRE-CREATION
> ql/src/test/results/clientnegative/exim_02_all_part_over_overlap.q.out PRE-CREATION
> ql/src/test/results/clientnegative/exim_03_nonpart_noncompat_colschema.q.out PRE-CREATION
> ql/src/test/results/clientnegative/exim_04_nonpart_noncompat_colnumber.q.out PRE-CREATION
> ql/src/test/results/clientnegative/exim_05_nonpart_noncompat_coltype.q.out PRE-CREATION
> ql/src/test/results/clientnegative/exim_06_nonpart_noncompat_storage.q.out PRE-CREATION
> ql/src/test/results/clientnegative/exim_07_nonpart_noncompat_ifof.q.out PRE-CREATION
> ql/src/test/results/clientnegative/exim_08_nonpart_noncompat_serde.q.out PRE-CREATION
> ql/src/test/results/clientnegative/exim_09_nonpart_noncompat_serdeparam.q.out PRE-CREATION
> ql/src/test/results/clientnegative/exim_10_nonpart_noncompat_bucketing.q.out PRE-CREATION
> ql/src/test/results/clientnegative/exim_11_nonpart_noncompat_sorting.q.out PRE-CREATION
> ql/src/test/results/clientnegative/exim_12_nonnative_export.q.out PRE-CREATION
> ql/src/test/results/clientnegative/exim_13_nonnative_import.q.out PRE-CREATION
> ql/src/test/results/clientnegative/exim_14_nonpart_part.q.out PRE-CREATION
> ql/src/test/results/clientnegative/exim_15_part_nonpart.q.out PRE-CREATION
> ql/src/test/results/clientnegative/exim_16_part_noncompat_schema.q.out PRE-CREATION
> ql/src/test/results/clientnegative/exim_17_part_spec_underspec.q.out PRE-CREATION
> ql/src/test/results/clientnegative/exim_18_part_spec_missing.q.out PRE-CREATION
> ql/src/test/results/clientnegative/exim_19_external_over_existing.q.out PRE-CREATION
> ql/src/test/results/clientnegative/exim_20_managed_location_over_existing.q.out PRE-CREATION
> ql/src/test/results/clientnegative/exim_21_part_managed_external.q.out PRE-CREATION
> ql/src/test/results/clientpositive/exim_00_nonpart_empty.q.out PRE-CREATION
> ql/src/test/results/clientpositive/exim_01_nonpart.q.out PRE-CREATION
> ql/src/test/results/clientpositive/exim_02_00_part_empty.q.out PRE-CREATION
> ql/src/test/results/clientpositive/exim_02_part.q.out PRE-CREATION
> ql/src/test/results/clientpositive/exim_03_nonpart_over_compat.q.out PRE-CREATION
> ql/src/test/results/clientpositive/exim_04_all_part.q.out PRE-CREATION
> ql/src/test/results/clientpositive/exim_04_evolved_parts.q.out PRE-CREATION
> ql/src/test/results/clientpositive/exim_05_some_part.q.out PRE-CREATION
> ql/src/test/results/clientpositive/exim_06_one_part.q.out PRE-CREATION
> ql/src/test/results/clientpositive/exim_07_all_part_over_nonoverlap.q.out PRE-CREATION
> ql/src/test/results/clientpositive/exim_08_nonpart_rename.q.out PRE-CREATION
> ql/src/test/results/clientpositive/exim_09_part_spec_nonoverlap.q.out PRE-CREATION
> ql/src/test/results/clientpositive/exim_10_external_managed.q.out PRE-CREATION
> ql/src/test/results/clientpositive/exim_11_managed_external.q.out PRE-CREATION
> ql/src/test/results/clientpositive/exim_12_external_location.q.out PRE-CREATION
> ql/src/test/results/clientpositive/exim_13_managed_location.q.out PRE-CREATION
> ql/src/test/results/clientpositive/exim_14_managed_location_over_existing.q.out PRE-CREATION
> ql/src/test/results/clientpositive/exim_15_external_part.q.out PRE-CREATION
> ql/src/test/results/clientpositive/exim_16_part_external.q.out PRE-CREATION
> ql/src/test/results/clientpositive/exim_17_part_managed.q.out PRE-CREATION
> ql/src/test/results/clientpositive/exim_18_part_external.q.out PRE-CREATION
> ql/src/test/results/clientpositive/exim_19_part_external_location.q.out PRE-CREATION
> ql/src/test/results/clientpositive/exim_20_part_managed_location.q.out PRE-CREATION
>
> Diff: https://reviews.apache.org/r/339/diff
>
>
> Testing
> -------
>
>
> Thanks,
>
> Carl
>
>