You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "Hive QA (JIRA)" <ji...@apache.org> on 2017/11/21 17:39:00 UTC
[jira] [Commented] (HIVE-18113) Remove mixed partitions/table schema support

    [ https://issues.apache.org/jira/browse/HIVE-18113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16261126#comment-16261126 ] 

Hive QA commented on HIVE-18113:
--------------------------------



Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12898683/HIVE-18113.01.patch

{color:red}ERROR:{color} -1 due to no test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 175 failed/errored test(s), 11388 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver (batchId=3)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver (batchId=59)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver (batchId=9)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_merge_stats] (batchId=59)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_numbuckets_partitioned_table2_h23] (batchId=14)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_numbuckets_partitioned_table_h23] (batchId=67)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_partition_change_col] (batchId=25)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_rename_partition_authorization] (batchId=59)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_table_cascade] (batchId=86)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_table_column_stats] (batchId=63)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_table_serde2] (batchId=26)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[authorization_6] (batchId=45)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[authorization_explain] (batchId=3)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[authorization_view_2] (batchId=59)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[authorization_view_disable_cbo_3] (batchId=9)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[autoColumnStats_5] (batchId=41)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join11] (batchId=9)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[avro_comments] (batchId=59)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[avro_date] (batchId=9)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[avro_partitioned] (batchId=3)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ba_table3] (batchId=9)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucket_map_join_spark2] (batchId=3)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketmapjoin10] (batchId=50)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketmapjoin11] (batchId=70)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketmapjoin12] (batchId=34)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketmapjoin13] (batchId=39)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketmapjoin8] (batchId=12)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketmapjoin9] (batchId=16)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketsortoptimize_insert_1] (batchId=59)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[compute_stats_decimal] (batchId=9)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[constGby] (batchId=9)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[create_like_view] (batchId=3)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[create_or_replace_view] (batchId=38)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[create_with_constraints] (batchId=67)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cte_mat_5] (batchId=3)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[database_drop] (batchId=59)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[dbtxnmgr_showlocks] (batchId=77)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[delete_whole_partition] (batchId=9)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[describe_table] (batchId=42)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[display_colstats_tbllvl] (batchId=3)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[druid_basic1] (batchId=9)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[druid_basic3] (batchId=59)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[druid_topn] (batchId=3)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[equal_ns] (batchId=9)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exim_04_evolved_parts] (batchId=30)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[extract] (batchId=3)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[filter_cond_pushdown] (batchId=59)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[groupby2_limit] (batchId=9)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[groupby4_noskew] (batchId=59)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[groupby7_noskew_multi_single_reducer] (batchId=59)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[groupby_grouping_sets_grouping] (batchId=3)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[hook_order] (batchId=3)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[implicit_cast1] (batchId=59)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[index_compact_binary_search] (batchId=59)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[input0] (batchId=9)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[input33] (batchId=59)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[input9] (batchId=59)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[lb_fs_stats] (batchId=88)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[list_bucket_query_multiskew_3] (batchId=78)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[list_bucket_query_oneskew_1] (batchId=59)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[list_bucket_query_oneskew_2] (batchId=3)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[list_bucket_query_oneskew_3] (batchId=9)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[llap_acid] (batchId=78)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[materialized_view_create_rewrite] (batchId=3)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[merge_dynamic_partition4] (batchId=33)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[merge_dynamic_partition5] (batchId=32)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mi] (batchId=9)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mm_buckets] (batchId=59)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mm_conversions] (batchId=74)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[orc_merge_incompat3] (batchId=9)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_map_of_arrays_of_ints] (batchId=9)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_mixed_partition_formats] (batchId=17)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_serde] (batchId=66)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[partition_schema1] (batchId=17)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[partition_wise_fileformat2] (batchId=85)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[partition_wise_fileformat3] (batchId=50)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[partition_wise_fileformat5] (batchId=3)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[partition_wise_fileformat] (batchId=18)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[pcr] (batchId=59)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ptf_decimal] (batchId=3)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[quotedid_basic] (batchId=59)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[reduce_deduplicate_extended2] (batchId=59)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[reloadJar] (batchId=9)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[rename_table_location] (batchId=54)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[skewjoin_mapjoin6] (batchId=59)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[smb_mapjoin_12] (batchId=9)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[sort_merge_join_desc_1] (batchId=9)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[split] (batchId=3)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[subquery_exists_having] (batchId=3)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[truncate_column_list_bucket] (batchId=83)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_hour] (batchId=3)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_like] (batchId=9)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_rlike] (batchId=59)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_trunc] (batchId=3)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[union15] (batchId=87)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[union19] (batchId=59)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[union23] (batchId=3)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[union24] (batchId=59)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[union31] (batchId=3)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[union_remove_5] (batchId=59)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_cast_constant] (batchId=9)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_if_expr_2] (batchId=59)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_null_projection] (batchId=9)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_string_decimal] (batchId=3)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[view_inputs] (batchId=3)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[unionDistinct_1] (batchId=146)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[authorization_2] (batchId=152)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_groupby] (batchId=167)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata] (batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid] (batchId=165)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid_fast] (batchId=157)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[mm_conversions] (batchId=164)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[orc_create] (batchId=160)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_orc_nonvec_part] (batchId=164)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_orc_nonvec_part_all_primitive] (batchId=167)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_orc_vec_part] (batchId=165)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_orc_vec_part_all_primitive] (batchId=165)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_text_nonvec_part] (batchId=152)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_text_nonvec_part_all_primitive] (batchId=159)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_text_vec_part] (batchId=160)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_text_vec_part_all_complex] (batchId=156)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_text_vec_part_all_primitive] (batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_text_vecrow_part] (batchId=166)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_text_vecrow_part_all_primitive] (batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_bmj_schema_evolution] (batchId=149)
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver[infer_bucket_sort_dyn_part] (batchId=89)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[alter_table_non_partitioned_table_cascade] (batchId=91)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_fail_2] (batchId=91)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_fail_3] (batchId=91)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_fail_4] (batchId=91)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_fail_5] (batchId=91)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_fail_6] (batchId=93)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_fail_7] (batchId=91)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_fail_create_db] (batchId=92)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_fail_drop_db] (batchId=92)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_part] (batchId=92)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_view_1] (batchId=91)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_view_2] (batchId=92)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_view_3] (batchId=91)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_view_4] (batchId=91)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_view_5] (batchId=92)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_view_6] (batchId=92)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_view_7] (batchId=92)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_view_disable_cbo_1] (batchId=92)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_view_disable_cbo_2] (batchId=91)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_view_disable_cbo_3] (batchId=91)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_view_disable_cbo_4] (batchId=91)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_view_disable_cbo_5] (batchId=92)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_view_disable_cbo_6] (batchId=91)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_view_disable_cbo_7] (batchId=92)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[exim_22_export_authfail] (batchId=92)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[exim_23_import_exist_authfail] (batchId=92)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[exim_24_import_part_authfail] (batchId=92)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[exim_25_import_nonexist_authfail] (batchId=91)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[load_exist_part_authfail] (batchId=91)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[load_nonpart_authfail] (batchId=92)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[load_part_authfail] (batchId=92)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[parquet_alter_part_table_drop_columns] (batchId=92)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[bucketmapjoin10] (batchId=125)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[bucketmapjoin11] (batchId=134)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[bucketmapjoin12] (batchId=118)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[bucketmapjoin13] (batchId=120)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[bucketmapjoin8] (batchId=108)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[bucketmapjoin9] (batchId=110)
org.apache.hadoop.hive.ql.TestAutoPurgeTables.testExternalPartitionedTable (batchId=225)
org.apache.hadoop.hive.ql.TestAutoPurgeTables.testPartitionedTable (batchId=225)
org.apache.hadoop.hive.ql.TestAutoPurgeTables.testTruncatePartitionedAutoPurge (batchId=225)
org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testAlters (batchId=223)
org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testConstraints (batchId=223)
org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testIncrementalRepeatEventOnMissingObject (batchId=223)
org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testStatus (batchId=223)
org.apache.hadoop.hive.ql.security.TestClientSideAuthorizationProvider.testSimplePrivileges (batchId=225)
org.apache.hive.jdbc.TestSSL.testConnectionMismatch (batchId=229)
org.apache.hive.jdbc.TestSSL.testConnectionWrongCertCN (batchId=229)
org.apache.hive.jdbc.TestSSL.testMetastoreConnectionWrongCertCN (batchId=229)
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7943/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7943/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7943/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 175 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12898683 - PreCommit-HIVE-Build

> Remove mixed partitions/table schema support
> --------------------------------------------
>
>                 Key: HIVE-18113
>                 URL: https://issues.apache.org/jira/browse/HIVE-18113
>             Project: Hive
>          Issue Type: Sub-task
>            Reporter: Zoltan Haindrich
>            Assignee: Zoltan Haindrich
>         Attachments: HIVE-18113.01.patch
>
>
> Currently it is possible to have misaligned table/partition schemas;
> [see here for example| https://github.com/apache/hive/blob/a5c2e15c7cc125d8cda2ee3a8ed64c116ff6b755/ql/src/test/queries/clientpositive/schema_evol_text_vec_part.q#L156]
> result of [this insert statement|https://github.com/apache/hive/blob/a5c2e15c7cc125d8cda2ee3a8ed64c116ff6b755/ql/src/test/queries/clientpositive/schema_evol_text_vec_part.q#L162] are these [null values|https://github.com/apache/hive/blob/a5c2e15c7cc125d8cda2ee3a8ed64c116ff6b755/ql/src/test/results/clientpositive/llap/schema_evol_text_vec_part.q.out#L660]
> This mixed partition setup can cause the stats aggregation to become quite confusing...I think it would be better to remove this thing....there is a {{CASCADE}} flag already; which changes the schema all over the table/partitions/etc.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)