You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "László Pintér (Jira)" <ji...@apache.org> on 2022/09/07 11:01:00 UTC
[jira] [Commented] (HIVE-26507) Do not allow hive to iceberg migration if source table contains CHAR or VARCHAR columns
[ https://issues.apache.org/jira/browse/HIVE-26507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17601246#comment-17601246 ]
László Pintér commented on HIVE-26507:
--------------------------------------
Merged into master. Thanks, [~szita] for the review!
> Do not allow hive to iceberg migration if source table contains CHAR or VARCHAR columns
> ---------------------------------------------------------------------------------------
>
> Key: HIVE-26507
> URL: https://issues.apache.org/jira/browse/HIVE-26507
> Project: Hive
> Issue Type: Bug
> Reporter: Rajesh Balamohan
> Assignee: László Pintér
> Priority: Major
> Labels: iceberg, pull-request-available
> Time Spent: 40m
> Remaining Estimate: 0h
>
> "alter table" statements can be used for generating iceberg metadata information (i.e for converting external tables -> iceberg tables).
> As part of this process, it also converts certain datatypes to iceberg compatible types (e.g char -> string). "iceberg.mr.schema.auto.conversion" enables this conversion.
> This could cause certain issues at runtime. Here is an example
> {noformat}
> Before conversion:
> ==================
> -- external table
> select count(*) from customer_demographics where cd_gender = 'F' and cd_marital_status = 'U' and cd_education_status = '2 yr Degree';
> 27440
> after conversion:
> =================
> -- iceberg table
> select count(*) from customer_demographics where cd_gender = 'F' and cd_marital_status = 'U' and cd_education_status = '2 yr Degree';
> 0
> select count(*) from customer_demographics where cd_gender = 'F' and cd_marital_status = 'U' and trim(cd_education_status) = '2 yr Degree';
> 27440
> {noformat}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)