You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2020/07/17 15:14:00 UTC
[jira] [Work logged] (HIVE-23871) ObjectStore should properly
handle MicroManaged Table properties
[ https://issues.apache.org/jira/browse/HIVE-23871?focusedWorklogId=460349&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-460349 ]
ASF GitHub Bot logged work on HIVE-23871:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 17/Jul/20 15:13
Start Date: 17/Jul/20 15:13
Worklog Time Spent: 10m
Work Description: pgaref opened a new pull request #1273:
URL: https://github.com/apache/hive/pull/1273
ObjectStore should properly handle MicroManaged Table properties
Change-Id: Ia5db047419a11504f3c6047a1eb63acd2a14bdc3
## NOTICE
Please create an issue in ASF JIRA before opening a pull request,
and you need to set the title of the pull request which starts with
the corresponding JIRA issue number. (e.g. HIVE-XXXXX: Fix a typo in YYY)
For more details, please see https://cwiki.apache.org/confluence/display/Hive/HowToContribute
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 460349)
Remaining Estimate: 0h
Time Spent: 10m
> ObjectStore should properly handle MicroManaged Table properties
> ----------------------------------------------------------------
>
> Key: HIVE-23871
> URL: https://issues.apache.org/jira/browse/HIVE-23871
> Project: Hive
> Issue Type: Bug
> Components: Metastore
> Reporter: Panagiotis Garefalakis
> Assignee: Panagiotis Garefalakis
> Priority: Major
> Attachments: table1
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> HIVE-23281 optimizes StorageDescriptor conversion as part of the ObjectStore by skipping particular Table properties like SkewInfo, bucketCols, ordering etc.
> However, it does that for all Transactional Tables – not only ACID – causing MicroManaged Tables to behave abnormally.
> MicroManaged (insert_only) tables may miss needed properties such as Storage Desc Params – that may define how lines are delimited (like in the example below):
> To repro the issue:
> {code:java}
> CREATE TRANSACTIONAL TABLE delim_table_trans(id INT, name STRING, safety INT) ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t' STORED AS TEXTFILE;
> LOAD DATA INPATH 'table1' OVERWRITE INTO TABLE delim_table_trans;
> describe formatted delim_table_trans;
> SELECT * FROM delim_table_trans;
> {code}
> Result:
> {code:java}
> Table Type: MANAGED_TABLE
> Table Parameters:
> bucketing_version 2
> numFiles 1
> numRows 0
> rawDataSize 0
> totalSize 72
> transactional true
> transactional_properties insert_only
> #### A masked pattern was here ####
>
> # Storage Information
> SerDe Library: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> InputFormat: org.apache.hadoop.mapred.TextInputFormat
> OutputFormat: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> Compressed: No
> Num Buckets: -1
> Bucket Columns: []
> Sort Columns: []
> PREHOOK: query: SELECT * FROM delim_table_trans
> PREHOOK: type: QUERY
> PREHOOK: Input: default@delim_table_trans
> #### A masked pattern was here ####
> POSTHOOK: query: SELECT * FROM delim_table_trans
> POSTHOOK: type: QUERY
> POSTHOOK: Input: default@delim_table_trans
> #### A masked pattern was here ####
> NULL NULL NULL
> NULL NULL NULL
> NULL NULL NULL
> NULL NULL NULL
> NULL NULL NULL
> NULL NULL NULL
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)