You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Navis (JIRA)" <ji...@apache.org> on 2014/01/22 06:11:20 UTC
[jira] [Updated] (HIVE-6083) User provided table properties are not
assigned to the TableDesc of the FileSinkDesc in a CTAS query
[ https://issues.apache.org/jira/browse/HIVE-6083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Navis updated HIVE-6083:
------------------------
Resolution: Fixed
Fix Version/s: 0.13.0
Status: Resolved (was: Patch Available)
Committed to trunk. Thanks Yin!
> User provided table properties are not assigned to the TableDesc of the FileSinkDesc in a CTAS query
> ----------------------------------------------------------------------------------------------------
>
> Key: HIVE-6083
> URL: https://issues.apache.org/jira/browse/HIVE-6083
> Project: Hive
> Issue Type: Bug
> Affects Versions: 0.12.0, 0.13.0
> Reporter: Yin Huai
> Assignee: Yin Huai
> Fix For: 0.13.0
>
> Attachments: HIVE-6083.1.patch.txt, HIVE-6083.2.patch.txt
>
>
> I was trying to use a CTAS query to create a table stored with ORC and orc.compress was set to SNAPPY. However, the table was still compressed as ZLIB (although the result of DESCRIBE still shows that this table is compressed by SNAPPY). For a CTAS query, SemanticAnalyzer.genFileSinkPlan uses CreateTableDesc to generate the TableDesc for the FileSinkDesc by calling PlanUtils.getTableDesc. However, in PlanUtils.getTableDesc, I do not see user provided table properties are assigned to the returned TableDesc (CreateTableDesc.getTblProps was not called in this method ).
> btw, I only checked the code of 0.12 and trunk.
> Two examples:
> * Snappy compression
> {code}
> create table web_sales_wrong_orc_snappy
> stored as orc tblproperties ("orc.compress"="SNAPPY")
> as select * from web_sales;
> {code}
> {code}
> describe formatted web_sales_wrong_orc_snappy;
> ....
> Location: hdfs://localhost:54310/user/hive/warehouse/web_sales_wrong_orc_snappy
> Table Type: MANAGED_TABLE
> Table Parameters:
> COLUMN_STATS_ACCURATE true
> numFiles 1
> numRows 719384
> orc.compress SNAPPY
> rawDataSize 97815412
> totalSize 40625243
> transient_lastDdlTime 1387566015
> ....
> {code}
> {code}
> bin/hive --orcfiledump /user/hive/warehouse/web_sales_wrong_orc_snappy/000000_0
> Rows: 719384
> Compression: ZLIB
> Compression size: 262144
> ...
> {code}
> * No compression
> {code}
> create table web_sales_wrong_orc_none
> stored as orc tblproperties ("orc.compress"="NONE")
> as select * from web_sales;
> {code}
> {code}
> describe formatted web_sales_wrong_orc_none;
> ....
> Location: hdfs://localhost:54310/user/hive/warehouse/web_sales_wrong_orc_none
> Table Type: MANAGED_TABLE
> Table Parameters:
> COLUMN_STATS_ACCURATE true
> numFiles 1
> numRows 719384
> orc.compress NONE
> rawDataSize 97815412
> totalSize 40625243
> transient_lastDdlTime 1387566064
> ....
> {code}
> {code}
> bin/hive --orcfiledump /user/hive/warehouse/web_sales_wrong_orc_none/000000_0
> Rows: 719384
> Compression: ZLIB
> Compression size: 262144
> ...
> {code}
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)