You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by Aihua Xu <ax...@cloudera.com> on 2017/03/03 15:01:08 UTC
Re: Review Request 56546: Allow user to update AVRO table schema via
command even if table has external schema
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56546/#review167827
-----------------------------------------------------------
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
Lines 1207 (patched)
<https://reviews.apache.org/r/56546/#comment239798>
I think finally I understand the approach you are taking. :)
You are removing the avro literal/url so the avro schemas are inferred from the table structure.
For avro table definition, we can define hive schema and/or avro schema and we can have both or either of them. If avro schema exists, seems the hive schema will be ignored.
I feel the approach to drop literal/url will cause some issues since the user could create a table using avro schema. If the avro file generated from other tools changes, the users can just update the avro schema itself to reflect in hive table. With this patch, it will break such logic.
My suggestion to this fix is: just simply give an error to mention that you can't add columns if url/literal is used (such add column operation is supported with hive schema and without avro schema), just like replace columns.
hive> alter table employees_avro replace columns (salary int);
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. Replace columns is not supported for table employees_avro. SerDe may be incompatible.
You need to use hive schema rather than avro schema to define the table in order to support alter table operations.
- Aihua Xu
On Feb. 22, 2017, 11:48 a.m., Adam Szita wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56546/
> -----------------------------------------------------------
>
> (Updated Feb. 22, 2017, 11:48 a.m.)
>
>
> Review request for hive, Aihua Xu, Peter Vary, and Sergio Pena.
>
>
> Bugs: HIVE-13780
> https://issues.apache.org/jira/browse/HIVE-13780
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> Allow user to update AVRO table schema via command even if table's definition was defined through schema file / literal in tblproperties
>
>
> Diffs
> -----
>
> common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 3777fa96ba8eac79e07e454ee437fb05583158c5
> ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java adabe70fa8f0fe1b990c6ac578a14ff5af06fc93
> ql/src/test/queries/clientnegative/avro_add_column_extschema.q PRE-CREATION
> ql/src/test/queries/clientpositive/avro_add_column_extschema.q PRE-CREATION
> ql/src/test/results/clientnegative/avro_add_column_extschema.q.out PRE-CREATION
> ql/src/test/results/clientpositive/avro_add_column_extschema.q.out PRE-CREATION
> serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroSerdeUtils.java f18585da1d108abdd500437362eb388b21030ec7
>
>
> Diff: https://reviews.apache.org/r/56546/diff/2/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Adam Szita
>
>