You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by Mohammad Islam <mi...@yahoo.com> on 2013/08/08 02:24:30 UTC
Re: Review Request 11925: Hive-3159 Update AvroSerde to determine schema of
new tables
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11925/
-----------------------------------------------------------
(Updated Aug. 8, 2013, 12:24 a.m.)
Review request for hive, Ashutosh Chauhan and Jakob Homan.
Changes
-------
Adding support for binary in type info to schema generation.
Bugs: HIVE-3159
https://issues.apache.org/jira/browse/HIVE-3159
Repository: hive-git
Description
-------
Problem:
Hive doesn't support to create a Avro-based table using HQL create table command. It currently requires to specify Avro schema literal or schema file name.
For multiple cases, it is very inconvenient for user.
Some of the un-supported use cases:
1. Create table ... <Avro-SERDE etc.> as SELECT ... from <NON-AVRO FILE>
2. Create table ... <Avro-SERDE etc.> as SELECT from <AVRO TABLE>
3. Create table without specifying Avro schema.
Diffs (updated)
-----
ql/src/test/queries/clientpositive/avro_create_as_select.q PRE-CREATION
ql/src/test/queries/clientpositive/avro_create_as_select2.q PRE-CREATION
ql/src/test/queries/clientpositive/avro_no_schema_test.q PRE-CREATION
ql/src/test/queries/clientpositive/avro_without_schema.q PRE-CREATION
ql/src/test/results/clientpositive/avro_create_as_select.q.out PRE-CREATION
ql/src/test/results/clientpositive/avro_create_as_select2.q.out PRE-CREATION
ql/src/test/results/clientpositive/avro_no_schema_test.q.out PRE-CREATION
ql/src/test/results/clientpositive/avro_without_schema.q.out PRE-CREATION
serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroSerdeUtils.java 13848b6
serde/src/java/org/apache/hadoop/hive/serde2/avro/TypeInfoToSchema.java PRE-CREATION
serde/src/test/org/apache/hadoop/hive/serde2/avro/TestAvroSerdeUtils.java 010f614
serde/src/test/org/apache/hadoop/hive/serde2/avro/TestTypeInfoToSchema.java PRE-CREATION
Diff: https://reviews.apache.org/r/11925/diff/
Testing
-------
Wrote a new java Test class for a new Java class. Added a new test case into existing java test class. In addition, there are 4 .q file for testing multiple use-cases.
Thanks,
Mohammad Islam
Re: Review Request 11925: Hive-3159 Update AvroSerde to determine schema of
new tables
Posted by Mohammad Islam <mi...@yahoo.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11925/
-----------------------------------------------------------
(Updated May 9, 2014, 12:23 a.m.)
Review request for hive, Ashutosh Chauhan and Jakob Homan.
Changes
-------
Rebased with the latest commit.
Bugs: HIVE-3159
https://issues.apache.org/jira/browse/HIVE-3159
Repository: hive-git
Description
-------
Problem:
Hive doesn't support to create a Avro-based table using HQL create table command. It currently requires to specify Avro schema literal or schema file name.
For multiple cases, it is very inconvenient for user.
Some of the un-supported use cases:
1. Create table ... <Avro-SERDE etc.> as SELECT ... from <NON-AVRO FILE>
2. Create table ... <Avro-SERDE etc.> as SELECT from <AVRO TABLE>
3. Create table without specifying Avro schema.
Diffs (updated)
-----
ql/src/test/queries/clientpositive/avro_create_as_select.q PRE-CREATION
ql/src/test/queries/clientpositive/avro_nested_complex.q PRE-CREATION
ql/src/test/queries/clientpositive/avro_nullable_fields.q f90ceb9
ql/src/test/queries/clientpositive/avro_without_schema.q PRE-CREATION
ql/src/test/results/clientpositive/avro_create_as_select.q.out PRE-CREATION
ql/src/test/results/clientpositive/avro_nested_complex.q.out PRE-CREATION
ql/src/test/results/clientpositive/avro_nullable_fields.q.out 77a6a2e
ql/src/test/results/clientpositive/avro_without_schema.q.out PRE-CREATION
serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroSerdeUtils.java 4564e75
serde/src/java/org/apache/hadoop/hive/serde2/avro/TypeInfoToSchema.java PRE-CREATION
serde/src/test/org/apache/hadoop/hive/serde2/avro/TestAvroSerdeUtils.java 67d5570
serde/src/test/org/apache/hadoop/hive/serde2/avro/TestTypeInfoToSchema.java PRE-CREATION
Diff: https://reviews.apache.org/r/11925/diff/
Testing
-------
Wrote a new java Test class for a new Java class. Added a new test case into existing java test class. In addition, there are 4 .q file for testing multiple use-cases.
Thanks,
Mohammad Islam
Re: Review Request 11925: Hive-3159 Update AvroSerde to determine schema of
new tables
Posted by Mohammad Islam <mi...@yahoo.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11925/
-----------------------------------------------------------
(Updated May 6, 2014, 10:06 p.m.)
Review request for hive, Ashutosh Chauhan and Jakob Homan.
Changes
-------
New patch that addressed Carl's review comments.
This patch addressed the following missing functions.
1. Create AVRO table from using HIVE schema ( w/o specifying Avro Schema).
2. Copy AVRO table structure and data from existing non-AVRO table using CTAS.
3. Copy AVRO table structure and data from existing AVRO table using CTAS.
Note: We can close dependent JIRA HIVE-5803 that is no longer required. Another JIRA has already taken care of this.
Bugs: HIVE-3159
https://issues.apache.org/jira/browse/HIVE-3159
Repository: hive-git
Description
-------
Problem:
Hive doesn't support to create a Avro-based table using HQL create table command. It currently requires to specify Avro schema literal or schema file name.
For multiple cases, it is very inconvenient for user.
Some of the un-supported use cases:
1. Create table ... <Avro-SERDE etc.> as SELECT ... from <NON-AVRO FILE>
2. Create table ... <Avro-SERDE etc.> as SELECT from <AVRO TABLE>
3. Create table without specifying Avro schema.
Diffs (updated)
-----
ql/src/test/queries/clientpositive/avro_create_as_select.q PRE-CREATION
ql/src/test/queries/clientpositive/avro_nested_complex.q PRE-CREATION
ql/src/test/queries/clientpositive/avro_nullable_fields.q f90ceb9
ql/src/test/queries/clientpositive/avro_without_schema.q PRE-CREATION
ql/src/test/results/clientpositive/avro_create_as_select.q.out PRE-CREATION
ql/src/test/results/clientpositive/avro_nested_complex.q.out PRE-CREATION
ql/src/test/results/clientpositive/avro_nullable_fields.q.out 77a6a2e
ql/src/test/results/clientpositive/avro_without_schema.q.out PRE-CREATION
serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroSerdeUtils.java 9d58d13
serde/src/java/org/apache/hadoop/hive/serde2/avro/TypeInfoToSchema.java PRE-CREATION
serde/src/test/org/apache/hadoop/hive/serde2/avro/TestAvroSerdeUtils.java 67d5570
serde/src/test/org/apache/hadoop/hive/serde2/avro/TestTypeInfoToSchema.java PRE-CREATION
Diff: https://reviews.apache.org/r/11925/diff/
Testing
-------
Wrote a new java Test class for a new Java class. Added a new test case into existing java test class. In addition, there are 4 .q file for testing multiple use-cases.
Thanks,
Mohammad Islam
Re: Review Request 11925: Hive-3159 Update AvroSerde to determine schema of
new tables
Posted by Mohammad Islam <mi...@yahoo.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11925/
-----------------------------------------------------------
(Updated Jan. 31, 2014, 2:18 a.m.)
Review request for hive, Ashutosh Chauhan and Jakob Homan.
Changes
-------
Addressed Carl's review comments
Bugs: HIVE-3159
https://issues.apache.org/jira/browse/HIVE-3159
Repository: hive-git
Description
-------
Problem:
Hive doesn't support to create a Avro-based table using HQL create table command. It currently requires to specify Avro schema literal or schema file name.
For multiple cases, it is very inconvenient for user.
Some of the un-supported use cases:
1. Create table ... <Avro-SERDE etc.> as SELECT ... from <NON-AVRO FILE>
2. Create table ... <Avro-SERDE etc.> as SELECT from <AVRO TABLE>
3. Create table without specifying Avro schema.
Diffs (updated)
-----
ql/src/test/queries/clientpositive/avro_create_as_select.q PRE-CREATION
ql/src/test/queries/clientpositive/avro_nested_complex.q PRE-CREATION
ql/src/test/queries/clientpositive/avro_nullable_fields.q f90ceb9
ql/src/test/queries/clientpositive/avro_without_schema.q PRE-CREATION
ql/src/test/results/clientpositive/avro_create_as_select.q.out PRE-CREATION
ql/src/test/results/clientpositive/avro_nested_complex.q.out PRE-CREATION
ql/src/test/results/clientpositive/avro_nullable_fields.q.out 81950b0
ql/src/test/results/clientpositive/avro_without_schema.q.out PRE-CREATION
serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroSerdeUtils.java 9d58d13
serde/src/java/org/apache/hadoop/hive/serde2/avro/TypeInfoToSchema.java PRE-CREATION
serde/src/test/org/apache/hadoop/hive/serde2/avro/TestAvroSerdeUtils.java 67d5570
serde/src/test/org/apache/hadoop/hive/serde2/avro/TestTypeInfoToSchema.java PRE-CREATION
Diff: https://reviews.apache.org/r/11925/diff/
Testing
-------
Wrote a new java Test class for a new Java class. Added a new test case into existing java test class. In addition, there are 4 .q file for testing multiple use-cases.
Thanks,
Mohammad Islam
Re: Review Request 11925: Hive-3159 Update AvroSerde to determine schema of
new tables
Posted by Mohammad Islam <mi...@yahoo.com>.
> On Jan. 11, 2014, 8:05 a.m., Carl Steinbach wrote:
> > serde/src/java/org/apache/hadoop/hive/serde2/avro/TypeInfoToSchema.java, line 146
> > <https://reviews.apache.org/r/11925/diff/6/?file=420187#file420187line146>
> >
> > wrapWithUnion is not used.
Used in updated code.
> On Jan. 11, 2014, 8:05 a.m., Carl Steinbach wrote:
> > serde/src/java/org/apache/hadoop/hive/serde2/avro/TypeInfoToSchema.java, line 163
> > <https://reviews.apache.org/r/11925/diff/6/?file=420187#file420187line163>
> >
> > wrapWithUnion is not used.
Used in updated code.
> On Jan. 11, 2014, 8:05 a.m., Carl Steinbach wrote:
> > serde/src/java/org/apache/hadoop/hive/serde2/avro/TypeInfoToSchema.java, line 174
> > <https://reviews.apache.org/r/11925/diff/6/?file=420187#file420187line174>
> >
> > wrapWithUnion is not used.
Used in updated code.
> On Jan. 11, 2014, 8:05 a.m., Carl Steinbach wrote:
> > ql/src/test/queries/clientpositive/avro_create_as_select.q, line 3
> > <https://reviews.apache.org/r/11925/diff/6/?file=420178#file420178line3>
> >
> > Please populate the source table with data before running a CTAS statement against it (the current example is functionally equivalent to CREATE TABLE LIKE -- does that work?). Better yet, please use the alltypesorc test table (which has columns for each type), as well as a query where only some columns are projected, and a query where nested fields are explicitly projected by name.
I added a new table creation using all avro types.
But load data into Avro table from non-Avro doesn't work (REF:HIVE-5803).
> On Jan. 11, 2014, 8:05 a.m., Carl Steinbach wrote:
> > ql/src/test/queries/clientpositive/avro_without_schema.q, line 1
> > <https://reviews.apache.org/r/11925/diff/6/?file=420181#file420181line1>
> >
> > Why have both avro_no_schema_test.q and avro_without_schema.q? Why does avro_without_schema.q create the table, but not load it with data or select from it?
remove one and put into one file with proper comments.
> On Jan. 11, 2014, 8:05 a.m., Carl Steinbach wrote:
> > serde/src/java/org/apache/hadoop/hive/serde2/avro/TypeInfoToSchema.java, line 53
> > <https://reviews.apache.org/r/11925/diff/6/?file=420187#file420187line53>
> >
> > "Moreover, since any type could potentially hold a NULL value, all corresponding Avro schema should be a union of null and the type (UNION<T, NULL>).
> >
> > This statement is contradicted by the code which won't generate a record or array field with nullable elements.
> >
> > Can you please add a test to show what happens when a non-avro table with record and array columns containing NULL elements is streamed into an Avro table using CTAS?
Added test case.
- Mohammad
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11925/#review31584
-----------------------------------------------------------
On Jan. 10, 2014, 7:58 p.m., Mohammad Islam wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/11925/
> -----------------------------------------------------------
>
> (Updated Jan. 10, 2014, 7:58 p.m.)
>
>
> Review request for hive, Ashutosh Chauhan and Jakob Homan.
>
>
> Bugs: HIVE-3159
> https://issues.apache.org/jira/browse/HIVE-3159
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> Problem:
> Hive doesn't support to create a Avro-based table using HQL create table command. It currently requires to specify Avro schema literal or schema file name.
> For multiple cases, it is very inconvenient for user.
> Some of the un-supported use cases:
> 1. Create table ... <Avro-SERDE etc.> as SELECT ... from <NON-AVRO FILE>
> 2. Create table ... <Avro-SERDE etc.> as SELECT from <AVRO TABLE>
> 3. Create table without specifying Avro schema.
>
>
> Diffs
> -----
>
> ql/src/test/queries/clientpositive/avro_create_as_select.q PRE-CREATION
> ql/src/test/queries/clientpositive/avro_create_as_select2.q PRE-CREATION
> ql/src/test/queries/clientpositive/avro_no_schema_test.q PRE-CREATION
> ql/src/test/queries/clientpositive/avro_without_schema.q PRE-CREATION
> ql/src/test/results/clientpositive/avro_create_as_select.q.out PRE-CREATION
> ql/src/test/results/clientpositive/avro_create_as_select2.q.out PRE-CREATION
> ql/src/test/results/clientpositive/avro_no_schema_test.q.out PRE-CREATION
> ql/src/test/results/clientpositive/avro_without_schema.q.out PRE-CREATION
> serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroSerdeUtils.java 9d58d13
> serde/src/java/org/apache/hadoop/hive/serde2/avro/TypeInfoToSchema.java PRE-CREATION
> serde/src/test/org/apache/hadoop/hive/serde2/avro/TestAvroSerdeUtils.java 67d5570
> serde/src/test/org/apache/hadoop/hive/serde2/avro/TestTypeInfoToSchema.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/11925/diff/
>
>
> Testing
> -------
>
> Wrote a new java Test class for a new Java class. Added a new test case into existing java test class. In addition, there are 4 .q file for testing multiple use-cases.
>
>
> Thanks,
>
> Mohammad Islam
>
>
Re: Review Request 11925: Hive-3159 Update AvroSerde to determine schema of
new tables
Posted by Carl Steinbach <cw...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11925/#review31584
-----------------------------------------------------------
ql/src/test/queries/clientpositive/avro_create_as_select.q
<https://reviews.apache.org/r/11925/#comment60123>
Please populate the source table with data before running a CTAS statement against it (the current example is functionally equivalent to CREATE TABLE LIKE -- does that work?). Better yet, please use the alltypesorc test table (which has columns for each type), as well as a query where only some columns are projected, and a query where nested fields are explicitly projected by name.
ql/src/test/queries/clientpositive/avro_without_schema.q
<https://reviews.apache.org/r/11925/#comment60124>
Why have both avro_no_schema_test.q and avro_without_schema.q? Why does avro_without_schema.q create the table, but not load it with data or select from it?
serde/src/java/org/apache/hadoop/hive/serde2/avro/TypeInfoToSchema.java
<https://reviews.apache.org/r/11925/#comment60122>
"Moreover, since any type could potentially hold a NULL value, all corresponding Avro schema should be a union of null and the type (UNION<T, NULL>).
This statement is contradicted by the code which won't generate a record or array field with nullable elements.
Can you please add a test to show what happens when a non-avro table with record and array columns containing NULL elements is streamed into an Avro table using CTAS?
serde/src/java/org/apache/hadoop/hive/serde2/avro/TypeInfoToSchema.java
<https://reviews.apache.org/r/11925/#comment60119>
wrapWithUnion is not used.
serde/src/java/org/apache/hadoop/hive/serde2/avro/TypeInfoToSchema.java
<https://reviews.apache.org/r/11925/#comment60120>
wrapWithUnion is not used.
serde/src/java/org/apache/hadoop/hive/serde2/avro/TypeInfoToSchema.java
<https://reviews.apache.org/r/11925/#comment60121>
wrapWithUnion is not used.
- Carl Steinbach
On Jan. 10, 2014, 7:58 p.m., Mohammad Islam wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/11925/
> -----------------------------------------------------------
>
> (Updated Jan. 10, 2014, 7:58 p.m.)
>
>
> Review request for hive, Ashutosh Chauhan and Jakob Homan.
>
>
> Bugs: HIVE-3159
> https://issues.apache.org/jira/browse/HIVE-3159
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> Problem:
> Hive doesn't support to create a Avro-based table using HQL create table command. It currently requires to specify Avro schema literal or schema file name.
> For multiple cases, it is very inconvenient for user.
> Some of the un-supported use cases:
> 1. Create table ... <Avro-SERDE etc.> as SELECT ... from <NON-AVRO FILE>
> 2. Create table ... <Avro-SERDE etc.> as SELECT from <AVRO TABLE>
> 3. Create table without specifying Avro schema.
>
>
> Diffs
> -----
>
> ql/src/test/queries/clientpositive/avro_create_as_select.q PRE-CREATION
> ql/src/test/queries/clientpositive/avro_create_as_select2.q PRE-CREATION
> ql/src/test/queries/clientpositive/avro_no_schema_test.q PRE-CREATION
> ql/src/test/queries/clientpositive/avro_without_schema.q PRE-CREATION
> ql/src/test/results/clientpositive/avro_create_as_select.q.out PRE-CREATION
> ql/src/test/results/clientpositive/avro_create_as_select2.q.out PRE-CREATION
> ql/src/test/results/clientpositive/avro_no_schema_test.q.out PRE-CREATION
> ql/src/test/results/clientpositive/avro_without_schema.q.out PRE-CREATION
> serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroSerdeUtils.java 9d58d13
> serde/src/java/org/apache/hadoop/hive/serde2/avro/TypeInfoToSchema.java PRE-CREATION
> serde/src/test/org/apache/hadoop/hive/serde2/avro/TestAvroSerdeUtils.java 67d5570
> serde/src/test/org/apache/hadoop/hive/serde2/avro/TestTypeInfoToSchema.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/11925/diff/
>
>
> Testing
> -------
>
> Wrote a new java Test class for a new Java class. Added a new test case into existing java test class. In addition, there are 4 .q file for testing multiple use-cases.
>
>
> Thanks,
>
> Mohammad Islam
>
>
Re: Review Request 11925: Hive-3159 Update AvroSerde to determine schema of
new tables
Posted by Carl Steinbach <cw...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11925/#review31583
-----------------------------------------------------------
ql/src/test/queries/clientpositive/avro_create_as_select.q
<https://reviews.apache.org/r/11925/#comment60098>
Please maintain reasonable limits on line length (80 or 100 characters works for me).
ql/src/test/queries/clientpositive/avro_create_as_select.q
<https://reviews.apache.org/r/11925/#comment60099>
Please put each clause on its own line:
CREATE TABLE xxx
ROW FORMAT SERDE '....'
STORED AS
INPUTFORMAT '....'
OUTPUTFORMAT '....'
AS
SELECT ......
ql/src/test/queries/clientpositive/avro_create_as_select2.q
<https://reviews.apache.org/r/11925/#comment60100>
Line length
ql/src/test/queries/clientpositive/avro_no_schema_test.q
<https://reviews.apache.org/r/11925/#comment60101>
Line length.
serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroSerdeUtils.java
<https://reviews.apache.org/r/11925/#comment60102>
Formatting
serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroSerdeUtils.java
<https://reviews.apache.org/r/11925/#comment60103>
Formatting
serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroSerdeUtils.java
<https://reviews.apache.org/r/11925/#comment60104>
Formatting
serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroSerdeUtils.java
<https://reviews.apache.org/r/11925/#comment60105>
s/Defintion/Definition/
serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroSerdeUtils.java
<https://reviews.apache.org/r/11925/#comment60106>
Please throw the exception here instead of returning null. This method should never return null.
serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroSerdeUtils.java
<https://reviews.apache.org/r/11925/#comment60107>
What's the significance of "org.apache.hive.auto_gen_schema"? I can't find any other references to this namespace in the code.
serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroSerdeUtils.java
<https://reviews.apache.org/r/11925/#comment60115>
table or partition properties, or both?
serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroSerdeUtils.java
<https://reviews.apache.org/r/11925/#comment60108>
Formatting
serde/src/java/org/apache/hadoop/hive/serde2/avro/TypeInfoToSchema.java
<https://reviews.apache.org/r/11925/#comment60109>
Missing ASF license header.
serde/src/java/org/apache/hadoop/hive/serde2/avro/TypeInfoToSchema.java
<https://reviews.apache.org/r/11925/#comment60110>
To me "TypeInfoToSchema" sounds like the name of method, not a class. Please either change the name to AvroSchemaUtils or merge this code into AvroSerdeUtils.
Also, there is an existing Hive class named "Schema" (org.apache.hadoop.hive.metastore.api.Schema). In order to avoid confusion please qualify "schema" with "avro" in the method names, e.g. generateAvroSchema, generateAvroUnionSchema, etc...
serde/src/java/org/apache/hadoop/hive/serde2/avro/TypeInfoToSchema.java
<https://reviews.apache.org/r/11925/#comment60118>
map<string, any-type>
serde/src/java/org/apache/hadoop/hive/serde2/avro/TypeInfoToSchema.java
<https://reviews.apache.org/r/11925/#comment60113>
Why Hashtable instead of HashMap?
serde/src/java/org/apache/hadoop/hive/serde2/avro/TypeInfoToSchema.java
<https://reviews.apache.org/r/11925/#comment60111>
People who don't already know how Avro encodes nullable values will find this method easier to understand if the parameter name "wrapWithUnion" is changed to "isNullable".
serde/src/java/org/apache/hadoop/hive/serde2/avro/TypeInfoToSchema.java
<https://reviews.apache.org/r/11925/#comment60116>
Instead of adding assertions to each of these methods I think it would be cleaner to specify the expected type in the parameter list and force the caller to do the cast before invoking the method.
serde/src/java/org/apache/hadoop/hive/serde2/avro/TypeInfoToSchema.java
<https://reviews.apache.org/r/11925/#comment60114>
The code snippet "tag + "_" + tInfo.getCategory().name()" gets repeated a lot. I think this should be moved to a helper method.
serde/src/java/org/apache/hadoop/hive/serde2/avro/TypeInfoToSchema.java
<https://reviews.apache.org/r/11925/#comment60112>
Is it possible to change the LHS to List?
serde/src/java/org/apache/hadoop/hive/serde2/avro/TypeInfoToSchema.java
<https://reviews.apache.org/r/11925/#comment60117>
Formatting.
serde/src/test/org/apache/hadoop/hive/serde2/avro/TestTypeInfoToSchema.java
<https://reviews.apache.org/r/11925/#comment60096>
Missing ASF license header.
serde/src/test/org/apache/hadoop/hive/serde2/avro/TestTypeInfoToSchema.java
<https://reviews.apache.org/r/11925/#comment60097>
Formatting.
- Carl Steinbach
On Jan. 10, 2014, 7:58 p.m., Mohammad Islam wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/11925/
> -----------------------------------------------------------
>
> (Updated Jan. 10, 2014, 7:58 p.m.)
>
>
> Review request for hive, Ashutosh Chauhan and Jakob Homan.
>
>
> Bugs: HIVE-3159
> https://issues.apache.org/jira/browse/HIVE-3159
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> Problem:
> Hive doesn't support to create a Avro-based table using HQL create table command. It currently requires to specify Avro schema literal or schema file name.
> For multiple cases, it is very inconvenient for user.
> Some of the un-supported use cases:
> 1. Create table ... <Avro-SERDE etc.> as SELECT ... from <NON-AVRO FILE>
> 2. Create table ... <Avro-SERDE etc.> as SELECT from <AVRO TABLE>
> 3. Create table without specifying Avro schema.
>
>
> Diffs
> -----
>
> ql/src/test/queries/clientpositive/avro_create_as_select.q PRE-CREATION
> ql/src/test/queries/clientpositive/avro_create_as_select2.q PRE-CREATION
> ql/src/test/queries/clientpositive/avro_no_schema_test.q PRE-CREATION
> ql/src/test/queries/clientpositive/avro_without_schema.q PRE-CREATION
> ql/src/test/results/clientpositive/avro_create_as_select.q.out PRE-CREATION
> ql/src/test/results/clientpositive/avro_create_as_select2.q.out PRE-CREATION
> ql/src/test/results/clientpositive/avro_no_schema_test.q.out PRE-CREATION
> ql/src/test/results/clientpositive/avro_without_schema.q.out PRE-CREATION
> serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroSerdeUtils.java 9d58d13
> serde/src/java/org/apache/hadoop/hive/serde2/avro/TypeInfoToSchema.java PRE-CREATION
> serde/src/test/org/apache/hadoop/hive/serde2/avro/TestAvroSerdeUtils.java 67d5570
> serde/src/test/org/apache/hadoop/hive/serde2/avro/TestTypeInfoToSchema.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/11925/diff/
>
>
> Testing
> -------
>
> Wrote a new java Test class for a new Java class. Added a new test case into existing java test class. In addition, there are 4 .q file for testing multiple use-cases.
>
>
> Thanks,
>
> Mohammad Islam
>
>
Re: Review Request 11925: Hive-3159 Update AvroSerde to determine schema of
new tables
Posted by Mohammad Islam <mi...@yahoo.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11925/
-----------------------------------------------------------
(Updated Jan. 10, 2014, 7:58 p.m.)
Review request for hive, Ashutosh Chauhan and Jakob Homan.
Changes
-------
Incorporating the reviewers' comments,
Bugs: HIVE-3159
https://issues.apache.org/jira/browse/HIVE-3159
Repository: hive-git
Description
-------
Problem:
Hive doesn't support to create a Avro-based table using HQL create table command. It currently requires to specify Avro schema literal or schema file name.
For multiple cases, it is very inconvenient for user.
Some of the un-supported use cases:
1. Create table ... <Avro-SERDE etc.> as SELECT ... from <NON-AVRO FILE>
2. Create table ... <Avro-SERDE etc.> as SELECT from <AVRO TABLE>
3. Create table without specifying Avro schema.
Diffs (updated)
-----
ql/src/test/queries/clientpositive/avro_create_as_select.q PRE-CREATION
ql/src/test/queries/clientpositive/avro_create_as_select2.q PRE-CREATION
ql/src/test/queries/clientpositive/avro_no_schema_test.q PRE-CREATION
ql/src/test/queries/clientpositive/avro_without_schema.q PRE-CREATION
ql/src/test/results/clientpositive/avro_create_as_select.q.out PRE-CREATION
ql/src/test/results/clientpositive/avro_create_as_select2.q.out PRE-CREATION
ql/src/test/results/clientpositive/avro_no_schema_test.q.out PRE-CREATION
ql/src/test/results/clientpositive/avro_without_schema.q.out PRE-CREATION
serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroSerdeUtils.java 9d58d13
serde/src/java/org/apache/hadoop/hive/serde2/avro/TypeInfoToSchema.java PRE-CREATION
serde/src/test/org/apache/hadoop/hive/serde2/avro/TestAvroSerdeUtils.java 67d5570
serde/src/test/org/apache/hadoop/hive/serde2/avro/TestTypeInfoToSchema.java PRE-CREATION
Diff: https://reviews.apache.org/r/11925/diff/
Testing
-------
Wrote a new java Test class for a new Java class. Added a new test case into existing java test class. In addition, there are 4 .q file for testing multiple use-cases.
Thanks,
Mohammad Islam