You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@avro.apache.org by Jegan <je...@gmail.com> on 2016/03/22 03:42:13 UTC
Fwd: Handling empty string in avro file
Hi All,
I am using avro file format to load the data in to hive and source system
is Teradata. In the source i have a both NULL and empty string. But when
it landed in the hive its storing both as NULL only. For example
{
"type" : "record",
"name" : "Personal",
"fields" : [ {
"name" : "Id",
"type" : [ "null", "int" ]
}, {
"name" : "name",
"type" : [ "null", "string" ]
}, {
"name" : "age",
"type" : [ "null", "int" ]
}, {
"name" : "zipcode",
"type" : [ "null", "int" ] --> have tried as "type" :int,
"default" : ""
}]
}
Since integer value is empty in source, by above one comes as null as
defined i tried give "type" : int as well and comes as NULL.
Is there any way where i can get the empty string as it is in source.
Because in my landing hive table should contain both NULL and emptystring
and it has a different meaning for us . Your help would be appreciable.
Thank you
Regards,
Jegan
Re: Handling empty string in avro file
Posted by Ryan Blue <rb...@netflix.com.INVALID>.
Jegan,
It looks like the problem is that your zipcode type can't be a String. An
empty string isn't a valid value for int. You probably want to use null or
string for that field.
rb
On Mon, Mar 21, 2016 at 7:42 PM, Jegan <je...@gmail.com> wrote:
> Hi All,
>
> I am using avro file format to load the data in to hive and source system
> is Teradata. In the source i have a both NULL and empty string. But when
> it landed in the hive its storing both as NULL only. For example
>
> {
> "type" : "record",
> "name" : "Personal",
> "fields" : [ {
> "name" : "Id",
> "type" : [ "null", "int" ]
> }, {
> "name" : "name",
> "type" : [ "null", "string" ]
> }, {
> "name" : "age",
> "type" : [ "null", "int" ]
> }, {
> "name" : "zipcode",
> "type" : [ "null", "int" ] --> have tried as "type" :int,
> "default" : ""
> }]
> }
>
> Since integer value is empty in source, by above one comes as null as
> defined i tried give "type" : int as well and comes as NULL.
>
> Is there any way where i can get the empty string as it is in source.
> Because in my landing hive table should contain both NULL and emptystring
> and it has a different meaning for us . Your help would be appreciable.
>
> Thank you
>
> Regards,
> Jegan
>
--
Ryan Blue
Software Engineer
Netflix