You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by "Raghunath, Ranjith" <Ra...@usaa.com> on 2011/05/03 05:12:24 UTC
Export data with column names
Does anyone know how to export data out with column names? Any help here is appreciated.
Thanks,
Ranjith
RE: Export data with column names
Posted by Steven Wong <sw...@netflix.com>.
I'm not aware.
-----Original Message-----
From: Raghunath, Ranjith [mailto:Ranjith.Raghunath1@usaa.com]
Sent: Tuesday, May 03, 2011 3:29 PM
To: user@hive.apache.org
Subject: RE: Export data with column names
Thanks for the help on this. The option to set the header worked out great. Is there a way that I can export data out to the local file system with the header information?
Thank you,
Ranjith N. Raghunath
-----Original Message-----
From: Steven Wong [mailto:swong@netflix.com]
Sent: Tuesday, May 03, 2011 3:20 PM
To: user@hive.apache.org
Subject: RE: Export data with column names
Loren, can you point me to the jira that describes the problem/symptom in more detail? If there is no jira yet, can you describe it?
Thanks.
-----Original Message-----
From: Loren Siebert [mailto:loren@siebert.org]
Sent: Tuesday, May 03, 2011 11:23 AM
To: user@hive.apache.org
Cc: user@hive.apache.org
Subject: Re: Export data with column names
Look out for NPEs if you happen to set this option to true before doing add/drop table statements.
On May 3, 2011, at 11:10, Steven Wong <sw...@netflix.com> wrote:
> set hive.cli.print.header=true;
>
>
> -----Original Message-----
> From: Raghunath, Ranjith [mailto:Ranjith.Raghunath1@usaa.com]
> Sent: Tuesday, May 03, 2011 11:03 AM
> To: 'user@hive.apache.org'
> Subject: Re: Export data with column names
>
> Thanks. Is there something I can do with cli?
>
> Thanks,
> Ranjith
>
> ----- Original Message -----
> From: Matthew Rathbone <ma...@foursquare.com>
> To: user@hive.apache.org <us...@hive.apache.org>
> Sent: Tue May 03 09:45:58 2011
> Subject: Re: Export data with column names
>
> If you use the thrift client you can call connection.getSchema()
>
> --
> Matthew Rathbone
> Foursquare | Software Engineer | Server Engineering Team
> matthew@foursquare.com | @rathboma | 4sq
>
> On Monday, May 2, 2011 at 11:12 PM, Raghunath, Ranjith wrote:
> Does anyone know how to export data out with column names? Any help here is appreciated.
>>
>> Thanks,
>> Ranjith
>>
>
>
RE: Export data with column names
Posted by "Raghunath, Ranjith" <Ra...@usaa.com>.
Thanks for the help on this. The option to set the header worked out great. Is there a way that I can export data out to the local file system with the header information?
Thank you,
Ranjith N. Raghunath
-----Original Message-----
From: Steven Wong [mailto:swong@netflix.com]
Sent: Tuesday, May 03, 2011 3:20 PM
To: user@hive.apache.org
Subject: RE: Export data with column names
Loren, can you point me to the jira that describes the problem/symptom in more detail? If there is no jira yet, can you describe it?
Thanks.
-----Original Message-----
From: Loren Siebert [mailto:loren@siebert.org]
Sent: Tuesday, May 03, 2011 11:23 AM
To: user@hive.apache.org
Cc: user@hive.apache.org
Subject: Re: Export data with column names
Look out for NPEs if you happen to set this option to true before doing add/drop table statements.
On May 3, 2011, at 11:10, Steven Wong <sw...@netflix.com> wrote:
> set hive.cli.print.header=true;
>
>
> -----Original Message-----
> From: Raghunath, Ranjith [mailto:Ranjith.Raghunath1@usaa.com]
> Sent: Tuesday, May 03, 2011 11:03 AM
> To: 'user@hive.apache.org'
> Subject: Re: Export data with column names
>
> Thanks. Is there something I can do with cli?
>
> Thanks,
> Ranjith
>
> ----- Original Message -----
> From: Matthew Rathbone <ma...@foursquare.com>
> To: user@hive.apache.org <us...@hive.apache.org>
> Sent: Tue May 03 09:45:58 2011
> Subject: Re: Export data with column names
>
> If you use the thrift client you can call connection.getSchema()
>
> --
> Matthew Rathbone
> Foursquare | Software Engineer | Server Engineering Team
> matthew@foursquare.com | @rathboma | 4sq
>
> On Monday, May 2, 2011 at 11:12 PM, Raghunath, Ranjith wrote:
> Does anyone know how to export data out with column names? Any help here is appreciated.
>>
>> Thanks,
>> Ranjith
>>
>
>
RE: Export data with column names
Posted by Steven Wong <sw...@netflix.com>.
Loren, can you point me to the jira that describes the problem/symptom in more detail? If there is no jira yet, can you describe it?
Thanks.
-----Original Message-----
From: Loren Siebert [mailto:loren@siebert.org]
Sent: Tuesday, May 03, 2011 11:23 AM
To: user@hive.apache.org
Cc: user@hive.apache.org
Subject: Re: Export data with column names
Look out for NPEs if you happen to set this option to true before doing add/drop table statements.
On May 3, 2011, at 11:10, Steven Wong <sw...@netflix.com> wrote:
> set hive.cli.print.header=true;
>
>
> -----Original Message-----
> From: Raghunath, Ranjith [mailto:Ranjith.Raghunath1@usaa.com]
> Sent: Tuesday, May 03, 2011 11:03 AM
> To: 'user@hive.apache.org'
> Subject: Re: Export data with column names
>
> Thanks. Is there something I can do with cli?
>
> Thanks,
> Ranjith
>
> ----- Original Message -----
> From: Matthew Rathbone <ma...@foursquare.com>
> To: user@hive.apache.org <us...@hive.apache.org>
> Sent: Tue May 03 09:45:58 2011
> Subject: Re: Export data with column names
>
> If you use the thrift client you can call connection.getSchema()
>
> --
> Matthew Rathbone
> Foursquare | Software Engineer | Server Engineering Team
> matthew@foursquare.com | @rathboma | 4sq
>
> On Monday, May 2, 2011 at 11:12 PM, Raghunath, Ranjith wrote:
> Does anyone know how to export data out with column names? Any help here is appreciated.
>>
>> Thanks,
>> Ranjith
>>
>
>
RE: Question about Pig UDF and sqoop export
Posted by "Sharma, Akash" <Ak...@searshc.com>.
Finally figured it out.
Flatten did the trick. Had to call flatten on the tuple returned by the
UDF as shown below.
B = FOREACH A GENERATE FLATTEN(PackageName.MyMethod());
Hope someone out there can make use of this behavior.
Thanks
-Akash
-----Original Message-----
From: Sharma, Akash [mailto:Akash.Sharma@searshc.com]
Sent: Tuesday, May 03, 2011 2:52 PM
To: user@hive.apache.org
Subject: Question about Pig UDF and sqoop export
New to Pig UDF so requesting help to resolve the issue
Created a Pig UDF to transform an input tuple and generate an
transformed tuple.
Executed a Store to save the transformed tuple into a file
B = FOREACH A GENERATE PackageName.MyMethod();
STORE B into 'output' USING PigStorage();
My question is that when sqoop export reads this data from output HDFS
file and inserts into MYSQL table, the curly braces "(" and ")" are also
saved into the first and the last column of mysql table. Is there a way
to avoid curly braces for getting into MySQL column as part of first and
last column?
I tried to delimit to tabs explicitly in PigStorage() but that did not
resolve the issue.
Any help is appreciated.
Thanks
-Akash
Question about Pig UDF and sqoop export
Posted by "Sharma, Akash" <Ak...@searshc.com>.
New to Pig UDF so requesting help to resolve the issue
Created a Pig UDF to transform an input tuple and generate an
transformed tuple.
Executed a Store to save the transformed tuple into a file
B = FOREACH A GENERATE PackageName.MyMethod();
STORE B into 'output' USING PigStorage();
My question is that when sqoop export reads this data from output HDFS
file and inserts into MYSQL table, the curly braces "(" and ")" are also
saved into the first and the last column of mysql table. Is there a way
to avoid curly braces for getting into MySQL column as part of first and
last column?
I tried to delimit to tabs explicitly in PigStorage() but that did not
resolve the issue.
Any help is appreciated.
Thanks
-Akash
Re: Export data with column names
Posted by Loren Siebert <lo...@siebert.org>.
Sorry, null pointer exception
On May 3, 2011, at 12:23, "Raghunath, Ranjith" <Ra...@usaa.com> wrote:
> Thanks Loren. Pardon my ignorance but what is an NPE?
>
> Thank you,
> Ranjith N. Raghunath
>
> -----Original Message-----
> From: Loren Siebert [mailto:loren@siebert.org]
> Sent: Tuesday, May 03, 2011 1:23 PM
> To: user@hive.apache.org
> Cc: user@hive.apache.org
> Subject: Re: Export data with column names
>
> Look out for NPEs if you happen to set this option to true before doing add/drop table statements.
>
>
> On May 3, 2011, at 11:10, Steven Wong <sw...@netflix.com> wrote:
>
>> set hive.cli.print.header=true;
>>
>>
>> -----Original Message-----
>> From: Raghunath, Ranjith [mailto:Ranjith.Raghunath1@usaa.com]
>> Sent: Tuesday, May 03, 2011 11:03 AM
>> To: 'user@hive.apache.org'
>> Subject: Re: Export data with column names
>>
>> Thanks. Is there something I can do with cli?
>>
>> Thanks,
>> Ranjith
>>
>> ----- Original Message -----
>> From: Matthew Rathbone <ma...@foursquare.com>
>> To: user@hive.apache.org <us...@hive.apache.org>
>> Sent: Tue May 03 09:45:58 2011
>> Subject: Re: Export data with column names
>>
>> If you use the thrift client you can call connection.getSchema()
>>
>> --
>> Matthew Rathbone
>> Foursquare | Software Engineer | Server Engineering Team
>> matthew@foursquare.com | @rathboma | 4sq
>>
>> On Monday, May 2, 2011 at 11:12 PM, Raghunath, Ranjith wrote:
>> Does anyone know how to export data out with column names? Any help here is appreciated.
>>>
>>> Thanks,
>>> Ranjith
>>>
>>
>>
RE: Export data with column names
Posted by "Raghunath, Ranjith" <Ra...@usaa.com>.
Thanks Loren. Pardon my ignorance but what is an NPE?
Thank you,
Ranjith N. Raghunath
-----Original Message-----
From: Loren Siebert [mailto:loren@siebert.org]
Sent: Tuesday, May 03, 2011 1:23 PM
To: user@hive.apache.org
Cc: user@hive.apache.org
Subject: Re: Export data with column names
Look out for NPEs if you happen to set this option to true before doing add/drop table statements.
On May 3, 2011, at 11:10, Steven Wong <sw...@netflix.com> wrote:
> set hive.cli.print.header=true;
>
>
> -----Original Message-----
> From: Raghunath, Ranjith [mailto:Ranjith.Raghunath1@usaa.com]
> Sent: Tuesday, May 03, 2011 11:03 AM
> To: 'user@hive.apache.org'
> Subject: Re: Export data with column names
>
> Thanks. Is there something I can do with cli?
>
> Thanks,
> Ranjith
>
> ----- Original Message -----
> From: Matthew Rathbone <ma...@foursquare.com>
> To: user@hive.apache.org <us...@hive.apache.org>
> Sent: Tue May 03 09:45:58 2011
> Subject: Re: Export data with column names
>
> If you use the thrift client you can call connection.getSchema()
>
> --
> Matthew Rathbone
> Foursquare | Software Engineer | Server Engineering Team
> matthew@foursquare.com | @rathboma | 4sq
>
> On Monday, May 2, 2011 at 11:12 PM, Raghunath, Ranjith wrote:
> Does anyone know how to export data out with column names? Any help here is appreciated.
>>
>> Thanks,
>> Ranjith
>>
>
>
Re: Export data with column names
Posted by Loren Siebert <lo...@siebert.org>.
Look out for NPEs if you happen to set this option to true before doing add/drop table statements.
On May 3, 2011, at 11:10, Steven Wong <sw...@netflix.com> wrote:
> set hive.cli.print.header=true;
>
>
> -----Original Message-----
> From: Raghunath, Ranjith [mailto:Ranjith.Raghunath1@usaa.com]
> Sent: Tuesday, May 03, 2011 11:03 AM
> To: 'user@hive.apache.org'
> Subject: Re: Export data with column names
>
> Thanks. Is there something I can do with cli?
>
> Thanks,
> Ranjith
>
> ----- Original Message -----
> From: Matthew Rathbone <ma...@foursquare.com>
> To: user@hive.apache.org <us...@hive.apache.org>
> Sent: Tue May 03 09:45:58 2011
> Subject: Re: Export data with column names
>
> If you use the thrift client you can call connection.getSchema()
>
> --
> Matthew Rathbone
> Foursquare | Software Engineer | Server Engineering Team
> matthew@foursquare.com | @rathboma | 4sq
>
> On Monday, May 2, 2011 at 11:12 PM, Raghunath, Ranjith wrote:
> Does anyone know how to export data out with column names? Any help here is appreciated.
>>
>> Thanks,
>> Ranjith
>>
>
>
RE: Export data with column names
Posted by Steven Wong <sw...@netflix.com>.
set hive.cli.print.header=true;
-----Original Message-----
From: Raghunath, Ranjith [mailto:Ranjith.Raghunath1@usaa.com]
Sent: Tuesday, May 03, 2011 11:03 AM
To: 'user@hive.apache.org'
Subject: Re: Export data with column names
Thanks. Is there something I can do with cli?
Thanks,
Ranjith
----- Original Message -----
From: Matthew Rathbone <ma...@foursquare.com>
To: user@hive.apache.org <us...@hive.apache.org>
Sent: Tue May 03 09:45:58 2011
Subject: Re: Export data with column names
If you use the thrift client you can call connection.getSchema()
--
Matthew Rathbone
Foursquare | Software Engineer | Server Engineering Team
matthew@foursquare.com | @rathboma | 4sq
On Monday, May 2, 2011 at 11:12 PM, Raghunath, Ranjith wrote:
Does anyone know how to export data out with column names? Any help here is appreciated.
>
> Thanks,
> Ranjith
>
Re: Export data with column names
Posted by "Raghunath, Ranjith" <Ra...@usaa.com>.
Thanks. Is there something I can do with cli?
Thanks,
Ranjith
----- Original Message -----
From: Matthew Rathbone <ma...@foursquare.com>
To: user@hive.apache.org <us...@hive.apache.org>
Sent: Tue May 03 09:45:58 2011
Subject: Re: Export data with column names
If you use the thrift client you can call connection.getSchema()
--
Matthew Rathbone
Foursquare | Software Engineer | Server Engineering Team
matthew@foursquare.com | @rathboma | 4sq
On Monday, May 2, 2011 at 11:12 PM, Raghunath, Ranjith wrote:
Does anyone know how to export data out with column names? Any help here is appreciated.
>
> Thanks,
> Ranjith
>
Re: Export data with column names
Posted by Matthew Rathbone <ma...@foursquare.com>.
If you use the thrift client you can call connection.getSchema()
--
Matthew Rathbone
Foursquare | Software Engineer | Server Engineering Team
matthew@foursquare.com | @rathboma | 4sq
On Monday, May 2, 2011 at 11:12 PM, Raghunath, Ranjith wrote:
Does anyone know how to export data out with column names? Any help here is appreciated.
>
> Thanks,
> Ranjith
>