You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by Mapred Learn <ma...@gmail.com> on 2011/02/17 21:00:19 UTC
does hive support Sequence File format ?
Hi,
I was wondering if hive supports Sequence File format. If yes, could me
point me to some documentation about how to use Seq files in hive.
Thanks,
-JJ
Re: does hive support Sequence File format ?
Posted by Mapred Learn <ma...@gmail.com>.
Thanks Tim !
On Tue, Jun 28, 2011 at 10:01 AM, Tim Spence <yo...@gmail.com>wrote:
> It should be here:
> https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL
>
> For everyone's benefit, the old wiki page you linked to has a link to a
> page directory on the new wiki here:
> https://cwiki.apache.org/confluence/pages/listpages-dirview.action?key=Hive
>
> Tim
>
>
>
>
>
> On Tue, Jun 28, 2011 at 9:54 AM, Mapred Learn <ma...@gmail.com>wrote:
>
>> Hi,
>> Looks the documentation link you guys provided earlier is moved to some
>> other location:
>> http://wiki.apache.org/hadoop/Hive/LanguageManual/DDL#Create_Table
>>
>>
>> On Thu, Feb 17, 2011 at 2:50 PM, Edward Capriolo <ed...@gmail.com>wrote:
>>
>>> On Thu, Feb 17, 2011 at 5:48 PM, Karthik <ka...@yahoo.com> wrote:
>>> > I have a requirement to support data from the SequenceFile "KEY" (not
>>> the
>>> > "VALUE") to be used by Hive table. How can I do this. From the code,
>>> it
>>> > looks like the "VALUE" part is available for Hive. Please help.
>>> >
>>> > Regards.
>>> >
>>> > ________________________________
>>> > From: Mapred Learn <ma...@gmail.com>
>>> > To: "user@hive.apache.org" <us...@hive.apache.org>
>>> > Cc: "user@hive.apache.org" <us...@hive.apache.org>
>>> > Sent: Thu, February 17, 2011 1:48:07 PM
>>> > Subject: Re: does hive support Sequence File format ?
>>> >
>>> > Thanks Ted !
>>> > Just found it few minutes ago.
>>> > On Feb 17, 2011, at 1:46 PM, Ted Yu <yu...@gmail.com> wrote:
>>> >
>>> > Look under
>>> > http://wiki.apache.org/hadoop/Hive/LanguageManual/DDL#Create_Table
>>> >
>>> > On Thu, Feb 17, 2011 at 12:00 PM, Mapred Learn <mapred.learn@gmail.com
>>> >
>>> > wrote:
>>> >>
>>> >> Hi,
>>> >> I was wondering if hive supports Sequence File format. If yes, could
>>> me
>>> >> point me to some documentation about how to use Seq files in hive.
>>> >>
>>> >> Thanks,
>>> >> -JJ
>>> >
>>>
>>> This has come up two or three times on the ML. It can be done with
>>> InputFormats.
>>>
>>> Edward
>>>
>>
>>
>
Re: does hive support Sequence File format ?
Posted by Tim Spence <yo...@gmail.com>.
It should be here:
https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL
For everyone's benefit, the old wiki page you linked to has a link to a page
directory on the new wiki here:
https://cwiki.apache.org/confluence/pages/listpages-dirview.action?key=Hive
Tim
On Tue, Jun 28, 2011 at 9:54 AM, Mapred Learn <ma...@gmail.com>wrote:
> Hi,
> Looks the documentation link you guys provided earlier is moved to some
> other location:
> http://wiki.apache.org/hadoop/Hive/LanguageManual/DDL#Create_Table
>
>
> On Thu, Feb 17, 2011 at 2:50 PM, Edward Capriolo <ed...@gmail.com>wrote:
>
>> On Thu, Feb 17, 2011 at 5:48 PM, Karthik <ka...@yahoo.com> wrote:
>> > I have a requirement to support data from the SequenceFile "KEY" (not
>> the
>> > "VALUE") to be used by Hive table. How can I do this. From the code,
>> it
>> > looks like the "VALUE" part is available for Hive. Please help.
>> >
>> > Regards.
>> >
>> > ________________________________
>> > From: Mapred Learn <ma...@gmail.com>
>> > To: "user@hive.apache.org" <us...@hive.apache.org>
>> > Cc: "user@hive.apache.org" <us...@hive.apache.org>
>> > Sent: Thu, February 17, 2011 1:48:07 PM
>> > Subject: Re: does hive support Sequence File format ?
>> >
>> > Thanks Ted !
>> > Just found it few minutes ago.
>> > On Feb 17, 2011, at 1:46 PM, Ted Yu <yu...@gmail.com> wrote:
>> >
>> > Look under
>> > http://wiki.apache.org/hadoop/Hive/LanguageManual/DDL#Create_Table
>> >
>> > On Thu, Feb 17, 2011 at 12:00 PM, Mapred Learn <ma...@gmail.com>
>> > wrote:
>> >>
>> >> Hi,
>> >> I was wondering if hive supports Sequence File format. If yes, could me
>> >> point me to some documentation about how to use Seq files in hive.
>> >>
>> >> Thanks,
>> >> -JJ
>> >
>>
>> This has come up two or three times on the ML. It can be done with
>> InputFormats.
>>
>> Edward
>>
>
>
Re: does hive support Sequence File format ?
Posted by Mapred Learn <ma...@gmail.com>.
Hi,
Looks the documentation link you guys provided earlier is moved to some
other location:
http://wiki.apache.org/hadoop/Hive/LanguageManual/DDL#Create_Table
On Thu, Feb 17, 2011 at 2:50 PM, Edward Capriolo <ed...@gmail.com>wrote:
> On Thu, Feb 17, 2011 at 5:48 PM, Karthik <ka...@yahoo.com> wrote:
> > I have a requirement to support data from the SequenceFile "KEY" (not the
> > "VALUE") to be used by Hive table. How can I do this. From the code, it
> > looks like the "VALUE" part is available for Hive. Please help.
> >
> > Regards.
> >
> > ________________________________
> > From: Mapred Learn <ma...@gmail.com>
> > To: "user@hive.apache.org" <us...@hive.apache.org>
> > Cc: "user@hive.apache.org" <us...@hive.apache.org>
> > Sent: Thu, February 17, 2011 1:48:07 PM
> > Subject: Re: does hive support Sequence File format ?
> >
> > Thanks Ted !
> > Just found it few minutes ago.
> > On Feb 17, 2011, at 1:46 PM, Ted Yu <yu...@gmail.com> wrote:
> >
> > Look under
> > http://wiki.apache.org/hadoop/Hive/LanguageManual/DDL#Create_Table
> >
> > On Thu, Feb 17, 2011 at 12:00 PM, Mapred Learn <ma...@gmail.com>
> > wrote:
> >>
> >> Hi,
> >> I was wondering if hive supports Sequence File format. If yes, could me
> >> point me to some documentation about how to use Seq files in hive.
> >>
> >> Thanks,
> >> -JJ
> >
>
> This has come up two or three times on the ML. It can be done with
> InputFormats.
>
> Edward
>
Re: does hive support Sequence File format ?
Posted by Edward Capriolo <ed...@gmail.com>.
On Thu, Feb 17, 2011 at 5:48 PM, Karthik <ka...@yahoo.com> wrote:
> I have a requirement to support data from the SequenceFile "KEY" (not the
> "VALUE") to be used by Hive table. How can I do this. From the code, it
> looks like the "VALUE" part is available for Hive. Please help.
>
> Regards.
>
> ________________________________
> From: Mapred Learn <ma...@gmail.com>
> To: "user@hive.apache.org" <us...@hive.apache.org>
> Cc: "user@hive.apache.org" <us...@hive.apache.org>
> Sent: Thu, February 17, 2011 1:48:07 PM
> Subject: Re: does hive support Sequence File format ?
>
> Thanks Ted !
> Just found it few minutes ago.
> On Feb 17, 2011, at 1:46 PM, Ted Yu <yu...@gmail.com> wrote:
>
> Look under
> http://wiki.apache.org/hadoop/Hive/LanguageManual/DDL#Create_Table
>
> On Thu, Feb 17, 2011 at 12:00 PM, Mapred Learn <ma...@gmail.com>
> wrote:
>>
>> Hi,
>> I was wondering if hive supports Sequence File format. If yes, could me
>> point me to some documentation about how to use Seq files in hive.
>>
>> Thanks,
>> -JJ
>
This has come up two or three times on the ML. It can be done with InputFormats.
Edward
Re: does hive support Sequence File format ?
Posted by Karthik <ka...@yahoo.com>.
I have a requirement to support data from the SequenceFile "KEY" (not the
"VALUE") to be used by Hive table. How can I do this. From the code, it looks
like the "VALUE" part is available for Hive. Please help.
Regards.
________________________________
From: Mapred Learn <ma...@gmail.com>
To: "user@hive.apache.org" <us...@hive.apache.org>
Cc: "user@hive.apache.org" <us...@hive.apache.org>
Sent: Thu, February 17, 2011 1:48:07 PM
Subject: Re: does hive support Sequence File format ?
Thanks Ted !
Just found it few minutes ago.
On Feb 17, 2011, at 1:46 PM, Ted Yu <yu...@gmail.com> wrote:
Look under http://wiki.apache.org/hadoop/Hive/LanguageManual/DDL#Create_Table
>
>
>On Thu, Feb 17, 2011 at 12:00 PM, Mapred Learn <ma...@gmail.com> wrote:
>
>Hi,
>>I was wondering if hive supports Sequence File format. If yes, could me point me
>>to some documentation about how to use Seq files in hive.
>>
>>Thanks,
>>-JJ
>
Re: does hive support Sequence File format ?
Posted by Mapred Learn <ma...@gmail.com>.
Thanks Ted !
Just found it few minutes ago.
On Feb 17, 2011, at 1:46 PM, Ted Yu <yu...@gmail.com> wrote:
> Look under http://wiki.apache.org/hadoop/Hive/LanguageManual/DDL#Create_Table
>
> On Thu, Feb 17, 2011 at 12:00 PM, Mapred Learn <ma...@gmail.com> wrote:
> Hi,
> I was wondering if hive supports Sequence File format. If yes, could me point me to some documentation about how to use Seq files in hive.
>
> Thanks,
> -JJ
>
Re: does hive support Sequence File format ?
Posted by Ted Yu <yu...@gmail.com>.
Look under
http://wiki.apache.org/hadoop/Hive/LanguageManual/DDL#Create_Table
On Thu, Feb 17, 2011 at 12:00 PM, Mapred Learn <ma...@gmail.com>wrote:
> Hi,
> I was wondering if hive supports Sequence File format. If yes, could me
> point me to some documentation about how to use Seq files in hive.
>
> Thanks,
> -JJ
>
RE: Hive Not Returning Column Names, even what not using 'When'??
Posted by "Sirota, Peter" <si...@amazon.com>.
Hi Mark,
Try this link for Hive .5 JDBC driver: http://aws.amazon.com/developertools/Elastic-MapReduce/0196055244487017
We are actually in Seattle office.
Best Regards,
Peter-
From: Sunderlin, Mark [mailto:mark.sunderlin@teamaol.com]
Sent: Friday, February 18, 2011 6:26 AM
To: 'user@hive.apache.org'
Subject: RE: Hive Not Returning Column Names, even what not using 'When'??
Hey Peter, this looks like it ought to work for me but .... The link to the hive 0.5 hive drivers ... seems broken???
http://buyitnw.appspot.com/aws.amazon.com/developertools/Elastic-MapReduce/0196055244487017
seems to be the link from the site you mention below, but it returns a blank page?
The link to the 0.7 drivers: http://aws.amazon.com/developertools/Elastic-MapReduce/1818074809286277
Seems to work just fine.
p.s. Wonder if by chance you work at the Amazon office right down the street from AOL here in N. Va.
---
Mark E. Sunderlin
Solutions Architect |AOL Data Warehouse
P: 703-256-6935 | C: 540-327-6222
AIM: MESunderlin
22000 AOL Way | Dulles, VA | 20166
From: Sirota, Peter [mailto:sirota@amazon.com]
Sent: Thursday, February 17, 2011 7:54 PM
To: user@hive.apache.org
Subject: RE: Hive Not Returning Column Names, even what not using 'When'??
Hi Mark,
You can use JDBC driver provided by Amazon Elastic MapReduce service. When you use that driver with SQL Squirrel it returns column names. Here are the docs on how to get that driver:
http://buyitnw.appspot.com/docs.amazonwebservices.com/ElasticMapReduce/latest/DeveloperGuide/index.html?UsingEMR_Hive.html
Regards,
Peter Sirota
From: Sunderlin, Mark [mailto:mark.sunderlin@teamaol.com]
Sent: Thursday, February 17, 2011 1:00 PM
To: user@hive.apache.org
Subject: Hive Not Returning Column Names, even what not using 'When'??
When using either Hue or using Squirrel and jdbc, I get "_col0, _col1, _col2 ..." as my columns names when querying my hive tables.
I see in https://issues.apache.org/jira/browse/HIVE-1346 that this is a known issue when using the where clause, but I get in circumstances other than using a where clause.
This query, to return counts of rows of a log file that are of a certain log record type value, also returns generic column names:
select
sum(case cast(raw.log_type_id as int) when 1 then 1 else 0 end) type_1,
sum(case cast(raw.log_type_id as int) when 2 then 1 else 0 end) type_2,
sum(case cast(raw.log_type_id as int) when 6 then 1 else 0 end) type_6,
from raw_log raw
Is this a variation on the same issue, or a new issue, or it is something I am doing wrong?
---
Mark E. Sunderlin
Solutions Architect |AOL Data Warehouse
P: 703-256-6935 | C: 540-327-6222
AIM: MESunderlin
22000 AOL Way | Dulles, VA | 20166
RE: Hive Not Returning Column Names, even what not using 'When'??
Posted by "Sunderlin, Mark" <ma...@teamaol.com>.
Hey Peter, this looks like it ought to work for me but .... The link to the hive 0.5 hive drivers ... seems broken???
http://buyitnw.appspot.com/aws.amazon.com/developertools/Elastic-MapReduce/0196055244487017
seems to be the link from the site you mention below, but it returns a blank page?
The link to the 0.7 drivers: http://aws.amazon.com/developertools/Elastic-MapReduce/1818074809286277
Seems to work just fine.
p.s. Wonder if by chance you work at the Amazon office right down the street from AOL here in N. Va.
---
Mark E. Sunderlin
Solutions Architect |AOL Data Warehouse
P: 703-256-6935 | C: 540-327-6222
AIM: MESunderlin
22000 AOL Way | Dulles, VA | 20166
From: Sirota, Peter [mailto:sirota@amazon.com]
Sent: Thursday, February 17, 2011 7:54 PM
To: user@hive.apache.org
Subject: RE: Hive Not Returning Column Names, even what not using 'When'??
Hi Mark,
You can use JDBC driver provided by Amazon Elastic MapReduce service. When you use that driver with SQL Squirrel it returns column names. Here are the docs on how to get that driver:
http://buyitnw.appspot.com/docs.amazonwebservices.com/ElasticMapReduce/latest/DeveloperGuide/index.html?UsingEMR_Hive.html
Regards,
Peter Sirota
From: Sunderlin, Mark [mailto:mark.sunderlin@teamaol.com]
Sent: Thursday, February 17, 2011 1:00 PM
To: user@hive.apache.org
Subject: Hive Not Returning Column Names, even what not using 'When'??
When using either Hue or using Squirrel and jdbc, I get "_col0, _col1, _col2 ..." as my columns names when querying my hive tables.
I see in https://issues.apache.org/jira/browse/HIVE-1346 that this is a known issue when using the where clause, but I get in circumstances other than using a where clause.
This query, to return counts of rows of a log file that are of a certain log record type value, also returns generic column names:
select
sum(case cast(raw.log_type_id as int) when 1 then 1 else 0 end) type_1,
sum(case cast(raw.log_type_id as int) when 2 then 1 else 0 end) type_2,
sum(case cast(raw.log_type_id as int) when 6 then 1 else 0 end) type_6,
from raw_log raw
Is this a variation on the same issue, or a new issue, or it is something I am doing wrong?
---
Mark E. Sunderlin
Solutions Architect |AOL Data Warehouse
P: 703-256-6935 | C: 540-327-6222
AIM: MESunderlin
22000 AOL Way | Dulles, VA | 20166
RE: Hive Not Returning Column Names, even what not using 'When'??
Posted by "Sirota, Peter" <si...@amazon.com>.
Hi Mark,
You can use JDBC driver provided by Amazon Elastic MapReduce service. When you use that driver with SQL Squirrel it returns column names. Here are the docs on how to get that driver:
http://buyitnw.appspot.com/docs.amazonwebservices.com/ElasticMapReduce/latest/DeveloperGuide/index.html?UsingEMR_Hive.html
Regards,
Peter Sirota
From: Sunderlin, Mark [mailto:mark.sunderlin@teamaol.com]
Sent: Thursday, February 17, 2011 1:00 PM
To: user@hive.apache.org
Subject: Hive Not Returning Column Names, even what not using 'When'??
When using either Hue or using Squirrel and jdbc, I get "_col0, _col1, _col2 ..." as my columns names when querying my hive tables.
I see in https://issues.apache.org/jira/browse/HIVE-1346 that this is a known issue when using the where clause, but I get in circumstances other than using a where clause.
This query, to return counts of rows of a log file that are of a certain log record type value, also returns generic column names:
select
sum(case cast(raw.log_type_id as int) when 1 then 1 else 0 end) type_1,
sum(case cast(raw.log_type_id as int) when 2 then 1 else 0 end) type_2,
sum(case cast(raw.log_type_id as int) when 6 then 1 else 0 end) type_6,
from raw_log raw
Is this a variation on the same issue, or a new issue, or it is something I am doing wrong?
---
Mark E. Sunderlin
Solutions Architect |AOL Data Warehouse
P: 703-256-6935 | C: 540-327-6222
AIM: MESunderlin
22000 AOL Way | Dulles, VA | 20166
Hive Not Returning Column Names, even what not using 'When'??
Posted by "Sunderlin, Mark" <ma...@teamaol.com>.
When using either Hue or using Squirrel and jdbc, I get "_col0, _col1, _col2 ..." as my columns names when querying my hive tables.
I see in https://issues.apache.org/jira/browse/HIVE-1346 that this is a known issue when using the where clause, but I get in circumstances other than using a where clause.
This query, to return counts of rows of a log file that are of a certain log record type value, also returns generic column names:
select
sum(case cast(raw.log_type_id as int) when 1 then 1 else 0 end) type_1,
sum(case cast(raw.log_type_id as int) when 2 then 1 else 0 end) type_2,
sum(case cast(raw.log_type_id as int) when 6 then 1 else 0 end) type_6,
from raw_log raw
Is this a variation on the same issue, or a new issue, or it is something I am doing wrong?
---
Mark E. Sunderlin
Solutions Architect |AOL Data Warehouse
P: 703-256-6935 | C: 540-327-6222
AIM: MESunderlin
22000 AOL Way | Dulles, VA | 20166