Question about query result storage

[Tom Brown <to...@gmail.com>]

Team,

I'm a new Hive user and I've just run my first large query (a few
hours). Unfortunately, I ran it from the CLI, and the output was
longer than my SSH client allowed for (scroll buffer) so I can't see
the first 1/2 of the result. (It also changes tabs to spaces so
properly aligning the columns of the result is difficult as well).

When a query is run through the CLI, is the result stored anywhere
(even temporarily)? I would love to not have to run the query again,
but will if I have to.

However, when I run the query again (from the CLI), I'm not sure how
to correctly store the results. Should I just pipe all output of the
CLI to a file and ignore the few hours of status updates? Can anyone
give any suggestions on what the most appropriate way to accomplish
this is?

Thanks in advance.

--Tom

RE: Question about query result storage

[Venkatesh Kavuluri <vk...@outlook.com>]

You can always do something like 
INSERT OVERWRITE LOCAL DIRECTORY '/path/....' SELECT [....] FROM [....]
which saves the result set on to the given path.
Check Hive wiki for more info.https://cwiki.apache.org/confluence/display/Hive/GettingStarted 

> Date: Thu, 9 Aug 2012 17:42:17 -0400
> From: pipehappy@gmail.com
> To: user@hive.apache.org
> Subject: Re: Question about query result storage
> 
> Oh, actually is
> hive -S -f some_query.q > some_query.log
> 
> On 08/09/2012 05:41 PM, Yue Guan wrote:
> > We always do something like this:
> > hive -f some_query.q > some_query.log
> >
> > If the output is large, just insert overwrite to some table.
> >
> > On 08/09/2012 05:32 PM, Tom Brown wrote:
> >> Team,
> >>
> >> I'm a new Hive user and I've just run my first large query (a few
> >> hours). Unfortunately, I ran it from the CLI, and the output was
> >> longer than my SSH client allowed for (scroll buffer) so I can't see
> >> the first 1/2 of the result. (It also changes tabs to spaces so
> >> properly aligning the columns of the result is difficult as well).
> >>
> >> When a query is run through the CLI, is the result stored anywhere
> >> (even temporarily)? I would love to not have to run the query again,
> >> but will if I have to.
> >>
> >> However, when I run the query again (from the CLI), I'm not sure how
> >> to correctly store the results. Should I just pipe all output of the
> >> CLI to a file and ignore the few hours of status updates? Can anyone
> >> give any suggestions on what the most appropriate way to accomplish
> >> this is?
> >>
> >> Thanks in advance.
> >>
> >> --Tom
> >
> 
 		 	   		  

Re: Question about query result storage

[Yue Guan <pi...@gmail.com>]

Oh, actually is
hive -S -f some_query.q > some_query.log

On 08/09/2012 05:41 PM, Yue Guan wrote:
> We always do something like this:
> hive -f some_query.q > some_query.log
>
> If the output is large, just insert overwrite to some table.
>
> On 08/09/2012 05:32 PM, Tom Brown wrote:
>> Team,
>>
>> I'm a new Hive user and I've just run my first large query (a few
>> hours). Unfortunately, I ran it from the CLI, and the output was
>> longer than my SSH client allowed for (scroll buffer) so I can't see
>> the first 1/2 of the result. (It also changes tabs to spaces so
>> properly aligning the columns of the result is difficult as well).
>>
>> When a query is run through the CLI, is the result stored anywhere
>> (even temporarily)? I would love to not have to run the query again,
>> but will if I have to.
>>
>> However, when I run the query again (from the CLI), I'm not sure how
>> to correctly store the results. Should I just pipe all output of the
>> CLI to a file and ignore the few hours of status updates? Can anyone
>> give any suggestions on what the most appropriate way to accomplish
>> this is?
>>
>> Thanks in advance.
>>
>> --Tom
>

Re: Question about query result storage

[Yue Guan <pi...@gmail.com>]

We always do something like this:
hive -f some_query.q > some_query.log

If the output is large, just insert overwrite to some table.

On 08/09/2012 05:32 PM, Tom Brown wrote:
> Team,
>
> I'm a new Hive user and I've just run my first large query (a few
> hours). Unfortunately, I ran it from the CLI, and the output was
> longer than my SSH client allowed for (scroll buffer) so I can't see
> the first 1/2 of the result. (It also changes tabs to spaces so
> properly aligning the columns of the result is difficult as well).
>
> When a query is run through the CLI, is the result stored anywhere
> (even temporarily)? I would love to not have to run the query again,
> but will if I have to.
>
> However, when I run the query again (from the CLI), I'm not sure how
> to correctly store the results. Should I just pipe all output of the
> CLI to a file and ignore the few hours of status updates? Can anyone
> give any suggestions on what the most appropriate way to accomplish
> this is?
>
> Thanks in advance.
>
> --Tom