Hive variables and assignment operator evaluations

[Ed Soniat <es...@liveperson.com>]

Hive variables do not seem to be  evaluated by the assigned expression, so
they seem to be macros not variables.

Let us say I want to have a variable called secondsInADay and for clarity
set it thus set secondsInADay=24*60*60;

To the best of my ability to determine secondsInADay will perform the
calculation each time it is referenced, it will not be bound to 86400 but
will be bound to 24*60*60 and could be evaluated billions of times.

This is has ramifications if I am using a date in a variable. Lets say I
start a set of selects using a variable set thus,

set priviousDay=<some code to use current date/time to calculate a previous
date/time>

And the queries referencing this previousDay variable were to execute
across the servers day. The queries would reference different previous days
because the variable is being evaluated in the query not at the assignment.

Is this an accurate assessment and can it be avoided?

Short of creating a table of values of variable evaluations, which wouldn't
be a very pretty thing with which to join, not to mention my hive is pre .8
and only has overwrite so I doubt even that is feasible without of creating
one table per variable (yuk).

Thank you for considering my question.

-- 
This message may contain confidential and/or privileged information. 
If you are not the addressee or authorized to receive this on behalf of the 
addressee you must not use, copy, disclose or take action based on this 
message or any information herein. 
If you have received this message in error, please advise the sender 
immediately by reply email and delete this message. Thank you.

Re: Hive variables and assignment operator evaluations

[Edward Capriolo <ed...@gmail.com>]

hive-variables are interpreted by the CLI before the query parser. If your
using hive 0.8. you should upgrade. You can benefit hive's macro system.


On Tue, Oct 15, 2013 at 5:49 AM, Ed Soniat <es...@liveperson.com> wrote:

> Hive variables do not seem to be  evaluated by the assigned expression, so
> they seem to be macros not variables.
>
> Let us say I want to have a variable called secondsInADay and for clarity
> set it thus set secondsInADay=24*60*60;
>
> To the best of my ability to determine secondsInADay will perform the
> calculation each time it is referenced, it will not be bound to 86400 but
> will be bound to 24*60*60 and could be evaluated billions of times.
>
> This is has ramifications if I am using a date in a variable. Lets say I
> start a set of selects using a variable set thus,
>
> set priviousDay=<some code to use current date/time to calculate a
> previous date/time>
>
> And the queries referencing this previousDay variable were to execute
> across the servers day. The queries would reference different previous days
> because the variable is being evaluated in the query not at the assignment.
>
> Is this an accurate assessment and can it be avoided?
>
> Short of creating a table of values of variable evaluations, which
> wouldn't be a very pretty thing with which to join, not to mention my hive
> is pre .8 and only has overwrite so I doubt even that is feasible without
> of creating one table per variable (yuk).
>
> Thank you for considering my question.
>
> This message may contain confidential and/or privileged information.
> If you are not the addressee or authorized to receive this on behalf of
> the addressee you must not use, copy, disclose or take action based on this
> message or any information herein.
> If you have received this message in error, please advise the sender
> immediately by reply email and delete this message. Thank you.
>