Passing date as command line arguments

[Techy Teck <co...@gmail.com>]

I have my below query in *test1.hql file. *I am trying to pass the date *
(dt)* as the command line argument.


*select * from lip_data_quality where dt = '${hiveconf: start_date}';*


So whenever I try to run the above *test1.hql file like this-*
*
*
* hive -f hivetest1.hql -hiveconf start_date=20120709*
*
*
I get zero records back. But the data is there in that table for that date.
Why is it so? Something wrong I am doing?

Can anyone help me out here? I was following Bejoy's Article here
http://kickstarthadoop.blogspot.com/2011/10/include-values-during-execution-time-in.html

Re: Passing date as command line arguments

[Techy Teck <co...@gmail.com>]

I am on Hive 0.6.

And when I tried executing using like this (*hive -f hivetest1.hql
-hiveconf start_date=20120709*). And after that when I tried viewing the
log file. I found only the below thing that's all. Why? Any idea?

*SessionStart SESSION_ID="rjamal_201208041409" TIME="1344064163296"*


On Sat, Aug 4, 2012 at 12:27 AM, Bejoy KS <be...@yahoo.com> wrote:

> **
> Hi Techy
>
> Which version of hive are you on?
>
> I'm on hive 0.9 and I'm sure I have executed similar scripts in hive 0.7
> as well.
>
> Regards
> Bejoy KS
>
> Sent from handheld, please excuse typos.
> ------------------------------
> *From: * "Bejoy KS" <be...@yahoo.com>
> *Date: *Sat, 4 Aug 2012 07:24:37 +0000
> *To: *<us...@hive.apache.org>
> *ReplyTo: * user@hive.apache.org
> *Subject: *Re: Passing date as command line arguments
>
> I tried the same query on my end, It is working fine for me without any
> issues.
>
> By de way the data type for 'dt' is String itself right?
> Regards
> Bejoy KS
>
> Sent from handheld, please excuse typos.
> ------------------------------
> *From: * Techy Teck <co...@gmail.com>
> *Date: *Sat, 4 Aug 2012 00:21:37 -0700
> *To: *<us...@hive.apache.org>; <be...@yahoo.com>
> *ReplyTo: * user@hive.apache.org
> *Subject: *Re: Passing date as command line arguments
>
> I was not able to understand. So what changes I need to make? This is my
> below *hivetest1.hql-*
> *
> *
> *select * from lip_data_quality where dt = '${hiveconf:start_date}';*
> *
> *
> And by this I am executing the above hql file-
>
> *hive -f hivetest1.hql -hiveconf start_date=20120709*
> *
> *
> *
> *
> Any thoughts what changes I need to make?
>
>
>
>
>
>
>
> On Sat, Aug 4, 2012 at 12:16 AM, Bejoy KS <be...@yahoo.com> wrote:
>
>> **
>> Yes.
>>
>> From the logs the query being executed is
>> select * from lip_data_quality where dt=20120709
>>
>> But here the dt is not in quotes. It should be like
>> select * from lip_data_quality where dt='20120709';
>> Regards
>> Bejoy KS
>>
>> Sent from handheld, please excuse typos.
>> ------------------------------
>> *From: * Techy Teck <co...@gmail.com>
>> *Date: *Sat, 4 Aug 2012 00:07:07 -0700
>> *To: *<us...@hive.apache.org>; Bejoy Ks<be...@yahoo.com>
>> *ReplyTo: * user@hive.apache.org
>> *Subject: *Re: Passing date as command line arguments
>>
>> I found this thing from the Hive log. This is what you are looking for?-
>>
>> *SessionStart SESSION_ID="rjamal_201208041359" TIME="1344063565655"*
>> *QueryStart QUERY_STRING="select * from lip_data_quality where
>> dt=20120709"
>> QUERY_ID="rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486"
>> TIME="1344063597488"*
>> *Counters
>> plan="{"queryId":"rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486","queryType":null,"queryAttributes":{"queryString":"select
>> * from lip_data_quality where dt=20120709"},"queryCo*
>> *unters":"null","stageGraph":{"nodeType":"1","roots":"null","adjacencyList":"]"},"stageList":"]","done":"false","started":"true"}"
>> TIME="1344063597501"*
>> *QueryEnd QUERY_STRING="select * from lip_data_quality where
>> dt=20120709"
>> QUERY_ID="rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486"
>> QUERY_RET_CODE="0" QUERY_NUM_TASKS="0" TIME="134406359*
>> *7503"*
>> *Counters
>> plan="{"queryId":"rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486","queryType":null,"queryAttributes":{"queryString":"select
>> * from lip_data_quality where dt=20120709"},"queryCo*
>> *unters":"null","stageGraph":{"nodeType":"1","roots":"null","adjacencyList":"]"},"stageList":"]","done":"true","started":"true"}"
>> TIME="1344063597504"*
>> *
>> *
>> *
>> *
>>
>>
>>
>> On Fri, Aug 3, 2012 at 11:58 PM, Techy Teck <co...@gmail.com>wrote:
>>
>>> Yes that query gives me the result back correctly.
>>>
>>> How can I see the hive log Bejoy?  Basically I am not sure which hive
>>> log and what is the path where I can see that thing?
>>>
>>>
>>>
>>>
>>>
>>>
>>> On Fri, Aug 3, 2012 at 11:52 PM, Bejoy Ks <be...@yahoo.com> wrote:
>>>
>>>>  Hi Techy
>>>>
>>>> Are you getting the results for the same query directly from CLI, I
>>>> mean does the below query return correct results
>>>> select * from lip_data_quality where dt = '20120709';
>>>>
>>>> If the above query returns results, can you check the hive logs and see
>>>> what is the actual query being executed?
>>>>
>>>> Regards,
>>>> Bejoy KS
>>>>
>>>>   ------------------------------
>>>> *From:* Techy Teck <co...@gmail.com>
>>>> *To:* user@hive.apache.org
>>>> *Cc:* Vijay Dirisala <te...@gmail.com>
>>>> *Sent:* Saturday, August 4, 2012 12:11 PM
>>>> *Subject:* Re: Passing date as command line arguments
>>>>
>>>> Thanks Vijay for the suggestion. I also tried that and it still didn't
>>>> worked for me.
>>>>
>>>> Any thoughts why it is not working?
>>>>
>>>>
>>>>
>>>>
>>>> On Fri, Aug 3, 2012 at 11:32 PM, Vijay <te...@gmail.com> wrote:
>>>>
>>>> There can't be a space in the variable name. Try this:
>>>> ${hiveconf:start_date}
>>>>
>>>> On Fri, Aug 3, 2012 at 11:06 PM, Techy Teck <co...@gmail.com>
>>>> wrote:
>>>> > Can anyone help me out on this?
>>>> >
>>>> > I am stuck on this and I have no clue what wrong I am doing?
>>>> >
>>>> >
>>>> >
>>>> >
>>>> > On Fri, Aug 3, 2012 at 3:48 PM, Techy Teck <co...@gmail.com>
>>>> wrote:
>>>> >>
>>>> >> I have my below query in test1.hql file. I am trying to pass the
>>>> date (dt)
>>>> >> as the command line argument.
>>>> >>
>>>> >>
>>>> >> select * from lip_data_quality where dt = '${hiveconf: start_date}';
>>>> >>
>>>> >>
>>>> >> So whenever I try to run the above test1.hql file like this-
>>>> >>
>>>> >>  hive -f hivetest1.hql -hiveconf start_date=20120709
>>>> >>
>>>> >> I get zero records back. But the data is there in that table for that
>>>> >> date. Why is it so? Something wrong I am doing?
>>>> >>
>>>> >> Can anyone help me out here? I was following Bejoy's Article here
>>>> >>
>>>> http://kickstarthadoop.blogspot.com/2011/10/include-values-during-execution-time-in.html
>>>> >>
>>>> >>
>>>> >
>>>>
>>>>
>>>>
>>>>
>>>>
>>>
>>
>

Re: Passing date as command line arguments

[Bejoy KS <be...@yahoo.com>]

Hi Techy

Which version of hive are you on?

I'm on hive 0.9 and I'm sure I have executed similar scripts in hive 0.7 as well.

Regards
Bejoy KS

Sent from handheld, please excuse typos.

-----Original Message-----
From: "Bejoy KS" <be...@yahoo.com>
Date: Sat, 4 Aug 2012 07:24:37 
To: <us...@hive.apache.org>
Reply-To: user@hive.apache.org
Subject: Re: Passing date as command line arguments

I tried the same query on my end, It is working fine for me without any issues.

By de way the data type for 'dt' is String itself right? 

Regards
Bejoy KS

Sent from handheld, please excuse typos.

-----Original Message-----
From: Techy Teck <co...@gmail.com>
Date: Sat, 4 Aug 2012 00:21:37 
To: <us...@hive.apache.org>; <be...@yahoo.com>
Reply-To: user@hive.apache.org
Subject: Re: Passing date as command line arguments

I was not able to understand. So what changes I need to make? This is my
below *hivetest1.hql-*
*
*
*select * from lip_data_quality where dt = '${hiveconf:start_date}';*
*
*
And by this I am executing the above hql file-

*hive -f hivetest1.hql -hiveconf start_date=20120709*
*
*
*
*
Any thoughts what changes I need to make?







On Sat, Aug 4, 2012 at 12:16 AM, Bejoy KS <be...@yahoo.com> wrote:

> **
> Yes.
>
> From the logs the query being executed is
> select * from lip_data_quality where dt=20120709
>
> But here the dt is not in quotes. It should be like
> select * from lip_data_quality where dt='20120709';
> Regards
> Bejoy KS
>
> Sent from handheld, please excuse typos.
> ------------------------------
> *From: * Techy Teck <co...@gmail.com>
> *Date: *Sat, 4 Aug 2012 00:07:07 -0700
> *To: *<us...@hive.apache.org>; Bejoy Ks<be...@yahoo.com>
> *ReplyTo: * user@hive.apache.org
> *Subject: *Re: Passing date as command line arguments
>
> I found this thing from the Hive log. This is what you are looking for?-
>
> *SessionStart SESSION_ID="rjamal_201208041359" TIME="1344063565655"*
> *QueryStart QUERY_STRING="select * from lip_data_quality where
> dt=20120709"
> QUERY_ID="rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486"
> TIME="1344063597488"*
> *Counters
> plan="{"queryId":"rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486","queryType":null,"queryAttributes":{"queryString":"select
> * from lip_data_quality where dt=20120709"},"queryCo*
> *unters":"null","stageGraph":{"nodeType":"1","roots":"null","adjacencyList":"]"},"stageList":"]","done":"false","started":"true"}"
> TIME="1344063597501"*
> *QueryEnd QUERY_STRING="select * from lip_data_quality where dt=20120709"
> QUERY_ID="rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486"
> QUERY_RET_CODE="0" QUERY_NUM_TASKS="0" TIME="134406359*
> *7503"*
> *Counters
> plan="{"queryId":"rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486","queryType":null,"queryAttributes":{"queryString":"select
> * from lip_data_quality where dt=20120709"},"queryCo*
> *unters":"null","stageGraph":{"nodeType":"1","roots":"null","adjacencyList":"]"},"stageList":"]","done":"true","started":"true"}"
> TIME="1344063597504"*
> *
> *
> *
> *
>
>
>
> On Fri, Aug 3, 2012 at 11:58 PM, Techy Teck <co...@gmail.com>wrote:
>
>> Yes that query gives me the result back correctly.
>>
>> How can I see the hive log Bejoy?  Basically I am not sure which hive log
>> and what is the path where I can see that thing?
>>
>>
>>
>>
>>
>>
>> On Fri, Aug 3, 2012 at 11:52 PM, Bejoy Ks <be...@yahoo.com> wrote:
>>
>>>  Hi Techy
>>>
>>> Are you getting the results for the same query directly from CLI, I mean
>>> does the below query return correct results
>>> select * from lip_data_quality where dt = '20120709';
>>>
>>> If the above query returns results, can you check the hive logs and see
>>> what is the actual query being executed?
>>>
>>> Regards,
>>> Bejoy KS
>>>
>>>   ------------------------------
>>> *From:* Techy Teck <co...@gmail.com>
>>> *To:* user@hive.apache.org
>>> *Cc:* Vijay Dirisala <te...@gmail.com>
>>> *Sent:* Saturday, August 4, 2012 12:11 PM
>>> *Subject:* Re: Passing date as command line arguments
>>>
>>> Thanks Vijay for the suggestion. I also tried that and it still didn't
>>> worked for me.
>>>
>>> Any thoughts why it is not working?
>>>
>>>
>>>
>>>
>>> On Fri, Aug 3, 2012 at 11:32 PM, Vijay <te...@gmail.com> wrote:
>>>
>>> There can't be a space in the variable name. Try this:
>>> ${hiveconf:start_date}
>>>
>>> On Fri, Aug 3, 2012 at 11:06 PM, Techy Teck <co...@gmail.com>
>>> wrote:
>>> > Can anyone help me out on this?
>>> >
>>> > I am stuck on this and I have no clue what wrong I am doing?
>>> >
>>> >
>>> >
>>> >
>>> > On Fri, Aug 3, 2012 at 3:48 PM, Techy Teck <co...@gmail.com>
>>> wrote:
>>> >>
>>> >> I have my below query in test1.hql file. I am trying to pass the date
>>> (dt)
>>> >> as the command line argument.
>>> >>
>>> >>
>>> >> select * from lip_data_quality where dt = '${hiveconf: start_date}';
>>> >>
>>> >>
>>> >> So whenever I try to run the above test1.hql file like this-
>>> >>
>>> >>  hive -f hivetest1.hql -hiveconf start_date=20120709
>>> >>
>>> >> I get zero records back. But the data is there in that table for that
>>> >> date. Why is it so? Something wrong I am doing?
>>> >>
>>> >> Can anyone help me out here? I was following Bejoy's Article here
>>> >>
>>> http://kickstarthadoop.blogspot.com/2011/10/include-values-during-execution-time-in.html
>>> >>
>>> >>
>>> >
>>>
>>>
>>>
>>>
>>>
>>
>

Re: Passing date as command line arguments

[Techy Teck <co...@gmail.com>]

Yup, I will try that out. I send out the latest email. Can you check why is
it happening like this?




On Sat, Aug 4, 2012 at 12:35 AM, Bejoy KS <be...@yahoo.com> wrote:

> **
> You can try it out on a local installation and test it against the later
> versions.
>
> As I pointed out, I just tested on hive 0.9 and it was working good for me.
>
> I guess you should recommend an upgrade of hive in your cluster as well.
> Hive has gone far too ahead after 0.6 . :)
> Regards
> Bejoy KS
>
> Sent from handheld, please excuse typos.
> ------------------------------
> *From: * Techy Teck <co...@gmail.com>
> *Date: *Sat, 4 Aug 2012 00:30:41 -0700
> *To: *<us...@hive.apache.org>; <be...@yahoo.com>
> *ReplyTo: * user@hive.apache.org
> *Subject: *Re: Passing date as command line arguments
>
> I cannot try that on Higher version of Hive as I am working in a company
> where they have installed only Hive 0.6 on the Ares cluster.
>
> :(
>
>
>
>
> On Sat, Aug 4, 2012 at 12:29 AM, Bejoy KS <be...@yahoo.com> wrote:
>
>> **
>>
>> Try it on a higher version of hive and let me know if that doesn't work
>> still. 0.9 should be good.
>>
>>
>> Regards
>> Bejoy KS
>>
>> Sent from handheld, please excuse typos.
>> ------------------------------
>> *From: * Techy Teck <co...@gmail.com>
>> *Date: *Sat, 4 Aug 2012 00:26:44 -0700
>> *To: *<us...@hive.apache.org>; <be...@yahoo.com>
>> *ReplyTo: * user@hive.apache.org
>> *Subject: *Re: Passing date as command line arguments
>>
>> Yes it is String for me here.
>>
>> This is the table structure by the way-
>>
>> *create table lip_data_quality*
>> *( buyer_id bigint,*
>> *  total_chkout bigint,*
>> *  total_errpds bigint*
>> * )*
>> * partitioned by (dt string)*
>> *row format delimited fields terminated by '\t'*
>> *stored as sequencefile*
>> *location '/apps/hdmi-technology/b_apdpds/lip-data-quality'*
>> *;*
>>
>> in which I am using dt in the where clause in my query. I am running *Hive
>> 0.6.*
>>
>>
>>
>> On Sat, Aug 4, 2012 at 12:24 AM, Bejoy KS <be...@yahoo.com> wrote:
>>
>>> **
>>> I tried the same query on my end, It is working fine for me without any
>>> issues.
>>>
>>> By de way the data type for 'dt' is String itself right?
>>> Regards
>>> Bejoy KS
>>>
>>> Sent from handheld, please excuse typos.
>>> ------------------------------
>>> *From: * Techy Teck <co...@gmail.com>
>>> *Date: *Sat, 4 Aug 2012 00:21:37 -0700
>>> *To: *<us...@hive.apache.org>; <be...@yahoo.com>
>>> *ReplyTo: * user@hive.apache.org
>>> *Subject: *Re: Passing date as command line arguments
>>>
>>> I was not able to understand. So what changes I need to make? This is my
>>> below *hivetest1.hql-*
>>> *
>>> *
>>> *select * from lip_data_quality where dt = '${hiveconf:start_date}';*
>>> *
>>> *
>>> And by this I am executing the above hql file-
>>>
>>> *hive -f hivetest1.hql -hiveconf start_date=20120709*
>>> *
>>> *
>>> *
>>> *
>>> Any thoughts what changes I need to make?
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> On Sat, Aug 4, 2012 at 12:16 AM, Bejoy KS <be...@yahoo.com> wrote:
>>>
>>>> **
>>>> Yes.
>>>>
>>>> From the logs the query being executed is
>>>> select * from lip_data_quality where dt=20120709
>>>>
>>>> But here the dt is not in quotes. It should be like
>>>> select * from lip_data_quality where dt='20120709';
>>>> Regards
>>>> Bejoy KS
>>>>
>>>> Sent from handheld, please excuse typos.
>>>> ------------------------------
>>>> *From: * Techy Teck <co...@gmail.com>
>>>> *Date: *Sat, 4 Aug 2012 00:07:07 -0700
>>>> *To: *<us...@hive.apache.org>; Bejoy Ks<be...@yahoo.com>
>>>> *ReplyTo: * user@hive.apache.org
>>>> *Subject: *Re: Passing date as command line arguments
>>>>
>>>> I found this thing from the Hive log. This is what you are looking for?-
>>>>
>>>> *SessionStart SESSION_ID="rjamal_201208041359" TIME="1344063565655"*
>>>> *QueryStart QUERY_STRING="select * from lip_data_quality where
>>>> dt=20120709"
>>>> QUERY_ID="rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486"
>>>> TIME="1344063597488"*
>>>> *Counters
>>>> plan="{"queryId":"rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486","queryType":null,"queryAttributes":{"queryString":"select
>>>> * from lip_data_quality where dt=20120709"},"queryCo*
>>>> *unters":"null","stageGraph":{"nodeType":"1","roots":"null","adjacencyList":"]"},"stageList":"]","done":"false","started":"true"}"
>>>> TIME="1344063597501"*
>>>> *QueryEnd QUERY_STRING="select * from lip_data_quality where
>>>> dt=20120709"
>>>> QUERY_ID="rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486"
>>>> QUERY_RET_CODE="0" QUERY_NUM_TASKS="0" TIME="134406359*
>>>> *7503"*
>>>> *Counters
>>>> plan="{"queryId":"rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486","queryType":null,"queryAttributes":{"queryString":"select
>>>> * from lip_data_quality where dt=20120709"},"queryCo*
>>>> *unters":"null","stageGraph":{"nodeType":"1","roots":"null","adjacencyList":"]"},"stageList":"]","done":"true","started":"true"}"
>>>> TIME="1344063597504"*
>>>> *
>>>> *
>>>> *
>>>> *
>>>>
>>>>
>>>>
>>>> On Fri, Aug 3, 2012 at 11:58 PM, Techy Teck <co...@gmail.com>wrote:
>>>>
>>>>> Yes that query gives me the result back correctly.
>>>>>
>>>>> How can I see the hive log Bejoy?  Basically I am not sure which hive
>>>>> log and what is the path where I can see that thing?
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Fri, Aug 3, 2012 at 11:52 PM, Bejoy Ks <be...@yahoo.com> wrote:
>>>>>
>>>>>>  Hi Techy
>>>>>>
>>>>>> Are you getting the results for the same query directly from CLI, I
>>>>>> mean does the below query return correct results
>>>>>> select * from lip_data_quality where dt = '20120709';
>>>>>>
>>>>>> If the above query returns results, can you check the hive logs and
>>>>>> see what is the actual query being executed?
>>>>>>
>>>>>> Regards,
>>>>>> Bejoy KS
>>>>>>
>>>>>>   ------------------------------
>>>>>> *From:* Techy Teck <co...@gmail.com>
>>>>>> *To:* user@hive.apache.org
>>>>>> *Cc:* Vijay Dirisala <te...@gmail.com>
>>>>>> *Sent:* Saturday, August 4, 2012 12:11 PM
>>>>>> *Subject:* Re: Passing date as command line arguments
>>>>>>
>>>>>> Thanks Vijay for the suggestion. I also tried that and it still
>>>>>> didn't worked for me.
>>>>>>
>>>>>> Any thoughts why it is not working?
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Fri, Aug 3, 2012 at 11:32 PM, Vijay <te...@gmail.com> wrote:
>>>>>>
>>>>>> There can't be a space in the variable name. Try this:
>>>>>> ${hiveconf:start_date}
>>>>>>
>>>>>> On Fri, Aug 3, 2012 at 11:06 PM, Techy Teck <co...@gmail.com>
>>>>>> wrote:
>>>>>> > Can anyone help me out on this?
>>>>>> >
>>>>>> > I am stuck on this and I have no clue what wrong I am doing?
>>>>>> >
>>>>>> >
>>>>>> >
>>>>>> >
>>>>>> > On Fri, Aug 3, 2012 at 3:48 PM, Techy Teck <co...@gmail.com>
>>>>>> wrote:
>>>>>> >>
>>>>>> >> I have my below query in test1.hql file. I am trying to pass the
>>>>>> date (dt)
>>>>>> >> as the command line argument.
>>>>>> >>
>>>>>> >>
>>>>>> >> select * from lip_data_quality where dt = '${hiveconf:
>>>>>> start_date}';
>>>>>> >>
>>>>>> >>
>>>>>> >> So whenever I try to run the above test1.hql file like this-
>>>>>> >>
>>>>>> >>  hive -f hivetest1.hql -hiveconf start_date=20120709
>>>>>> >>
>>>>>> >> I get zero records back. But the data is there in that table for
>>>>>> that
>>>>>> >> date. Why is it so? Something wrong I am doing?
>>>>>> >>
>>>>>> >> Can anyone help me out here? I was following Bejoy's Article here
>>>>>> >>
>>>>>> http://kickstarthadoop.blogspot.com/2011/10/include-values-during-execution-time-in.html
>>>>>> >>
>>>>>> >>
>>>>>> >
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>

Re: Passing date as command line arguments

[Bejoy KS <be...@yahoo.com>]

You can try it out on a local installation and test it against the later versions.

As I pointed out, I just tested on hive 0.9 and it was working good for me.

I guess you should recommend an upgrade of hive in your cluster as well. Hive has gone far too ahead after 0.6 . :)
Regards
Bejoy KS

Sent from handheld, please excuse typos.

-----Original Message-----
From: Techy Teck <co...@gmail.com>
Date: Sat, 4 Aug 2012 00:30:41 
To: <us...@hive.apache.org>; <be...@yahoo.com>
Reply-To: user@hive.apache.org
Subject: Re: Passing date as command line arguments

I cannot try that on Higher version of Hive as I am working in a company
where they have installed only Hive 0.6 on the Ares cluster.

:(




On Sat, Aug 4, 2012 at 12:29 AM, Bejoy KS <be...@yahoo.com> wrote:

> **
>
> Try it on a higher version of hive and let me know if that doesn't work
> still. 0.9 should be good.
>
>
> Regards
> Bejoy KS
>
> Sent from handheld, please excuse typos.
> ------------------------------
> *From: * Techy Teck <co...@gmail.com>
> *Date: *Sat, 4 Aug 2012 00:26:44 -0700
> *To: *<us...@hive.apache.org>; <be...@yahoo.com>
> *ReplyTo: * user@hive.apache.org
> *Subject: *Re: Passing date as command line arguments
>
> Yes it is String for me here.
>
> This is the table structure by the way-
>
> *create table lip_data_quality*
> *( buyer_id bigint,*
> *  total_chkout bigint,*
> *  total_errpds bigint*
> * )*
> * partitioned by (dt string)*
> *row format delimited fields terminated by '\t'*
> *stored as sequencefile*
> *location '/apps/hdmi-technology/b_apdpds/lip-data-quality'*
> *;*
>
> in which I am using dt in the where clause in my query. I am running *Hive
> 0.6.*
>
>
>
> On Sat, Aug 4, 2012 at 12:24 AM, Bejoy KS <be...@yahoo.com> wrote:
>
>> **
>> I tried the same query on my end, It is working fine for me without any
>> issues.
>>
>> By de way the data type for 'dt' is String itself right?
>> Regards
>> Bejoy KS
>>
>> Sent from handheld, please excuse typos.
>> ------------------------------
>> *From: * Techy Teck <co...@gmail.com>
>> *Date: *Sat, 4 Aug 2012 00:21:37 -0700
>> *To: *<us...@hive.apache.org>; <be...@yahoo.com>
>> *ReplyTo: * user@hive.apache.org
>> *Subject: *Re: Passing date as command line arguments
>>
>> I was not able to understand. So what changes I need to make? This is my
>> below *hivetest1.hql-*
>> *
>> *
>> *select * from lip_data_quality where dt = '${hiveconf:start_date}';*
>> *
>> *
>> And by this I am executing the above hql file-
>>
>> *hive -f hivetest1.hql -hiveconf start_date=20120709*
>> *
>> *
>> *
>> *
>> Any thoughts what changes I need to make?
>>
>>
>>
>>
>>
>>
>>
>> On Sat, Aug 4, 2012 at 12:16 AM, Bejoy KS <be...@yahoo.com> wrote:
>>
>>> **
>>> Yes.
>>>
>>> From the logs the query being executed is
>>> select * from lip_data_quality where dt=20120709
>>>
>>> But here the dt is not in quotes. It should be like
>>> select * from lip_data_quality where dt='20120709';
>>> Regards
>>> Bejoy KS
>>>
>>> Sent from handheld, please excuse typos.
>>> ------------------------------
>>> *From: * Techy Teck <co...@gmail.com>
>>> *Date: *Sat, 4 Aug 2012 00:07:07 -0700
>>> *To: *<us...@hive.apache.org>; Bejoy Ks<be...@yahoo.com>
>>> *ReplyTo: * user@hive.apache.org
>>> *Subject: *Re: Passing date as command line arguments
>>>
>>> I found this thing from the Hive log. This is what you are looking for?-
>>>
>>> *SessionStart SESSION_ID="rjamal_201208041359" TIME="1344063565655"*
>>> *QueryStart QUERY_STRING="select * from lip_data_quality where
>>> dt=20120709"
>>> QUERY_ID="rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486"
>>> TIME="1344063597488"*
>>> *Counters
>>> plan="{"queryId":"rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486","queryType":null,"queryAttributes":{"queryString":"select
>>> * from lip_data_quality where dt=20120709"},"queryCo*
>>> *unters":"null","stageGraph":{"nodeType":"1","roots":"null","adjacencyList":"]"},"stageList":"]","done":"false","started":"true"}"
>>> TIME="1344063597501"*
>>> *QueryEnd QUERY_STRING="select * from lip_data_quality where
>>> dt=20120709"
>>> QUERY_ID="rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486"
>>> QUERY_RET_CODE="0" QUERY_NUM_TASKS="0" TIME="134406359*
>>> *7503"*
>>> *Counters
>>> plan="{"queryId":"rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486","queryType":null,"queryAttributes":{"queryString":"select
>>> * from lip_data_quality where dt=20120709"},"queryCo*
>>> *unters":"null","stageGraph":{"nodeType":"1","roots":"null","adjacencyList":"]"},"stageList":"]","done":"true","started":"true"}"
>>> TIME="1344063597504"*
>>> *
>>> *
>>> *
>>> *
>>>
>>>
>>>
>>> On Fri, Aug 3, 2012 at 11:58 PM, Techy Teck <co...@gmail.com>wrote:
>>>
>>>> Yes that query gives me the result back correctly.
>>>>
>>>> How can I see the hive log Bejoy?  Basically I am not sure which hive
>>>> log and what is the path where I can see that thing?
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Fri, Aug 3, 2012 at 11:52 PM, Bejoy Ks <be...@yahoo.com> wrote:
>>>>
>>>>>  Hi Techy
>>>>>
>>>>> Are you getting the results for the same query directly from CLI, I
>>>>> mean does the below query return correct results
>>>>> select * from lip_data_quality where dt = '20120709';
>>>>>
>>>>> If the above query returns results, can you check the hive logs and
>>>>> see what is the actual query being executed?
>>>>>
>>>>> Regards,
>>>>> Bejoy KS
>>>>>
>>>>>   ------------------------------
>>>>> *From:* Techy Teck <co...@gmail.com>
>>>>> *To:* user@hive.apache.org
>>>>> *Cc:* Vijay Dirisala <te...@gmail.com>
>>>>> *Sent:* Saturday, August 4, 2012 12:11 PM
>>>>> *Subject:* Re: Passing date as command line arguments
>>>>>
>>>>> Thanks Vijay for the suggestion. I also tried that and it still didn't
>>>>> worked for me.
>>>>>
>>>>> Any thoughts why it is not working?
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Fri, Aug 3, 2012 at 11:32 PM, Vijay <te...@gmail.com> wrote:
>>>>>
>>>>> There can't be a space in the variable name. Try this:
>>>>> ${hiveconf:start_date}
>>>>>
>>>>> On Fri, Aug 3, 2012 at 11:06 PM, Techy Teck <co...@gmail.com>
>>>>> wrote:
>>>>> > Can anyone help me out on this?
>>>>> >
>>>>> > I am stuck on this and I have no clue what wrong I am doing?
>>>>> >
>>>>> >
>>>>> >
>>>>> >
>>>>> > On Fri, Aug 3, 2012 at 3:48 PM, Techy Teck <co...@gmail.com>
>>>>> wrote:
>>>>> >>
>>>>> >> I have my below query in test1.hql file. I am trying to pass the
>>>>> date (dt)
>>>>> >> as the command line argument.
>>>>> >>
>>>>> >>
>>>>> >> select * from lip_data_quality where dt = '${hiveconf: start_date}';
>>>>> >>
>>>>> >>
>>>>> >> So whenever I try to run the above test1.hql file like this-
>>>>> >>
>>>>> >>  hive -f hivetest1.hql -hiveconf start_date=20120709
>>>>> >>
>>>>> >> I get zero records back. But the data is there in that table for
>>>>> that
>>>>> >> date. Why is it so? Something wrong I am doing?
>>>>> >>
>>>>> >> Can anyone help me out here? I was following Bejoy's Article here
>>>>> >>
>>>>> http://kickstarthadoop.blogspot.com/2011/10/include-values-during-execution-time-in.html
>>>>> >>
>>>>> >>
>>>>> >
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>

Re: Passing date as command line arguments

[Edward Capriolo <ed...@gmail.com>]

Variable support was not added until hive 0.7.0


On Sat, Aug 4, 2012 at 3:58 AM, Lefty Leverenz <le...@hortonworks.com> wrote:
> Does Hive 0.6 support underscores in variable names?
>
> Just a thought.  The examples I've seen contain letters and dots (".") but
> no other characters.
>
> – Lefty
>
>
>
>
> On Sat, Aug 4, 2012 at 12:39 AM, Bejoy KS <be...@yahoo.com> wrote:
>>
>> Yes that is the right issue. Variable substitution is not happening. I
>> can't say much here as I haven't tried out this on 0.6 . The code on my blog
>> post is based on 0.7 or higher version I guess.
>> Regards
>> Bejoy KS
>>
>> Sent from handheld, please excuse typos.
>> ________________________________
>> From: Techy Teck <co...@gmail.com>
>> Date: Sat, 4 Aug 2012 00:35:20 -0700
>> To: <us...@hive.apache.org>; <be...@yahoo.com>
>> ReplyTo: user@hive.apache.org
>> Subject: Re: Passing date as command line arguments
>>
>> Hi Bejoy,
>>
>> I found something very strange. When I actually viewed the Hive log file,
>> I found something like this- See the thing in red color. Why is it like so?
>>
>> SessionStart SESSION_ID="rjamal_201208041433" TIME="1344065620956"
>> QueryStart QUERY_STRING=" select * from lip_data_quality where dt =
>> '${hiveconf:start_date}'"
>> QUERY_ID="rjamal_20120804143333_3719afde-117d-4c6b-a322-0cff6e4b4c5d"
>> TIME="1344065629740"
>> Counters
>> plan="{"queryId":"rjamal_20120804143333_3719afde-117d-4c6b-a322-0cff6e4b4c5d","queryType":null,"queryAttributes":{"queryString":"
>> select * from lip_data_quality where dt =
>> '${hiveconf:start_date}'"},"queryCounters":"null","stageGraph":{"nodeType":"1","roots":"null","adjacencyList":"]"},"stageList":"]","done":"false","started":"true"}"
>> TIME="1344065629743"
>> QueryEnd QUERY_STRING=" select * from lip_data_quality where dt =
>> '${hiveconf:start_date}'"
>> QUERY_ID="rjamal_20120804143333_3719afde-117d-4c6b-a322-0cff6e4b4c5d"
>> QUERY_RET_CODE="0" QUERY_NUM_TASKS="0" TIME="1344065629745"
>> Counters
>> plan="{"queryId":"rjamal_20120804143333_3719afde-117d-4c6b-a322-0cff6e4b4c5d","queryType":null,"queryAttributes":{"queryString":"
>> select * from lip_data_quality where dt =
>> '${hiveconf:start_date}'"},"queryCounters":"null","stageGraph":{"nodeType":"1","roots":"null","adjacencyList":"]"},"stageList":"]","done":"true","started":"true"}"
>> TIME="1344065629745"
>>
>> On Sat, Aug 4, 2012 at 12:30 AM, Techy Teck <co...@gmail.com>
>> wrote:
>>>
>>> I cannot try that on Higher version of Hive as I am working in a company
>>> where they have installed only Hive 0.6 on the Ares cluster.
>>>
>>> :(
>>>
>>>
>>>
>>>
>>> On Sat, Aug 4, 2012 at 12:29 AM, Bejoy KS <be...@yahoo.com> wrote:
>>>>
>>>>
>>>> Try it on a higher version of hive and let me know if that doesn't work
>>>> still. 0.9 should be good.
>>>>
>>>>
>>>> Regards
>>>> Bejoy KS
>>>>
>>>> Sent from handheld, please excuse typos.
>>>> ________________________________
>>>> From: Techy Teck <co...@gmail.com>
>>>> Date: Sat, 4 Aug 2012 00:26:44 -0700
>>>> To: <us...@hive.apache.org>; <be...@yahoo.com>
>>>> ReplyTo: user@hive.apache.org
>>>> Subject: Re: Passing date as command line arguments
>>>>
>>>> Yes it is String for me here.
>>>>
>>>> This is the table structure by the way-
>>>>
>>>> create table lip_data_quality
>>>> ( buyer_id bigint,
>>>>   total_chkout bigint,
>>>>   total_errpds bigint
>>>>  )
>>>>  partitioned by (dt string)
>>>> row format delimited fields terminated by '\t'
>>>> stored as sequencefile
>>>> location '/apps/hdmi-technology/b_apdpds/lip-data-quality'
>>>> ;
>>>>
>>>> in which I am using dt in the where clause in my query. I am running
>>>> Hive 0.6.
>>>>
>>>>
>>>>
>>>> On Sat, Aug 4, 2012 at 12:24 AM, Bejoy KS <be...@yahoo.com> wrote:
>>>>>
>>>>> I tried the same query on my end, It is working fine for me without any
>>>>> issues.
>>>>>
>>>>> By de way the data type for 'dt' is String itself right?
>>>>> Regards
>>>>> Bejoy KS
>>>>>
>>>>> Sent from handheld, please excuse typos.
>>>>> ________________________________
>>>>> From: Techy Teck <co...@gmail.com>
>>>>> Date: Sat, 4 Aug 2012 00:21:37 -0700
>>>>> To: <us...@hive.apache.org>; <be...@yahoo.com>
>>>>> ReplyTo: user@hive.apache.org
>>>>> Subject: Re: Passing date as command line arguments
>>>>>
>>>>> I was not able to understand. So what changes I need to make? This is
>>>>> my below hivetest1.hql-
>>>>>
>>>>> select * from lip_data_quality where dt = '${hiveconf:start_date}';
>>>>>
>>>>> And by this I am executing the above hql file-
>>>>>
>>>>> hive -f hivetest1.hql -hiveconf start_date=20120709
>>>>>
>>>>>
>>>>> Any thoughts what changes I need to make?
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Sat, Aug 4, 2012 at 12:16 AM, Bejoy KS <be...@yahoo.com> wrote:
>>>>>>
>>>>>> Yes.
>>>>>>
>>>>>> From the logs the query being executed is
>>>>>> select * from lip_data_quality where dt=20120709
>>>>>>
>>>>>> But here the dt is not in quotes. It should be like
>>>>>> select * from lip_data_quality where dt='20120709';
>>>>>> Regards
>>>>>> Bejoy KS
>>>>>>
>>>>>> Sent from handheld, please excuse typos.
>>>>>> ________________________________
>>>>>> From: Techy Teck <co...@gmail.com>
>>>>>> Date: Sat, 4 Aug 2012 00:07:07 -0700
>>>>>> To: <us...@hive.apache.org>; Bejoy Ks<be...@yahoo.com>
>>>>>> ReplyTo: user@hive.apache.org
>>>>>> Subject: Re: Passing date as command line arguments
>>>>>>
>>>>>> I found this thing from the Hive log. This is what you are looking
>>>>>> for?-
>>>>>>
>>>>>> SessionStart SESSION_ID="rjamal_201208041359" TIME="1344063565655"
>>>>>> QueryStart QUERY_STRING="select * from lip_data_quality where
>>>>>> dt=20120709"
>>>>>> QUERY_ID="rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486"
>>>>>> TIME="1344063597488"
>>>>>> Counters
>>>>>> plan="{"queryId":"rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486","queryType":null,"queryAttributes":{"queryString":"select
>>>>>> * from lip_data_quality where dt=20120709"},"queryCo
>>>>>>
>>>>>> unters":"null","stageGraph":{"nodeType":"1","roots":"null","adjacencyList":"]"},"stageList":"]","done":"false","started":"true"}"
>>>>>> TIME="1344063597501"
>>>>>> QueryEnd QUERY_STRING="select * from lip_data_quality where
>>>>>> dt=20120709"
>>>>>> QUERY_ID="rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486"
>>>>>> QUERY_RET_CODE="0" QUERY_NUM_TASKS="0" TIME="134406359
>>>>>> 7503"
>>>>>> Counters
>>>>>> plan="{"queryId":"rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486","queryType":null,"queryAttributes":{"queryString":"select
>>>>>> * from lip_data_quality where dt=20120709"},"queryCo
>>>>>>
>>>>>> unters":"null","stageGraph":{"nodeType":"1","roots":"null","adjacencyList":"]"},"stageList":"]","done":"true","started":"true"}"
>>>>>> TIME="1344063597504"
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Fri, Aug 3, 2012 at 11:58 PM, Techy Teck <co...@gmail.com>
>>>>>> wrote:
>>>>>>>
>>>>>>> Yes that query gives me the result back correctly.
>>>>>>>
>>>>>>> How can I see the hive log Bejoy?  Basically I am not sure which hive
>>>>>>> log and what is the path where I can see that thing?
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Fri, Aug 3, 2012 at 11:52 PM, Bejoy Ks <be...@yahoo.com> wrote:
>>>>>>>>
>>>>>>>> Hi Techy
>>>>>>>>
>>>>>>>> Are you getting the results for the same query directly from CLI, I
>>>>>>>> mean does the below query return correct results
>>>>>>>> select * from lip_data_quality where dt = '20120709';
>>>>>>>>
>>>>>>>> If the above query returns results, can you check the hive logs and
>>>>>>>> see what is the actual query being executed?
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>> Bejoy KS
>>>>>>>>
>>>>>>>> ________________________________
>>>>>>>> From: Techy Teck <co...@gmail.com>
>>>>>>>> To: user@hive.apache.org
>>>>>>>> Cc: Vijay Dirisala <te...@gmail.com>
>>>>>>>> Sent: Saturday, August 4, 2012 12:11 PM
>>>>>>>> Subject: Re: Passing date as command line arguments
>>>>>>>>
>>>>>>>> Thanks Vijay for the suggestion. I also tried that and it still
>>>>>>>> didn't worked for me.
>>>>>>>>
>>>>>>>> Any thoughts why it is not working?
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Fri, Aug 3, 2012 at 11:32 PM, Vijay <te...@gmail.com> wrote:
>>>>>>>>
>>>>>>>> There can't be a space in the variable name. Try this:
>>>>>>>> ${hiveconf:start_date}
>>>>>>>>
>>>>>>>> On Fri, Aug 3, 2012 at 11:06 PM, Techy Teck
>>>>>>>> <co...@gmail.com> wrote:
>>>>>>>> > Can anyone help me out on this?
>>>>>>>> >
>>>>>>>> > I am stuck on this and I have no clue what wrong I am doing?
>>>>>>>> >
>>>>>>>> >
>>>>>>>> >
>>>>>>>> >
>>>>>>>> > On Fri, Aug 3, 2012 at 3:48 PM, Techy Teck
>>>>>>>> > <co...@gmail.com> wrote:
>>>>>>>> >>
>>>>>>>> >> I have my below query in test1.hql file. I am trying to pass the
>>>>>>>> >> date (dt)
>>>>>>>> >> as the command line argument.
>>>>>>>> >>
>>>>>>>> >>
>>>>>>>> >> select * from lip_data_quality where dt = '${hiveconf:
>>>>>>>> >> start_date}';
>>>>>>>> >>
>>>>>>>> >>
>>>>>>>> >> So whenever I try to run the above test1.hql file like this-
>>>>>>>> >>
>>>>>>>> >>  hive -f hivetest1.hql -hiveconf start_date=20120709
>>>>>>>> >>
>>>>>>>> >> I get zero records back. But the data is there in that table for
>>>>>>>> >> that
>>>>>>>> >> date. Why is it so? Something wrong I am doing?
>>>>>>>> >>
>>>>>>>> >> Can anyone help me out here? I was following Bejoy's Article here
>>>>>>>> >>
>>>>>>>> >> http://kickstarthadoop.blogspot.com/2011/10/include-values-during-execution-time-in.html
>>>>>>>> >>
>>>>>>>> >>
>>>>>>>> >
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>

Re: Passing date as command line arguments

[Lefty Leverenz <le...@hortonworks.com>]

Does Hive 0.6 support underscores in variable names?

Just a thought.  The examples I've seen contain letters and dots (".") but
no other characters.

– Lefty




On Sat, Aug 4, 2012 at 12:39 AM, Bejoy KS <be...@yahoo.com> wrote:

> **
> Yes that is the right issue. Variable substitution is not happening. I
> can't say much here as I haven't tried out this on 0.6 . The code on my
> blog post is based on 0.7 or higher version I guess.
> Regards
> Bejoy KS
>
> Sent from handheld, please excuse typos.
> ------------------------------
> *From: * Techy Teck <co...@gmail.com>
> *Date: *Sat, 4 Aug 2012 00:35:20 -0700
> *To: *<us...@hive.apache.org>; <be...@yahoo.com>
> *ReplyTo: * user@hive.apache.org
> *Subject: *Re: Passing date as command line arguments
>
> Hi Bejoy,
>
> I found something very strange. When I actually viewed the Hive log file,
> I found something like this- See the thing in red color. Why is it like so?
>
> *SessionStart SESSION_ID="rjamal_201208041433" TIME="1344065620956"*
> *QueryStart QUERY_STRING=" select * from lip_data_quality where dt = '
> ${hiveconf:start_date}'"
> QUERY_ID="rjamal_20120804143333_3719afde-117d-4c6b-a322-0cff6e4b4c5d"
> TIME="1344065629740"*
> *Counters
> plan="{"queryId":"rjamal_20120804143333_3719afde-117d-4c6b-a322-0cff6e4b4c5d","queryType":null,"queryAttributes":{"queryString":"
> select * from lip_data_quality where dt =
> '${hiveconf:start_date}'"},"queryCounters":"null","stageGraph":{"nodeType":"1","roots":"null","adjacencyList":"]"},"stageList":"]","done":"false","started":"true"}"
> TIME="1344065629743"*
> *QueryEnd QUERY_STRING=" select * from lip_data_quality where dt =
> '${hiveconf:start_date}'"
> QUERY_ID="rjamal_20120804143333_3719afde-117d-4c6b-a322-0cff6e4b4c5d"
> QUERY_RET_CODE="0" QUERY_NUM_TASKS="0" TIME="1344065629745"*
> *Counters
> plan="{"queryId":"rjamal_20120804143333_3719afde-117d-4c6b-a322-0cff6e4b4c5d","queryType":null,"queryAttributes":{"queryString":"
> select * from lip_data_quality where dt =
> '${hiveconf:start_date}'"},"queryCounters":"null","stageGraph":{"nodeType":"1","roots":"null","adjacencyList":"]"},"stageList":"]","done":"true","started":"true"}"
> TIME="1344065629745"*
>
> On Sat, Aug 4, 2012 at 12:30 AM, Techy Teck <co...@gmail.com>wrote:
>
>> I cannot try that on Higher version of Hive as I am working in a company
>> where they have installed only Hive 0.6 on the Ares cluster.
>>
>> :(
>>
>>
>>
>>
>> On Sat, Aug 4, 2012 at 12:29 AM, Bejoy KS <be...@yahoo.com> wrote:
>>
>>> **
>>>
>>> Try it on a higher version of hive and let me know if that doesn't work
>>> still. 0.9 should be good.
>>>
>>>
>>> Regards
>>> Bejoy KS
>>>
>>> Sent from handheld, please excuse typos.
>>> ------------------------------
>>> *From: * Techy Teck <co...@gmail.com>
>>> *Date: *Sat, 4 Aug 2012 00:26:44 -0700
>>> *To: *<us...@hive.apache.org>; <be...@yahoo.com>
>>> *ReplyTo: * user@hive.apache.org
>>> *Subject: *Re: Passing date as command line arguments
>>>
>>> Yes it is String for me here.
>>>
>>> This is the table structure by the way-
>>>
>>> *create table lip_data_quality*
>>> *( buyer_id bigint,*
>>> *  total_chkout bigint,*
>>> *  total_errpds bigint*
>>> * )*
>>> * partitioned by (dt string)*
>>> *row format delimited fields terminated by '\t'*
>>> *stored as sequencefile*
>>> *location '/apps/hdmi-technology/b_apdpds/lip-data-quality'*
>>> *;*
>>>
>>> in which I am using dt in the where clause in my query. I am running *Hive
>>> 0.6.*
>>>
>>>
>>>
>>> On Sat, Aug 4, 2012 at 12:24 AM, Bejoy KS <be...@yahoo.com> wrote:
>>>
>>>> **
>>>> I tried the same query on my end, It is working fine for me without any
>>>> issues.
>>>>
>>>> By de way the data type for 'dt' is String itself right?
>>>> Regards
>>>> Bejoy KS
>>>>
>>>> Sent from handheld, please excuse typos.
>>>> ------------------------------
>>>> *From: * Techy Teck <co...@gmail.com>
>>>> *Date: *Sat, 4 Aug 2012 00:21:37 -0700
>>>> *To: *<us...@hive.apache.org>; <be...@yahoo.com>
>>>> *ReplyTo: * user@hive.apache.org
>>>> *Subject: *Re: Passing date as command line arguments
>>>>
>>>> I was not able to understand. So what changes I need to make? This is
>>>> my below *hivetest1.hql-*
>>>> *
>>>> *
>>>> *select * from lip_data_quality where dt = '${hiveconf:start_date}';*
>>>> *
>>>> *
>>>> And by this I am executing the above hql file-
>>>>
>>>> *hive -f hivetest1.hql -hiveconf start_date=20120709*
>>>> *
>>>> *
>>>> *
>>>> *
>>>> Any thoughts what changes I need to make?
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Sat, Aug 4, 2012 at 12:16 AM, Bejoy KS <be...@yahoo.com> wrote:
>>>>
>>>>> **
>>>>> Yes.
>>>>>
>>>>> From the logs the query being executed is
>>>>> select * from lip_data_quality where dt=20120709
>>>>>
>>>>> But here the dt is not in quotes. It should be like
>>>>> select * from lip_data_quality where dt='20120709';
>>>>> Regards
>>>>> Bejoy KS
>>>>>
>>>>> Sent from handheld, please excuse typos.
>>>>> ------------------------------
>>>>> *From: * Techy Teck <co...@gmail.com>
>>>>> *Date: *Sat, 4 Aug 2012 00:07:07 -0700
>>>>> *To: *<us...@hive.apache.org>; Bejoy Ks<be...@yahoo.com>
>>>>> *ReplyTo: * user@hive.apache.org
>>>>> *Subject: *Re: Passing date as command line arguments
>>>>>
>>>>> I found this thing from the Hive log. This is what you are looking
>>>>> for?-
>>>>>
>>>>> *SessionStart SESSION_ID="rjamal_201208041359" TIME="1344063565655"*
>>>>> *QueryStart QUERY_STRING="select * from lip_data_quality where
>>>>> dt=20120709"
>>>>> QUERY_ID="rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486"
>>>>> TIME="1344063597488"*
>>>>> *Counters
>>>>> plan="{"queryId":"rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486","queryType":null,"queryAttributes":{"queryString":"select
>>>>> * from lip_data_quality where dt=20120709"},"queryCo*
>>>>> *unters":"null","stageGraph":{"nodeType":"1","roots":"null","adjacencyList":"]"},"stageList":"]","done":"false","started":"true"}"
>>>>> TIME="1344063597501"*
>>>>> *QueryEnd QUERY_STRING="select * from lip_data_quality where
>>>>> dt=20120709"
>>>>> QUERY_ID="rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486"
>>>>> QUERY_RET_CODE="0" QUERY_NUM_TASKS="0" TIME="134406359*
>>>>> *7503"*
>>>>> *Counters
>>>>> plan="{"queryId":"rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486","queryType":null,"queryAttributes":{"queryString":"select
>>>>> * from lip_data_quality where dt=20120709"},"queryCo*
>>>>> *unters":"null","stageGraph":{"nodeType":"1","roots":"null","adjacencyList":"]"},"stageList":"]","done":"true","started":"true"}"
>>>>> TIME="1344063597504"*
>>>>> *
>>>>> *
>>>>> *
>>>>> *
>>>>>
>>>>>
>>>>>
>>>>> On Fri, Aug 3, 2012 at 11:58 PM, Techy Teck <co...@gmail.com>wrote:
>>>>>
>>>>>> Yes that query gives me the result back correctly.
>>>>>>
>>>>>> How can I see the hive log Bejoy?  Basically I am not sure which hive
>>>>>> log and what is the path where I can see that thing?
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Fri, Aug 3, 2012 at 11:52 PM, Bejoy Ks <be...@yahoo.com> wrote:
>>>>>>
>>>>>>>  Hi Techy
>>>>>>>
>>>>>>> Are you getting the results for the same query directly from CLI, I
>>>>>>> mean does the below query return correct results
>>>>>>> select * from lip_data_quality where dt = '20120709';
>>>>>>>
>>>>>>> If the above query returns results, can you check the hive logs and
>>>>>>> see what is the actual query being executed?
>>>>>>>
>>>>>>> Regards,
>>>>>>> Bejoy KS
>>>>>>>
>>>>>>>   ------------------------------
>>>>>>> *From:* Techy Teck <co...@gmail.com>
>>>>>>> *To:* user@hive.apache.org
>>>>>>> *Cc:* Vijay Dirisala <te...@gmail.com>
>>>>>>> *Sent:* Saturday, August 4, 2012 12:11 PM
>>>>>>> *Subject:* Re: Passing date as command line arguments
>>>>>>>
>>>>>>> Thanks Vijay for the suggestion. I also tried that and it still
>>>>>>> didn't worked for me.
>>>>>>>
>>>>>>> Any thoughts why it is not working?
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Fri, Aug 3, 2012 at 11:32 PM, Vijay <te...@gmail.com> wrote:
>>>>>>>
>>>>>>> There can't be a space in the variable name. Try this:
>>>>>>> ${hiveconf:start_date}
>>>>>>>
>>>>>>> On Fri, Aug 3, 2012 at 11:06 PM, Techy Teck <co...@gmail.com>
>>>>>>> wrote:
>>>>>>> > Can anyone help me out on this?
>>>>>>> >
>>>>>>> > I am stuck on this and I have no clue what wrong I am doing?
>>>>>>> >
>>>>>>> >
>>>>>>> >
>>>>>>> >
>>>>>>> > On Fri, Aug 3, 2012 at 3:48 PM, Techy Teck <
>>>>>>> comptechgeeky@gmail.com> wrote:
>>>>>>> >>
>>>>>>> >> I have my below query in test1.hql file. I am trying to pass the
>>>>>>> date (dt)
>>>>>>> >> as the command line argument.
>>>>>>> >>
>>>>>>> >>
>>>>>>> >> select * from lip_data_quality where dt = '${hiveconf:
>>>>>>> start_date}';
>>>>>>> >>
>>>>>>> >>
>>>>>>> >> So whenever I try to run the above test1.hql file like this-
>>>>>>> >>
>>>>>>> >>  hive -f hivetest1.hql -hiveconf start_date=20120709
>>>>>>> >>
>>>>>>> >> I get zero records back. But the data is there in that table for
>>>>>>> that
>>>>>>> >> date. Why is it so? Something wrong I am doing?
>>>>>>> >>
>>>>>>> >> Can anyone help me out here? I was following Bejoy's Article here
>>>>>>> >>
>>>>>>> http://kickstarthadoop.blogspot.com/2011/10/include-values-during-execution-time-in.html
>>>>>>> >>
>>>>>>> >>
>>>>>>> >
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>

Re: Passing date as command line arguments

[Bejoy KS <be...@yahoo.com>]

Yes that is the right issue. Variable substitution is not happening. I can't say much here as I haven't tried out this on 0.6 . The code on my blog post is based on 0.7 or higher version I guess.  

Regards
Bejoy KS

Sent from handheld, please excuse typos.

-----Original Message-----
From: Techy Teck <co...@gmail.com>
Date: Sat, 4 Aug 2012 00:35:20 
To: <us...@hive.apache.org>; <be...@yahoo.com>
Reply-To: user@hive.apache.org
Subject: Re: Passing date as command line arguments

Hi Bejoy,

I found something very strange. When I actually viewed the Hive log file, I
found something like this- See the thing in red color. Why is it like so?

*SessionStart SESSION_ID="rjamal_201208041433" TIME="1344065620956"*
*QueryStart QUERY_STRING=" select * from lip_data_quality where dt = '
${hiveconf:start_date}'"
QUERY_ID="rjamal_20120804143333_3719afde-117d-4c6b-a322-0cff6e4b4c5d"
TIME="1344065629740"*
*Counters
plan="{"queryId":"rjamal_20120804143333_3719afde-117d-4c6b-a322-0cff6e4b4c5d","queryType":null,"queryAttributes":{"queryString":"
select * from lip_data_quality where dt =
'${hiveconf:start_date}'"},"queryCounters":"null","stageGraph":{"nodeType":"1","roots":"null","adjacencyList":"]"},"stageList":"]","done":"false","started":"true"}"
TIME="1344065629743"*
*QueryEnd QUERY_STRING=" select * from lip_data_quality where dt =
'${hiveconf:start_date}'"
QUERY_ID="rjamal_20120804143333_3719afde-117d-4c6b-a322-0cff6e4b4c5d"
QUERY_RET_CODE="0" QUERY_NUM_TASKS="0" TIME="1344065629745"*
*Counters
plan="{"queryId":"rjamal_20120804143333_3719afde-117d-4c6b-a322-0cff6e4b4c5d","queryType":null,"queryAttributes":{"queryString":"
select * from lip_data_quality where dt =
'${hiveconf:start_date}'"},"queryCounters":"null","stageGraph":{"nodeType":"1","roots":"null","adjacencyList":"]"},"stageList":"]","done":"true","started":"true"}"
TIME="1344065629745"*

On Sat, Aug 4, 2012 at 12:30 AM, Techy Teck <co...@gmail.com> wrote:

> I cannot try that on Higher version of Hive as I am working in a company
> where they have installed only Hive 0.6 on the Ares cluster.
>
> :(
>
>
>
>
> On Sat, Aug 4, 2012 at 12:29 AM, Bejoy KS <be...@yahoo.com> wrote:
>
>> **
>>
>> Try it on a higher version of hive and let me know if that doesn't work
>> still. 0.9 should be good.
>>
>>
>> Regards
>> Bejoy KS
>>
>> Sent from handheld, please excuse typos.
>> ------------------------------
>> *From: * Techy Teck <co...@gmail.com>
>> *Date: *Sat, 4 Aug 2012 00:26:44 -0700
>> *To: *<us...@hive.apache.org>; <be...@yahoo.com>
>> *ReplyTo: * user@hive.apache.org
>> *Subject: *Re: Passing date as command line arguments
>>
>> Yes it is String for me here.
>>
>> This is the table structure by the way-
>>
>> *create table lip_data_quality*
>> *( buyer_id bigint,*
>> *  total_chkout bigint,*
>> *  total_errpds bigint*
>> * )*
>> * partitioned by (dt string)*
>> *row format delimited fields terminated by '\t'*
>> *stored as sequencefile*
>> *location '/apps/hdmi-technology/b_apdpds/lip-data-quality'*
>> *;*
>>
>> in which I am using dt in the where clause in my query. I am running *Hive
>> 0.6.*
>>
>>
>>
>> On Sat, Aug 4, 2012 at 12:24 AM, Bejoy KS <be...@yahoo.com> wrote:
>>
>>> **
>>> I tried the same query on my end, It is working fine for me without any
>>> issues.
>>>
>>> By de way the data type for 'dt' is String itself right?
>>> Regards
>>> Bejoy KS
>>>
>>> Sent from handheld, please excuse typos.
>>> ------------------------------
>>> *From: * Techy Teck <co...@gmail.com>
>>> *Date: *Sat, 4 Aug 2012 00:21:37 -0700
>>> *To: *<us...@hive.apache.org>; <be...@yahoo.com>
>>> *ReplyTo: * user@hive.apache.org
>>> *Subject: *Re: Passing date as command line arguments
>>>
>>> I was not able to understand. So what changes I need to make? This is my
>>> below *hivetest1.hql-*
>>> *
>>> *
>>> *select * from lip_data_quality where dt = '${hiveconf:start_date}';*
>>> *
>>> *
>>> And by this I am executing the above hql file-
>>>
>>> *hive -f hivetest1.hql -hiveconf start_date=20120709*
>>> *
>>> *
>>> *
>>> *
>>> Any thoughts what changes I need to make?
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> On Sat, Aug 4, 2012 at 12:16 AM, Bejoy KS <be...@yahoo.com> wrote:
>>>
>>>> **
>>>> Yes.
>>>>
>>>> From the logs the query being executed is
>>>> select * from lip_data_quality where dt=20120709
>>>>
>>>> But here the dt is not in quotes. It should be like
>>>> select * from lip_data_quality where dt='20120709';
>>>> Regards
>>>> Bejoy KS
>>>>
>>>> Sent from handheld, please excuse typos.
>>>> ------------------------------
>>>> *From: * Techy Teck <co...@gmail.com>
>>>> *Date: *Sat, 4 Aug 2012 00:07:07 -0700
>>>> *To: *<us...@hive.apache.org>; Bejoy Ks<be...@yahoo.com>
>>>> *ReplyTo: * user@hive.apache.org
>>>> *Subject: *Re: Passing date as command line arguments
>>>>
>>>> I found this thing from the Hive log. This is what you are looking for?-
>>>>
>>>> *SessionStart SESSION_ID="rjamal_201208041359" TIME="1344063565655"*
>>>> *QueryStart QUERY_STRING="select * from lip_data_quality where
>>>> dt=20120709"
>>>> QUERY_ID="rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486"
>>>> TIME="1344063597488"*
>>>> *Counters
>>>> plan="{"queryId":"rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486","queryType":null,"queryAttributes":{"queryString":"select
>>>> * from lip_data_quality where dt=20120709"},"queryCo*
>>>> *unters":"null","stageGraph":{"nodeType":"1","roots":"null","adjacencyList":"]"},"stageList":"]","done":"false","started":"true"}"
>>>> TIME="1344063597501"*
>>>> *QueryEnd QUERY_STRING="select * from lip_data_quality where
>>>> dt=20120709"
>>>> QUERY_ID="rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486"
>>>> QUERY_RET_CODE="0" QUERY_NUM_TASKS="0" TIME="134406359*
>>>> *7503"*
>>>> *Counters
>>>> plan="{"queryId":"rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486","queryType":null,"queryAttributes":{"queryString":"select
>>>> * from lip_data_quality where dt=20120709"},"queryCo*
>>>> *unters":"null","stageGraph":{"nodeType":"1","roots":"null","adjacencyList":"]"},"stageList":"]","done":"true","started":"true"}"
>>>> TIME="1344063597504"*
>>>> *
>>>> *
>>>> *
>>>> *
>>>>
>>>>
>>>>
>>>> On Fri, Aug 3, 2012 at 11:58 PM, Techy Teck <co...@gmail.com>wrote:
>>>>
>>>>> Yes that query gives me the result back correctly.
>>>>>
>>>>> How can I see the hive log Bejoy?  Basically I am not sure which hive
>>>>> log and what is the path where I can see that thing?
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Fri, Aug 3, 2012 at 11:52 PM, Bejoy Ks <be...@yahoo.com> wrote:
>>>>>
>>>>>>  Hi Techy
>>>>>>
>>>>>> Are you getting the results for the same query directly from CLI, I
>>>>>> mean does the below query return correct results
>>>>>> select * from lip_data_quality where dt = '20120709';
>>>>>>
>>>>>> If the above query returns results, can you check the hive logs and
>>>>>> see what is the actual query being executed?
>>>>>>
>>>>>> Regards,
>>>>>> Bejoy KS
>>>>>>
>>>>>>   ------------------------------
>>>>>> *From:* Techy Teck <co...@gmail.com>
>>>>>> *To:* user@hive.apache.org
>>>>>> *Cc:* Vijay Dirisala <te...@gmail.com>
>>>>>> *Sent:* Saturday, August 4, 2012 12:11 PM
>>>>>> *Subject:* Re: Passing date as command line arguments
>>>>>>
>>>>>> Thanks Vijay for the suggestion. I also tried that and it still
>>>>>> didn't worked for me.
>>>>>>
>>>>>> Any thoughts why it is not working?
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Fri, Aug 3, 2012 at 11:32 PM, Vijay <te...@gmail.com> wrote:
>>>>>>
>>>>>> There can't be a space in the variable name. Try this:
>>>>>> ${hiveconf:start_date}
>>>>>>
>>>>>> On Fri, Aug 3, 2012 at 11:06 PM, Techy Teck <co...@gmail.com>
>>>>>> wrote:
>>>>>> > Can anyone help me out on this?
>>>>>> >
>>>>>> > I am stuck on this and I have no clue what wrong I am doing?
>>>>>> >
>>>>>> >
>>>>>> >
>>>>>> >
>>>>>> > On Fri, Aug 3, 2012 at 3:48 PM, Techy Teck <co...@gmail.com>
>>>>>> wrote:
>>>>>> >>
>>>>>> >> I have my below query in test1.hql file. I am trying to pass the
>>>>>> date (dt)
>>>>>> >> as the command line argument.
>>>>>> >>
>>>>>> >>
>>>>>> >> select * from lip_data_quality where dt = '${hiveconf:
>>>>>> start_date}';
>>>>>> >>
>>>>>> >>
>>>>>> >> So whenever I try to run the above test1.hql file like this-
>>>>>> >>
>>>>>> >>  hive -f hivetest1.hql -hiveconf start_date=20120709
>>>>>> >>
>>>>>> >> I get zero records back. But the data is there in that table for
>>>>>> that
>>>>>> >> date. Why is it so? Something wrong I am doing?
>>>>>> >>
>>>>>> >> Can anyone help me out here? I was following Bejoy's Article here
>>>>>> >>
>>>>>> http://kickstarthadoop.blogspot.com/2011/10/include-values-during-execution-time-in.html
>>>>>> >>
>>>>>> >>
>>>>>> >
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>

Re: Passing date as command line arguments

[Techy Teck <co...@gmail.com>]

Hi Bejoy,

I found something very strange. When I actually viewed the Hive log file, I
found something like this- See the thing in red color. Why is it like so?

*SessionStart SESSION_ID="rjamal_201208041433" TIME="1344065620956"*
*QueryStart QUERY_STRING=" select * from lip_data_quality where dt = '
${hiveconf:start_date}'"
QUERY_ID="rjamal_20120804143333_3719afde-117d-4c6b-a322-0cff6e4b4c5d"
TIME="1344065629740"*
*Counters
plan="{"queryId":"rjamal_20120804143333_3719afde-117d-4c6b-a322-0cff6e4b4c5d","queryType":null,"queryAttributes":{"queryString":"
select * from lip_data_quality where dt =
'${hiveconf:start_date}'"},"queryCounters":"null","stageGraph":{"nodeType":"1","roots":"null","adjacencyList":"]"},"stageList":"]","done":"false","started":"true"}"
TIME="1344065629743"*
*QueryEnd QUERY_STRING=" select * from lip_data_quality where dt =
'${hiveconf:start_date}'"
QUERY_ID="rjamal_20120804143333_3719afde-117d-4c6b-a322-0cff6e4b4c5d"
QUERY_RET_CODE="0" QUERY_NUM_TASKS="0" TIME="1344065629745"*
*Counters
plan="{"queryId":"rjamal_20120804143333_3719afde-117d-4c6b-a322-0cff6e4b4c5d","queryType":null,"queryAttributes":{"queryString":"
select * from lip_data_quality where dt =
'${hiveconf:start_date}'"},"queryCounters":"null","stageGraph":{"nodeType":"1","roots":"null","adjacencyList":"]"},"stageList":"]","done":"true","started":"true"}"
TIME="1344065629745"*

On Sat, Aug 4, 2012 at 12:30 AM, Techy Teck <co...@gmail.com> wrote:

> I cannot try that on Higher version of Hive as I am working in a company
> where they have installed only Hive 0.6 on the Ares cluster.
>
> :(
>
>
>
>
> On Sat, Aug 4, 2012 at 12:29 AM, Bejoy KS <be...@yahoo.com> wrote:
>
>> **
>>
>> Try it on a higher version of hive and let me know if that doesn't work
>> still. 0.9 should be good.
>>
>>
>> Regards
>> Bejoy KS
>>
>> Sent from handheld, please excuse typos.
>> ------------------------------
>> *From: * Techy Teck <co...@gmail.com>
>> *Date: *Sat, 4 Aug 2012 00:26:44 -0700
>> *To: *<us...@hive.apache.org>; <be...@yahoo.com>
>> *ReplyTo: * user@hive.apache.org
>> *Subject: *Re: Passing date as command line arguments
>>
>> Yes it is String for me here.
>>
>> This is the table structure by the way-
>>
>> *create table lip_data_quality*
>> *( buyer_id bigint,*
>> *  total_chkout bigint,*
>> *  total_errpds bigint*
>> * )*
>> * partitioned by (dt string)*
>> *row format delimited fields terminated by '\t'*
>> *stored as sequencefile*
>> *location '/apps/hdmi-technology/b_apdpds/lip-data-quality'*
>> *;*
>>
>> in which I am using dt in the where clause in my query. I am running *Hive
>> 0.6.*
>>
>>
>>
>> On Sat, Aug 4, 2012 at 12:24 AM, Bejoy KS <be...@yahoo.com> wrote:
>>
>>> **
>>> I tried the same query on my end, It is working fine for me without any
>>> issues.
>>>
>>> By de way the data type for 'dt' is String itself right?
>>> Regards
>>> Bejoy KS
>>>
>>> Sent from handheld, please excuse typos.
>>> ------------------------------
>>> *From: * Techy Teck <co...@gmail.com>
>>> *Date: *Sat, 4 Aug 2012 00:21:37 -0700
>>> *To: *<us...@hive.apache.org>; <be...@yahoo.com>
>>> *ReplyTo: * user@hive.apache.org
>>> *Subject: *Re: Passing date as command line arguments
>>>
>>> I was not able to understand. So what changes I need to make? This is my
>>> below *hivetest1.hql-*
>>> *
>>> *
>>> *select * from lip_data_quality where dt = '${hiveconf:start_date}';*
>>> *
>>> *
>>> And by this I am executing the above hql file-
>>>
>>> *hive -f hivetest1.hql -hiveconf start_date=20120709*
>>> *
>>> *
>>> *
>>> *
>>> Any thoughts what changes I need to make?
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> On Sat, Aug 4, 2012 at 12:16 AM, Bejoy KS <be...@yahoo.com> wrote:
>>>
>>>> **
>>>> Yes.
>>>>
>>>> From the logs the query being executed is
>>>> select * from lip_data_quality where dt=20120709
>>>>
>>>> But here the dt is not in quotes. It should be like
>>>> select * from lip_data_quality where dt='20120709';
>>>> Regards
>>>> Bejoy KS
>>>>
>>>> Sent from handheld, please excuse typos.
>>>> ------------------------------
>>>> *From: * Techy Teck <co...@gmail.com>
>>>> *Date: *Sat, 4 Aug 2012 00:07:07 -0700
>>>> *To: *<us...@hive.apache.org>; Bejoy Ks<be...@yahoo.com>
>>>> *ReplyTo: * user@hive.apache.org
>>>> *Subject: *Re: Passing date as command line arguments
>>>>
>>>> I found this thing from the Hive log. This is what you are looking for?-
>>>>
>>>> *SessionStart SESSION_ID="rjamal_201208041359" TIME="1344063565655"*
>>>> *QueryStart QUERY_STRING="select * from lip_data_quality where
>>>> dt=20120709"
>>>> QUERY_ID="rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486"
>>>> TIME="1344063597488"*
>>>> *Counters
>>>> plan="{"queryId":"rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486","queryType":null,"queryAttributes":{"queryString":"select
>>>> * from lip_data_quality where dt=20120709"},"queryCo*
>>>> *unters":"null","stageGraph":{"nodeType":"1","roots":"null","adjacencyList":"]"},"stageList":"]","done":"false","started":"true"}"
>>>> TIME="1344063597501"*
>>>> *QueryEnd QUERY_STRING="select * from lip_data_quality where
>>>> dt=20120709"
>>>> QUERY_ID="rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486"
>>>> QUERY_RET_CODE="0" QUERY_NUM_TASKS="0" TIME="134406359*
>>>> *7503"*
>>>> *Counters
>>>> plan="{"queryId":"rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486","queryType":null,"queryAttributes":{"queryString":"select
>>>> * from lip_data_quality where dt=20120709"},"queryCo*
>>>> *unters":"null","stageGraph":{"nodeType":"1","roots":"null","adjacencyList":"]"},"stageList":"]","done":"true","started":"true"}"
>>>> TIME="1344063597504"*
>>>> *
>>>> *
>>>> *
>>>> *
>>>>
>>>>
>>>>
>>>> On Fri, Aug 3, 2012 at 11:58 PM, Techy Teck <co...@gmail.com>wrote:
>>>>
>>>>> Yes that query gives me the result back correctly.
>>>>>
>>>>> How can I see the hive log Bejoy?  Basically I am not sure which hive
>>>>> log and what is the path where I can see that thing?
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Fri, Aug 3, 2012 at 11:52 PM, Bejoy Ks <be...@yahoo.com> wrote:
>>>>>
>>>>>>  Hi Techy
>>>>>>
>>>>>> Are you getting the results for the same query directly from CLI, I
>>>>>> mean does the below query return correct results
>>>>>> select * from lip_data_quality where dt = '20120709';
>>>>>>
>>>>>> If the above query returns results, can you check the hive logs and
>>>>>> see what is the actual query being executed?
>>>>>>
>>>>>> Regards,
>>>>>> Bejoy KS
>>>>>>
>>>>>>   ------------------------------
>>>>>> *From:* Techy Teck <co...@gmail.com>
>>>>>> *To:* user@hive.apache.org
>>>>>> *Cc:* Vijay Dirisala <te...@gmail.com>
>>>>>> *Sent:* Saturday, August 4, 2012 12:11 PM
>>>>>> *Subject:* Re: Passing date as command line arguments
>>>>>>
>>>>>> Thanks Vijay for the suggestion. I also tried that and it still
>>>>>> didn't worked for me.
>>>>>>
>>>>>> Any thoughts why it is not working?
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Fri, Aug 3, 2012 at 11:32 PM, Vijay <te...@gmail.com> wrote:
>>>>>>
>>>>>> There can't be a space in the variable name. Try this:
>>>>>> ${hiveconf:start_date}
>>>>>>
>>>>>> On Fri, Aug 3, 2012 at 11:06 PM, Techy Teck <co...@gmail.com>
>>>>>> wrote:
>>>>>> > Can anyone help me out on this?
>>>>>> >
>>>>>> > I am stuck on this and I have no clue what wrong I am doing?
>>>>>> >
>>>>>> >
>>>>>> >
>>>>>> >
>>>>>> > On Fri, Aug 3, 2012 at 3:48 PM, Techy Teck <co...@gmail.com>
>>>>>> wrote:
>>>>>> >>
>>>>>> >> I have my below query in test1.hql file. I am trying to pass the
>>>>>> date (dt)
>>>>>> >> as the command line argument.
>>>>>> >>
>>>>>> >>
>>>>>> >> select * from lip_data_quality where dt = '${hiveconf:
>>>>>> start_date}';
>>>>>> >>
>>>>>> >>
>>>>>> >> So whenever I try to run the above test1.hql file like this-
>>>>>> >>
>>>>>> >>  hive -f hivetest1.hql -hiveconf start_date=20120709
>>>>>> >>
>>>>>> >> I get zero records back. But the data is there in that table for
>>>>>> that
>>>>>> >> date. Why is it so? Something wrong I am doing?
>>>>>> >>
>>>>>> >> Can anyone help me out here? I was following Bejoy's Article here
>>>>>> >>
>>>>>> http://kickstarthadoop.blogspot.com/2011/10/include-values-during-execution-time-in.html
>>>>>> >>
>>>>>> >>
>>>>>> >
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>

Re: Passing date as command line arguments

[Techy Teck <co...@gmail.com>]

I cannot try that on Higher version of Hive as I am working in a company
where they have installed only Hive 0.6 on the Ares cluster.

:(




On Sat, Aug 4, 2012 at 12:29 AM, Bejoy KS <be...@yahoo.com> wrote:

> **
>
> Try it on a higher version of hive and let me know if that doesn't work
> still. 0.9 should be good.
>
>
> Regards
> Bejoy KS
>
> Sent from handheld, please excuse typos.
> ------------------------------
> *From: * Techy Teck <co...@gmail.com>
> *Date: *Sat, 4 Aug 2012 00:26:44 -0700
> *To: *<us...@hive.apache.org>; <be...@yahoo.com>
> *ReplyTo: * user@hive.apache.org
> *Subject: *Re: Passing date as command line arguments
>
> Yes it is String for me here.
>
> This is the table structure by the way-
>
> *create table lip_data_quality*
> *( buyer_id bigint,*
> *  total_chkout bigint,*
> *  total_errpds bigint*
> * )*
> * partitioned by (dt string)*
> *row format delimited fields terminated by '\t'*
> *stored as sequencefile*
> *location '/apps/hdmi-technology/b_apdpds/lip-data-quality'*
> *;*
>
> in which I am using dt in the where clause in my query. I am running *Hive
> 0.6.*
>
>
>
> On Sat, Aug 4, 2012 at 12:24 AM, Bejoy KS <be...@yahoo.com> wrote:
>
>> **
>> I tried the same query on my end, It is working fine for me without any
>> issues.
>>
>> By de way the data type for 'dt' is String itself right?
>> Regards
>> Bejoy KS
>>
>> Sent from handheld, please excuse typos.
>> ------------------------------
>> *From: * Techy Teck <co...@gmail.com>
>> *Date: *Sat, 4 Aug 2012 00:21:37 -0700
>> *To: *<us...@hive.apache.org>; <be...@yahoo.com>
>> *ReplyTo: * user@hive.apache.org
>> *Subject: *Re: Passing date as command line arguments
>>
>> I was not able to understand. So what changes I need to make? This is my
>> below *hivetest1.hql-*
>> *
>> *
>> *select * from lip_data_quality where dt = '${hiveconf:start_date}';*
>> *
>> *
>> And by this I am executing the above hql file-
>>
>> *hive -f hivetest1.hql -hiveconf start_date=20120709*
>> *
>> *
>> *
>> *
>> Any thoughts what changes I need to make?
>>
>>
>>
>>
>>
>>
>>
>> On Sat, Aug 4, 2012 at 12:16 AM, Bejoy KS <be...@yahoo.com> wrote:
>>
>>> **
>>> Yes.
>>>
>>> From the logs the query being executed is
>>> select * from lip_data_quality where dt=20120709
>>>
>>> But here the dt is not in quotes. It should be like
>>> select * from lip_data_quality where dt='20120709';
>>> Regards
>>> Bejoy KS
>>>
>>> Sent from handheld, please excuse typos.
>>> ------------------------------
>>> *From: * Techy Teck <co...@gmail.com>
>>> *Date: *Sat, 4 Aug 2012 00:07:07 -0700
>>> *To: *<us...@hive.apache.org>; Bejoy Ks<be...@yahoo.com>
>>> *ReplyTo: * user@hive.apache.org
>>> *Subject: *Re: Passing date as command line arguments
>>>
>>> I found this thing from the Hive log. This is what you are looking for?-
>>>
>>> *SessionStart SESSION_ID="rjamal_201208041359" TIME="1344063565655"*
>>> *QueryStart QUERY_STRING="select * from lip_data_quality where
>>> dt=20120709"
>>> QUERY_ID="rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486"
>>> TIME="1344063597488"*
>>> *Counters
>>> plan="{"queryId":"rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486","queryType":null,"queryAttributes":{"queryString":"select
>>> * from lip_data_quality where dt=20120709"},"queryCo*
>>> *unters":"null","stageGraph":{"nodeType":"1","roots":"null","adjacencyList":"]"},"stageList":"]","done":"false","started":"true"}"
>>> TIME="1344063597501"*
>>> *QueryEnd QUERY_STRING="select * from lip_data_quality where
>>> dt=20120709"
>>> QUERY_ID="rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486"
>>> QUERY_RET_CODE="0" QUERY_NUM_TASKS="0" TIME="134406359*
>>> *7503"*
>>> *Counters
>>> plan="{"queryId":"rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486","queryType":null,"queryAttributes":{"queryString":"select
>>> * from lip_data_quality where dt=20120709"},"queryCo*
>>> *unters":"null","stageGraph":{"nodeType":"1","roots":"null","adjacencyList":"]"},"stageList":"]","done":"true","started":"true"}"
>>> TIME="1344063597504"*
>>> *
>>> *
>>> *
>>> *
>>>
>>>
>>>
>>> On Fri, Aug 3, 2012 at 11:58 PM, Techy Teck <co...@gmail.com>wrote:
>>>
>>>> Yes that query gives me the result back correctly.
>>>>
>>>> How can I see the hive log Bejoy?  Basically I am not sure which hive
>>>> log and what is the path where I can see that thing?
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Fri, Aug 3, 2012 at 11:52 PM, Bejoy Ks <be...@yahoo.com> wrote:
>>>>
>>>>>  Hi Techy
>>>>>
>>>>> Are you getting the results for the same query directly from CLI, I
>>>>> mean does the below query return correct results
>>>>> select * from lip_data_quality where dt = '20120709';
>>>>>
>>>>> If the above query returns results, can you check the hive logs and
>>>>> see what is the actual query being executed?
>>>>>
>>>>> Regards,
>>>>> Bejoy KS
>>>>>
>>>>>   ------------------------------
>>>>> *From:* Techy Teck <co...@gmail.com>
>>>>> *To:* user@hive.apache.org
>>>>> *Cc:* Vijay Dirisala <te...@gmail.com>
>>>>> *Sent:* Saturday, August 4, 2012 12:11 PM
>>>>> *Subject:* Re: Passing date as command line arguments
>>>>>
>>>>> Thanks Vijay for the suggestion. I also tried that and it still didn't
>>>>> worked for me.
>>>>>
>>>>> Any thoughts why it is not working?
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Fri, Aug 3, 2012 at 11:32 PM, Vijay <te...@gmail.com> wrote:
>>>>>
>>>>> There can't be a space in the variable name. Try this:
>>>>> ${hiveconf:start_date}
>>>>>
>>>>> On Fri, Aug 3, 2012 at 11:06 PM, Techy Teck <co...@gmail.com>
>>>>> wrote:
>>>>> > Can anyone help me out on this?
>>>>> >
>>>>> > I am stuck on this and I have no clue what wrong I am doing?
>>>>> >
>>>>> >
>>>>> >
>>>>> >
>>>>> > On Fri, Aug 3, 2012 at 3:48 PM, Techy Teck <co...@gmail.com>
>>>>> wrote:
>>>>> >>
>>>>> >> I have my below query in test1.hql file. I am trying to pass the
>>>>> date (dt)
>>>>> >> as the command line argument.
>>>>> >>
>>>>> >>
>>>>> >> select * from lip_data_quality where dt = '${hiveconf: start_date}';
>>>>> >>
>>>>> >>
>>>>> >> So whenever I try to run the above test1.hql file like this-
>>>>> >>
>>>>> >>  hive -f hivetest1.hql -hiveconf start_date=20120709
>>>>> >>
>>>>> >> I get zero records back. But the data is there in that table for
>>>>> that
>>>>> >> date. Why is it so? Something wrong I am doing?
>>>>> >>
>>>>> >> Can anyone help me out here? I was following Bejoy's Article here
>>>>> >>
>>>>> http://kickstarthadoop.blogspot.com/2011/10/include-values-during-execution-time-in.html
>>>>> >>
>>>>> >>
>>>>> >
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>

Re: Passing date as command line arguments

[Bejoy KS <be...@yahoo.com>]

Try it on a higher version of hive and let me know if that doesn't work still. 0.9 should be good.


Regards
Bejoy KS

Sent from handheld, please excuse typos.

-----Original Message-----
From: Techy Teck <co...@gmail.com>
Date: Sat, 4 Aug 2012 00:26:44 
To: <us...@hive.apache.org>; <be...@yahoo.com>
Reply-To: user@hive.apache.org
Subject: Re: Passing date as command line arguments

Yes it is String for me here.

This is the table structure by the way-

*create table lip_data_quality*
*( buyer_id bigint,*
*  total_chkout bigint,*
*  total_errpds bigint*
* )*
* partitioned by (dt string)*
*row format delimited fields terminated by '\t'*
*stored as sequencefile*
*location '/apps/hdmi-technology/b_apdpds/lip-data-quality'*
*;*

in which I am using dt in the where clause in my query. I am running *Hive
0.6.*



On Sat, Aug 4, 2012 at 12:24 AM, Bejoy KS <be...@yahoo.com> wrote:

> **
> I tried the same query on my end, It is working fine for me without any
> issues.
>
> By de way the data type for 'dt' is String itself right?
> Regards
> Bejoy KS
>
> Sent from handheld, please excuse typos.
> ------------------------------
> *From: * Techy Teck <co...@gmail.com>
> *Date: *Sat, 4 Aug 2012 00:21:37 -0700
> *To: *<us...@hive.apache.org>; <be...@yahoo.com>
> *ReplyTo: * user@hive.apache.org
> *Subject: *Re: Passing date as command line arguments
>
> I was not able to understand. So what changes I need to make? This is my
> below *hivetest1.hql-*
> *
> *
> *select * from lip_data_quality where dt = '${hiveconf:start_date}';*
> *
> *
> And by this I am executing the above hql file-
>
> *hive -f hivetest1.hql -hiveconf start_date=20120709*
> *
> *
> *
> *
> Any thoughts what changes I need to make?
>
>
>
>
>
>
>
> On Sat, Aug 4, 2012 at 12:16 AM, Bejoy KS <be...@yahoo.com> wrote:
>
>> **
>> Yes.
>>
>> From the logs the query being executed is
>> select * from lip_data_quality where dt=20120709
>>
>> But here the dt is not in quotes. It should be like
>> select * from lip_data_quality where dt='20120709';
>> Regards
>> Bejoy KS
>>
>> Sent from handheld, please excuse typos.
>> ------------------------------
>> *From: * Techy Teck <co...@gmail.com>
>> *Date: *Sat, 4 Aug 2012 00:07:07 -0700
>> *To: *<us...@hive.apache.org>; Bejoy Ks<be...@yahoo.com>
>> *ReplyTo: * user@hive.apache.org
>> *Subject: *Re: Passing date as command line arguments
>>
>> I found this thing from the Hive log. This is what you are looking for?-
>>
>> *SessionStart SESSION_ID="rjamal_201208041359" TIME="1344063565655"*
>> *QueryStart QUERY_STRING="select * from lip_data_quality where
>> dt=20120709"
>> QUERY_ID="rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486"
>> TIME="1344063597488"*
>> *Counters
>> plan="{"queryId":"rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486","queryType":null,"queryAttributes":{"queryString":"select
>> * from lip_data_quality where dt=20120709"},"queryCo*
>> *unters":"null","stageGraph":{"nodeType":"1","roots":"null","adjacencyList":"]"},"stageList":"]","done":"false","started":"true"}"
>> TIME="1344063597501"*
>> *QueryEnd QUERY_STRING="select * from lip_data_quality where
>> dt=20120709"
>> QUERY_ID="rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486"
>> QUERY_RET_CODE="0" QUERY_NUM_TASKS="0" TIME="134406359*
>> *7503"*
>> *Counters
>> plan="{"queryId":"rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486","queryType":null,"queryAttributes":{"queryString":"select
>> * from lip_data_quality where dt=20120709"},"queryCo*
>> *unters":"null","stageGraph":{"nodeType":"1","roots":"null","adjacencyList":"]"},"stageList":"]","done":"true","started":"true"}"
>> TIME="1344063597504"*
>> *
>> *
>> *
>> *
>>
>>
>>
>> On Fri, Aug 3, 2012 at 11:58 PM, Techy Teck <co...@gmail.com>wrote:
>>
>>> Yes that query gives me the result back correctly.
>>>
>>> How can I see the hive log Bejoy?  Basically I am not sure which hive
>>> log and what is the path where I can see that thing?
>>>
>>>
>>>
>>>
>>>
>>>
>>> On Fri, Aug 3, 2012 at 11:52 PM, Bejoy Ks <be...@yahoo.com> wrote:
>>>
>>>>  Hi Techy
>>>>
>>>> Are you getting the results for the same query directly from CLI, I
>>>> mean does the below query return correct results
>>>> select * from lip_data_quality where dt = '20120709';
>>>>
>>>> If the above query returns results, can you check the hive logs and see
>>>> what is the actual query being executed?
>>>>
>>>> Regards,
>>>> Bejoy KS
>>>>
>>>>   ------------------------------
>>>> *From:* Techy Teck <co...@gmail.com>
>>>> *To:* user@hive.apache.org
>>>> *Cc:* Vijay Dirisala <te...@gmail.com>
>>>> *Sent:* Saturday, August 4, 2012 12:11 PM
>>>> *Subject:* Re: Passing date as command line arguments
>>>>
>>>> Thanks Vijay for the suggestion. I also tried that and it still didn't
>>>> worked for me.
>>>>
>>>> Any thoughts why it is not working?
>>>>
>>>>
>>>>
>>>>
>>>> On Fri, Aug 3, 2012 at 11:32 PM, Vijay <te...@gmail.com> wrote:
>>>>
>>>> There can't be a space in the variable name. Try this:
>>>> ${hiveconf:start_date}
>>>>
>>>> On Fri, Aug 3, 2012 at 11:06 PM, Techy Teck <co...@gmail.com>
>>>> wrote:
>>>> > Can anyone help me out on this?
>>>> >
>>>> > I am stuck on this and I have no clue what wrong I am doing?
>>>> >
>>>> >
>>>> >
>>>> >
>>>> > On Fri, Aug 3, 2012 at 3:48 PM, Techy Teck <co...@gmail.com>
>>>> wrote:
>>>> >>
>>>> >> I have my below query in test1.hql file. I am trying to pass the
>>>> date (dt)
>>>> >> as the command line argument.
>>>> >>
>>>> >>
>>>> >> select * from lip_data_quality where dt = '${hiveconf: start_date}';
>>>> >>
>>>> >>
>>>> >> So whenever I try to run the above test1.hql file like this-
>>>> >>
>>>> >>  hive -f hivetest1.hql -hiveconf start_date=20120709
>>>> >>
>>>> >> I get zero records back. But the data is there in that table for that
>>>> >> date. Why is it so? Something wrong I am doing?
>>>> >>
>>>> >> Can anyone help me out here? I was following Bejoy's Article here
>>>> >>
>>>> http://kickstarthadoop.blogspot.com/2011/10/include-values-during-execution-time-in.html
>>>> >>
>>>> >>
>>>> >
>>>>
>>>>
>>>>
>>>>
>>>>
>>>
>>
>

Re: Passing date as command line arguments

[Techy Teck <co...@gmail.com>]

Yes it is String for me here.

This is the table structure by the way-

*create table lip_data_quality*
*( buyer_id bigint,*
*  total_chkout bigint,*
*  total_errpds bigint*
* )*
* partitioned by (dt string)*
*row format delimited fields terminated by '\t'*
*stored as sequencefile*
*location '/apps/hdmi-technology/b_apdpds/lip-data-quality'*
*;*

in which I am using dt in the where clause in my query. I am running *Hive
0.6.*



On Sat, Aug 4, 2012 at 12:24 AM, Bejoy KS <be...@yahoo.com> wrote:

> **
> I tried the same query on my end, It is working fine for me without any
> issues.
>
> By de way the data type for 'dt' is String itself right?
> Regards
> Bejoy KS
>
> Sent from handheld, please excuse typos.
> ------------------------------
> *From: * Techy Teck <co...@gmail.com>
> *Date: *Sat, 4 Aug 2012 00:21:37 -0700
> *To: *<us...@hive.apache.org>; <be...@yahoo.com>
> *ReplyTo: * user@hive.apache.org
> *Subject: *Re: Passing date as command line arguments
>
> I was not able to understand. So what changes I need to make? This is my
> below *hivetest1.hql-*
> *
> *
> *select * from lip_data_quality where dt = '${hiveconf:start_date}';*
> *
> *
> And by this I am executing the above hql file-
>
> *hive -f hivetest1.hql -hiveconf start_date=20120709*
> *
> *
> *
> *
> Any thoughts what changes I need to make?
>
>
>
>
>
>
>
> On Sat, Aug 4, 2012 at 12:16 AM, Bejoy KS <be...@yahoo.com> wrote:
>
>> **
>> Yes.
>>
>> From the logs the query being executed is
>> select * from lip_data_quality where dt=20120709
>>
>> But here the dt is not in quotes. It should be like
>> select * from lip_data_quality where dt='20120709';
>> Regards
>> Bejoy KS
>>
>> Sent from handheld, please excuse typos.
>> ------------------------------
>> *From: * Techy Teck <co...@gmail.com>
>> *Date: *Sat, 4 Aug 2012 00:07:07 -0700
>> *To: *<us...@hive.apache.org>; Bejoy Ks<be...@yahoo.com>
>> *ReplyTo: * user@hive.apache.org
>> *Subject: *Re: Passing date as command line arguments
>>
>> I found this thing from the Hive log. This is what you are looking for?-
>>
>> *SessionStart SESSION_ID="rjamal_201208041359" TIME="1344063565655"*
>> *QueryStart QUERY_STRING="select * from lip_data_quality where
>> dt=20120709"
>> QUERY_ID="rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486"
>> TIME="1344063597488"*
>> *Counters
>> plan="{"queryId":"rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486","queryType":null,"queryAttributes":{"queryString":"select
>> * from lip_data_quality where dt=20120709"},"queryCo*
>> *unters":"null","stageGraph":{"nodeType":"1","roots":"null","adjacencyList":"]"},"stageList":"]","done":"false","started":"true"}"
>> TIME="1344063597501"*
>> *QueryEnd QUERY_STRING="select * from lip_data_quality where
>> dt=20120709"
>> QUERY_ID="rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486"
>> QUERY_RET_CODE="0" QUERY_NUM_TASKS="0" TIME="134406359*
>> *7503"*
>> *Counters
>> plan="{"queryId":"rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486","queryType":null,"queryAttributes":{"queryString":"select
>> * from lip_data_quality where dt=20120709"},"queryCo*
>> *unters":"null","stageGraph":{"nodeType":"1","roots":"null","adjacencyList":"]"},"stageList":"]","done":"true","started":"true"}"
>> TIME="1344063597504"*
>> *
>> *
>> *
>> *
>>
>>
>>
>> On Fri, Aug 3, 2012 at 11:58 PM, Techy Teck <co...@gmail.com>wrote:
>>
>>> Yes that query gives me the result back correctly.
>>>
>>> How can I see the hive log Bejoy?  Basically I am not sure which hive
>>> log and what is the path where I can see that thing?
>>>
>>>
>>>
>>>
>>>
>>>
>>> On Fri, Aug 3, 2012 at 11:52 PM, Bejoy Ks <be...@yahoo.com> wrote:
>>>
>>>>  Hi Techy
>>>>
>>>> Are you getting the results for the same query directly from CLI, I
>>>> mean does the below query return correct results
>>>> select * from lip_data_quality where dt = '20120709';
>>>>
>>>> If the above query returns results, can you check the hive logs and see
>>>> what is the actual query being executed?
>>>>
>>>> Regards,
>>>> Bejoy KS
>>>>
>>>>   ------------------------------
>>>> *From:* Techy Teck <co...@gmail.com>
>>>> *To:* user@hive.apache.org
>>>> *Cc:* Vijay Dirisala <te...@gmail.com>
>>>> *Sent:* Saturday, August 4, 2012 12:11 PM
>>>> *Subject:* Re: Passing date as command line arguments
>>>>
>>>> Thanks Vijay for the suggestion. I also tried that and it still didn't
>>>> worked for me.
>>>>
>>>> Any thoughts why it is not working?
>>>>
>>>>
>>>>
>>>>
>>>> On Fri, Aug 3, 2012 at 11:32 PM, Vijay <te...@gmail.com> wrote:
>>>>
>>>> There can't be a space in the variable name. Try this:
>>>> ${hiveconf:start_date}
>>>>
>>>> On Fri, Aug 3, 2012 at 11:06 PM, Techy Teck <co...@gmail.com>
>>>> wrote:
>>>> > Can anyone help me out on this?
>>>> >
>>>> > I am stuck on this and I have no clue what wrong I am doing?
>>>> >
>>>> >
>>>> >
>>>> >
>>>> > On Fri, Aug 3, 2012 at 3:48 PM, Techy Teck <co...@gmail.com>
>>>> wrote:
>>>> >>
>>>> >> I have my below query in test1.hql file. I am trying to pass the
>>>> date (dt)
>>>> >> as the command line argument.
>>>> >>
>>>> >>
>>>> >> select * from lip_data_quality where dt = '${hiveconf: start_date}';
>>>> >>
>>>> >>
>>>> >> So whenever I try to run the above test1.hql file like this-
>>>> >>
>>>> >>  hive -f hivetest1.hql -hiveconf start_date=20120709
>>>> >>
>>>> >> I get zero records back. But the data is there in that table for that
>>>> >> date. Why is it so? Something wrong I am doing?
>>>> >>
>>>> >> Can anyone help me out here? I was following Bejoy's Article here
>>>> >>
>>>> http://kickstarthadoop.blogspot.com/2011/10/include-values-during-execution-time-in.html
>>>> >>
>>>> >>
>>>> >
>>>>
>>>>
>>>>
>>>>
>>>>
>>>
>>
>

Re: Passing date as command line arguments

[Bejoy KS <be...@yahoo.com>]

I tried the same query on my end, It is working fine for me without any issues.

By de way the data type for 'dt' is String itself right? 

Regards
Bejoy KS

Sent from handheld, please excuse typos.

-----Original Message-----
From: Techy Teck <co...@gmail.com>
Date: Sat, 4 Aug 2012 00:21:37 
To: <us...@hive.apache.org>; <be...@yahoo.com>
Reply-To: user@hive.apache.org
Subject: Re: Passing date as command line arguments

I was not able to understand. So what changes I need to make? This is my
below *hivetest1.hql-*
*
*
*select * from lip_data_quality where dt = '${hiveconf:start_date}';*
*
*
And by this I am executing the above hql file-

*hive -f hivetest1.hql -hiveconf start_date=20120709*
*
*
*
*
Any thoughts what changes I need to make?







On Sat, Aug 4, 2012 at 12:16 AM, Bejoy KS <be...@yahoo.com> wrote:

> **
> Yes.
>
> From the logs the query being executed is
> select * from lip_data_quality where dt=20120709
>
> But here the dt is not in quotes. It should be like
> select * from lip_data_quality where dt='20120709';
> Regards
> Bejoy KS
>
> Sent from handheld, please excuse typos.
> ------------------------------
> *From: * Techy Teck <co...@gmail.com>
> *Date: *Sat, 4 Aug 2012 00:07:07 -0700
> *To: *<us...@hive.apache.org>; Bejoy Ks<be...@yahoo.com>
> *ReplyTo: * user@hive.apache.org
> *Subject: *Re: Passing date as command line arguments
>
> I found this thing from the Hive log. This is what you are looking for?-
>
> *SessionStart SESSION_ID="rjamal_201208041359" TIME="1344063565655"*
> *QueryStart QUERY_STRING="select * from lip_data_quality where
> dt=20120709"
> QUERY_ID="rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486"
> TIME="1344063597488"*
> *Counters
> plan="{"queryId":"rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486","queryType":null,"queryAttributes":{"queryString":"select
> * from lip_data_quality where dt=20120709"},"queryCo*
> *unters":"null","stageGraph":{"nodeType":"1","roots":"null","adjacencyList":"]"},"stageList":"]","done":"false","started":"true"}"
> TIME="1344063597501"*
> *QueryEnd QUERY_STRING="select * from lip_data_quality where dt=20120709"
> QUERY_ID="rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486"
> QUERY_RET_CODE="0" QUERY_NUM_TASKS="0" TIME="134406359*
> *7503"*
> *Counters
> plan="{"queryId":"rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486","queryType":null,"queryAttributes":{"queryString":"select
> * from lip_data_quality where dt=20120709"},"queryCo*
> *unters":"null","stageGraph":{"nodeType":"1","roots":"null","adjacencyList":"]"},"stageList":"]","done":"true","started":"true"}"
> TIME="1344063597504"*
> *
> *
> *
> *
>
>
>
> On Fri, Aug 3, 2012 at 11:58 PM, Techy Teck <co...@gmail.com>wrote:
>
>> Yes that query gives me the result back correctly.
>>
>> How can I see the hive log Bejoy?  Basically I am not sure which hive log
>> and what is the path where I can see that thing?
>>
>>
>>
>>
>>
>>
>> On Fri, Aug 3, 2012 at 11:52 PM, Bejoy Ks <be...@yahoo.com> wrote:
>>
>>>  Hi Techy
>>>
>>> Are you getting the results for the same query directly from CLI, I mean
>>> does the below query return correct results
>>> select * from lip_data_quality where dt = '20120709';
>>>
>>> If the above query returns results, can you check the hive logs and see
>>> what is the actual query being executed?
>>>
>>> Regards,
>>> Bejoy KS
>>>
>>>   ------------------------------
>>> *From:* Techy Teck <co...@gmail.com>
>>> *To:* user@hive.apache.org
>>> *Cc:* Vijay Dirisala <te...@gmail.com>
>>> *Sent:* Saturday, August 4, 2012 12:11 PM
>>> *Subject:* Re: Passing date as command line arguments
>>>
>>> Thanks Vijay for the suggestion. I also tried that and it still didn't
>>> worked for me.
>>>
>>> Any thoughts why it is not working?
>>>
>>>
>>>
>>>
>>> On Fri, Aug 3, 2012 at 11:32 PM, Vijay <te...@gmail.com> wrote:
>>>
>>> There can't be a space in the variable name. Try this:
>>> ${hiveconf:start_date}
>>>
>>> On Fri, Aug 3, 2012 at 11:06 PM, Techy Teck <co...@gmail.com>
>>> wrote:
>>> > Can anyone help me out on this?
>>> >
>>> > I am stuck on this and I have no clue what wrong I am doing?
>>> >
>>> >
>>> >
>>> >
>>> > On Fri, Aug 3, 2012 at 3:48 PM, Techy Teck <co...@gmail.com>
>>> wrote:
>>> >>
>>> >> I have my below query in test1.hql file. I am trying to pass the date
>>> (dt)
>>> >> as the command line argument.
>>> >>
>>> >>
>>> >> select * from lip_data_quality where dt = '${hiveconf: start_date}';
>>> >>
>>> >>
>>> >> So whenever I try to run the above test1.hql file like this-
>>> >>
>>> >>  hive -f hivetest1.hql -hiveconf start_date=20120709
>>> >>
>>> >> I get zero records back. But the data is there in that table for that
>>> >> date. Why is it so? Something wrong I am doing?
>>> >>
>>> >> Can anyone help me out here? I was following Bejoy's Article here
>>> >>
>>> http://kickstarthadoop.blogspot.com/2011/10/include-values-during-execution-time-in.html
>>> >>
>>> >>
>>> >
>>>
>>>
>>>
>>>
>>>
>>
>

Re: Passing date as command line arguments

[Techy Teck <co...@gmail.com>]

I was not able to understand. So what changes I need to make? This is my
below *hivetest1.hql-*
*
*
*select * from lip_data_quality where dt = '${hiveconf:start_date}';*
*
*
And by this I am executing the above hql file-

*hive -f hivetest1.hql -hiveconf start_date=20120709*
*
*
*
*
Any thoughts what changes I need to make?







On Sat, Aug 4, 2012 at 12:16 AM, Bejoy KS <be...@yahoo.com> wrote:

> **
> Yes.
>
> From the logs the query being executed is
> select * from lip_data_quality where dt=20120709
>
> But here the dt is not in quotes. It should be like
> select * from lip_data_quality where dt='20120709';
> Regards
> Bejoy KS
>
> Sent from handheld, please excuse typos.
> ------------------------------
> *From: * Techy Teck <co...@gmail.com>
> *Date: *Sat, 4 Aug 2012 00:07:07 -0700
> *To: *<us...@hive.apache.org>; Bejoy Ks<be...@yahoo.com>
> *ReplyTo: * user@hive.apache.org
> *Subject: *Re: Passing date as command line arguments
>
> I found this thing from the Hive log. This is what you are looking for?-
>
> *SessionStart SESSION_ID="rjamal_201208041359" TIME="1344063565655"*
> *QueryStart QUERY_STRING="select * from lip_data_quality where
> dt=20120709"
> QUERY_ID="rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486"
> TIME="1344063597488"*
> *Counters
> plan="{"queryId":"rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486","queryType":null,"queryAttributes":{"queryString":"select
> * from lip_data_quality where dt=20120709"},"queryCo*
> *unters":"null","stageGraph":{"nodeType":"1","roots":"null","adjacencyList":"]"},"stageList":"]","done":"false","started":"true"}"
> TIME="1344063597501"*
> *QueryEnd QUERY_STRING="select * from lip_data_quality where dt=20120709"
> QUERY_ID="rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486"
> QUERY_RET_CODE="0" QUERY_NUM_TASKS="0" TIME="134406359*
> *7503"*
> *Counters
> plan="{"queryId":"rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486","queryType":null,"queryAttributes":{"queryString":"select
> * from lip_data_quality where dt=20120709"},"queryCo*
> *unters":"null","stageGraph":{"nodeType":"1","roots":"null","adjacencyList":"]"},"stageList":"]","done":"true","started":"true"}"
> TIME="1344063597504"*
> *
> *
> *
> *
>
>
>
> On Fri, Aug 3, 2012 at 11:58 PM, Techy Teck <co...@gmail.com>wrote:
>
>> Yes that query gives me the result back correctly.
>>
>> How can I see the hive log Bejoy?  Basically I am not sure which hive log
>> and what is the path where I can see that thing?
>>
>>
>>
>>
>>
>>
>> On Fri, Aug 3, 2012 at 11:52 PM, Bejoy Ks <be...@yahoo.com> wrote:
>>
>>>  Hi Techy
>>>
>>> Are you getting the results for the same query directly from CLI, I mean
>>> does the below query return correct results
>>> select * from lip_data_quality where dt = '20120709';
>>>
>>> If the above query returns results, can you check the hive logs and see
>>> what is the actual query being executed?
>>>
>>> Regards,
>>> Bejoy KS
>>>
>>>   ------------------------------
>>> *From:* Techy Teck <co...@gmail.com>
>>> *To:* user@hive.apache.org
>>> *Cc:* Vijay Dirisala <te...@gmail.com>
>>> *Sent:* Saturday, August 4, 2012 12:11 PM
>>> *Subject:* Re: Passing date as command line arguments
>>>
>>> Thanks Vijay for the suggestion. I also tried that and it still didn't
>>> worked for me.
>>>
>>> Any thoughts why it is not working?
>>>
>>>
>>>
>>>
>>> On Fri, Aug 3, 2012 at 11:32 PM, Vijay <te...@gmail.com> wrote:
>>>
>>> There can't be a space in the variable name. Try this:
>>> ${hiveconf:start_date}
>>>
>>> On Fri, Aug 3, 2012 at 11:06 PM, Techy Teck <co...@gmail.com>
>>> wrote:
>>> > Can anyone help me out on this?
>>> >
>>> > I am stuck on this and I have no clue what wrong I am doing?
>>> >
>>> >
>>> >
>>> >
>>> > On Fri, Aug 3, 2012 at 3:48 PM, Techy Teck <co...@gmail.com>
>>> wrote:
>>> >>
>>> >> I have my below query in test1.hql file. I am trying to pass the date
>>> (dt)
>>> >> as the command line argument.
>>> >>
>>> >>
>>> >> select * from lip_data_quality where dt = '${hiveconf: start_date}';
>>> >>
>>> >>
>>> >> So whenever I try to run the above test1.hql file like this-
>>> >>
>>> >>  hive -f hivetest1.hql -hiveconf start_date=20120709
>>> >>
>>> >> I get zero records back. But the data is there in that table for that
>>> >> date. Why is it so? Something wrong I am doing?
>>> >>
>>> >> Can anyone help me out here? I was following Bejoy's Article here
>>> >>
>>> http://kickstarthadoop.blogspot.com/2011/10/include-values-during-execution-time-in.html
>>> >>
>>> >>
>>> >
>>>
>>>
>>>
>>>
>>>
>>
>

Re: Passing date as command line arguments

[Bejoy KS <be...@yahoo.com>]

Yes.

From the logs the query being executed is 
select * from lip_data_quality where dt=20120709

But here the dt is not in quotes. It should be like
select * from lip_data_quality where dt='20120709';
 
Regards
Bejoy KS

Sent from handheld, please excuse typos.

-----Original Message-----
From: Techy Teck <co...@gmail.com>
Date: Sat, 4 Aug 2012 00:07:07 
To: <us...@hive.apache.org>; Bejoy Ks<be...@yahoo.com>
Reply-To: user@hive.apache.org
Subject: Re: Passing date as command line arguments

I found this thing from the Hive log. This is what you are looking for?-

*SessionStart SESSION_ID="rjamal_201208041359" TIME="1344063565655"*
*QueryStart QUERY_STRING="select * from lip_data_quality where dt=20120709"
QUERY_ID="rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486"
TIME="1344063597488"*
*Counters
plan="{"queryId":"rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486","queryType":null,"queryAttributes":{"queryString":"select
* from lip_data_quality where dt=20120709"},"queryCo*
*unters":"null","stageGraph":{"nodeType":"1","roots":"null","adjacencyList":"]"},"stageList":"]","done":"false","started":"true"}"
TIME="1344063597501"*
*QueryEnd QUERY_STRING="select * from lip_data_quality where dt=20120709"
QUERY_ID="rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486"
QUERY_RET_CODE="0" QUERY_NUM_TASKS="0" TIME="134406359*
*7503"*
*Counters
plan="{"queryId":"rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486","queryType":null,"queryAttributes":{"queryString":"select
* from lip_data_quality where dt=20120709"},"queryCo*
*unters":"null","stageGraph":{"nodeType":"1","roots":"null","adjacencyList":"]"},"stageList":"]","done":"true","started":"true"}"
TIME="1344063597504"*
*
*
*
*



On Fri, Aug 3, 2012 at 11:58 PM, Techy Teck <co...@gmail.com> wrote:

> Yes that query gives me the result back correctly.
>
> How can I see the hive log Bejoy?  Basically I am not sure which hive log
> and what is the path where I can see that thing?
>
>
>
>
>
>
> On Fri, Aug 3, 2012 at 11:52 PM, Bejoy Ks <be...@yahoo.com> wrote:
>
>>  Hi Techy
>>
>> Are you getting the results for the same query directly from CLI, I mean
>> does the below query return correct results
>> select * from lip_data_quality where dt = '20120709';
>>
>> If the above query returns results, can you check the hive logs and see
>> what is the actual query being executed?
>>
>> Regards,
>> Bejoy KS
>>
>>   ------------------------------
>> *From:* Techy Teck <co...@gmail.com>
>> *To:* user@hive.apache.org
>> *Cc:* Vijay Dirisala <te...@gmail.com>
>> *Sent:* Saturday, August 4, 2012 12:11 PM
>> *Subject:* Re: Passing date as command line arguments
>>
>> Thanks Vijay for the suggestion. I also tried that and it still didn't
>> worked for me.
>>
>> Any thoughts why it is not working?
>>
>>
>>
>>
>> On Fri, Aug 3, 2012 at 11:32 PM, Vijay <te...@gmail.com> wrote:
>>
>> There can't be a space in the variable name. Try this:
>> ${hiveconf:start_date}
>>
>> On Fri, Aug 3, 2012 at 11:06 PM, Techy Teck <co...@gmail.com>
>> wrote:
>> > Can anyone help me out on this?
>> >
>> > I am stuck on this and I have no clue what wrong I am doing?
>> >
>> >
>> >
>> >
>> > On Fri, Aug 3, 2012 at 3:48 PM, Techy Teck <co...@gmail.com>
>> wrote:
>> >>
>> >> I have my below query in test1.hql file. I am trying to pass the date
>> (dt)
>> >> as the command line argument.
>> >>
>> >>
>> >> select * from lip_data_quality where dt = '${hiveconf: start_date}';
>> >>
>> >>
>> >> So whenever I try to run the above test1.hql file like this-
>> >>
>> >>  hive -f hivetest1.hql -hiveconf start_date=20120709
>> >>
>> >> I get zero records back. But the data is there in that table for that
>> >> date. Why is it so? Something wrong I am doing?
>> >>
>> >> Can anyone help me out here? I was following Bejoy's Article here
>> >>
>> http://kickstarthadoop.blogspot.com/2011/10/include-values-during-execution-time-in.html
>> >>
>> >>
>> >
>>
>>
>>
>>
>>
>

Re: Passing date as command line arguments

[Techy Teck <co...@gmail.com>]

I found this thing from the Hive log. This is what you are looking for?-

*SessionStart SESSION_ID="rjamal_201208041359" TIME="1344063565655"*
*QueryStart QUERY_STRING="select * from lip_data_quality where dt=20120709"
QUERY_ID="rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486"
TIME="1344063597488"*
*Counters
plan="{"queryId":"rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486","queryType":null,"queryAttributes":{"queryString":"select
* from lip_data_quality where dt=20120709"},"queryCo*
*unters":"null","stageGraph":{"nodeType":"1","roots":"null","adjacencyList":"]"},"stageList":"]","done":"false","started":"true"}"
TIME="1344063597501"*
*QueryEnd QUERY_STRING="select * from lip_data_quality where dt=20120709"
QUERY_ID="rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486"
QUERY_RET_CODE="0" QUERY_NUM_TASKS="0" TIME="134406359*
*7503"*
*Counters
plan="{"queryId":"rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486","queryType":null,"queryAttributes":{"queryString":"select
* from lip_data_quality where dt=20120709"},"queryCo*
*unters":"null","stageGraph":{"nodeType":"1","roots":"null","adjacencyList":"]"},"stageList":"]","done":"true","started":"true"}"
TIME="1344063597504"*
*
*
*
*



On Fri, Aug 3, 2012 at 11:58 PM, Techy Teck <co...@gmail.com> wrote:

> Yes that query gives me the result back correctly.
>
> How can I see the hive log Bejoy?  Basically I am not sure which hive log
> and what is the path where I can see that thing?
>
>
>
>
>
>
> On Fri, Aug 3, 2012 at 11:52 PM, Bejoy Ks <be...@yahoo.com> wrote:
>
>>  Hi Techy
>>
>> Are you getting the results for the same query directly from CLI, I mean
>> does the below query return correct results
>> select * from lip_data_quality where dt = '20120709';
>>
>> If the above query returns results, can you check the hive logs and see
>> what is the actual query being executed?
>>
>> Regards,
>> Bejoy KS
>>
>>   ------------------------------
>> *From:* Techy Teck <co...@gmail.com>
>> *To:* user@hive.apache.org
>> *Cc:* Vijay Dirisala <te...@gmail.com>
>> *Sent:* Saturday, August 4, 2012 12:11 PM
>> *Subject:* Re: Passing date as command line arguments
>>
>> Thanks Vijay for the suggestion. I also tried that and it still didn't
>> worked for me.
>>
>> Any thoughts why it is not working?
>>
>>
>>
>>
>> On Fri, Aug 3, 2012 at 11:32 PM, Vijay <te...@gmail.com> wrote:
>>
>> There can't be a space in the variable name. Try this:
>> ${hiveconf:start_date}
>>
>> On Fri, Aug 3, 2012 at 11:06 PM, Techy Teck <co...@gmail.com>
>> wrote:
>> > Can anyone help me out on this?
>> >
>> > I am stuck on this and I have no clue what wrong I am doing?
>> >
>> >
>> >
>> >
>> > On Fri, Aug 3, 2012 at 3:48 PM, Techy Teck <co...@gmail.com>
>> wrote:
>> >>
>> >> I have my below query in test1.hql file. I am trying to pass the date
>> (dt)
>> >> as the command line argument.
>> >>
>> >>
>> >> select * from lip_data_quality where dt = '${hiveconf: start_date}';
>> >>
>> >>
>> >> So whenever I try to run the above test1.hql file like this-
>> >>
>> >>  hive -f hivetest1.hql -hiveconf start_date=20120709
>> >>
>> >> I get zero records back. But the data is there in that table for that
>> >> date. Why is it so? Something wrong I am doing?
>> >>
>> >> Can anyone help me out here? I was following Bejoy's Article here
>> >>
>> http://kickstarthadoop.blogspot.com/2011/10/include-values-during-execution-time-in.html
>> >>
>> >>
>> >
>>
>>
>>
>>
>>
>

Re: Passing date as command line arguments

[Techy Teck <co...@gmail.com>]

Yes that query gives me the result back correctly.

How can I see the hive log Bejoy?  Basically I am not sure which hive log
and what is the path where I can see that thing?






On Fri, Aug 3, 2012 at 11:52 PM, Bejoy Ks <be...@yahoo.com> wrote:

> Hi Techy
>
> Are you getting the results for the same query directly from CLI, I mean
> does the below query return correct results
> select * from lip_data_quality where dt = '20120709';
>
> If the above query returns results, can you check the hive logs and see
> what is the actual query being executed?
>
> Regards,
> Bejoy KS
>
>   ------------------------------
> *From:* Techy Teck <co...@gmail.com>
> *To:* user@hive.apache.org
> *Cc:* Vijay Dirisala <te...@gmail.com>
> *Sent:* Saturday, August 4, 2012 12:11 PM
> *Subject:* Re: Passing date as command line arguments
>
> Thanks Vijay for the suggestion. I also tried that and it still didn't
> worked for me.
>
> Any thoughts why it is not working?
>
>
>
>
> On Fri, Aug 3, 2012 at 11:32 PM, Vijay <te...@gmail.com> wrote:
>
> There can't be a space in the variable name. Try this:
> ${hiveconf:start_date}
>
> On Fri, Aug 3, 2012 at 11:06 PM, Techy Teck <co...@gmail.com>
> wrote:
> > Can anyone help me out on this?
> >
> > I am stuck on this and I have no clue what wrong I am doing?
> >
> >
> >
> >
> > On Fri, Aug 3, 2012 at 3:48 PM, Techy Teck <co...@gmail.com>
> wrote:
> >>
> >> I have my below query in test1.hql file. I am trying to pass the date
> (dt)
> >> as the command line argument.
> >>
> >>
> >> select * from lip_data_quality where dt = '${hiveconf: start_date}';
> >>
> >>
> >> So whenever I try to run the above test1.hql file like this-
> >>
> >>  hive -f hivetest1.hql -hiveconf start_date=20120709
> >>
> >> I get zero records back. But the data is there in that table for that
> >> date. Why is it so? Something wrong I am doing?
> >>
> >> Can anyone help me out here? I was following Bejoy's Article here
> >>
> http://kickstarthadoop.blogspot.com/2011/10/include-values-during-execution-time-in.html
> >>
> >>
> >
>
>
>
>
>

Re: Passing date as command line arguments

[Bejoy Ks <be...@yahoo.com>]

Hi Techy

Are you getting the results for the same query directly from CLI, I mean does the below query return correct results
select * from lip_data_quality where dt = '20120709'; 


If the above query returns results, can you check the hive logs and see what is the actual query being executed?
 
Regards,
Bejoy KS


________________________________
 From: Techy Teck <co...@gmail.com>
To: user@hive.apache.org 
Cc: Vijay Dirisala <te...@gmail.com> 
Sent: Saturday, August 4, 2012 12:11 PM
Subject: Re: Passing date as command line arguments
 

Thanks Vijay for the suggestion. I also tried that and it still didn't worked for me.

Any thoughts why it is not working?





On Fri, Aug 3, 2012 at 11:32 PM, Vijay <te...@gmail.com> wrote:

There can't be a space in the variable name. Try this: ${hiveconf:start_date}
>
>
>On Fri, Aug 3, 2012 at 11:06 PM, Techy Teck <co...@gmail.com> wrote:
>> Can anyone help me out on this?
>>
>> I am stuck on this and I have no clue what wrong I am doing?
>>
>>
>>
>>
>> On Fri, Aug 3, 2012 at 3:48 PM, Techy Teck <co...@gmail.com> wrote:
>>>
>>> I have my below query in test1.hql file. I am trying to pass the date (dt)
>>> as the command line argument.
>>>
>>>
>>> select * from lip_data_quality where dt = '${hiveconf: start_date}';
>>>
>>>
>>> So whenever I try to run the above test1.hql file like this-
>>>
>>>  hive -f hivetest1.hql -hiveconf start_date=20120709
>>>
>>> I get zero records back. But the data is there in that table for that
>>> date. Why is it so? Something wrong I am doing?
>>>
>>> Can anyone help me out here? I was following Bejoy's Article here
>>> http://kickstarthadoop.blogspot.com/2011/10/include-values-during-execution-time-in.html
>>>
>>>
>>
>

Re: Passing date as command line arguments

[Techy Teck <co...@gmail.com>]

Thanks Vijay for the suggestion. I also tried that and it still didn't
worked for me.

Any thoughts why it is not working?




On Fri, Aug 3, 2012 at 11:32 PM, Vijay <te...@gmail.com> wrote:

> There can't be a space in the variable name. Try this:
> ${hiveconf:start_date}
>
> On Fri, Aug 3, 2012 at 11:06 PM, Techy Teck <co...@gmail.com>
> wrote:
> > Can anyone help me out on this?
> >
> > I am stuck on this and I have no clue what wrong I am doing?
> >
> >
> >
> >
> > On Fri, Aug 3, 2012 at 3:48 PM, Techy Teck <co...@gmail.com>
> wrote:
> >>
> >> I have my below query in test1.hql file. I am trying to pass the date
> (dt)
> >> as the command line argument.
> >>
> >>
> >> select * from lip_data_quality where dt = '${hiveconf: start_date}';
> >>
> >>
> >> So whenever I try to run the above test1.hql file like this-
> >>
> >>  hive -f hivetest1.hql -hiveconf start_date=20120709
> >>
> >> I get zero records back. But the data is there in that table for that
> >> date. Why is it so? Something wrong I am doing?
> >>
> >> Can anyone help me out here? I was following Bejoy's Article here
> >>
> http://kickstarthadoop.blogspot.com/2011/10/include-values-during-execution-time-in.html
> >>
> >>
> >
>

Re: Passing date as command line arguments

[Vijay <te...@gmail.com>]

There can't be a space in the variable name. Try this: ${hiveconf:start_date}

On Fri, Aug 3, 2012 at 11:06 PM, Techy Teck <co...@gmail.com> wrote:
> Can anyone help me out on this?
>
> I am stuck on this and I have no clue what wrong I am doing?
>
>
>
>
> On Fri, Aug 3, 2012 at 3:48 PM, Techy Teck <co...@gmail.com> wrote:
>>
>> I have my below query in test1.hql file. I am trying to pass the date (dt)
>> as the command line argument.
>>
>>
>> select * from lip_data_quality where dt = '${hiveconf: start_date}';
>>
>>
>> So whenever I try to run the above test1.hql file like this-
>>
>>  hive -f hivetest1.hql -hiveconf start_date=20120709
>>
>> I get zero records back. But the data is there in that table for that
>> date. Why is it so? Something wrong I am doing?
>>
>> Can anyone help me out here? I was following Bejoy's Article here
>> http://kickstarthadoop.blogspot.com/2011/10/include-values-during-execution-time-in.html
>>
>>
>

Re: Passing date as command line arguments

[Techy Teck <co...@gmail.com>]

Can anyone help me out on this?

I am stuck on this and I have no clue what wrong I am doing?




On Fri, Aug 3, 2012 at 3:48 PM, Techy Teck <co...@gmail.com> wrote:

> I have my below query in *test1.hql file. *I am trying to pass the date *
> (dt)* as the command line argument.
>
>
> *select * from lip_data_quality where dt = '${hiveconf: start_date}';*
>
>
> So whenever I try to run the above *test1.hql file like this-*
> *
> *
> * hive -f hivetest1.hql -hiveconf start_date=20120709*
> *
> *
> I get zero records back. But the data is there in that table for that
> date. Why is it so? Something wrong I am doing?
>
> Can anyone help me out here? I was following Bejoy's Article here
> http://kickstarthadoop.blogspot.com/2011/10/include-values-during-execution-time-in.html
>
>
>

Re: Passing date as command line arguments

[Techy Teck <co...@gmail.com>]

I tried that. But it's not working for me. I didn't got any result.

Any other suggestion?


On Fri, Aug 3, 2012 at 7:41 PM, Matt Tucker <ma...@gmail.com> wrote:

> Yes, that should do the trick for you.
>
>
>
> On Aug 3, 2012, at 10:17 PM, Techy Teck <co...@gmail.com> wrote:
>
> So you are saying, I should invoke like this below?
>
> * hive -f hivetest1.hql -hiveconf start_date="20120709"*
> *
> *
> *
> *
>
>
>
> On Fri, Aug 3, 2012 at 7:15 PM, Matt Tucker <ma...@gmail.com> wrote:
>
>> Hi,
>>
>> In the command line, you want to wrap 20120709 in double-quotes, as they
>> get stripped when being passed into the hiveconf variable.
>>
>> Matt
>>
>>
>>
>> On Aug 3, 2012, at 6:48 PM, Techy Teck <co...@gmail.com> wrote:
>>
>> I have my below query in *test1.hql file. *I am trying to pass the date *
>> (dt)* as the command line argument.
>>
>>
>> *select * from lip_data_quality where dt = '${hiveconf: start_date}';*
>>
>>
>> So whenever I try to run the above *test1.hql file like this-*
>> *
>> *
>> * hive -f hivetest1.hql -hiveconf start_date=20120709*
>> *
>> *
>> I get zero records back. But the data is there in that table for that
>> date. Why is it so? Something wrong I am doing?
>>
>> Can anyone help me out here? I was following Bejoy's Article here
>> http://kickstarthadoop.blogspot.com/2011/10/include-values-during-execution-time-in.html
>>
>>
>>
>

Re: Passing date as command line arguments

[Matt Tucker <ma...@gmail.com>]

Hi,

In the command line, you want to wrap 20120709 in double-quotes, as they get stripped when being passed into the hiveconf variable.

Matt



On Aug 3, 2012, at 6:48 PM, Techy Teck <co...@gmail.com> wrote:

> I have my below query in test1.hql file. I am trying to pass the date (dt) as the command line argument.
> 
> 
> select * from lip_data_quality where dt = '${hiveconf: start_date}';
> 
> 
> So whenever I try to run the above test1.hql file like this-
> 
>  hive -f hivetest1.hql -hiveconf start_date=20120709
> 
> I get zero records back. But the data is there in that table for that date. Why is it so? Something wrong I am doing?
> 
> Can anyone help me out here? I was following Bejoy's Article here http://kickstarthadoop.blogspot.com/2011/10/include-values-during-execution-time-in.html
> 
>