You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by Alex Rovner <al...@gmail.com> on 2012/08/22 20:28:11 UTC
Issues with Bincond
I am having trouble with bincond in pig 11.
Sample input:
1234
0
1234
Sample pig script:
a = LOAD 'input.txt' as (col1:int);
b = FOREACH a GENERATE col1, (col1 == null ? 'null' : 'not-null') as col2;
dump b;
Output:
(1234,)
(0,)
(1234,)
Certainly not what you expect to see... I expected to see 'not-null' string
in the second column.
If I change the bincond to look for a particular value then everything
works as expected:
b = FOREACH a GENERATE col1, (col1 == 1234 ? 'null' : 'not-null') as col2;
Output:
(1234,null)
(0,not-null)
(1234,null)
Any ideas? I did not get a chance to test this with prior versions.
Thanks
Alex
Re: Issues with Bincond
Posted by Alex Rovner <al...@gmail.com>.
Thanks Alan for the tip.
Sent from my iPhone
On Aug 22, 2012, at 4:42 PM, Alan Gates <ga...@hortonworks.com> wrote:
> Use "is null" instead of "== null". Equality, inequality, boolean, and arithmetic operators that encounter a null returning null is standard trinary logic. The only possible answer to "is this equal to an unknown" is "unknown".
>
> Alan.
>
> On Aug 22, 2012, at 11:43 AM, Alex Rovner wrote:
>
>> Thanks Cheolsoo,
>>
>> Not very intuitive but makes sense.
>>
>>
>> On Wed, Aug 22, 2012 at 2:37 PM, Cheolsoo Park <ch...@cloudera.com>wrote:
>>
>>> Hi Alex,
>>>
>>> I think that that's expected. The Pig manual says the following
>>> regarding comparison
>>> operators (e.g. ==):
>>>
>>> If either sub-expression is null, the result is null.
>>>
>>>
>>> So "col1 == null" is null.
>>>
>>> Now it also says the following regarding arithmetic operators (e.g. ?):
>>>
>>> If either sub-expression is null, the resulting expression is null.
>>>
>>>
>>> So "col1 == null ? 'null' : 'not-null'" is null as "col1 == null" is null.
>>>
>>> Here is the link:
>>> http://pig.apache.org/docs/r0.10.0/basic.html#nulls
>>>
>>> Thanks,
>>> Cheolsoo
>>>
>>> On Wed, Aug 22, 2012 at 11:28 AM, Alex Rovner <al...@gmail.com>
>>> wrote:
>>>
>>>> I am having trouble with bincond in pig 11.
>>>>
>>>> Sample input:
>>>> 1234
>>>> 0
>>>> 1234
>>>>
>>>> Sample pig script:
>>>> a = LOAD 'input.txt' as (col1:int);
>>>>
>>>> b = FOREACH a GENERATE col1, (col1 == null ? 'null' : 'not-null') as
>>> col2;
>>>>
>>>> dump b;
>>>>
>>>>
>>>> Output:
>>>> (1234,)
>>>> (0,)
>>>> (1234,)
>>>>
>>>>
>>>> Certainly not what you expect to see... I expected to see 'not-null'
>>> string
>>>> in the second column.
>>>> If I change the bincond to look for a particular value then everything
>>>> works as expected:
>>>>
>>>> b = FOREACH a GENERATE col1, (col1 == 1234 ? 'null' : 'not-null') as
>>> col2;
>>>>
>>>> Output:
>>>> (1234,null)
>>>> (0,not-null)
>>>> (1234,null)
>>>>
>>>>
>>>> Any ideas? I did not get a chance to test this with prior versions.
>>>>
>>>> Thanks
>>>> Alex
>>>>
>>>
>
Re: Issues with Bincond
Posted by Alan Gates <ga...@hortonworks.com>.
Use "is null" instead of "== null". Equality, inequality, boolean, and arithmetic operators that encounter a null returning null is standard trinary logic. The only possible answer to "is this equal to an unknown" is "unknown".
Alan.
On Aug 22, 2012, at 11:43 AM, Alex Rovner wrote:
> Thanks Cheolsoo,
>
> Not very intuitive but makes sense.
>
>
> On Wed, Aug 22, 2012 at 2:37 PM, Cheolsoo Park <ch...@cloudera.com>wrote:
>
>> Hi Alex,
>>
>> I think that that's expected. The Pig manual says the following
>> regarding comparison
>> operators (e.g. ==):
>>
>> If either sub-expression is null, the result is null.
>>
>>
>> So "col1 == null" is null.
>>
>> Now it also says the following regarding arithmetic operators (e.g. ?):
>>
>> If either sub-expression is null, the resulting expression is null.
>>
>>
>> So "col1 == null ? 'null' : 'not-null'" is null as "col1 == null" is null.
>>
>> Here is the link:
>> http://pig.apache.org/docs/r0.10.0/basic.html#nulls
>>
>> Thanks,
>> Cheolsoo
>>
>> On Wed, Aug 22, 2012 at 11:28 AM, Alex Rovner <al...@gmail.com>
>> wrote:
>>
>>> I am having trouble with bincond in pig 11.
>>>
>>> Sample input:
>>> 1234
>>> 0
>>> 1234
>>>
>>> Sample pig script:
>>> a = LOAD 'input.txt' as (col1:int);
>>>
>>> b = FOREACH a GENERATE col1, (col1 == null ? 'null' : 'not-null') as
>> col2;
>>>
>>> dump b;
>>>
>>>
>>> Output:
>>> (1234,)
>>> (0,)
>>> (1234,)
>>>
>>>
>>> Certainly not what you expect to see... I expected to see 'not-null'
>> string
>>> in the second column.
>>> If I change the bincond to look for a particular value then everything
>>> works as expected:
>>>
>>> b = FOREACH a GENERATE col1, (col1 == 1234 ? 'null' : 'not-null') as
>> col2;
>>>
>>> Output:
>>> (1234,null)
>>> (0,not-null)
>>> (1234,null)
>>>
>>>
>>> Any ideas? I did not get a chance to test this with prior versions.
>>>
>>> Thanks
>>> Alex
>>>
>>
Re: Issues with Bincond
Posted by Alex Rovner <al...@gmail.com>.
Thanks Cheolsoo,
Not very intuitive but makes sense.
On Wed, Aug 22, 2012 at 2:37 PM, Cheolsoo Park <ch...@cloudera.com>wrote:
> Hi Alex,
>
> I think that that's expected. The Pig manual says the following
> regarding comparison
> operators (e.g. ==):
>
> If either sub-expression is null, the result is null.
>
>
> So "col1 == null" is null.
>
> Now it also says the following regarding arithmetic operators (e.g. ?):
>
> If either sub-expression is null, the resulting expression is null.
>
>
> So "col1 == null ? 'null' : 'not-null'" is null as "col1 == null" is null.
>
> Here is the link:
> http://pig.apache.org/docs/r0.10.0/basic.html#nulls
>
> Thanks,
> Cheolsoo
>
> On Wed, Aug 22, 2012 at 11:28 AM, Alex Rovner <al...@gmail.com>
> wrote:
>
> > I am having trouble with bincond in pig 11.
> >
> > Sample input:
> > 1234
> > 0
> > 1234
> >
> > Sample pig script:
> > a = LOAD 'input.txt' as (col1:int);
> >
> > b = FOREACH a GENERATE col1, (col1 == null ? 'null' : 'not-null') as
> col2;
> >
> > dump b;
> >
> >
> > Output:
> > (1234,)
> > (0,)
> > (1234,)
> >
> >
> > Certainly not what you expect to see... I expected to see 'not-null'
> string
> > in the second column.
> > If I change the bincond to look for a particular value then everything
> > works as expected:
> >
> > b = FOREACH a GENERATE col1, (col1 == 1234 ? 'null' : 'not-null') as
> col2;
> >
> > Output:
> > (1234,null)
> > (0,not-null)
> > (1234,null)
> >
> >
> > Any ideas? I did not get a chance to test this with prior versions.
> >
> > Thanks
> > Alex
> >
>
Re: Issues with Bincond
Posted by Cheolsoo Park <ch...@cloudera.com>.
Hi Alex,
I think that that's expected. The Pig manual says the following
regarding comparison
operators (e.g. ==):
If either sub-expression is null, the result is null.
So "col1 == null" is null.
Now it also says the following regarding arithmetic operators (e.g. ?):
If either sub-expression is null, the resulting expression is null.
So "col1 == null ? 'null' : 'not-null'" is null as "col1 == null" is null.
Here is the link:
http://pig.apache.org/docs/r0.10.0/basic.html#nulls
Thanks,
Cheolsoo
On Wed, Aug 22, 2012 at 11:28 AM, Alex Rovner <al...@gmail.com> wrote:
> I am having trouble with bincond in pig 11.
>
> Sample input:
> 1234
> 0
> 1234
>
> Sample pig script:
> a = LOAD 'input.txt' as (col1:int);
>
> b = FOREACH a GENERATE col1, (col1 == null ? 'null' : 'not-null') as col2;
>
> dump b;
>
>
> Output:
> (1234,)
> (0,)
> (1234,)
>
>
> Certainly not what you expect to see... I expected to see 'not-null' string
> in the second column.
> If I change the bincond to look for a particular value then everything
> works as expected:
>
> b = FOREACH a GENERATE col1, (col1 == 1234 ? 'null' : 'not-null') as col2;
>
> Output:
> (1234,null)
> (0,not-null)
> (1234,null)
>
>
> Any ideas? I did not get a chance to test this with prior versions.
>
> Thanks
> Alex
>