You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Daniel Dai (JIRA)" <ji...@apache.org> on 2015/06/13 00:24:01 UTC
[jira] [Commented] (PIG-4590) The function StringConcat returns a
null value in certain cases even when there are chars to concatinate.
[ https://issues.apache.org/jira/browse/PIG-4590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14584187#comment-14584187 ]
Daniel Dai commented on PIG-4590:
---------------------------------
You need to do a bincond to make sure $21 is not null:
StringConcat($0,$1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12,$13,$14,$15,$16,$17,$18,$19,$20,($21 is null?name:''))
> The function StringConcat returns a null value in certain cases even when there are chars to concatinate.
> ---------------------------------------------------------------------------------------------------------
>
> Key: PIG-4590
> URL: https://issues.apache.org/jira/browse/PIG-4590
> Project: Pig
> Issue Type: Bug
> Components: grunt
> Affects Versions: 0.12.0
> Environment: Linux server
> Reporter: Jordan Babb
>
> The basic issue is I want to scan a whole row for a value. In the case below I want to take ref::tin and scan each column in combined or sub_load for that value.
> Is there a better way to do this?
> The way it is done below, a null value is returned for some of the rows when StringConcat is used. it seems to be if there is no value or null in the last column($21).
> any help would be appreciated.
> ------------------------------------------
> There should be 8 matches when files are combined.
> When e is ran there should be 8 matches.
> pig
> ref_load = LOAD '/user/sditmpoc/pop/reference/TC4/RU99.KM40' USING PigStorage(',') AS (clnt_id:chararray, death_dt, birth_dt:chararray, tin:chararray, pref_first_name, cmprs_full_pref_first_name);
> ref = FILTER ref_load BY (tin matches '^[^ ].*');
> sub_load = LOAD '/user/sditmpoc/pop/Subject/TC4/RU99.KM40' using PigStorage(',') AS (clnt_id:chararray,two,three,four,birth_dt:chararray,six,tin:chararray,eight,nine,ten,eleven,twelve,thirteen,fourteen,fifteen,sixteen,seventeen,eighteen,nineteen,twenty,twentyone,twentytwo);
> combined = JOIN sub_load BY (clnt_id), ref BY (clnt_id);
> sub = foreach combined generate StringConcat($0,$1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12,$13,$14,$15,$16,$17,$18,$19,$20,$21),CONCAT(CONCAT('.*',ref::tin),'.*');
> e = filter sub by $0 matches $1;
> example data for /user/sditmpoc/pop/reference/TC4/RU99.KM40:
> BBBBB4WZ008,null,1933-07-28,402404671 ,,
> BBBBB5WJ00P,null,1984-02-06, ,,
> BBBBB6NHT04,null,1964-06-10, ,,
> example data for /user/sditmpoc/pop/Subject/TC4/RU99.KM40:
> BBBBB4WZ008,2013-08-12,F,null,1953-08-06,M, ,2,1,1,1,Y,null,N,N,402404671 ,2013-08-12 13:53:34.300944,2013-08-12 13:53:34.300993,N,4$,,a
> BBBBB4WZ008,2013-08-12,F,null,1953-08-06,M, ,2,1,1,1,Y,null,N,N,402404671 ,2013-08-12 13:53:34.300944,2013-08-12 13:53:34.300993,N,4$,a,
> BBBBB4WZ008,2013-08-12,F,null,1953-08-06,M, ,2,1,1,1,Y,null,N,N,402404671 ,2013-08-12 13:53:34.300944,2013-08-12 13:53:34.300993,N,4$,,
> 999986ZMYGF,2013-05-21,F,null,1989-12-21, , ,2,1,1,1,Y,2013-05-21,N,C,null,2013-05-21 14:11:33.300959,2013-06-23 09:27:20.608754,N,4$,,
> BBBBB4WZ008,2013-08-27,M,null,1980-01-29,S, ,2,1,1,1,Y,2013-08-27,N,N,null,2013-08-27 14:12:09.30093,2013-08-27 14:12:09.300982,N,4$,,
> 9998975LHGF,2013-11-22,M,null,1963-10-20, , ,2,1,1,1,Y,2013-11-22,N,N,null,2013-11-22 13:55:49.300929,2013-11-22 13:55:49.300958,N,4$,,
> BBBBB4WZ008,2013-08-12,F,null,1953-08-06,M, ,2,1,1,1,Y,null,N,N,402404671 ,2013-08-12 13:53:34.300944,2013-08-12 13:53:34.300993,N,4$,,
> BBBBB4WZ008,2012-07-24,F,null,1972-01-07,M, ,2,28,1,1,Y,2012-07-24,N,N,null,2012-07-24 18:00:21.41841,2012-07-24 18:00:21.418635,N,04,402404671 ,
> 9998K71G9AL,2013-07-17, ,null,null, , ,2,1,1,1,Y,null,N,N,null,2013-07-17 13:59:51.301,2013-07-17 13:59:51.301082,N,4$,,
> 9998K6Y1TGF,2013-04-02, ,null,null, , ,3,1,1,1,Y,null,N,N,null,2013-04-02 14:00:07.30094,2013-04-02 14:00:07.301012,N,4$,,
> 9998K6YBQAL,2013-04-11, ,null,null, , ,2,1,1,1,Y,null,N,N,null,2013-04-11 14:00:04.301045,2013-04-11 14:00:04.301125,N,4$,,
> 9998H710HGF,2013-07-03, ,null,null, , ,2,1,1,1,Y,null,N,N,null,2013-07-03 13:58:55.301051,2013-07-03 13:58:55.301156,N,4$,,
> BBBBB4WZ008,2012-08-07,M,null,1972-02-20,S, ,2,28,1,1,Y,2012-08-07,N,N,null,2012-08-07 17:58:40.055748,2012-08-07 17:58:40.055826,N,17,,
> 9998D6PWW0X,2012-08-23,F,null,1985-07-14,M, ,2,1,1,1,Y,2012-08-23,N,N,null,2012-08-23 17:57:36.472213,2012-08-23 17:57:36.472273,N,28,,
> BBBBB4WZ008,2012-10-10,M,null,1980-04-16, , ,2,28,1,1,Y,2012-10-10,N,N,null,2012-10-10 18:50:12.545438,2012-10-10 18:50:12.545489,N,08,402404671 ,
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)