You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by praveenesh kumar <pr...@gmail.com> on 2014/03/21 17:52:05 UTC
STRSPLIT should return a bag rather than a tuple
Hi there,
I had a scenario, which required me to change the STRSPLIT code. The
scenario was as follows:
I have a data like:
1 A|1|1 some
2 B|2|2 data
3 C|3|3 hadoop
Need output like this :
1 A some
1 1 some
1 1 some
2 B data
2 2 data
2 2 data
3 C hadoop
3 3 hadoop
3 3 hadoop
I was trying to use STRSPLIT($1,'\\|') which was returning a tuple, If I do
flatten on it, it converts the data into columns.
If we return a bag of tuples, we can easily use flatten() to convert it
into rows, plus can also convert that into Tuple using TOTUPLE() UDF (if
someone just want to use it as tuple.
Please suggest if this looks like a valid change. I am happy to file a JIRA
ticket and work accordingly on that.
Regards
Praveenesh
Re: STRSPLIT should return a bag rather than a tuple
Posted by Daniel Dai <da...@hortonworks.com>.
It is be better but need to change in a backward compatible way (eg,
create a new UDF instead of changing existing one).
On Fri, Mar 21, 2014 at 9:52 AM, praveenesh kumar <pr...@gmail.com> wrote:
> Hi there,
>
> I had a scenario, which required me to change the STRSPLIT code. The
> scenario was as follows:
>
> I have a data like:
> 1 A|1|1 some
> 2 B|2|2 data
> 3 C|3|3 hadoop
>
>
> Need output like this :
>
> 1 A some
> 1 1 some
> 1 1 some
> 2 B data
> 2 2 data
> 2 2 data
> 3 C hadoop
> 3 3 hadoop
> 3 3 hadoop
>
> I was trying to use STRSPLIT($1,'\\|') which was returning a tuple, If I do
> flatten on it, it converts the data into columns.
>
> If we return a bag of tuples, we can easily use flatten() to convert it
> into rows, plus can also convert that into Tuple using TOTUPLE() UDF (if
> someone just want to use it as tuple.
>
> Please suggest if this looks like a valid change. I am happy to file a JIRA
> ticket and work accordingly on that.
>
> Regards
> Praveenesh
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.