You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by Fabian Alenius <fa...@gmail.com> on 2012/08/23 10:53:07 UTC
Project the last field of a tuple
Hi,
is there anyway to project the last field of a tuple (when you don't
know how many fields there are) without creating a UDF?
Thanks,
Fabian
Re: Project the last field of a tuple
Posted by Russell Jurney <ru...@gmail.com>.
This is neat. This should maybe be a built in: Tuple.first and
Tuple.last. Very convenient in Ruby.
Russell Jurney http://datasyndrome.com
On Aug 23, 2012, at 1:53 PM, Jonathan Coveney <jc...@gmail.com> wrote:
> here's a UDF to do it that took me about 10s to write, so may have errors:
>
> import java.io.IOException;
> import org.apache.pig.EvalFunc;
> import org.apache.pig.data.Tuple;
> import org.apache.pig.impl.logicalLayer.schema.Schema;
>
> public class LastInTuple extends EvalFunc<Object> {
> public Object exec(Tuple input) throws IOException {
> int size = input.size();
> if (size > 0) {
> return input.get(size - 1);
> }
> return null;
> }
>
> public Schema outputSchema(Schema input) {
> try {
> int size = input.size();
> if (size > 0) {
> return new Schema(input.getField(size - 1));
> }
> } catch (Exception e) {}
> return null;
> }
> }
>
> 2012/8/23 Ruslan Al-Fakikh <ru...@jalent.ru>
>
>> Hi Fabian,
>>
>> I don't know whether there is a built-in feature for this, but here is the
>> idea:
>> try to load the whole line as one field (ignoring the delimiter at
>> this step) and then try to extract the last part using substring,
>> regex, etc.
>>
>> Ruslan
>>
>> On Thu, Aug 23, 2012 at 12:53 PM, Fabian Alenius
>> <fa...@gmail.com> wrote:
>>> Hi,
>>>
>>> is there anyway to project the last field of a tuple (when you don't
>>> know how many fields there are) without creating a UDF?
>>>
>>>
>>> Thanks,
>>>
>>> Fabian
>>
>>
>>
>> --
>> Best Regards,
>> Ruslan Al-Fakikh
>>
Re: Project the last field of a tuple
Posted by Jonathan Coveney <jc...@gmail.com>.
here's a UDF to do it that took me about 10s to write, so may have errors:
import java.io.IOException;
import org.apache.pig.EvalFunc;
import org.apache.pig.data.Tuple;
import org.apache.pig.impl.logicalLayer.schema.Schema;
public class LastInTuple extends EvalFunc<Object> {
public Object exec(Tuple input) throws IOException {
int size = input.size();
if (size > 0) {
return input.get(size - 1);
}
return null;
}
public Schema outputSchema(Schema input) {
try {
int size = input.size();
if (size > 0) {
return new Schema(input.getField(size - 1));
}
} catch (Exception e) {}
return null;
}
}
2012/8/23 Ruslan Al-Fakikh <ru...@jalent.ru>
> Hi Fabian,
>
> I don't know whether there is a built-in feature for this, but here is the
> idea:
> try to load the whole line as one field (ignoring the delimiter at
> this step) and then try to extract the last part using substring,
> regex, etc.
>
> Ruslan
>
> On Thu, Aug 23, 2012 at 12:53 PM, Fabian Alenius
> <fa...@gmail.com> wrote:
> > Hi,
> >
> > is there anyway to project the last field of a tuple (when you don't
> > know how many fields there are) without creating a UDF?
> >
> >
> > Thanks,
> >
> > Fabian
>
>
>
> --
> Best Regards,
> Ruslan Al-Fakikh
>
Re: Project the last field of a tuple
Posted by Ruslan Al-Fakikh <ru...@jalent.ru>.
Hi Fabian,
I don't know whether there is a built-in feature for this, but here is the idea:
try to load the whole line as one field (ignoring the delimiter at
this step) and then try to extract the last part using substring,
regex, etc.
Ruslan
On Thu, Aug 23, 2012 at 12:53 PM, Fabian Alenius
<fa...@gmail.com> wrote:
> Hi,
>
> is there anyway to project the last field of a tuple (when you don't
> know how many fields there are) without creating a UDF?
>
>
> Thanks,
>
> Fabian
--
Best Regards,
Ruslan Al-Fakikh