You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by Fabian Alenius <fa...@gmail.com> on 2012/08/23 10:53:07 UTC

Project the last field of a tuple

Hi,

is there anyway to project the last field of a tuple (when you don't
know how many fields there are) without creating a UDF?


Thanks,

Fabian

Re: Project the last field of a tuple

Posted by Russell Jurney <ru...@gmail.com>.
This is neat. This should maybe be a built in: Tuple.first and
Tuple.last. Very convenient in Ruby.

Russell Jurney http://datasyndrome.com

On Aug 23, 2012, at 1:53 PM, Jonathan Coveney <jc...@gmail.com> wrote:

> here's a UDF to do it that took me about 10s to write, so may have errors:
>
> import java.io.IOException;
> import org.apache.pig.EvalFunc;
> import org.apache.pig.data.Tuple;
> import org.apache.pig.impl.logicalLayer.schema.Schema;
>
> public class LastInTuple extends EvalFunc<Object> {
>    public Object exec(Tuple input) throws IOException {
>        int size = input.size();
>        if (size > 0) {
>            return input.get(size - 1);
>        }
>        return null;
>    }
>
>    public Schema outputSchema(Schema input) {
>        try {
>        int size = input.size();
>            if (size > 0) {
>                return new Schema(input.getField(size - 1));
>            }
>        } catch (Exception e) {}
>        return null;
>    }
> }
>
> 2012/8/23 Ruslan Al-Fakikh <ru...@jalent.ru>
>
>> Hi Fabian,
>>
>> I don't know whether there is a built-in feature for this, but here is the
>> idea:
>> try to load the whole line as one field (ignoring the delimiter at
>> this step) and then try to extract the last part using substring,
>> regex, etc.
>>
>> Ruslan
>>
>> On Thu, Aug 23, 2012 at 12:53 PM, Fabian Alenius
>> <fa...@gmail.com> wrote:
>>> Hi,
>>>
>>> is there anyway to project the last field of a tuple (when you don't
>>> know how many fields there are) without creating a UDF?
>>>
>>>
>>> Thanks,
>>>
>>> Fabian
>>
>>
>>
>> --
>> Best Regards,
>> Ruslan Al-Fakikh
>>

Re: Project the last field of a tuple

Posted by Jonathan Coveney <jc...@gmail.com>.
here's a UDF to do it that took me about 10s to write, so may have errors:

import java.io.IOException;
import org.apache.pig.EvalFunc;
import org.apache.pig.data.Tuple;
import org.apache.pig.impl.logicalLayer.schema.Schema;

public class LastInTuple extends EvalFunc<Object> {
    public Object exec(Tuple input) throws IOException {
        int size = input.size();
        if (size > 0) {
            return input.get(size - 1);
        }
        return null;
    }

    public Schema outputSchema(Schema input) {
        try {
        int size = input.size();
            if (size > 0) {
                return new Schema(input.getField(size - 1));
            }
        } catch (Exception e) {}
        return null;
    }
}

2012/8/23 Ruslan Al-Fakikh <ru...@jalent.ru>

> Hi Fabian,
>
> I don't know whether there is a built-in feature for this, but here is the
> idea:
> try to load the whole line as one field (ignoring the delimiter at
> this step) and then try to extract the last part using substring,
> regex, etc.
>
> Ruslan
>
> On Thu, Aug 23, 2012 at 12:53 PM, Fabian Alenius
> <fa...@gmail.com> wrote:
> > Hi,
> >
> > is there anyway to project the last field of a tuple (when you don't
> > know how many fields there are) without creating a UDF?
> >
> >
> > Thanks,
> >
> > Fabian
>
>
>
> --
> Best Regards,
> Ruslan Al-Fakikh
>

Re: Project the last field of a tuple

Posted by Ruslan Al-Fakikh <ru...@jalent.ru>.
Hi Fabian,

I don't know whether there is a built-in feature for this, but here is the idea:
try to load the whole line as one field (ignoring the delimiter at
this step) and then try to extract the last part using substring,
regex, etc.

Ruslan

On Thu, Aug 23, 2012 at 12:53 PM, Fabian Alenius
<fa...@gmail.com> wrote:
> Hi,
>
> is there anyway to project the last field of a tuple (when you don't
> know how many fields there are) without creating a UDF?
>
>
> Thanks,
>
> Fabian



-- 
Best Regards,
Ruslan Al-Fakikh