[jira] Commented: (PIG-476) given a date that can match a SimpleDateFormat want to be able to extract arbitrary SimpleDateFormat data, like day or year

["Alan Gates (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/PIG-476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12638594#action_12638594 ] 

Alan Gates commented on PIG-476:
--------------------------------

Pig Latin has a way that you can define constructors for a UDF:

{code}
define MyDateExtractor org.apache.pig.piggybank.evaluation.util.apachelogparser.DateExtractor("MM-dd-yyyy");
...
A = FOREACH raw GENERATE DateExtractor(dayTime);
{code}

Whatever you pass as an argument in the define method is passed to the constructor of the UDF.  In your case, this would allow you to pass the date format up front, parse it once, and avoid parsing it on every tuple passed to the UDF.  This should give you a significant performance boost.  The downside of this is that if you want to use the same UDF with different date formats in the same query you'd have to alias it different ways.  

It's up to you whether to choose flexibility or performance here.

> given a date that can match a SimpleDateFormat want to be able to extract arbitrary SimpleDateFormat data, like day or year
> ---------------------------------------------------------------------------------------------------------------------------
>
>                 Key: PIG-476
>                 URL: https://issues.apache.org/jira/browse/PIG-476
>             Project: Pig
>          Issue Type: New Feature
>            Reporter: Earl Cahill
>         Attachments: DateExtractor-PIG-476
>
>
> Want to be able to do something like
> A = FOREACH raw GENERATE org.apache.pig.piggybank.evaluation.util.apachelogparser.DateExtractor(dayTime, "yyyy", "dd/MMM/yyyy:HH:mm:ss");
> to extract the year, or if your date is formatted as
> dd/MMM/yyyy:HH:mm:ss Z
> you could do something like
> A = FOREACH raw GENERATE org.apache.pig.piggybank.evaluation.util.apachelogparser.DateExtractor(dayTime, "MM-dd-yyyy");
> to grab out the day

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.