You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by Pradeep Kamath <pr...@yahoo-inc.com> on 2010/07/08 01:35:39 UTC
UDF which takes entire row as arg
Hi,
I wanted to implement a UDF which takes the entire row of input as
argument. Unfortunately when I call it using select myudf(*) from my
table, I get :
FAILED: Parse Error: line 1:16 mismatched input '*' expecting ) in
function specification
Is this not allowed?
Thanks,
Pradeep
Re: UDF which takes entire row as arg
Posted by John Sichi <js...@facebook.com>.
As Zheng mentioned, I think a separate JIRA issue needs to be created for the SemanticAnalyzer star expansions for both UDF and UDAF; see the comment I just added on HIVE-287.
JVS
On Jul 7, 2010, at 5:44 PM, Pradeep Kamath wrote:
Hi Arvind,
Before I dig deeper with the patch on that jira, I was wondering if it is relevant that what I am trying is not an aggregate function (no group by involved). I am writing a row level UDF but would like all fields in the row to be presented to the udf. So my query literally is:
“select myudf(*) from mytable;”
Do you think the patch might address my use case even though my UDF is not a UDAF?
Thanks,
Pradeep
________________________________
From: Arvind Prabhakar [mailto:arvind@cloudera.com]
Sent: Wednesday, July 07, 2010 5:26 PM
To: hive-user@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: Re: UDF which takes entire row as arg
Hi Pradeep,
See HIVE-287<https://issues.apache.org/jira/browse/HIVE-287>. The most recent patch supplied with this Jira allows you to create UDAFs that support the FUNCTION(STAR) syntax. You can apply this patch to your local working copy and see if that addresses your problem.
Also feel free to ask questions regarding anything that this patch provides.
Arvind
On Wed, Jul 7, 2010 at 4:35 PM, Pradeep Kamath <pr...@yahoo-inc.com>> wrote:
Hi,
I wanted to implement a UDF which takes the entire row of input as argument. Unfortunately when I call it using select myudf(*) from my table, I get :
FAILED: Parse Error: line 1:16 mismatched input '*' expecting ) in function specification
Is this not allowed?
Thanks,
Pradeep
Re: UDF which takes entire row as arg
Posted by Zheng Shao <zs...@gmail.com>.
Yes. Even a normal (non-generic) UDF might work if all columns can be
converted to the same type. UDF can accept variable-length of
arguments of the same type.
it will be a great addition to let UDF/UDAF handle * (as well as `regex`).
The change is all compile-time, and is relatively simple.
Zheng
On Wed, Jul 7, 2010 at 8:31 PM, Edward Capriolo <ed...@gmail.com> wrote:
>
> You could write a generic UDF since they accept arbitrary signature,
> but you would have to pass each column specifically (no * support)
>
--
Yours,
Zheng
http://www.linkedin.com/in/zshao
Re: UDF which takes entire row as arg
Posted by Edward Capriolo <ed...@gmail.com>.
On Wed, Jul 7, 2010 at 8:44 PM, Pradeep Kamath <pr...@yahoo-inc.com> wrote:
> Hi Arvind,
>
> Before I dig deeper with the patch on that jira, I was wondering if it is
> relevant that what I am trying is not an aggregate function (no group by
> involved). I am writing a row level UDF but would like all fields in the row
> to be presented to the udf. So my query literally is:
>
> “select myudf(*) from mytable;”
>
>
>
> Do you think the patch might address my use case even though my UDF is not a
> UDAF?
>
>
>
> Thanks,
>
> Pradeep
>
>
>
> ________________________________
>
> From: Arvind Prabhakar [mailto:arvind@cloudera.com]
> Sent: Wednesday, July 07, 2010 5:26 PM
> To: hive-user@hadoop.apache.org
> Subject: Re: UDF which takes entire row as arg
>
>
>
> Hi Pradeep,
>
>
>
> See HIVE-287. The most recent patch supplied with this Jira allows you to
> create UDAFs that support the FUNCTION(STAR) syntax. You can apply this
> patch to your local working copy and see if that addresses your problem.
>
>
>
> Also feel free to ask questions regarding anything that this patch provides.
>
>
>
> Arvind
>
>
>
> On Wed, Jul 7, 2010 at 4:35 PM, Pradeep Kamath <pr...@yahoo-inc.com>
> wrote:
>
> Hi,
>
> I wanted to implement a UDF which takes the entire row of input as
> argument. Unfortunately when I call it using select myudf(*) from my table,
> I get :
>
> FAILED: Parse Error: line 1:16 mismatched input '*' expecting ) in function
> specification
>
>
>
> Is this not allowed?
>
>
>
> Thanks,
>
> Pradeep
>
>
You could write a generic UDF since they accept arbitrary signature,
but you would have to pass each column specifically (no * support)
RE: UDF which takes entire row as arg
Posted by Pradeep Kamath <pr...@yahoo-inc.com>.
Hi Arvind,
Before I dig deeper with the patch on that jira, I was wondering if it
is relevant that what I am trying is not an aggregate function (no group
by involved). I am writing a row level UDF but would like all fields in
the row to be presented to the udf. So my query literally is:
"select myudf(*) from mytable;"
Do you think the patch might address my use case even though my UDF is
not a UDAF?
Thanks,
Pradeep
________________________________
From: Arvind Prabhakar [mailto:arvind@cloudera.com]
Sent: Wednesday, July 07, 2010 5:26 PM
To: hive-user@hadoop.apache.org
Subject: Re: UDF which takes entire row as arg
Hi Pradeep,
See HIVE-287 <https://issues.apache.org/jira/browse/HIVE-287> . The most
recent patch supplied with this Jira allows you to create UDAFs that
support the FUNCTION(STAR) syntax. You can apply this patch to your
local working copy and see if that addresses your problem.
Also feel free to ask questions regarding anything that this patch
provides.
Arvind
On Wed, Jul 7, 2010 at 4:35 PM, Pradeep Kamath <pr...@yahoo-inc.com>
wrote:
Hi,
I wanted to implement a UDF which takes the entire row of input as
argument. Unfortunately when I call it using select myudf(*) from my
table, I get :
FAILED: Parse Error: line 1:16 mismatched input '*' expecting ) in
function specification
Is this not allowed?
Thanks,
Pradeep
Re: UDF which takes entire row as arg
Posted by Arvind Prabhakar <ar...@cloudera.com>.
Hi Pradeep,
See HIVE-287 <https://issues.apache.org/jira/browse/HIVE-287>. The most
recent patch supplied with this Jira allows you to create UDAFs that support
the FUNCTION(STAR) syntax. You can apply this patch to your local working
copy and see if that addresses your problem.
Also feel free to ask questions regarding anything that this patch provides.
Arvind
On Wed, Jul 7, 2010 at 4:35 PM, Pradeep Kamath <pr...@yahoo-inc.com>wrote:
> Hi,
>
> I wanted to implement a UDF which takes the entire row of input as
> argument. Unfortunately when I call it using select myudf(*) from my table,
> I get :
>
> FAILED: Parse Error: line 1:16 mismatched input '*' expecting ) in function
> specification
>
>
>
> Is this not allowed?
>
>
>
> Thanks,
>
> Pradeep
>