User Define Function (UDF) in AsterixDB

[Xikui Wang <xi...@uci.edu>]

Hi Devs,

I want to use UDF to process the Tweets that I got from the feed, and I met
following two questions. Hope you guys can help me or point me to the right
documentation.

1. How to install UDF on instance running from
Eclipse+AsterixHyracksIntegrationUtil?

Website only mentioned how to install with Managix. I am wondering if there
is a way for me to install it on instance running in Eclipse, which is
easier for debugging.

2. Implementation of UDF

I found several UDFs in
asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/library,
like SumFunction, ParseTweetFunction. I assume if I want to implement new
UDF, it needs to implement IExternalScalarFunction interface and to be put
under the same directory? I also found 'aggregate' and 'unnest' type which
is not implemented yet. Just out of curiosity, what is the difference
between them?

Thanks in advance! :)

Best,
Xikui

Re: User Define Function (UDF) in AsterixDB

[abdullah alamoudi <ba...@gmail.com>]

Yes.
It can only be tested through the test framework.

The web interface only understands query languages (AQL and SQL++).

Cheers,
Abdullah.

On Fri, Jun 3, 2016 at 12:44 AM, Xikui Wang <xi...@uci.edu> wrote:

> Hi Abdullah,
>
> Thanks for your help. I met an error when I was trying to execute 'install
> externallibtest testlib PATH/TO/testlib-zip-binary-assembly.zip' from the
> web query interface. Probably I used this in a wrong way?
>
> Best,
> Xikui
>
> On Thu, Jun 2, 2016 at 2:26 PM, abdullah alamoudi <ba...@gmail.com>
> wrote:
>
> > Hi Xikui,
> > 1. How to install UDF on instance running from Eclipse+
> > AsterixHyracksIntegrationUtil?
> >
> > There are a few external library test cases, you can look at them and see
> > how we test those. One thing you will notice is that we only test a few
> > examples. Clearly, we can do better. You can find the test cases in:
> >
> >
> >
> asterixdb/asterixdb/asterix-app/src/test/resources/runtimets/queries/external-library
> >
> > As for the difference between scalar, aggregate, and unnest functions,
> here
> > is the way I see it:
> > 1. Scalar: one input to one output.
> > 2. Aggregate: 0 or more inputs to one output.
> > 3. Unnest: one input to 0 or more outputs.
> >
> > Hope that helps,
> > Abdullah.
> >
> > On Thu, Jun 2, 2016 at 11:40 PM, Xikui Wang <xi...@uci.edu> wrote:
> >
> > > Hi Devs,
> > >
> > > I want to use UDF to process the Tweets that I got from the feed, and I
> > met
> > > following two questions. Hope you guys can help me or point me to the
> > right
> > > documentation.
> > >
> > > 1. How to install UDF on instance running from
> > > Eclipse+AsterixHyracksIntegrationUtil?
> > >
> > > Website only mentioned how to install with Managix. I am wondering if
> > there
> > > is a way for me to install it on instance running in Eclipse, which is
> > > easier for debugging.
> > >
> > > 2. Implementation of UDF
> > >
> > > I found several UDFs in
> > >
> > >
> >
> asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/library,
> > > like SumFunction, ParseTweetFunction. I assume if I want to implement
> new
> > > UDF, it needs to implement IExternalScalarFunction interface and to be
> > put
> > > under the same directory? I also found 'aggregate' and 'unnest' type
> > which
> > > is not implemented yet. Just out of curiosity, what is the difference
> > > between them?
> > >
> > > Thanks in advance! :)
> > >
> > > Best,
> > > Xikui
> > >
> >
>

Re: User Define Function (UDF) in AsterixDB

[Chen Li <ch...@gmail.com>]

We had several separate discussions about UDF.  Is it possible to take
this chance to polish the documentation so that other users can rely
on the documentation to get started?

Chen

On Fri, Jun 3, 2016 at 12:08 AM, Xikui Wang <xi...@uci.edu> wrote:
> Hi Heri,
>
> Thanks for sharing the document. It is useful as the general structure of
> UDF remains the same.
>
> Best,
> Xikui
>
> On Thu, Jun 2, 2016 at 11:34 PM, Heri Ramampiaro <he...@gmail.com> wrote:
>
>> Xikui,
>>
>> Enclosed is an instruction based on the older version of feeds and UDF
>> that perhaps could help you
>> figur out the principle behind installing external libs in AsterixDB
>>
>> Best,
>> -heri
>>
>>
>>
>>
>> > On Jun 2, 2016, at 23:44, Xikui Wang <xi...@uci.edu> wrote:
>> >
>> > Hi Abdullah,
>> >
>> > Thanks for your help. I met an error when I was trying to execute
>> 'install
>> > externallibtest testlib PATH/TO/testlib-zip-binary-assembly.zip' from the
>> > web query interface. Probably I used this in a wrong way?
>> >
>> > Best,
>> > Xikui
>> >
>> > On Thu, Jun 2, 2016 at 2:26 PM, abdullah alamoudi <ba...@gmail.com>
>> > wrote:
>> >
>> >> Hi Xikui,
>> >> 1. How to install UDF on instance running from Eclipse+
>> >> AsterixHyracksIntegrationUtil?
>> >>
>> >> There are a few external library test cases, you can look at them and
>> see
>> >> how we test those. One thing you will notice is that we only test a few
>> >> examples. Clearly, we can do better. You can find the test cases in:
>> >>
>> >>
>> >>
>> asterixdb/asterixdb/asterix-app/src/test/resources/runtimets/queries/external-library
>> >>
>> >> As for the difference between scalar, aggregate, and unnest functions,
>> here
>> >> is the way I see it:
>> >> 1. Scalar: one input to one output.
>> >> 2. Aggregate: 0 or more inputs to one output.
>> >> 3. Unnest: one input to 0 or more outputs.
>> >>
>> >> Hope that helps,
>> >> Abdullah.
>> >>
>> >> On Thu, Jun 2, 2016 at 11:40 PM, Xikui Wang <xi...@uci.edu> wrote:
>> >>
>> >>> Hi Devs,
>> >>>
>> >>> I want to use UDF to process the Tweets that I got from the feed, and I
>> >> met
>> >>> following two questions. Hope you guys can help me or point me to the
>> >> right
>> >>> documentation.
>> >>>
>> >>> 1. How to install UDF on instance running from
>> >>> Eclipse+AsterixHyracksIntegrationUtil?
>> >>>
>> >>> Website only mentioned how to install with Managix. I am wondering if
>> >> there
>> >>> is a way for me to install it on instance running in Eclipse, which is
>> >>> easier for debugging.
>> >>>
>> >>> 2. Implementation of UDF
>> >>>
>> >>> I found several UDFs in
>> >>>
>> >>>
>> >>
>> asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/library,
>> >>> like SumFunction, ParseTweetFunction. I assume if I want to implement
>> new
>> >>> UDF, it needs to implement IExternalScalarFunction interface and to be
>> >> put
>> >>> under the same directory? I also found 'aggregate' and 'unnest' type
>> >> which
>> >>> is not implemented yet. Just out of curiosity, what is the difference
>> >>> between them?
>> >>>
>> >>> Thanks in advance! :)
>> >>>
>> >>> Best,
>> >>> Xikui
>> >>>
>> >>
>>
>>
>>

Re: User Define Function (UDF) in AsterixDB

[Xikui Wang <xi...@uci.edu>]

Hi Heri,

Thanks for sharing the document. It is useful as the general structure of
UDF remains the same.

Best,
Xikui

On Thu, Jun 2, 2016 at 11:34 PM, Heri Ramampiaro <he...@gmail.com> wrote:

> Xikui,
>
> Enclosed is an instruction based on the older version of feeds and UDF
> that perhaps could help you
> figur out the principle behind installing external libs in AsterixDB
>
> Best,
> -heri
>
>
>
>
> > On Jun 2, 2016, at 23:44, Xikui Wang <xi...@uci.edu> wrote:
> >
> > Hi Abdullah,
> >
> > Thanks for your help. I met an error when I was trying to execute
> 'install
> > externallibtest testlib PATH/TO/testlib-zip-binary-assembly.zip' from the
> > web query interface. Probably I used this in a wrong way?
> >
> > Best,
> > Xikui
> >
> > On Thu, Jun 2, 2016 at 2:26 PM, abdullah alamoudi <ba...@gmail.com>
> > wrote:
> >
> >> Hi Xikui,
> >> 1. How to install UDF on instance running from Eclipse+
> >> AsterixHyracksIntegrationUtil?
> >>
> >> There are a few external library test cases, you can look at them and
> see
> >> how we test those. One thing you will notice is that we only test a few
> >> examples. Clearly, we can do better. You can find the test cases in:
> >>
> >>
> >>
> asterixdb/asterixdb/asterix-app/src/test/resources/runtimets/queries/external-library
> >>
> >> As for the difference between scalar, aggregate, and unnest functions,
> here
> >> is the way I see it:
> >> 1. Scalar: one input to one output.
> >> 2. Aggregate: 0 or more inputs to one output.
> >> 3. Unnest: one input to 0 or more outputs.
> >>
> >> Hope that helps,
> >> Abdullah.
> >>
> >> On Thu, Jun 2, 2016 at 11:40 PM, Xikui Wang <xi...@uci.edu> wrote:
> >>
> >>> Hi Devs,
> >>>
> >>> I want to use UDF to process the Tweets that I got from the feed, and I
> >> met
> >>> following two questions. Hope you guys can help me or point me to the
> >> right
> >>> documentation.
> >>>
> >>> 1. How to install UDF on instance running from
> >>> Eclipse+AsterixHyracksIntegrationUtil?
> >>>
> >>> Website only mentioned how to install with Managix. I am wondering if
> >> there
> >>> is a way for me to install it on instance running in Eclipse, which is
> >>> easier for debugging.
> >>>
> >>> 2. Implementation of UDF
> >>>
> >>> I found several UDFs in
> >>>
> >>>
> >>
> asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/library,
> >>> like SumFunction, ParseTweetFunction. I assume if I want to implement
> new
> >>> UDF, it needs to implement IExternalScalarFunction interface and to be
> >> put
> >>> under the same directory? I also found 'aggregate' and 'unnest' type
> >> which
> >>> is not implemented yet. Just out of curiosity, what is the difference
> >>> between them?
> >>>
> >>> Thanks in advance! :)
> >>>
> >>> Best,
> >>> Xikui
> >>>
> >>
>
>
>

Re: User Define Function (UDF) in AsterixDB

[Heri Ramampiaro <he...@gmail.com>]

Xikui,

Enclosed is an instruction based on the older version of feeds and UDF that perhaps could help you
figur out the principle behind installing external libs in AsterixDB

Best,
-heri

Re: User Define Function (UDF) in AsterixDB

[Xikui Wang <xi...@uci.edu>]

Hi Abdullah,

Thanks for your help. I met an error when I was trying to execute 'install
externallibtest testlib PATH/TO/testlib-zip-binary-assembly.zip' from the
web query interface. Probably I used this in a wrong way?

Best,
Xikui

On Thu, Jun 2, 2016 at 2:26 PM, abdullah alamoudi <ba...@gmail.com>
wrote:

> Hi Xikui,
> 1. How to install UDF on instance running from Eclipse+
> AsterixHyracksIntegrationUtil?
>
> There are a few external library test cases, you can look at them and see
> how we test those. One thing you will notice is that we only test a few
> examples. Clearly, we can do better. You can find the test cases in:
>
>
> asterixdb/asterixdb/asterix-app/src/test/resources/runtimets/queries/external-library
>
> As for the difference between scalar, aggregate, and unnest functions, here
> is the way I see it:
> 1. Scalar: one input to one output.
> 2. Aggregate: 0 or more inputs to one output.
> 3. Unnest: one input to 0 or more outputs.
>
> Hope that helps,
> Abdullah.
>
> On Thu, Jun 2, 2016 at 11:40 PM, Xikui Wang <xi...@uci.edu> wrote:
>
> > Hi Devs,
> >
> > I want to use UDF to process the Tweets that I got from the feed, and I
> met
> > following two questions. Hope you guys can help me or point me to the
> right
> > documentation.
> >
> > 1. How to install UDF on instance running from
> > Eclipse+AsterixHyracksIntegrationUtil?
> >
> > Website only mentioned how to install with Managix. I am wondering if
> there
> > is a way for me to install it on instance running in Eclipse, which is
> > easier for debugging.
> >
> > 2. Implementation of UDF
> >
> > I found several UDFs in
> >
> >
> asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/library,
> > like SumFunction, ParseTweetFunction. I assume if I want to implement new
> > UDF, it needs to implement IExternalScalarFunction interface and to be
> put
> > under the same directory? I also found 'aggregate' and 'unnest' type
> which
> > is not implemented yet. Just out of curiosity, what is the difference
> > between them?
> >
> > Thanks in advance! :)
> >
> > Best,
> > Xikui
> >
>

Re: User Define Function (UDF) in AsterixDB

[abdullah alamoudi <ba...@gmail.com>]

Hi Xikui,
1. How to install UDF on instance running from Eclipse+
AsterixHyracksIntegrationUtil?

There are a few external library test cases, you can look at them and see
how we test those. One thing you will notice is that we only test a few
examples. Clearly, we can do better. You can find the test cases in:

asterixdb/asterixdb/asterix-app/src/test/resources/runtimets/queries/external-library

As for the difference between scalar, aggregate, and unnest functions, here
is the way I see it:
1. Scalar: one input to one output.
2. Aggregate: 0 or more inputs to one output.
3. Unnest: one input to 0 or more outputs.

Hope that helps,
Abdullah.

On Thu, Jun 2, 2016 at 11:40 PM, Xikui Wang <xi...@uci.edu> wrote:

> Hi Devs,
>
> I want to use UDF to process the Tweets that I got from the feed, and I met
> following two questions. Hope you guys can help me or point me to the right
> documentation.
>
> 1. How to install UDF on instance running from
> Eclipse+AsterixHyracksIntegrationUtil?
>
> Website only mentioned how to install with Managix. I am wondering if there
> is a way for me to install it on instance running in Eclipse, which is
> easier for debugging.
>
> 2. Implementation of UDF
>
> I found several UDFs in
>
> asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/library,
> like SumFunction, ParseTweetFunction. I assume if I want to implement new
> UDF, it needs to implement IExternalScalarFunction interface and to be put
> under the same directory? I also found 'aggregate' and 'unnest' type which
> is not implemented yet. Just out of curiosity, what is the difference
> between them?
>
> Thanks in advance! :)
>
> Best,
> Xikui
>