You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@drill.apache.org by Stefán Baxter <st...@activitystream.com> on 2015/07/19 19:20:17 UTC
Drill not picking up a UDF
Hi,
I'm trying to deploy a UDF that I have written according to the
documentation.
I have also:
1. Copied the jar file
to jars/3rdparty
2. Changed the config "conf/drill-override.conf" to include:
drill.logical.function.package +=
["org.apache.drill.exec.expr.fn.impl","com.activitystream.drill.udfs"]
3. Rerun the Drillbit
bin/drillbit.sh start
The log shows no sign of the the UDF being loaded but if I use it in a SQL
statement then this error is thrown:
Error: PARSE ERROR: From line 1, column 184 to line 1, column 219: No match
found for function signature asRoundTimeStamp(<ANY>, <CHARACTER>)
The relevant part of the UDF source:
package com.activitystream.drill.udfs;
public class ASUserDefinedFunctions {
@FunctionTemplate(name = "asRoundTimeStamp", scope =
FunctionTemplate.FunctionScope.SIMPLE, nulls =
FunctionTemplate.NullHandling.NULL_IF_NULL)
public static class RoundTimeStamp implements DrillSimpleFunc
Can anyone please assist?
Regards,
-Stefan
Re: Drill not picking up a UDF
Posted by Jim Bates <jb...@maprtech.com>.
give me a few min and I'll post one.
On Sun, Jul 19, 2015 at 4:14 PM, Stefán Baxter <st...@activitystream.com>
wrote:
> Hi Jim,
>
> I'm still not able to make this work. Do you have a sample .jar file with a
> small example that you are running?
>
> Regards,
> -Stefan
>
> On Sun, Jul 19, 2015 at 6:46 PM, Ted Dunning <te...@gmail.com>
> wrote:
>
> >
> > Sounds like a fine example, not because of sophistication but because it
> > deals with dates.
> >
> > Check the drill logs. It is likely that drill is grumpy about something
> > in your udf or packaging.
> >
> > Also, feel free to snitch the pom from the simple examples in order to
> get
> > the pieces assembled and packaged correctly.
> >
> > Sent from my iPhone
> >
> > > On Jul 19, 2015, at 11:25, Stefán Baxter <st...@activitystream.com>
> > wrote:
> > >
> > > Hi Jim,
> > >
> > > My UDF is currently so simple that I'm not sure you need it (or want
> it).
> > >
> > > It basically just rounds a timestamp value with ISO 8601 periods
> > > asRoundTimestamp(timestampvalue,'PT10M').
> > >
> > > I would be more than happy to contribute to your project rather than
> > > building our own :).
> > >
> > > Is the repo public?
> > >
> > > Regards,
> > > -Stefan
> > >
> > >
> > >
> > >> On Sun, Jul 19, 2015 at 6:18 PM, Jim Bates <jb...@maprtech.com>
> wrote:
> > >>
> > >> Maven will typically create a jar for class and a jar for source when
> > told
> > >> to do so. I just include the source files in the same jar as the class
> > >> files. There is a github example drill udf project we are working on
> to
> > >> include several examples to simplify the learning curve. If your
> > >> interested... I'd love to have you add your udf.
> > >> On Jul 19, 2015 12:59 PM, "Stefán Baxter" <st...@activitystream.com>
> > >> wrote:
> > >>
> > >>> Hi again,
> > >>>
> > >>> Going over the documentation once more I came across this:
> > >>>
> > >>> - Add the sources and classes JAR files to Drill’s classpath.
> > >>>
> > >>> I'm only including a standard .jar (with classes no sources) and I
> > wonder
> > >>> why this dependency is listed.
> > >>>
> > >>> Anyways, that it the next step.
> > >>>
> > >>> A sample .jar that works with the drill-config counterpart would be a
> > >>> welcome addition to the documentation.
> > >>>
> > >>> Regards,
> > >>> -Stefan
> > >>>
> > >>>
> > >>> On Sun, Jul 19, 2015 at 5:53 PM, Stefán Baxter <
> > >> stefan@activitystream.com>
> > >>> wrote:
> > >>>
> > >>>> Hi Jim,
> > >>>>
> > >>>> Now I have added the file to the jar (both root and resources
> folder)
> > >> but
> > >>>> that does not seem to change anything.
> > >>>>
> > >>>> Any additional ideas?
> > >>>>
> > >>>> Regards,
> > >>>> -Stefan
> > >>>>
> > >>>> On Sun, Jul 19, 2015 at 5:40 PM, Stefán Baxter <
> > >>> stefan@activitystream.com>
> > >>>> wrote:
> > >>>>
> > >>>>> NO!
> > >>>>>
> > >>>>> Thank you , will do that right now :)
> > >>>>>
> > >>>>> On Sun, Jul 19, 2015 at 5:38 PM, Jim Bates <jb...@maprtech.com>
> > >> wrote:
> > >>>>>
> > >>>>>> Did you include a file drill-module.conf in your jar along with
> > >> source
> > >>>>>> files?
> > >>>>>> On Jul 19, 2015 12:20 PM, "Stefán Baxter" <
> > stefan@activitystream.com
> > >>>
> > >>>>>> wrote:
> > >>>>>>
> > >>>>>>> Hi,
> > >>>>>>>
> > >>>>>>> I'm trying to deploy a UDF that I have written according to the
> > >>>>>>> documentation.
> > >>>>>>>
> > >>>>>>> I have also:
> > >>>>>>>
> > >>>>>>> 1. Copied the jar file
> > >>>>>>> to jars/3rdparty
> > >>>>>>>
> > >>>>>>> 2. Changed the config "conf/drill-override.conf" to include:
> > >>>>>>> drill.logical.function.package +=
> > >> ["org.apache.drill.exec.expr.fn.impl","com.activitystream.drill.udfs"]
> > >>>>>>>
> > >>>>>>> 3. Rerun the Drillbit
> > >>>>>>> bin/drillbit.sh start
> > >>>>>>>
> > >>>>>>> The log shows no sign of the the UDF being loaded but if I use it
> > >> in
> > >>> a
> > >>>>>> SQL
> > >>>>>>> statement then this error is thrown:
> > >>>>>>>
> > >>>>>>> Error: PARSE ERROR: From line 1, column 184 to line 1, column
> 219:
> > >> No
> > >>>>>> match
> > >>>>>>> found for function signature asRoundTimeStamp(<ANY>, <CHARACTER>)
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> The relevant part of the UDF source:
> > >>>>>>>
> > >>>>>>> package com.activitystream.drill.udfs;
> > >>>>>>>
> > >>>>>>> public class ASUserDefinedFunctions {
> > >>>>>>>
> > >>>>>>> @FunctionTemplate(name = "asRoundTimeStamp", scope =
> > >>>>>>> FunctionTemplate.FunctionScope.SIMPLE, nulls =
> > >>>>>>> FunctionTemplate.NullHandling.NULL_IF_NULL)
> > >>>>>>>
> > >>>>>>> public static class RoundTimeStamp implements DrillSimpleFunc
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> Can anyone please assist?
> > >>>>>>>
> > >>>>>>> Regards,
> > >>>>>>> -Stefan
> > >>
> >
>
Re: Drill not picking up a UDF
Posted by Jim Bates <jb...@maprtech.com>.
Stefán, in the logs/drillbit.log file, are there any complaints before it
gives you the number of loaded functions?
INFO o.a.d.e.e.f.FunctionImplementationRegistry - Function registry
loaded. 2372 functions loaded in 5214 ms
On Sun, Jul 19, 2015 at 6:20 PM, Stefán Baxter <st...@activitystream.com>
wrote:
> Hi Ted,
>
> I fetched this, built it and deployed it without problems.
> I can not see any real difference other than this deploys two .jar (I tried
> that as well earlier).
>
> I'm still trying to figure out why Drill is not picking up my UDFs
>
> Regards,
> -Stefán
>
> On Sun, Jul 19, 2015 at 10:45 PM, Ted Dunning <te...@gmail.com>
> wrote:
>
> > Stefan,
> >
> > Have you seen this github project:
> >
> > https://github.com/mapr-demos/simple-drill-functions
> >
> > ?
> >
> >
> > On Sun, Jul 19, 2015 at 2:14 PM, Stefán Baxter <
> stefan@activitystream.com>
> > wrote:
> >
> > > Hi Jim,
> > >
> > > I'm still not able to make this work. Do you have a sample .jar file
> > with a
> > > small example that you are running?
> > >
> > > Regards,
> > > -Stefan
> > >
> > > On Sun, Jul 19, 2015 at 6:46 PM, Ted Dunning <te...@gmail.com>
> > > wrote:
> > >
> > > >
> > > > Sounds like a fine example, not because of sophistication but because
> > it
> > > > deals with dates.
> > > >
> > > > Check the drill logs. It is likely that drill is grumpy about
> > something
> > > > in your udf or packaging.
> > > >
> > > > Also, feel free to snitch the pom from the simple examples in order
> to
> > > get
> > > > the pieces assembled and packaged correctly.
> > > >
> > > > Sent from my iPhone
> > > >
> > > > > On Jul 19, 2015, at 11:25, Stefán Baxter <
> stefan@activitystream.com>
> > > > wrote:
> > > > >
> > > > > Hi Jim,
> > > > >
> > > > > My UDF is currently so simple that I'm not sure you need it (or
> want
> > > it).
> > > > >
> > > > > It basically just rounds a timestamp value with ISO 8601 periods
> > > > > asRoundTimestamp(timestampvalue,'PT10M').
> > > > >
> > > > > I would be more than happy to contribute to your project rather
> than
> > > > > building our own :).
> > > > >
> > > > > Is the repo public?
> > > > >
> > > > > Regards,
> > > > > -Stefan
> > > > >
> > > > >
> > > > >
> > > > >> On Sun, Jul 19, 2015 at 6:18 PM, Jim Bates <jb...@maprtech.com>
> > > wrote:
> > > > >>
> > > > >> Maven will typically create a jar for class and a jar for source
> > when
> > > > told
> > > > >> to do so. I just include the source files in the same jar as the
> > class
> > > > >> files. There is a github example drill udf project we are working
> on
> > > to
> > > > >> include several examples to simplify the learning curve. If your
> > > > >> interested... I'd love to have you add your udf.
> > > > >> On Jul 19, 2015 12:59 PM, "Stefán Baxter" <
> > stefan@activitystream.com>
> > > > >> wrote:
> > > > >>
> > > > >>> Hi again,
> > > > >>>
> > > > >>> Going over the documentation once more I came across this:
> > > > >>>
> > > > >>> - Add the sources and classes JAR files to Drill’s classpath.
> > > > >>>
> > > > >>> I'm only including a standard .jar (with classes no sources) and
> I
> > > > wonder
> > > > >>> why this dependency is listed.
> > > > >>>
> > > > >>> Anyways, that it the next step.
> > > > >>>
> > > > >>> A sample .jar that works with the drill-config counterpart would
> > be a
> > > > >>> welcome addition to the documentation.
> > > > >>>
> > > > >>> Regards,
> > > > >>> -Stefan
> > > > >>>
> > > > >>>
> > > > >>> On Sun, Jul 19, 2015 at 5:53 PM, Stefán Baxter <
> > > > >> stefan@activitystream.com>
> > > > >>> wrote:
> > > > >>>
> > > > >>>> Hi Jim,
> > > > >>>>
> > > > >>>> Now I have added the file to the jar (both root and resources
> > > folder)
> > > > >> but
> > > > >>>> that does not seem to change anything.
> > > > >>>>
> > > > >>>> Any additional ideas?
> > > > >>>>
> > > > >>>> Regards,
> > > > >>>> -Stefan
> > > > >>>>
> > > > >>>> On Sun, Jul 19, 2015 at 5:40 PM, Stefán Baxter <
> > > > >>> stefan@activitystream.com>
> > > > >>>> wrote:
> > > > >>>>
> > > > >>>>> NO!
> > > > >>>>>
> > > > >>>>> Thank you , will do that right now :)
> > > > >>>>>
> > > > >>>>> On Sun, Jul 19, 2015 at 5:38 PM, Jim Bates <
> jbates@maprtech.com>
> > > > >> wrote:
> > > > >>>>>
> > > > >>>>>> Did you include a file drill-module.conf in your jar along
> with
> > > > >> source
> > > > >>>>>> files?
> > > > >>>>>> On Jul 19, 2015 12:20 PM, "Stefán Baxter" <
> > > > stefan@activitystream.com
> > > > >>>
> > > > >>>>>> wrote:
> > > > >>>>>>
> > > > >>>>>>> Hi,
> > > > >>>>>>>
> > > > >>>>>>> I'm trying to deploy a UDF that I have written according to
> the
> > > > >>>>>>> documentation.
> > > > >>>>>>>
> > > > >>>>>>> I have also:
> > > > >>>>>>>
> > > > >>>>>>> 1. Copied the jar file
> > > > >>>>>>> to jars/3rdparty
> > > > >>>>>>>
> > > > >>>>>>> 2. Changed the config "conf/drill-override.conf" to
> include:
> > > > >>>>>>> drill.logical.function.package +=
> > > > >>
> > ["org.apache.drill.exec.expr.fn.impl","com.activitystream.drill.udfs"]
> > > > >>>>>>>
> > > > >>>>>>> 3. Rerun the Drillbit
> > > > >>>>>>> bin/drillbit.sh start
> > > > >>>>>>>
> > > > >>>>>>> The log shows no sign of the the UDF being loaded but if I
> use
> > it
> > > > >> in
> > > > >>> a
> > > > >>>>>> SQL
> > > > >>>>>>> statement then this error is thrown:
> > > > >>>>>>>
> > > > >>>>>>> Error: PARSE ERROR: From line 1, column 184 to line 1, column
> > > 219:
> > > > >> No
> > > > >>>>>> match
> > > > >>>>>>> found for function signature asRoundTimeStamp(<ANY>,
> > <CHARACTER>)
> > > > >>>>>>>
> > > > >>>>>>>
> > > > >>>>>>> The relevant part of the UDF source:
> > > > >>>>>>>
> > > > >>>>>>> package com.activitystream.drill.udfs;
> > > > >>>>>>>
> > > > >>>>>>> public class ASUserDefinedFunctions {
> > > > >>>>>>>
> > > > >>>>>>> @FunctionTemplate(name = "asRoundTimeStamp", scope =
> > > > >>>>>>> FunctionTemplate.FunctionScope.SIMPLE, nulls =
> > > > >>>>>>> FunctionTemplate.NullHandling.NULL_IF_NULL)
> > > > >>>>>>>
> > > > >>>>>>> public static class RoundTimeStamp implements
> > DrillSimpleFunc
> > > > >>>>>>>
> > > > >>>>>>>
> > > > >>>>>>> Can anyone please assist?
> > > > >>>>>>>
> > > > >>>>>>> Regards,
> > > > >>>>>>> -Stefan
> > > > >>
> > > >
> > >
> >
>
Re: Drill not picking up a UDF
Posted by Stefán Baxter <st...@activitystream.com>.
Thank you.
I understand the import limitation and the whole custom-drillbit premise.
Will make the changes to see if this starts running :).
Regards,
-Stefan
On Mon, Jul 20, 2015 at 1:25 AM, Jim Bates <jb...@maprtech.com> wrote:
> I pulled out your udf class and threw it into my package. It worked for me
> with a few modifications.
>
> You can not have imported classes or method references in your eval or
> setup methods as the code will get pulled out and executed somewhere else
> and it won't be able to find it. With that in mind, Period will need to be
> org.joda.time.Period, DateTime will need to be org.joda.time.DateTime and
> roundTimeStamp will need to be
> com.activitystream.drill.udfs.ASUserDefinedFunctions.roundTimeStamp.
>
>
>
> On Sun, Jul 19, 2015 at 7:02 PM, Stefán Baxter <st...@activitystream.com>
> wrote:
>
> > Hi,
> >
> > The project can be found here:
> > https://github.com/acmeguy/asdrill
> >
> > Thank you,
> > -Stefán
> >
> > On Sun, Jul 19, 2015 at 11:57 PM, Stefán Baxter <
> stefan@activitystream.com
> > >
> > wrote:
> >
> > > Hi,
> > >
> > > I'm more than happy to share the little that is there (I will publish
> it
> > > on github and send link tomorrow).
> > >
> > > I ended up copying my UDF (singl-file.java) into the
> > simple-drill-function
> > > project where it got picked up.
> > >
> > > Then I discovered a whole new set of dependencies/limitations
> > >
> > > - The UDF are recompiled - any imports are invalid or at least
> > > overwritten
> > > - import org.joda.time.Period; (means that Period class is not
> > > resolved on runtime)
> > > - Error: SYSTEM ERROR: CompileException: Line 71, Column 26: Cannot
> > > determine simple type name "Period"
> > >
> > > - Calling any "outside" functions, like I was calling a static
> > > function of the new class (same file), leads to an errors as they
> are
> > not
> > > not resolved
> > > - Error: SYSTEM ERROR: CompileException: Line 70, Column 35: A
> method
> > > named "roundTimeStamp" is not declared in any enclosing class nor
> any
> > > supertype, nor through a static import
> > >
> > > Perhaps this was mentioned in the documentation but this is, at the
> very
> > > least, not straight forward and super-inviting.
> > >
> > > Thank you for your assistance, we will keep trying :)
> > >
> > > Regards,
> > > -Stefan
> > >
> > >
> > > On Sun, Jul 19, 2015 at 11:34 PM, Tugdual Grall <tu...@gmail.com>
> > wrote:
> > >
> > >> Hi Stefan,
> > >>
> > >> Do you think you can share your complete project ?
> > >>
> > >> This will help to debug it for you.
> > >>
> > >> T
> > >>
> > >> On Sunday, July 19, 2015, Stefán Baxter <st...@activitystream.com>
> > >> wrote:
> > >>
> > >> > Hi Ted,
> > >> >
> > >> > I fetched this, built it and deployed it without problems.
> > >> > I can not see any real difference other than this deploys two .jar
> (I
> > >> tried
> > >> > that as well earlier).
> > >> >
> > >> > I'm still trying to figure out why Drill is not picking up my UDFs
> > >> >
> > >> > Regards,
> > >> > -Stefán
> > >> >
> > >> > On Sun, Jul 19, 2015 at 10:45 PM, Ted Dunning <
> ted.dunning@gmail.com
> > >> > <javascript:;>> wrote:
> > >> >
> > >> > > Stefan,
> > >> > >
> > >> > > Have you seen this github project:
> > >> > >
> > >> > > https://github.com/mapr-demos/simple-drill-functions
> > >> > >
> > >> > > ?
> > >> > >
> > >> > >
> > >> > > On Sun, Jul 19, 2015 at 2:14 PM, Stefán Baxter <
> > >> > stefan@activitystream.com <javascript:;>>
> > >> > > wrote:
> > >> > >
> > >> > > > Hi Jim,
> > >> > > >
> > >> > > > I'm still not able to make this work. Do you have a sample .jar
> > file
> > >> > > with a
> > >> > > > small example that you are running?
> > >> > > >
> > >> > > > Regards,
> > >> > > > -Stefan
> > >> > > >
> > >> > > > On Sun, Jul 19, 2015 at 6:46 PM, Ted Dunning <
> > ted.dunning@gmail.com
> > >> > <javascript:;>>
> > >> > > > wrote:
> > >> > > >
> > >> > > > >
> > >> > > > > Sounds like a fine example, not because of sophistication but
> > >> because
> > >> > > it
> > >> > > > > deals with dates.
> > >> > > > >
> > >> > > > > Check the drill logs. It is likely that drill is grumpy about
> > >> > > something
> > >> > > > > in your udf or packaging.
> > >> > > > >
> > >> > > > > Also, feel free to snitch the pom from the simple examples in
> > >> order
> > >> > to
> > >> > > > get
> > >> > > > > the pieces assembled and packaged correctly.
> > >> > > > >
> > >> > > > > Sent from my iPhone
> > >> > > > >
> > >> > > > > > On Jul 19, 2015, at 11:25, Stefán Baxter <
> > >> > stefan@activitystream.com <javascript:;>>
> > >> > > > > wrote:
> > >> > > > > >
> > >> > > > > > Hi Jim,
> > >> > > > > >
> > >> > > > > > My UDF is currently so simple that I'm not sure you need it
> > (or
> > >> > want
> > >> > > > it).
> > >> > > > > >
> > >> > > > > > It basically just rounds a timestamp value with ISO 8601
> > periods
> > >> > > > > > asRoundTimestamp(timestampvalue,'PT10M').
> > >> > > > > >
> > >> > > > > > I would be more than happy to contribute to your project
> > rather
> > >> > than
> > >> > > > > > building our own :).
> > >> > > > > >
> > >> > > > > > Is the repo public?
> > >> > > > > >
> > >> > > > > > Regards,
> > >> > > > > > -Stefan
> > >> > > > > >
> > >> > > > > >
> > >> > > > > >
> > >> > > > > >> On Sun, Jul 19, 2015 at 6:18 PM, Jim Bates <
> > >> jbates@maprtech.com
> > >> > <javascript:;>>
> > >> > > > wrote:
> > >> > > > > >>
> > >> > > > > >> Maven will typically create a jar for class and a jar for
> > >> source
> > >> > > when
> > >> > > > > told
> > >> > > > > >> to do so. I just include the source files in the same jar
> as
> > >> the
> > >> > > class
> > >> > > > > >> files. There is a github example drill udf project we are
> > >> working
> > >> > on
> > >> > > > to
> > >> > > > > >> include several examples to simplify the learning curve. If
> > >> your
> > >> > > > > >> interested... I'd love to have you add your udf.
> > >> > > > > >> On Jul 19, 2015 12:59 PM, "Stefán Baxter" <
> > >> > > stefan@activitystream.com <javascript:;>>
> > >> > > > > >> wrote:
> > >> > > > > >>
> > >> > > > > >>> Hi again,
> > >> > > > > >>>
> > >> > > > > >>> Going over the documentation once more I came across this:
> > >> > > > > >>>
> > >> > > > > >>> - Add the sources and classes JAR files to Drill’s
> > >> classpath.
> > >> > > > > >>>
> > >> > > > > >>> I'm only including a standard .jar (with classes no
> sources)
> > >> and
> > >> > I
> > >> > > > > wonder
> > >> > > > > >>> why this dependency is listed.
> > >> > > > > >>>
> > >> > > > > >>> Anyways, that it the next step.
> > >> > > > > >>>
> > >> > > > > >>> A sample .jar that works with the drill-config counterpart
> > >> would
> > >> > > be a
> > >> > > > > >>> welcome addition to the documentation.
> > >> > > > > >>>
> > >> > > > > >>> Regards,
> > >> > > > > >>> -Stefan
> > >> > > > > >>>
> > >> > > > > >>>
> > >> > > > > >>> On Sun, Jul 19, 2015 at 5:53 PM, Stefán Baxter <
> > >> > > > > >> stefan@activitystream.com <javascript:;>>
> > >> > > > > >>> wrote:
> > >> > > > > >>>
> > >> > > > > >>>> Hi Jim,
> > >> > > > > >>>>
> > >> > > > > >>>> Now I have added the file to the jar (both root and
> > resources
> > >> > > > folder)
> > >> > > > > >> but
> > >> > > > > >>>> that does not seem to change anything.
> > >> > > > > >>>>
> > >> > > > > >>>> Any additional ideas?
> > >> > > > > >>>>
> > >> > > > > >>>> Regards,
> > >> > > > > >>>> -Stefan
> > >> > > > > >>>>
> > >> > > > > >>>> On Sun, Jul 19, 2015 at 5:40 PM, Stefán Baxter <
> > >> > > > > >>> stefan@activitystream.com <javascript:;>>
> > >> > > > > >>>> wrote:
> > >> > > > > >>>>
> > >> > > > > >>>>> NO!
> > >> > > > > >>>>>
> > >> > > > > >>>>> Thank you , will do that right now :)
> > >> > > > > >>>>>
> > >> > > > > >>>>> On Sun, Jul 19, 2015 at 5:38 PM, Jim Bates <
> > >> > jbates@maprtech.com <javascript:;>>
> > >> > > > > >> wrote:
> > >> > > > > >>>>>
> > >> > > > > >>>>>> Did you include a file drill-module.conf in your jar
> > along
> > >> > with
> > >> > > > > >> source
> > >> > > > > >>>>>> files?
> > >> > > > > >>>>>> On Jul 19, 2015 12:20 PM, "Stefán Baxter" <
> > >> > > > > stefan@activitystream.com <javascript:;>
> > >>
> > >> > > > > >>>
> > >> > > > > >>>>>> wrote:
> > >> > > > > >>>>>>
> > >> > > > > >>>>>>> Hi,
> > >> > > > > >>>>>>>
> > >> > > > > >>>>>>> I'm trying to deploy a UDF that I have written
> according
> > >> to
> > >> > the
> > >> > > > > >>>>>>> documentation.
> > >> > > > > >>>>>>>
> > >> > > > > >>>>>>> I have also:
> > >> > > > > >>>>>>>
> > >> > > > > >>>>>>> 1. Copied the jar file
> > >> > > > > >>>>>>> to jars/3rdparty
> > >> > > > > >>>>>>>
> > >> > > > > >>>>>>> 2. Changed the config "conf/drill-override.conf" to
> > >> > include:
> > >> > > > > >>>>>>> drill.logical.function.package +=
> > >> > > > > >>
> > >> > >
> > ["org.apache.drill.exec.expr.fn.impl","com.activitystream.drill.udfs"]
> > >> > > > > >>>>>>>
> > >> > > > > >>>>>>> 3. Rerun the Drillbit
> > >> > > > > >>>>>>> bin/drillbit.sh start
> > >> > > > > >>>>>>>
> > >> > > > > >>>>>>> The log shows no sign of the the UDF being loaded but
> > if I
> > >> > use
> > >> > > it
> > >> > > > > >> in
> > >> > > > > >>> a
> > >> > > > > >>>>>> SQL
> > >> > > > > >>>>>>> statement then this error is thrown:
> > >> > > > > >>>>>>>
> > >> > > > > >>>>>>> Error: PARSE ERROR: From line 1, column 184 to line 1,
> > >> column
> > >> > > > 219:
> > >> > > > > >> No
> > >> > > > > >>>>>> match
> > >> > > > > >>>>>>> found for function signature asRoundTimeStamp(<ANY>,
> > >> > > <CHARACTER>)
> > >> > > > > >>>>>>>
> > >> > > > > >>>>>>>
> > >> > > > > >>>>>>> The relevant part of the UDF source:
> > >> > > > > >>>>>>>
> > >> > > > > >>>>>>> package com.activitystream.drill.udfs;
> > >> > > > > >>>>>>>
> > >> > > > > >>>>>>> public class ASUserDefinedFunctions {
> > >> > > > > >>>>>>>
> > >> > > > > >>>>>>> @FunctionTemplate(name = "asRoundTimeStamp", scope
> =
> > >> > > > > >>>>>>> FunctionTemplate.FunctionScope.SIMPLE, nulls =
> > >> > > > > >>>>>>> FunctionTemplate.NullHandling.NULL_IF_NULL)
> > >> > > > > >>>>>>>
> > >> > > > > >>>>>>> public static class RoundTimeStamp implements
> > >> > > DrillSimpleFunc
> > >> > > > > >>>>>>>
> > >> > > > > >>>>>>>
> > >> > > > > >>>>>>> Can anyone please assist?
> > >> > > > > >>>>>>>
> > >> > > > > >>>>>>> Regards,
> > >> > > > > >>>>>>> -Stefan
> > >> > > > > >>
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> > >
> > >
> >
>
Re: Drill not picking up a UDF
Posted by Tugdual Grall <tu...@gmail.com>.
Hi,
I have created this content, hope this will help people to develop new
function:
http://tgrall.github.io/blog/2015/07/21/apache-drill-how-to-create-a-new-function/
Regards
Tug
@tgrall
On Mon, Jul 20, 2015 at 2:28 PM, Ted Dunning <te...@gmail.com> wrote:
> No apologies needed. We welcome constructive criticism. I just wanted to
> put it in context for the rest of the readers of this list.
>
>
>
> On Mon, Jul 20, 2015 at 1:18 PM, Andrew Brust <
> andrew.brust@bluebadgeinsights.com> wrote:
>
> > Yes, agreed. Fair point and apologies for not articulating same in my
> > comment.
> >
> >
> >
> >
> > On 7/20/15, 4:10 PM, "Ted Dunning" <te...@gmail.com> wrote:
> >
> > >On Mon, Jul 20, 2015 at 8:22 AM, Andrew Brust <
> > >andrew.brust@bluebadgeinsights.com> wrote:
> > >
> > >> 1. It seems to me like Drill is at a point where, if you thread the
> > needle
> > >> perfectly, things generally work as advertised. That’s certainly an
> > >> advance over the old, old days, where stuff that should have worked
> > >> sometimes just didn’t.
> > >> 2. Threading that needle can be super-hard, even for an experienced
> Java
> > >> developer.
> > >>
> > >
> > >This is definitely true for the problem of *extending* Drill.
> > >
> > >It is much less of a problem for *using* Drill, which is what the team
> has
> > >spent much more effort on. See the Drill in 10 minutes article in the
> > docs.
> > >
> > >To my mind, this prioritization is correct. But it shouldn't be
> exclusive
> > >either.
> > >
> > >This is not to say that user-level fit and finish is done by any means,
> > >just that people who extend the system will get lots more splinters.
> >
>
Re: Drill not picking up a UDF
Posted by Ted Dunning <te...@gmail.com>.
No apologies needed. We welcome constructive criticism. I just wanted to
put it in context for the rest of the readers of this list.
On Mon, Jul 20, 2015 at 1:18 PM, Andrew Brust <
andrew.brust@bluebadgeinsights.com> wrote:
> Yes, agreed. Fair point and apologies for not articulating same in my
> comment.
>
>
>
>
> On 7/20/15, 4:10 PM, "Ted Dunning" <te...@gmail.com> wrote:
>
> >On Mon, Jul 20, 2015 at 8:22 AM, Andrew Brust <
> >andrew.brust@bluebadgeinsights.com> wrote:
> >
> >> 1. It seems to me like Drill is at a point where, if you thread the
> needle
> >> perfectly, things generally work as advertised. That’s certainly an
> >> advance over the old, old days, where stuff that should have worked
> >> sometimes just didn’t.
> >> 2. Threading that needle can be super-hard, even for an experienced Java
> >> developer.
> >>
> >
> >This is definitely true for the problem of *extending* Drill.
> >
> >It is much less of a problem for *using* Drill, which is what the team has
> >spent much more effort on. See the Drill in 10 minutes article in the
> docs.
> >
> >To my mind, this prioritization is correct. But it shouldn't be exclusive
> >either.
> >
> >This is not to say that user-level fit and finish is done by any means,
> >just that people who extend the system will get lots more splinters.
>
Re: Drill not picking up a UDF
Posted by Andrew Brust <an...@bluebadgeinsights.com>.
Yes, agreed. Fair point and apologies for not articulating same in my comment.
On 7/20/15, 4:10 PM, "Ted Dunning" <te...@gmail.com> wrote:
>On Mon, Jul 20, 2015 at 8:22 AM, Andrew Brust <
>andrew.brust@bluebadgeinsights.com> wrote:
>
>> 1. It seems to me like Drill is at a point where, if you thread the needle
>> perfectly, things generally work as advertised. That’s certainly an
>> advance over the old, old days, where stuff that should have worked
>> sometimes just didn’t.
>> 2. Threading that needle can be super-hard, even for an experienced Java
>> developer.
>>
>
>This is definitely true for the problem of *extending* Drill.
>
>It is much less of a problem for *using* Drill, which is what the team has
>spent much more effort on. See the Drill in 10 minutes article in the docs.
>
>To my mind, this prioritization is correct. But it shouldn't be exclusive
>either.
>
>This is not to say that user-level fit and finish is done by any means,
>just that people who extend the system will get lots more splinters.
Re: Drill not picking up a UDF
Posted by Ted Dunning <te...@gmail.com>.
On Mon, Jul 20, 2015 at 8:22 AM, Andrew Brust <
andrew.brust@bluebadgeinsights.com> wrote:
> 1. It seems to me like Drill is at a point where, if you thread the needle
> perfectly, things generally work as advertised. That’s certainly an
> advance over the old, old days, where stuff that should have worked
> sometimes just didn’t.
> 2. Threading that needle can be super-hard, even for an experienced Java
> developer.
>
This is definitely true for the problem of *extending* Drill.
It is much less of a problem for *using* Drill, which is what the team has
spent much more effort on. See the Drill in 10 minutes article in the docs.
To my mind, this prioritization is correct. But it shouldn't be exclusive
either.
This is not to say that user-level fit and finish is done by any means,
just that people who extend the system will get lots more splinters.
Re: Drill not picking up a UDF
Posted by Andrew Brust <an...@bluebadgeinsights.com>.
(Heh, sorry, guess that was *five* observations :-) )
On 7/20/15, 11:22 AM, "Andrew Brust" <an...@bluebadgeinsights.com> wrote:
>Hi, all. I’m just a fly on the wall here, as usual. This was an interesting thread to follow. Two observations:
>
>1. It seems to me like Drill is at a point where, if you thread the needle perfectly, things generally work as advertised. That’s certainly an advance over the old, old days, where stuff that should have worked sometimes just didn’t.
>2. Threading that needle can be super-hard, even for an experienced Java developer.
>3. If I were to compare all of this to something from my world — .NET CLR-based UDFs in SQL Server (2005 and later) — the fit and finish on the Drill side isn’t anywhere close. That’s not a fair comparison, of course. But it does seem to point out a goal, especially because it seems the harder work is done. Another thing to consider is that SQL Server CLR features are often disabled by DBAs, and Drill won’t have that problem. So the Drill project could take a familiar feature from the RDBMS world, and make it much more useful.
>4. Overall, I’m encouraged to see all this and think it’s great that tenacious people like Stefan are willing to slog through until things work.
>5. I’m seeing greater traction for Drill across the industry and expect it will grow. Keep up the good work!
>
>
>
>On 7/20/15, 10:23 AM, "Jim Bates" <jb...@maprtech.com> wrote:
>
>>That's an apt description. The Holders that have a fixed size value objects
>>are simple to get at but the ones that have variable length objects are
>>pulled via the buffer.
>>
>>You can also use the StringFunctionHelpers.
>>
>>org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(in
>>.start, in.end, in.buffer)
>>
>>Where 'in' is
>>
>>@Param NullableVarCharHolder in
>>
>>or
>>
>>@Param VarCharHolder in
>>
>>On Mon, Jul 20, 2015 at 9:07 AM, Stefán Baxter <st...@activitystream.com>
>>wrote:
>>
>>> Hi,
>>>
>>> After going through the log this is clear what is happening (once the Drill
>>> picked up the UDF a bit earlier this morning).
>>>
>>> I'm calling the VarCharHolder.toString() to get the text value for the
>>> parameter and that is throwing this exception.
>>>
>>> Two observations:
>>>
>>> 1. Calling a deprecated function, even though that its not optimal,
>>> usually does not cause such drastic results.
>>> 2. As far as I can see there is no easy way to get the string value of a
>>> property without carving out a piece of the buffer.
>>>
>>> What am I missing here?
>>>
>>> Regards,
>>> -Stefan
>>>
>>>
>>> On Mon, Jul 20, 2015 at 12:58 PM, Jacques Nadeau <ja...@apache.org>
>>> wrote:
>>>
>>> > Can you enable verbose errors at the session level? It may reveal more
>>> > about what is failing.
>>> > On Jul 20, 2015 5:32 AM, "Stefán Baxter" <st...@activitystream.com>
>>> > wrote:
>>> >
>>> > > Hi Jim,
>>> > >
>>> > > I have made those changes and I'm wondering if you can runs this be
>>> using
>>> > > the two .jar files that the "mvn package" places in the target
>>> directory?
>>> > >
>>> > > I have tried to have Drill pick those up but the error now is:
>>> > >
>>> > > Error: SYSTEM ERROR: UnsupportedOperationException
>>> > > Fragment 0:0
>>> > > [Error Id: da589dd4-4cfd-4659-8b93-219074ab8c72 on localhost:31010]
>>> > > (state=,code=0)
>>> > >
>>> > > It seems to indicate that it's picking up the functions but that they
>>> can
>>> > > not be run.
>>> > >
>>> > > Regards,
>>> > > - Stefán
>>> > >
>>> > > On Mon, Jul 20, 2015 at 1:25 AM, Jim Bates <jb...@maprtech.com>
>>> wrote:
>>> > >
>>> > > > I pulled out your udf class and threw it into my package. It worked
>>> for
>>> > > me
>>> > > > with a few modifications.
>>> > > >
>>> > > > You can not have imported classes or method references in your eval
>>> or
>>> > > > setup methods as the code will get pulled out and executed somewhere
>>> > else
>>> > > > and it won't be able to find it. With that in mind, Period will need
>>> > to
>>> > > be
>>> > > > org.joda.time.Period, DateTime will need to be org.joda.time.DateTime
>>> > and
>>> > > > roundTimeStamp will need to be
>>> > > > com.activitystream.drill.udfs.ASUserDefinedFunctions.roundTimeStamp.
>>> > > >
>>> > > >
>>> > > >
>>> > > > On Sun, Jul 19, 2015 at 7:02 PM, Stefán Baxter <
>>> > > stefan@activitystream.com>
>>> > > > wrote:
>>> > > >
>>> > > > > Hi,
>>> > > > >
>>> > > > > The project can be found here:
>>> > > > > https://github.com/acmeguy/asdrill
>>> > > > >
>>> > > > > Thank you,
>>> > > > > -Stefán
>>> > > > >
>>> > > > > On Sun, Jul 19, 2015 at 11:57 PM, Stefán Baxter <
>>> > > > stefan@activitystream.com
>>> > > > > >
>>> > > > > wrote:
>>> > > > >
>>> > > > > > Hi,
>>> > > > > >
>>> > > > > > I'm more than happy to share the little that is there (I will
>>> > publish
>>> > > > it
>>> > > > > > on github and send link tomorrow).
>>> > > > > >
>>> > > > > > I ended up copying my UDF (singl-file.java) into the
>>> > > > > simple-drill-function
>>> > > > > > project where it got picked up.
>>> > > > > >
>>> > > > > > Then I discovered a whole new set of dependencies/limitations
>>> > > > > >
>>> > > > > > - The UDF are recompiled - any imports are invalid or at least
>>> > > > > > overwritten
>>> > > > > > - import org.joda.time.Period; (means that Period class is
>>> not
>>> > > > > > resolved on runtime)
>>> > > > > > - Error: SYSTEM ERROR: CompileException: Line 71, Column 26:
>>> > > Cannot
>>> > > > > > determine simple type name "Period"
>>> > > > > >
>>> > > > > > - Calling any "outside" functions, like I was calling a static
>>> > > > > > function of the new class (same file), leads to an errors as
>>> > they
>>> > > > are
>>> > > > > not
>>> > > > > > not resolved
>>> > > > > > - Error: SYSTEM ERROR: CompileException: Line 70, Column 35:
>>> A
>>> > > > method
>>> > > > > > named "roundTimeStamp" is not declared in any enclosing class
>>> > nor
>>> > > > any
>>> > > > > > supertype, nor through a static import
>>> > > > > >
>>> > > > > > Perhaps this was mentioned in the documentation but this is, at
>>> the
>>> > > > very
>>> > > > > > least, not straight forward and super-inviting.
>>> > > > > >
>>> > > > > > Thank you for your assistance, we will keep trying :)
>>> > > > > >
>>> > > > > > Regards,
>>> > > > > > -Stefan
>>> > > > > >
>>> > > > > >
>>> > > > > > On Sun, Jul 19, 2015 at 11:34 PM, Tugdual Grall <
>>> tugdual@gmail.com
>>> > >
>>> > > > > wrote:
>>> > > > > >
>>> > > > > >> Hi Stefan,
>>> > > > > >>
>>> > > > > >> Do you think you can share your complete project ?
>>> > > > > >>
>>> > > > > >> This will help to debug it for you.
>>> > > > > >>
>>> > > > > >> T
>>> > > > > >>
>>> > > > > >> On Sunday, July 19, 2015, Stefán Baxter <
>>> > stefan@activitystream.com>
>>> > > > > >> wrote:
>>> > > > > >>
>>> > > > > >> > Hi Ted,
>>> > > > > >> >
>>> > > > > >> > I fetched this, built it and deployed it without problems.
>>> > > > > >> > I can not see any real difference other than this deploys two
>>> > .jar
>>> > > > (I
>>> > > > > >> tried
>>> > > > > >> > that as well earlier).
>>> > > > > >> >
>>> > > > > >> > I'm still trying to figure out why Drill is not picking up my
>>> > UDFs
>>> > > > > >> >
>>> > > > > >> > Regards,
>>> > > > > >> > -Stefán
>>> > > > > >> >
>>> > > > > >> > On Sun, Jul 19, 2015 at 10:45 PM, Ted Dunning <
>>> > > > ted.dunning@gmail.com
>>> > > > > >> > <javascript:;>> wrote:
>>> > > > > >> >
>>> > > > > >> > > Stefan,
>>> > > > > >> > >
>>> > > > > >> > > Have you seen this github project:
>>> > > > > >> > >
>>> > > > > >> > > https://github.com/mapr-demos/simple-drill-functions
>>> > > > > >> > >
>>> > > > > >> > > ?
>>> > > > > >> > >
>>> > > > > >> > >
>>> > > > > >> > > On Sun, Jul 19, 2015 at 2:14 PM, Stefán Baxter <
>>> > > > > >> > stefan@activitystream.com <javascript:;>>
>>> > > > > >> > > wrote:
>>> > > > > >> > >
>>> > > > > >> > > > Hi Jim,
>>> > > > > >> > > >
>>> > > > > >> > > > I'm still not able to make this work. Do you have a sample
>>> > > .jar
>>> > > > > file
>>> > > > > >> > > with a
>>> > > > > >> > > > small example that you are running?
>>> > > > > >> > > >
>>> > > > > >> > > > Regards,
>>> > > > > >> > > > -Stefan
>>> > > > > >> > > >
>>> > > > > >> > > > On Sun, Jul 19, 2015 at 6:46 PM, Ted Dunning <
>>> > > > > ted.dunning@gmail.com
>>> > > > > >> > <javascript:;>>
>>> > > > > >> > > > wrote:
>>> > > > > >> > > >
>>> > > > > >> > > > >
>>> > > > > >> > > > > Sounds like a fine example, not because of
>>> sophistication
>>> > > but
>>> > > > > >> because
>>> > > > > >> > > it
>>> > > > > >> > > > > deals with dates.
>>> > > > > >> > > > >
>>> > > > > >> > > > > Check the drill logs. It is likely that drill is grumpy
>>> > > about
>>> > > > > >> > > something
>>> > > > > >> > > > > in your udf or packaging.
>>> > > > > >> > > > >
>>> > > > > >> > > > > Also, feel free to snitch the pom from the simple
>>> examples
>>> > > in
>>> > > > > >> order
>>> > > > > >> > to
>>> > > > > >> > > > get
>>> > > > > >> > > > > the pieces assembled and packaged correctly.
>>> > > > > >> > > > >
>>> > > > > >> > > > > Sent from my iPhone
>>> > > > > >> > > > >
>>> > > > > >> > > > > > On Jul 19, 2015, at 11:25, Stefán Baxter <
>>> > > > > >> > stefan@activitystream.com <javascript:;>>
>>> > > > > >> > > > > wrote:
>>> > > > > >> > > > > >
>>> > > > > >> > > > > > Hi Jim,
>>> > > > > >> > > > > >
>>> > > > > >> > > > > > My UDF is currently so simple that I'm not sure you
>>> need
>>> > > it
>>> > > > > (or
>>> > > > > >> > want
>>> > > > > >> > > > it).
>>> > > > > >> > > > > >
>>> > > > > >> > > > > > It basically just rounds a timestamp value with ISO
>>> 8601
>>> > > > > periods
>>> > > > > >> > > > > > asRoundTimestamp(timestampvalue,'PT10M').
>>> > > > > >> > > > > >
>>> > > > > >> > > > > > I would be more than happy to contribute to your
>>> project
>>> > > > > rather
>>> > > > > >> > than
>>> > > > > >> > > > > > building our own :).
>>> > > > > >> > > > > >
>>> > > > > >> > > > > > Is the repo public?
>>> > > > > >> > > > > >
>>> > > > > >> > > > > > Regards,
>>> > > > > >> > > > > > -Stefan
>>> > > > > >> > > > > >
>>> > > > > >> > > > > >
>>> > > > > >> > > > > >
>>> > > > > >> > > > > >> On Sun, Jul 19, 2015 at 6:18 PM, Jim Bates <
>>> > > > > >> jbates@maprtech.com
>>> > > > > >> > <javascript:;>>
>>> > > > > >> > > > wrote:
>>> > > > > >> > > > > >>
>>> > > > > >> > > > > >> Maven will typically create a jar for class and a jar
>>> > for
>>> > > > > >> source
>>> > > > > >> > > when
>>> > > > > >> > > > > told
>>> > > > > >> > > > > >> to do so. I just include the source files in the same
>>> > jar
>>> > > > as
>>> > > > > >> the
>>> > > > > >> > > class
>>> > > > > >> > > > > >> files. There is a github example drill udf project we
>>> > are
>>> > > > > >> working
>>> > > > > >> > on
>>> > > > > >> > > > to
>>> > > > > >> > > > > >> include several examples to simplify the learning
>>> > curve.
>>> > > If
>>> > > > > >> your
>>> > > > > >> > > > > >> interested... I'd love to have you add your udf.
>>> > > > > >> > > > > >> On Jul 19, 2015 12:59 PM, "Stefán Baxter" <
>>> > > > > >> > > stefan@activitystream.com <javascript:;>>
>>> > > > > >> > > > > >> wrote:
>>> > > > > >> > > > > >>
>>> > > > > >> > > > > >>> Hi again,
>>> > > > > >> > > > > >>>
>>> > > > > >> > > > > >>> Going over the documentation once more I came across
>>> > > this:
>>> > > > > >> > > > > >>>
>>> > > > > >> > > > > >>> - Add the sources and classes JAR files to Drill’s
>>> > > > > >> classpath.
>>> > > > > >> > > > > >>>
>>> > > > > >> > > > > >>> I'm only including a standard .jar (with classes no
>>> > > > sources)
>>> > > > > >> and
>>> > > > > >> > I
>>> > > > > >> > > > > wonder
>>> > > > > >> > > > > >>> why this dependency is listed.
>>> > > > > >> > > > > >>>
>>> > > > > >> > > > > >>> Anyways, that it the next step.
>>> > > > > >> > > > > >>>
>>> > > > > >> > > > > >>> A sample .jar that works with the drill-config
>>> > > counterpart
>>> > > > > >> would
>>> > > > > >> > > be a
>>> > > > > >> > > > > >>> welcome addition to the documentation.
>>> > > > > >> > > > > >>>
>>> > > > > >> > > > > >>> Regards,
>>> > > > > >> > > > > >>> -Stefan
>>> > > > > >> > > > > >>>
>>> > > > > >> > > > > >>>
>>> > > > > >> > > > > >>> On Sun, Jul 19, 2015 at 5:53 PM, Stefán Baxter <
>>> > > > > >> > > > > >> stefan@activitystream.com <javascript:;>>
>>> > > > > >> > > > > >>> wrote:
>>> > > > > >> > > > > >>>
>>> > > > > >> > > > > >>>> Hi Jim,
>>> > > > > >> > > > > >>>>
>>> > > > > >> > > > > >>>> Now I have added the file to the jar (both root and
>>> > > > > resources
>>> > > > > >> > > > folder)
>>> > > > > >> > > > > >> but
>>> > > > > >> > > > > >>>> that does not seem to change anything.
>>> > > > > >> > > > > >>>>
>>> > > > > >> > > > > >>>> Any additional ideas?
>>> > > > > >> > > > > >>>>
>>> > > > > >> > > > > >>>> Regards,
>>> > > > > >> > > > > >>>> -Stefan
>>> > > > > >> > > > > >>>>
>>> > > > > >> > > > > >>>> On Sun, Jul 19, 2015 at 5:40 PM, Stefán Baxter <
>>> > > > > >> > > > > >>> stefan@activitystream.com <javascript:;>>
>>> > > > > >> > > > > >>>> wrote:
>>> > > > > >> > > > > >>>>
>>> > > > > >> > > > > >>>>> NO!
>>> > > > > >> > > > > >>>>>
>>> > > > > >> > > > > >>>>> Thank you , will do that right now :)
>>> > > > > >> > > > > >>>>>
>>> > > > > >> > > > > >>>>> On Sun, Jul 19, 2015 at 5:38 PM, Jim Bates <
>>> > > > > >> > jbates@maprtech.com <javascript:;>>
>>> > > > > >> > > > > >> wrote:
>>> > > > > >> > > > > >>>>>
>>> > > > > >> > > > > >>>>>> Did you include a file drill-module.conf in your
>>> > jar
>>> > > > > along
>>> > > > > >> > with
>>> > > > > >> > > > > >> source
>>> > > > > >> > > > > >>>>>> files?
>>> > > > > >> > > > > >>>>>> On Jul 19, 2015 12:20 PM, "Stefán Baxter" <
>>> > > > > >> > > > > stefan@activitystream.com <javascript:;>
>>> > > > > >>
>>> > > > > >> > > > > >>>
>>> > > > > >> > > > > >>>>>> wrote:
>>> > > > > >> > > > > >>>>>>
>>> > > > > >> > > > > >>>>>>> Hi,
>>> > > > > >> > > > > >>>>>>>
>>> > > > > >> > > > > >>>>>>> I'm trying to deploy a UDF that I have written
>>> > > > according
>>> > > > > >> to
>>> > > > > >> > the
>>> > > > > >> > > > > >>>>>>> documentation.
>>> > > > > >> > > > > >>>>>>>
>>> > > > > >> > > > > >>>>>>> I have also:
>>> > > > > >> > > > > >>>>>>>
>>> > > > > >> > > > > >>>>>>> 1. Copied the jar file
>>> > > > > >> > > > > >>>>>>> to jars/3rdparty
>>> > > > > >> > > > > >>>>>>>
>>> > > > > >> > > > > >>>>>>> 2. Changed the config
>>> "conf/drill-override.conf"
>>> > > to
>>> > > > > >> > include:
>>> > > > > >> > > > > >>>>>>> drill.logical.function.package +=
>>> > > > > >> > > > > >>
>>> > > > > >> > >
>>> > > > >
>>> > ["org.apache.drill.exec.expr.fn.impl","com.activitystream.drill.udfs"]
>>> > > > > >> > > > > >>>>>>>
>>> > > > > >> > > > > >>>>>>> 3. Rerun the Drillbit
>>> > > > > >> > > > > >>>>>>> bin/drillbit.sh start
>>> > > > > >> > > > > >>>>>>>
>>> > > > > >> > > > > >>>>>>> The log shows no sign of the the UDF being
>>> loaded
>>> > > but
>>> > > > > if I
>>> > > > > >> > use
>>> > > > > >> > > it
>>> > > > > >> > > > > >> in
>>> > > > > >> > > > > >>> a
>>> > > > > >> > > > > >>>>>> SQL
>>> > > > > >> > > > > >>>>>>> statement then this error is thrown:
>>> > > > > >> > > > > >>>>>>>
>>> > > > > >> > > > > >>>>>>> Error: PARSE ERROR: From line 1, column 184 to
>>> > line
>>> > > 1,
>>> > > > > >> column
>>> > > > > >> > > > 219:
>>> > > > > >> > > > > >> No
>>> > > > > >> > > > > >>>>>> match
>>> > > > > >> > > > > >>>>>>> found for function signature
>>> > asRoundTimeStamp(<ANY>,
>>> > > > > >> > > <CHARACTER>)
>>> > > > > >> > > > > >>>>>>>
>>> > > > > >> > > > > >>>>>>>
>>> > > > > >> > > > > >>>>>>> The relevant part of the UDF source:
>>> > > > > >> > > > > >>>>>>>
>>> > > > > >> > > > > >>>>>>> package com.activitystream.drill.udfs;
>>> > > > > >> > > > > >>>>>>>
>>> > > > > >> > > > > >>>>>>> public class ASUserDefinedFunctions {
>>> > > > > >> > > > > >>>>>>>
>>> > > > > >> > > > > >>>>>>> @FunctionTemplate(name = "asRoundTimeStamp",
>>> > > scope
>>> > > > =
>>> > > > > >> > > > > >>>>>>> FunctionTemplate.FunctionScope.SIMPLE, nulls =
>>> > > > > >> > > > > >>>>>>> FunctionTemplate.NullHandling.NULL_IF_NULL)
>>> > > > > >> > > > > >>>>>>>
>>> > > > > >> > > > > >>>>>>> public static class RoundTimeStamp implements
>>> > > > > >> > > DrillSimpleFunc
>>> > > > > >> > > > > >>>>>>>
>>> > > > > >> > > > > >>>>>>>
>>> > > > > >> > > > > >>>>>>> Can anyone please assist?
>>> > > > > >> > > > > >>>>>>>
>>> > > > > >> > > > > >>>>>>> Regards,
>>> > > > > >> > > > > >>>>>>> -Stefan
>>> > > > > >> > > > > >>
>>> > > > > >> > > > >
>>> > > > > >> > > >
>>> > > > > >> > >
>>> > > > > >> >
>>> > > > > >>
>>> > > > > >
>>> > > > > >
>>> > > > >
>>> > > >
>>> > >
>>> >
>>>
Re: Drill not picking up a UDF
Posted by Andrew Brust <an...@bluebadgeinsights.com>.
Hi, all. I’m just a fly on the wall here, as usual. This was an interesting thread to follow. Two observations:
1. It seems to me like Drill is at a point where, if you thread the needle perfectly, things generally work as advertised. That’s certainly an advance over the old, old days, where stuff that should have worked sometimes just didn’t.
2. Threading that needle can be super-hard, even for an experienced Java developer.
3. If I were to compare all of this to something from my world — .NET CLR-based UDFs in SQL Server (2005 and later) — the fit and finish on the Drill side isn’t anywhere close. That’s not a fair comparison, of course. But it does seem to point out a goal, especially because it seems the harder work is done. Another thing to consider is that SQL Server CLR features are often disabled by DBAs, and Drill won’t have that problem. So the Drill project could take a familiar feature from the RDBMS world, and make it much more useful.
4. Overall, I’m encouraged to see all this and think it’s great that tenacious people like Stefan are willing to slog through until things work.
5. I’m seeing greater traction for Drill across the industry and expect it will grow. Keep up the good work!
On 7/20/15, 10:23 AM, "Jim Bates" <jb...@maprtech.com> wrote:
>That's an apt description. The Holders that have a fixed size value objects
>are simple to get at but the ones that have variable length objects are
>pulled via the buffer.
>
>You can also use the StringFunctionHelpers.
>
>org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(in
>.start, in.end, in.buffer)
>
>Where 'in' is
>
>@Param NullableVarCharHolder in
>
>or
>
>@Param VarCharHolder in
>
>On Mon, Jul 20, 2015 at 9:07 AM, Stefán Baxter <st...@activitystream.com>
>wrote:
>
>> Hi,
>>
>> After going through the log this is clear what is happening (once the Drill
>> picked up the UDF a bit earlier this morning).
>>
>> I'm calling the VarCharHolder.toString() to get the text value for the
>> parameter and that is throwing this exception.
>>
>> Two observations:
>>
>> 1. Calling a deprecated function, even though that its not optimal,
>> usually does not cause such drastic results.
>> 2. As far as I can see there is no easy way to get the string value of a
>> property without carving out a piece of the buffer.
>>
>> What am I missing here?
>>
>> Regards,
>> -Stefan
>>
>>
>> On Mon, Jul 20, 2015 at 12:58 PM, Jacques Nadeau <ja...@apache.org>
>> wrote:
>>
>> > Can you enable verbose errors at the session level? It may reveal more
>> > about what is failing.
>> > On Jul 20, 2015 5:32 AM, "Stefán Baxter" <st...@activitystream.com>
>> > wrote:
>> >
>> > > Hi Jim,
>> > >
>> > > I have made those changes and I'm wondering if you can runs this be
>> using
>> > > the two .jar files that the "mvn package" places in the target
>> directory?
>> > >
>> > > I have tried to have Drill pick those up but the error now is:
>> > >
>> > > Error: SYSTEM ERROR: UnsupportedOperationException
>> > > Fragment 0:0
>> > > [Error Id: da589dd4-4cfd-4659-8b93-219074ab8c72 on localhost:31010]
>> > > (state=,code=0)
>> > >
>> > > It seems to indicate that it's picking up the functions but that they
>> can
>> > > not be run.
>> > >
>> > > Regards,
>> > > - Stefán
>> > >
>> > > On Mon, Jul 20, 2015 at 1:25 AM, Jim Bates <jb...@maprtech.com>
>> wrote:
>> > >
>> > > > I pulled out your udf class and threw it into my package. It worked
>> for
>> > > me
>> > > > with a few modifications.
>> > > >
>> > > > You can not have imported classes or method references in your eval
>> or
>> > > > setup methods as the code will get pulled out and executed somewhere
>> > else
>> > > > and it won't be able to find it. With that in mind, Period will need
>> > to
>> > > be
>> > > > org.joda.time.Period, DateTime will need to be org.joda.time.DateTime
>> > and
>> > > > roundTimeStamp will need to be
>> > > > com.activitystream.drill.udfs.ASUserDefinedFunctions.roundTimeStamp.
>> > > >
>> > > >
>> > > >
>> > > > On Sun, Jul 19, 2015 at 7:02 PM, Stefán Baxter <
>> > > stefan@activitystream.com>
>> > > > wrote:
>> > > >
>> > > > > Hi,
>> > > > >
>> > > > > The project can be found here:
>> > > > > https://github.com/acmeguy/asdrill
>> > > > >
>> > > > > Thank you,
>> > > > > -Stefán
>> > > > >
>> > > > > On Sun, Jul 19, 2015 at 11:57 PM, Stefán Baxter <
>> > > > stefan@activitystream.com
>> > > > > >
>> > > > > wrote:
>> > > > >
>> > > > > > Hi,
>> > > > > >
>> > > > > > I'm more than happy to share the little that is there (I will
>> > publish
>> > > > it
>> > > > > > on github and send link tomorrow).
>> > > > > >
>> > > > > > I ended up copying my UDF (singl-file.java) into the
>> > > > > simple-drill-function
>> > > > > > project where it got picked up.
>> > > > > >
>> > > > > > Then I discovered a whole new set of dependencies/limitations
>> > > > > >
>> > > > > > - The UDF are recompiled - any imports are invalid or at least
>> > > > > > overwritten
>> > > > > > - import org.joda.time.Period; (means that Period class is
>> not
>> > > > > > resolved on runtime)
>> > > > > > - Error: SYSTEM ERROR: CompileException: Line 71, Column 26:
>> > > Cannot
>> > > > > > determine simple type name "Period"
>> > > > > >
>> > > > > > - Calling any "outside" functions, like I was calling a static
>> > > > > > function of the new class (same file), leads to an errors as
>> > they
>> > > > are
>> > > > > not
>> > > > > > not resolved
>> > > > > > - Error: SYSTEM ERROR: CompileException: Line 70, Column 35:
>> A
>> > > > method
>> > > > > > named "roundTimeStamp" is not declared in any enclosing class
>> > nor
>> > > > any
>> > > > > > supertype, nor through a static import
>> > > > > >
>> > > > > > Perhaps this was mentioned in the documentation but this is, at
>> the
>> > > > very
>> > > > > > least, not straight forward and super-inviting.
>> > > > > >
>> > > > > > Thank you for your assistance, we will keep trying :)
>> > > > > >
>> > > > > > Regards,
>> > > > > > -Stefan
>> > > > > >
>> > > > > >
>> > > > > > On Sun, Jul 19, 2015 at 11:34 PM, Tugdual Grall <
>> tugdual@gmail.com
>> > >
>> > > > > wrote:
>> > > > > >
>> > > > > >> Hi Stefan,
>> > > > > >>
>> > > > > >> Do you think you can share your complete project ?
>> > > > > >>
>> > > > > >> This will help to debug it for you.
>> > > > > >>
>> > > > > >> T
>> > > > > >>
>> > > > > >> On Sunday, July 19, 2015, Stefán Baxter <
>> > stefan@activitystream.com>
>> > > > > >> wrote:
>> > > > > >>
>> > > > > >> > Hi Ted,
>> > > > > >> >
>> > > > > >> > I fetched this, built it and deployed it without problems.
>> > > > > >> > I can not see any real difference other than this deploys two
>> > .jar
>> > > > (I
>> > > > > >> tried
>> > > > > >> > that as well earlier).
>> > > > > >> >
>> > > > > >> > I'm still trying to figure out why Drill is not picking up my
>> > UDFs
>> > > > > >> >
>> > > > > >> > Regards,
>> > > > > >> > -Stefán
>> > > > > >> >
>> > > > > >> > On Sun, Jul 19, 2015 at 10:45 PM, Ted Dunning <
>> > > > ted.dunning@gmail.com
>> > > > > >> > <javascript:;>> wrote:
>> > > > > >> >
>> > > > > >> > > Stefan,
>> > > > > >> > >
>> > > > > >> > > Have you seen this github project:
>> > > > > >> > >
>> > > > > >> > > https://github.com/mapr-demos/simple-drill-functions
>> > > > > >> > >
>> > > > > >> > > ?
>> > > > > >> > >
>> > > > > >> > >
>> > > > > >> > > On Sun, Jul 19, 2015 at 2:14 PM, Stefán Baxter <
>> > > > > >> > stefan@activitystream.com <javascript:;>>
>> > > > > >> > > wrote:
>> > > > > >> > >
>> > > > > >> > > > Hi Jim,
>> > > > > >> > > >
>> > > > > >> > > > I'm still not able to make this work. Do you have a sample
>> > > .jar
>> > > > > file
>> > > > > >> > > with a
>> > > > > >> > > > small example that you are running?
>> > > > > >> > > >
>> > > > > >> > > > Regards,
>> > > > > >> > > > -Stefan
>> > > > > >> > > >
>> > > > > >> > > > On Sun, Jul 19, 2015 at 6:46 PM, Ted Dunning <
>> > > > > ted.dunning@gmail.com
>> > > > > >> > <javascript:;>>
>> > > > > >> > > > wrote:
>> > > > > >> > > >
>> > > > > >> > > > >
>> > > > > >> > > > > Sounds like a fine example, not because of
>> sophistication
>> > > but
>> > > > > >> because
>> > > > > >> > > it
>> > > > > >> > > > > deals with dates.
>> > > > > >> > > > >
>> > > > > >> > > > > Check the drill logs. It is likely that drill is grumpy
>> > > about
>> > > > > >> > > something
>> > > > > >> > > > > in your udf or packaging.
>> > > > > >> > > > >
>> > > > > >> > > > > Also, feel free to snitch the pom from the simple
>> examples
>> > > in
>> > > > > >> order
>> > > > > >> > to
>> > > > > >> > > > get
>> > > > > >> > > > > the pieces assembled and packaged correctly.
>> > > > > >> > > > >
>> > > > > >> > > > > Sent from my iPhone
>> > > > > >> > > > >
>> > > > > >> > > > > > On Jul 19, 2015, at 11:25, Stefán Baxter <
>> > > > > >> > stefan@activitystream.com <javascript:;>>
>> > > > > >> > > > > wrote:
>> > > > > >> > > > > >
>> > > > > >> > > > > > Hi Jim,
>> > > > > >> > > > > >
>> > > > > >> > > > > > My UDF is currently so simple that I'm not sure you
>> need
>> > > it
>> > > > > (or
>> > > > > >> > want
>> > > > > >> > > > it).
>> > > > > >> > > > > >
>> > > > > >> > > > > > It basically just rounds a timestamp value with ISO
>> 8601
>> > > > > periods
>> > > > > >> > > > > > asRoundTimestamp(timestampvalue,'PT10M').
>> > > > > >> > > > > >
>> > > > > >> > > > > > I would be more than happy to contribute to your
>> project
>> > > > > rather
>> > > > > >> > than
>> > > > > >> > > > > > building our own :).
>> > > > > >> > > > > >
>> > > > > >> > > > > > Is the repo public?
>> > > > > >> > > > > >
>> > > > > >> > > > > > Regards,
>> > > > > >> > > > > > -Stefan
>> > > > > >> > > > > >
>> > > > > >> > > > > >
>> > > > > >> > > > > >
>> > > > > >> > > > > >> On Sun, Jul 19, 2015 at 6:18 PM, Jim Bates <
>> > > > > >> jbates@maprtech.com
>> > > > > >> > <javascript:;>>
>> > > > > >> > > > wrote:
>> > > > > >> > > > > >>
>> > > > > >> > > > > >> Maven will typically create a jar for class and a jar
>> > for
>> > > > > >> source
>> > > > > >> > > when
>> > > > > >> > > > > told
>> > > > > >> > > > > >> to do so. I just include the source files in the same
>> > jar
>> > > > as
>> > > > > >> the
>> > > > > >> > > class
>> > > > > >> > > > > >> files. There is a github example drill udf project we
>> > are
>> > > > > >> working
>> > > > > >> > on
>> > > > > >> > > > to
>> > > > > >> > > > > >> include several examples to simplify the learning
>> > curve.
>> > > If
>> > > > > >> your
>> > > > > >> > > > > >> interested... I'd love to have you add your udf.
>> > > > > >> > > > > >> On Jul 19, 2015 12:59 PM, "Stefán Baxter" <
>> > > > > >> > > stefan@activitystream.com <javascript:;>>
>> > > > > >> > > > > >> wrote:
>> > > > > >> > > > > >>
>> > > > > >> > > > > >>> Hi again,
>> > > > > >> > > > > >>>
>> > > > > >> > > > > >>> Going over the documentation once more I came across
>> > > this:
>> > > > > >> > > > > >>>
>> > > > > >> > > > > >>> - Add the sources and classes JAR files to Drill’s
>> > > > > >> classpath.
>> > > > > >> > > > > >>>
>> > > > > >> > > > > >>> I'm only including a standard .jar (with classes no
>> > > > sources)
>> > > > > >> and
>> > > > > >> > I
>> > > > > >> > > > > wonder
>> > > > > >> > > > > >>> why this dependency is listed.
>> > > > > >> > > > > >>>
>> > > > > >> > > > > >>> Anyways, that it the next step.
>> > > > > >> > > > > >>>
>> > > > > >> > > > > >>> A sample .jar that works with the drill-config
>> > > counterpart
>> > > > > >> would
>> > > > > >> > > be a
>> > > > > >> > > > > >>> welcome addition to the documentation.
>> > > > > >> > > > > >>>
>> > > > > >> > > > > >>> Regards,
>> > > > > >> > > > > >>> -Stefan
>> > > > > >> > > > > >>>
>> > > > > >> > > > > >>>
>> > > > > >> > > > > >>> On Sun, Jul 19, 2015 at 5:53 PM, Stefán Baxter <
>> > > > > >> > > > > >> stefan@activitystream.com <javascript:;>>
>> > > > > >> > > > > >>> wrote:
>> > > > > >> > > > > >>>
>> > > > > >> > > > > >>>> Hi Jim,
>> > > > > >> > > > > >>>>
>> > > > > >> > > > > >>>> Now I have added the file to the jar (both root and
>> > > > > resources
>> > > > > >> > > > folder)
>> > > > > >> > > > > >> but
>> > > > > >> > > > > >>>> that does not seem to change anything.
>> > > > > >> > > > > >>>>
>> > > > > >> > > > > >>>> Any additional ideas?
>> > > > > >> > > > > >>>>
>> > > > > >> > > > > >>>> Regards,
>> > > > > >> > > > > >>>> -Stefan
>> > > > > >> > > > > >>>>
>> > > > > >> > > > > >>>> On Sun, Jul 19, 2015 at 5:40 PM, Stefán Baxter <
>> > > > > >> > > > > >>> stefan@activitystream.com <javascript:;>>
>> > > > > >> > > > > >>>> wrote:
>> > > > > >> > > > > >>>>
>> > > > > >> > > > > >>>>> NO!
>> > > > > >> > > > > >>>>>
>> > > > > >> > > > > >>>>> Thank you , will do that right now :)
>> > > > > >> > > > > >>>>>
>> > > > > >> > > > > >>>>> On Sun, Jul 19, 2015 at 5:38 PM, Jim Bates <
>> > > > > >> > jbates@maprtech.com <javascript:;>>
>> > > > > >> > > > > >> wrote:
>> > > > > >> > > > > >>>>>
>> > > > > >> > > > > >>>>>> Did you include a file drill-module.conf in your
>> > jar
>> > > > > along
>> > > > > >> > with
>> > > > > >> > > > > >> source
>> > > > > >> > > > > >>>>>> files?
>> > > > > >> > > > > >>>>>> On Jul 19, 2015 12:20 PM, "Stefán Baxter" <
>> > > > > >> > > > > stefan@activitystream.com <javascript:;>
>> > > > > >>
>> > > > > >> > > > > >>>
>> > > > > >> > > > > >>>>>> wrote:
>> > > > > >> > > > > >>>>>>
>> > > > > >> > > > > >>>>>>> Hi,
>> > > > > >> > > > > >>>>>>>
>> > > > > >> > > > > >>>>>>> I'm trying to deploy a UDF that I have written
>> > > > according
>> > > > > >> to
>> > > > > >> > the
>> > > > > >> > > > > >>>>>>> documentation.
>> > > > > >> > > > > >>>>>>>
>> > > > > >> > > > > >>>>>>> I have also:
>> > > > > >> > > > > >>>>>>>
>> > > > > >> > > > > >>>>>>> 1. Copied the jar file
>> > > > > >> > > > > >>>>>>> to jars/3rdparty
>> > > > > >> > > > > >>>>>>>
>> > > > > >> > > > > >>>>>>> 2. Changed the config
>> "conf/drill-override.conf"
>> > > to
>> > > > > >> > include:
>> > > > > >> > > > > >>>>>>> drill.logical.function.package +=
>> > > > > >> > > > > >>
>> > > > > >> > >
>> > > > >
>> > ["org.apache.drill.exec.expr.fn.impl","com.activitystream.drill.udfs"]
>> > > > > >> > > > > >>>>>>>
>> > > > > >> > > > > >>>>>>> 3. Rerun the Drillbit
>> > > > > >> > > > > >>>>>>> bin/drillbit.sh start
>> > > > > >> > > > > >>>>>>>
>> > > > > >> > > > > >>>>>>> The log shows no sign of the the UDF being
>> loaded
>> > > but
>> > > > > if I
>> > > > > >> > use
>> > > > > >> > > it
>> > > > > >> > > > > >> in
>> > > > > >> > > > > >>> a
>> > > > > >> > > > > >>>>>> SQL
>> > > > > >> > > > > >>>>>>> statement then this error is thrown:
>> > > > > >> > > > > >>>>>>>
>> > > > > >> > > > > >>>>>>> Error: PARSE ERROR: From line 1, column 184 to
>> > line
>> > > 1,
>> > > > > >> column
>> > > > > >> > > > 219:
>> > > > > >> > > > > >> No
>> > > > > >> > > > > >>>>>> match
>> > > > > >> > > > > >>>>>>> found for function signature
>> > asRoundTimeStamp(<ANY>,
>> > > > > >> > > <CHARACTER>)
>> > > > > >> > > > > >>>>>>>
>> > > > > >> > > > > >>>>>>>
>> > > > > >> > > > > >>>>>>> The relevant part of the UDF source:
>> > > > > >> > > > > >>>>>>>
>> > > > > >> > > > > >>>>>>> package com.activitystream.drill.udfs;
>> > > > > >> > > > > >>>>>>>
>> > > > > >> > > > > >>>>>>> public class ASUserDefinedFunctions {
>> > > > > >> > > > > >>>>>>>
>> > > > > >> > > > > >>>>>>> @FunctionTemplate(name = "asRoundTimeStamp",
>> > > scope
>> > > > =
>> > > > > >> > > > > >>>>>>> FunctionTemplate.FunctionScope.SIMPLE, nulls =
>> > > > > >> > > > > >>>>>>> FunctionTemplate.NullHandling.NULL_IF_NULL)
>> > > > > >> > > > > >>>>>>>
>> > > > > >> > > > > >>>>>>> public static class RoundTimeStamp implements
>> > > > > >> > > DrillSimpleFunc
>> > > > > >> > > > > >>>>>>>
>> > > > > >> > > > > >>>>>>>
>> > > > > >> > > > > >>>>>>> Can anyone please assist?
>> > > > > >> > > > > >>>>>>>
>> > > > > >> > > > > >>>>>>> Regards,
>> > > > > >> > > > > >>>>>>> -Stefan
>> > > > > >> > > > > >>
>> > > > > >> > > > >
>> > > > > >> > > >
>> > > > > >> > >
>> > > > > >> >
>> > > > > >>
>> > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
Re: Drill not picking up a UDF
Posted by Stefán Baxter <st...@activitystream.com>.
Hi,
Just wanted to thank those that helped.
While I'm happy that the UDF is running I feel like it could have taken a
lost shorter.
I will contribute to the documentation but here is my main takeaway:
- Source code needs to be included in the jar
- it's used when the drillbits/queries are built/orchistrated
- Always use full class qualifiers in the Eval() segment of your UDF
- Don't forget to add the drill-module.conf to the resources folder of
your project (should end up in th root of the jar)
- Adding your udf package to drill-override.conf does not seem to matter
- just copy the jar(s) with the .class and .java files to the
jars/3rdparty directory
- Feel free to user this code anyway you wish:
- https://github.com/acmeguy/asdrill
- Know about this project:
- https://github.com/mapr-demos/simple-drill-functions
Regards,
-Stefán
On Mon, Jul 20, 2015 at 2:23 PM, Jim Bates <jb...@maprtech.com> wrote:
> That's an apt description. The Holders that have a fixed size value objects
> are simple to get at but the ones that have variable length objects are
> pulled via the buffer.
>
> You can also use the StringFunctionHelpers.
>
>
> org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(in
> .start, in.end, in.buffer)
>
> Where 'in' is
>
> @Param NullableVarCharHolder in
>
> or
>
> @Param VarCharHolder in
>
> On Mon, Jul 20, 2015 at 9:07 AM, Stefán Baxter <st...@activitystream.com>
> wrote:
>
> > Hi,
> >
> > After going through the log this is clear what is happening (once the
> Drill
> > picked up the UDF a bit earlier this morning).
> >
> > I'm calling the VarCharHolder.toString() to get the text value for the
> > parameter and that is throwing this exception.
> >
> > Two observations:
> >
> > 1. Calling a deprecated function, even though that its not optimal,
> > usually does not cause such drastic results.
> > 2. As far as I can see there is no easy way to get the string value
> of a
> > property without carving out a piece of the buffer.
> >
> > What am I missing here?
> >
> > Regards,
> > -Stefan
> >
> >
> > On Mon, Jul 20, 2015 at 12:58 PM, Jacques Nadeau <ja...@apache.org>
> > wrote:
> >
> > > Can you enable verbose errors at the session level? It may reveal more
> > > about what is failing.
> > > On Jul 20, 2015 5:32 AM, "Stefán Baxter" <st...@activitystream.com>
> > > wrote:
> > >
> > > > Hi Jim,
> > > >
> > > > I have made those changes and I'm wondering if you can runs this be
> > using
> > > > the two .jar files that the "mvn package" places in the target
> > directory?
> > > >
> > > > I have tried to have Drill pick those up but the error now is:
> > > >
> > > > Error: SYSTEM ERROR: UnsupportedOperationException
> > > > Fragment 0:0
> > > > [Error Id: da589dd4-4cfd-4659-8b93-219074ab8c72 on localhost:31010]
> > > > (state=,code=0)
> > > >
> > > > It seems to indicate that it's picking up the functions but that they
> > can
> > > > not be run.
> > > >
> > > > Regards,
> > > > - Stefán
> > > >
> > > > On Mon, Jul 20, 2015 at 1:25 AM, Jim Bates <jb...@maprtech.com>
> > wrote:
> > > >
> > > > > I pulled out your udf class and threw it into my package. It worked
> > for
> > > > me
> > > > > with a few modifications.
> > > > >
> > > > > You can not have imported classes or method references in your eval
> > or
> > > > > setup methods as the code will get pulled out and executed
> somewhere
> > > else
> > > > > and it won't be able to find it. With that in mind, Period will
> need
> > > to
> > > > be
> > > > > org.joda.time.Period, DateTime will need to be
> org.joda.time.DateTime
> > > and
> > > > > roundTimeStamp will need to be
> > > > >
> com.activitystream.drill.udfs.ASUserDefinedFunctions.roundTimeStamp.
> > > > >
> > > > >
> > > > >
> > > > > On Sun, Jul 19, 2015 at 7:02 PM, Stefán Baxter <
> > > > stefan@activitystream.com>
> > > > > wrote:
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > The project can be found here:
> > > > > > https://github.com/acmeguy/asdrill
> > > > > >
> > > > > > Thank you,
> > > > > > -Stefán
> > > > > >
> > > > > > On Sun, Jul 19, 2015 at 11:57 PM, Stefán Baxter <
> > > > > stefan@activitystream.com
> > > > > > >
> > > > > > wrote:
> > > > > >
> > > > > > > Hi,
> > > > > > >
> > > > > > > I'm more than happy to share the little that is there (I will
> > > publish
> > > > > it
> > > > > > > on github and send link tomorrow).
> > > > > > >
> > > > > > > I ended up copying my UDF (singl-file.java) into the
> > > > > > simple-drill-function
> > > > > > > project where it got picked up.
> > > > > > >
> > > > > > > Then I discovered a whole new set of dependencies/limitations
> > > > > > >
> > > > > > > - The UDF are recompiled - any imports are invalid or at
> least
> > > > > > > overwritten
> > > > > > > - import org.joda.time.Period; (means that Period class is
> > not
> > > > > > > resolved on runtime)
> > > > > > > - Error: SYSTEM ERROR: CompileException: Line 71, Column 26:
> > > > Cannot
> > > > > > > determine simple type name "Period"
> > > > > > >
> > > > > > > - Calling any "outside" functions, like I was calling a
> static
> > > > > > > function of the new class (same file), leads to an errors as
> > > they
> > > > > are
> > > > > > not
> > > > > > > not resolved
> > > > > > > - Error: SYSTEM ERROR: CompileException: Line 70, Column
> 35:
> > A
> > > > > method
> > > > > > > named "roundTimeStamp" is not declared in any enclosing
> class
> > > nor
> > > > > any
> > > > > > > supertype, nor through a static import
> > > > > > >
> > > > > > > Perhaps this was mentioned in the documentation but this is, at
> > the
> > > > > very
> > > > > > > least, not straight forward and super-inviting.
> > > > > > >
> > > > > > > Thank you for your assistance, we will keep trying :)
> > > > > > >
> > > > > > > Regards,
> > > > > > > -Stefan
> > > > > > >
> > > > > > >
> > > > > > > On Sun, Jul 19, 2015 at 11:34 PM, Tugdual Grall <
> > tugdual@gmail.com
> > > >
> > > > > > wrote:
> > > > > > >
> > > > > > >> Hi Stefan,
> > > > > > >>
> > > > > > >> Do you think you can share your complete project ?
> > > > > > >>
> > > > > > >> This will help to debug it for you.
> > > > > > >>
> > > > > > >> T
> > > > > > >>
> > > > > > >> On Sunday, July 19, 2015, Stefán Baxter <
> > > stefan@activitystream.com>
> > > > > > >> wrote:
> > > > > > >>
> > > > > > >> > Hi Ted,
> > > > > > >> >
> > > > > > >> > I fetched this, built it and deployed it without problems.
> > > > > > >> > I can not see any real difference other than this deploys
> two
> > > .jar
> > > > > (I
> > > > > > >> tried
> > > > > > >> > that as well earlier).
> > > > > > >> >
> > > > > > >> > I'm still trying to figure out why Drill is not picking up
> my
> > > UDFs
> > > > > > >> >
> > > > > > >> > Regards,
> > > > > > >> > -Stefán
> > > > > > >> >
> > > > > > >> > On Sun, Jul 19, 2015 at 10:45 PM, Ted Dunning <
> > > > > ted.dunning@gmail.com
> > > > > > >> > <javascript:;>> wrote:
> > > > > > >> >
> > > > > > >> > > Stefan,
> > > > > > >> > >
> > > > > > >> > > Have you seen this github project:
> > > > > > >> > >
> > > > > > >> > > https://github.com/mapr-demos/simple-drill-functions
> > > > > > >> > >
> > > > > > >> > > ?
> > > > > > >> > >
> > > > > > >> > >
> > > > > > >> > > On Sun, Jul 19, 2015 at 2:14 PM, Stefán Baxter <
> > > > > > >> > stefan@activitystream.com <javascript:;>>
> > > > > > >> > > wrote:
> > > > > > >> > >
> > > > > > >> > > > Hi Jim,
> > > > > > >> > > >
> > > > > > >> > > > I'm still not able to make this work. Do you have a
> sample
> > > > .jar
> > > > > > file
> > > > > > >> > > with a
> > > > > > >> > > > small example that you are running?
> > > > > > >> > > >
> > > > > > >> > > > Regards,
> > > > > > >> > > > -Stefan
> > > > > > >> > > >
> > > > > > >> > > > On Sun, Jul 19, 2015 at 6:46 PM, Ted Dunning <
> > > > > > ted.dunning@gmail.com
> > > > > > >> > <javascript:;>>
> > > > > > >> > > > wrote:
> > > > > > >> > > >
> > > > > > >> > > > >
> > > > > > >> > > > > Sounds like a fine example, not because of
> > sophistication
> > > > but
> > > > > > >> because
> > > > > > >> > > it
> > > > > > >> > > > > deals with dates.
> > > > > > >> > > > >
> > > > > > >> > > > > Check the drill logs. It is likely that drill is
> grumpy
> > > > about
> > > > > > >> > > something
> > > > > > >> > > > > in your udf or packaging.
> > > > > > >> > > > >
> > > > > > >> > > > > Also, feel free to snitch the pom from the simple
> > examples
> > > > in
> > > > > > >> order
> > > > > > >> > to
> > > > > > >> > > > get
> > > > > > >> > > > > the pieces assembled and packaged correctly.
> > > > > > >> > > > >
> > > > > > >> > > > > Sent from my iPhone
> > > > > > >> > > > >
> > > > > > >> > > > > > On Jul 19, 2015, at 11:25, Stefán Baxter <
> > > > > > >> > stefan@activitystream.com <javascript:;>>
> > > > > > >> > > > > wrote:
> > > > > > >> > > > > >
> > > > > > >> > > > > > Hi Jim,
> > > > > > >> > > > > >
> > > > > > >> > > > > > My UDF is currently so simple that I'm not sure you
> > need
> > > > it
> > > > > > (or
> > > > > > >> > want
> > > > > > >> > > > it).
> > > > > > >> > > > > >
> > > > > > >> > > > > > It basically just rounds a timestamp value with ISO
> > 8601
> > > > > > periods
> > > > > > >> > > > > > asRoundTimestamp(timestampvalue,'PT10M').
> > > > > > >> > > > > >
> > > > > > >> > > > > > I would be more than happy to contribute to your
> > project
> > > > > > rather
> > > > > > >> > than
> > > > > > >> > > > > > building our own :).
> > > > > > >> > > > > >
> > > > > > >> > > > > > Is the repo public?
> > > > > > >> > > > > >
> > > > > > >> > > > > > Regards,
> > > > > > >> > > > > > -Stefan
> > > > > > >> > > > > >
> > > > > > >> > > > > >
> > > > > > >> > > > > >
> > > > > > >> > > > > >> On Sun, Jul 19, 2015 at 6:18 PM, Jim Bates <
> > > > > > >> jbates@maprtech.com
> > > > > > >> > <javascript:;>>
> > > > > > >> > > > wrote:
> > > > > > >> > > > > >>
> > > > > > >> > > > > >> Maven will typically create a jar for class and a
> jar
> > > for
> > > > > > >> source
> > > > > > >> > > when
> > > > > > >> > > > > told
> > > > > > >> > > > > >> to do so. I just include the source files in the
> same
> > > jar
> > > > > as
> > > > > > >> the
> > > > > > >> > > class
> > > > > > >> > > > > >> files. There is a github example drill udf project
> we
> > > are
> > > > > > >> working
> > > > > > >> > on
> > > > > > >> > > > to
> > > > > > >> > > > > >> include several examples to simplify the learning
> > > curve.
> > > > If
> > > > > > >> your
> > > > > > >> > > > > >> interested... I'd love to have you add your udf.
> > > > > > >> > > > > >> On Jul 19, 2015 12:59 PM, "Stefán Baxter" <
> > > > > > >> > > stefan@activitystream.com <javascript:;>>
> > > > > > >> > > > > >> wrote:
> > > > > > >> > > > > >>
> > > > > > >> > > > > >>> Hi again,
> > > > > > >> > > > > >>>
> > > > > > >> > > > > >>> Going over the documentation once more I came
> across
> > > > this:
> > > > > > >> > > > > >>>
> > > > > > >> > > > > >>> - Add the sources and classes JAR files to
> Drill’s
> > > > > > >> classpath.
> > > > > > >> > > > > >>>
> > > > > > >> > > > > >>> I'm only including a standard .jar (with classes
> no
> > > > > sources)
> > > > > > >> and
> > > > > > >> > I
> > > > > > >> > > > > wonder
> > > > > > >> > > > > >>> why this dependency is listed.
> > > > > > >> > > > > >>>
> > > > > > >> > > > > >>> Anyways, that it the next step.
> > > > > > >> > > > > >>>
> > > > > > >> > > > > >>> A sample .jar that works with the drill-config
> > > > counterpart
> > > > > > >> would
> > > > > > >> > > be a
> > > > > > >> > > > > >>> welcome addition to the documentation.
> > > > > > >> > > > > >>>
> > > > > > >> > > > > >>> Regards,
> > > > > > >> > > > > >>> -Stefan
> > > > > > >> > > > > >>>
> > > > > > >> > > > > >>>
> > > > > > >> > > > > >>> On Sun, Jul 19, 2015 at 5:53 PM, Stefán Baxter <
> > > > > > >> > > > > >> stefan@activitystream.com <javascript:;>>
> > > > > > >> > > > > >>> wrote:
> > > > > > >> > > > > >>>
> > > > > > >> > > > > >>>> Hi Jim,
> > > > > > >> > > > > >>>>
> > > > > > >> > > > > >>>> Now I have added the file to the jar (both root
> and
> > > > > > resources
> > > > > > >> > > > folder)
> > > > > > >> > > > > >> but
> > > > > > >> > > > > >>>> that does not seem to change anything.
> > > > > > >> > > > > >>>>
> > > > > > >> > > > > >>>> Any additional ideas?
> > > > > > >> > > > > >>>>
> > > > > > >> > > > > >>>> Regards,
> > > > > > >> > > > > >>>> -Stefan
> > > > > > >> > > > > >>>>
> > > > > > >> > > > > >>>> On Sun, Jul 19, 2015 at 5:40 PM, Stefán Baxter <
> > > > > > >> > > > > >>> stefan@activitystream.com <javascript:;>>
> > > > > > >> > > > > >>>> wrote:
> > > > > > >> > > > > >>>>
> > > > > > >> > > > > >>>>> NO!
> > > > > > >> > > > > >>>>>
> > > > > > >> > > > > >>>>> Thank you , will do that right now :)
> > > > > > >> > > > > >>>>>
> > > > > > >> > > > > >>>>> On Sun, Jul 19, 2015 at 5:38 PM, Jim Bates <
> > > > > > >> > jbates@maprtech.com <javascript:;>>
> > > > > > >> > > > > >> wrote:
> > > > > > >> > > > > >>>>>
> > > > > > >> > > > > >>>>>> Did you include a file drill-module.conf in
> your
> > > jar
> > > > > > along
> > > > > > >> > with
> > > > > > >> > > > > >> source
> > > > > > >> > > > > >>>>>> files?
> > > > > > >> > > > > >>>>>> On Jul 19, 2015 12:20 PM, "Stefán Baxter" <
> > > > > > >> > > > > stefan@activitystream.com <javascript:;>
> > > > > > >>
> > > > > > >> > > > > >>>
> > > > > > >> > > > > >>>>>> wrote:
> > > > > > >> > > > > >>>>>>
> > > > > > >> > > > > >>>>>>> Hi,
> > > > > > >> > > > > >>>>>>>
> > > > > > >> > > > > >>>>>>> I'm trying to deploy a UDF that I have written
> > > > > according
> > > > > > >> to
> > > > > > >> > the
> > > > > > >> > > > > >>>>>>> documentation.
> > > > > > >> > > > > >>>>>>>
> > > > > > >> > > > > >>>>>>> I have also:
> > > > > > >> > > > > >>>>>>>
> > > > > > >> > > > > >>>>>>> 1. Copied the jar file
> > > > > > >> > > > > >>>>>>> to jars/3rdparty
> > > > > > >> > > > > >>>>>>>
> > > > > > >> > > > > >>>>>>> 2. Changed the config
> > "conf/drill-override.conf"
> > > > to
> > > > > > >> > include:
> > > > > > >> > > > > >>>>>>> drill.logical.function.package +=
> > > > > > >> > > > > >>
> > > > > > >> > >
> > > > > >
> > > ["org.apache.drill.exec.expr.fn.impl","com.activitystream.drill.udfs"]
> > > > > > >> > > > > >>>>>>>
> > > > > > >> > > > > >>>>>>> 3. Rerun the Drillbit
> > > > > > >> > > > > >>>>>>> bin/drillbit.sh start
> > > > > > >> > > > > >>>>>>>
> > > > > > >> > > > > >>>>>>> The log shows no sign of the the UDF being
> > loaded
> > > > but
> > > > > > if I
> > > > > > >> > use
> > > > > > >> > > it
> > > > > > >> > > > > >> in
> > > > > > >> > > > > >>> a
> > > > > > >> > > > > >>>>>> SQL
> > > > > > >> > > > > >>>>>>> statement then this error is thrown:
> > > > > > >> > > > > >>>>>>>
> > > > > > >> > > > > >>>>>>> Error: PARSE ERROR: From line 1, column 184 to
> > > line
> > > > 1,
> > > > > > >> column
> > > > > > >> > > > 219:
> > > > > > >> > > > > >> No
> > > > > > >> > > > > >>>>>> match
> > > > > > >> > > > > >>>>>>> found for function signature
> > > asRoundTimeStamp(<ANY>,
> > > > > > >> > > <CHARACTER>)
> > > > > > >> > > > > >>>>>>>
> > > > > > >> > > > > >>>>>>>
> > > > > > >> > > > > >>>>>>> The relevant part of the UDF source:
> > > > > > >> > > > > >>>>>>>
> > > > > > >> > > > > >>>>>>> package com.activitystream.drill.udfs;
> > > > > > >> > > > > >>>>>>>
> > > > > > >> > > > > >>>>>>> public class ASUserDefinedFunctions {
> > > > > > >> > > > > >>>>>>>
> > > > > > >> > > > > >>>>>>> @FunctionTemplate(name =
> "asRoundTimeStamp",
> > > > scope
> > > > > =
> > > > > > >> > > > > >>>>>>> FunctionTemplate.FunctionScope.SIMPLE, nulls =
> > > > > > >> > > > > >>>>>>> FunctionTemplate.NullHandling.NULL_IF_NULL)
> > > > > > >> > > > > >>>>>>>
> > > > > > >> > > > > >>>>>>> public static class RoundTimeStamp
> implements
> > > > > > >> > > DrillSimpleFunc
> > > > > > >> > > > > >>>>>>>
> > > > > > >> > > > > >>>>>>>
> > > > > > >> > > > > >>>>>>> Can anyone please assist?
> > > > > > >> > > > > >>>>>>>
> > > > > > >> > > > > >>>>>>> Regards,
> > > > > > >> > > > > >>>>>>> -Stefan
> > > > > > >> > > > > >>
> > > > > > >> > > > >
> > > > > > >> > > >
> > > > > > >> > >
> > > > > > >> >
> > > > > > >>
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: Drill not picking up a UDF
Posted by Jim Bates <jb...@maprtech.com>.
That's an apt description. The Holders that have a fixed size value objects
are simple to get at but the ones that have variable length objects are
pulled via the buffer.
You can also use the StringFunctionHelpers.
org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(in
.start, in.end, in.buffer)
Where 'in' is
@Param NullableVarCharHolder in
or
@Param VarCharHolder in
On Mon, Jul 20, 2015 at 9:07 AM, Stefán Baxter <st...@activitystream.com>
wrote:
> Hi,
>
> After going through the log this is clear what is happening (once the Drill
> picked up the UDF a bit earlier this morning).
>
> I'm calling the VarCharHolder.toString() to get the text value for the
> parameter and that is throwing this exception.
>
> Two observations:
>
> 1. Calling a deprecated function, even though that its not optimal,
> usually does not cause such drastic results.
> 2. As far as I can see there is no easy way to get the string value of a
> property without carving out a piece of the buffer.
>
> What am I missing here?
>
> Regards,
> -Stefan
>
>
> On Mon, Jul 20, 2015 at 12:58 PM, Jacques Nadeau <ja...@apache.org>
> wrote:
>
> > Can you enable verbose errors at the session level? It may reveal more
> > about what is failing.
> > On Jul 20, 2015 5:32 AM, "Stefán Baxter" <st...@activitystream.com>
> > wrote:
> >
> > > Hi Jim,
> > >
> > > I have made those changes and I'm wondering if you can runs this be
> using
> > > the two .jar files that the "mvn package" places in the target
> directory?
> > >
> > > I have tried to have Drill pick those up but the error now is:
> > >
> > > Error: SYSTEM ERROR: UnsupportedOperationException
> > > Fragment 0:0
> > > [Error Id: da589dd4-4cfd-4659-8b93-219074ab8c72 on localhost:31010]
> > > (state=,code=0)
> > >
> > > It seems to indicate that it's picking up the functions but that they
> can
> > > not be run.
> > >
> > > Regards,
> > > - Stefán
> > >
> > > On Mon, Jul 20, 2015 at 1:25 AM, Jim Bates <jb...@maprtech.com>
> wrote:
> > >
> > > > I pulled out your udf class and threw it into my package. It worked
> for
> > > me
> > > > with a few modifications.
> > > >
> > > > You can not have imported classes or method references in your eval
> or
> > > > setup methods as the code will get pulled out and executed somewhere
> > else
> > > > and it won't be able to find it. With that in mind, Period will need
> > to
> > > be
> > > > org.joda.time.Period, DateTime will need to be org.joda.time.DateTime
> > and
> > > > roundTimeStamp will need to be
> > > > com.activitystream.drill.udfs.ASUserDefinedFunctions.roundTimeStamp.
> > > >
> > > >
> > > >
> > > > On Sun, Jul 19, 2015 at 7:02 PM, Stefán Baxter <
> > > stefan@activitystream.com>
> > > > wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > The project can be found here:
> > > > > https://github.com/acmeguy/asdrill
> > > > >
> > > > > Thank you,
> > > > > -Stefán
> > > > >
> > > > > On Sun, Jul 19, 2015 at 11:57 PM, Stefán Baxter <
> > > > stefan@activitystream.com
> > > > > >
> > > > > wrote:
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > I'm more than happy to share the little that is there (I will
> > publish
> > > > it
> > > > > > on github and send link tomorrow).
> > > > > >
> > > > > > I ended up copying my UDF (singl-file.java) into the
> > > > > simple-drill-function
> > > > > > project where it got picked up.
> > > > > >
> > > > > > Then I discovered a whole new set of dependencies/limitations
> > > > > >
> > > > > > - The UDF are recompiled - any imports are invalid or at least
> > > > > > overwritten
> > > > > > - import org.joda.time.Period; (means that Period class is
> not
> > > > > > resolved on runtime)
> > > > > > - Error: SYSTEM ERROR: CompileException: Line 71, Column 26:
> > > Cannot
> > > > > > determine simple type name "Period"
> > > > > >
> > > > > > - Calling any "outside" functions, like I was calling a static
> > > > > > function of the new class (same file), leads to an errors as
> > they
> > > > are
> > > > > not
> > > > > > not resolved
> > > > > > - Error: SYSTEM ERROR: CompileException: Line 70, Column 35:
> A
> > > > method
> > > > > > named "roundTimeStamp" is not declared in any enclosing class
> > nor
> > > > any
> > > > > > supertype, nor through a static import
> > > > > >
> > > > > > Perhaps this was mentioned in the documentation but this is, at
> the
> > > > very
> > > > > > least, not straight forward and super-inviting.
> > > > > >
> > > > > > Thank you for your assistance, we will keep trying :)
> > > > > >
> > > > > > Regards,
> > > > > > -Stefan
> > > > > >
> > > > > >
> > > > > > On Sun, Jul 19, 2015 at 11:34 PM, Tugdual Grall <
> tugdual@gmail.com
> > >
> > > > > wrote:
> > > > > >
> > > > > >> Hi Stefan,
> > > > > >>
> > > > > >> Do you think you can share your complete project ?
> > > > > >>
> > > > > >> This will help to debug it for you.
> > > > > >>
> > > > > >> T
> > > > > >>
> > > > > >> On Sunday, July 19, 2015, Stefán Baxter <
> > stefan@activitystream.com>
> > > > > >> wrote:
> > > > > >>
> > > > > >> > Hi Ted,
> > > > > >> >
> > > > > >> > I fetched this, built it and deployed it without problems.
> > > > > >> > I can not see any real difference other than this deploys two
> > .jar
> > > > (I
> > > > > >> tried
> > > > > >> > that as well earlier).
> > > > > >> >
> > > > > >> > I'm still trying to figure out why Drill is not picking up my
> > UDFs
> > > > > >> >
> > > > > >> > Regards,
> > > > > >> > -Stefán
> > > > > >> >
> > > > > >> > On Sun, Jul 19, 2015 at 10:45 PM, Ted Dunning <
> > > > ted.dunning@gmail.com
> > > > > >> > <javascript:;>> wrote:
> > > > > >> >
> > > > > >> > > Stefan,
> > > > > >> > >
> > > > > >> > > Have you seen this github project:
> > > > > >> > >
> > > > > >> > > https://github.com/mapr-demos/simple-drill-functions
> > > > > >> > >
> > > > > >> > > ?
> > > > > >> > >
> > > > > >> > >
> > > > > >> > > On Sun, Jul 19, 2015 at 2:14 PM, Stefán Baxter <
> > > > > >> > stefan@activitystream.com <javascript:;>>
> > > > > >> > > wrote:
> > > > > >> > >
> > > > > >> > > > Hi Jim,
> > > > > >> > > >
> > > > > >> > > > I'm still not able to make this work. Do you have a sample
> > > .jar
> > > > > file
> > > > > >> > > with a
> > > > > >> > > > small example that you are running?
> > > > > >> > > >
> > > > > >> > > > Regards,
> > > > > >> > > > -Stefan
> > > > > >> > > >
> > > > > >> > > > On Sun, Jul 19, 2015 at 6:46 PM, Ted Dunning <
> > > > > ted.dunning@gmail.com
> > > > > >> > <javascript:;>>
> > > > > >> > > > wrote:
> > > > > >> > > >
> > > > > >> > > > >
> > > > > >> > > > > Sounds like a fine example, not because of
> sophistication
> > > but
> > > > > >> because
> > > > > >> > > it
> > > > > >> > > > > deals with dates.
> > > > > >> > > > >
> > > > > >> > > > > Check the drill logs. It is likely that drill is grumpy
> > > about
> > > > > >> > > something
> > > > > >> > > > > in your udf or packaging.
> > > > > >> > > > >
> > > > > >> > > > > Also, feel free to snitch the pom from the simple
> examples
> > > in
> > > > > >> order
> > > > > >> > to
> > > > > >> > > > get
> > > > > >> > > > > the pieces assembled and packaged correctly.
> > > > > >> > > > >
> > > > > >> > > > > Sent from my iPhone
> > > > > >> > > > >
> > > > > >> > > > > > On Jul 19, 2015, at 11:25, Stefán Baxter <
> > > > > >> > stefan@activitystream.com <javascript:;>>
> > > > > >> > > > > wrote:
> > > > > >> > > > > >
> > > > > >> > > > > > Hi Jim,
> > > > > >> > > > > >
> > > > > >> > > > > > My UDF is currently so simple that I'm not sure you
> need
> > > it
> > > > > (or
> > > > > >> > want
> > > > > >> > > > it).
> > > > > >> > > > > >
> > > > > >> > > > > > It basically just rounds a timestamp value with ISO
> 8601
> > > > > periods
> > > > > >> > > > > > asRoundTimestamp(timestampvalue,'PT10M').
> > > > > >> > > > > >
> > > > > >> > > > > > I would be more than happy to contribute to your
> project
> > > > > rather
> > > > > >> > than
> > > > > >> > > > > > building our own :).
> > > > > >> > > > > >
> > > > > >> > > > > > Is the repo public?
> > > > > >> > > > > >
> > > > > >> > > > > > Regards,
> > > > > >> > > > > > -Stefan
> > > > > >> > > > > >
> > > > > >> > > > > >
> > > > > >> > > > > >
> > > > > >> > > > > >> On Sun, Jul 19, 2015 at 6:18 PM, Jim Bates <
> > > > > >> jbates@maprtech.com
> > > > > >> > <javascript:;>>
> > > > > >> > > > wrote:
> > > > > >> > > > > >>
> > > > > >> > > > > >> Maven will typically create a jar for class and a jar
> > for
> > > > > >> source
> > > > > >> > > when
> > > > > >> > > > > told
> > > > > >> > > > > >> to do so. I just include the source files in the same
> > jar
> > > > as
> > > > > >> the
> > > > > >> > > class
> > > > > >> > > > > >> files. There is a github example drill udf project we
> > are
> > > > > >> working
> > > > > >> > on
> > > > > >> > > > to
> > > > > >> > > > > >> include several examples to simplify the learning
> > curve.
> > > If
> > > > > >> your
> > > > > >> > > > > >> interested... I'd love to have you add your udf.
> > > > > >> > > > > >> On Jul 19, 2015 12:59 PM, "Stefán Baxter" <
> > > > > >> > > stefan@activitystream.com <javascript:;>>
> > > > > >> > > > > >> wrote:
> > > > > >> > > > > >>
> > > > > >> > > > > >>> Hi again,
> > > > > >> > > > > >>>
> > > > > >> > > > > >>> Going over the documentation once more I came across
> > > this:
> > > > > >> > > > > >>>
> > > > > >> > > > > >>> - Add the sources and classes JAR files to Drill’s
> > > > > >> classpath.
> > > > > >> > > > > >>>
> > > > > >> > > > > >>> I'm only including a standard .jar (with classes no
> > > > sources)
> > > > > >> and
> > > > > >> > I
> > > > > >> > > > > wonder
> > > > > >> > > > > >>> why this dependency is listed.
> > > > > >> > > > > >>>
> > > > > >> > > > > >>> Anyways, that it the next step.
> > > > > >> > > > > >>>
> > > > > >> > > > > >>> A sample .jar that works with the drill-config
> > > counterpart
> > > > > >> would
> > > > > >> > > be a
> > > > > >> > > > > >>> welcome addition to the documentation.
> > > > > >> > > > > >>>
> > > > > >> > > > > >>> Regards,
> > > > > >> > > > > >>> -Stefan
> > > > > >> > > > > >>>
> > > > > >> > > > > >>>
> > > > > >> > > > > >>> On Sun, Jul 19, 2015 at 5:53 PM, Stefán Baxter <
> > > > > >> > > > > >> stefan@activitystream.com <javascript:;>>
> > > > > >> > > > > >>> wrote:
> > > > > >> > > > > >>>
> > > > > >> > > > > >>>> Hi Jim,
> > > > > >> > > > > >>>>
> > > > > >> > > > > >>>> Now I have added the file to the jar (both root and
> > > > > resources
> > > > > >> > > > folder)
> > > > > >> > > > > >> but
> > > > > >> > > > > >>>> that does not seem to change anything.
> > > > > >> > > > > >>>>
> > > > > >> > > > > >>>> Any additional ideas?
> > > > > >> > > > > >>>>
> > > > > >> > > > > >>>> Regards,
> > > > > >> > > > > >>>> -Stefan
> > > > > >> > > > > >>>>
> > > > > >> > > > > >>>> On Sun, Jul 19, 2015 at 5:40 PM, Stefán Baxter <
> > > > > >> > > > > >>> stefan@activitystream.com <javascript:;>>
> > > > > >> > > > > >>>> wrote:
> > > > > >> > > > > >>>>
> > > > > >> > > > > >>>>> NO!
> > > > > >> > > > > >>>>>
> > > > > >> > > > > >>>>> Thank you , will do that right now :)
> > > > > >> > > > > >>>>>
> > > > > >> > > > > >>>>> On Sun, Jul 19, 2015 at 5:38 PM, Jim Bates <
> > > > > >> > jbates@maprtech.com <javascript:;>>
> > > > > >> > > > > >> wrote:
> > > > > >> > > > > >>>>>
> > > > > >> > > > > >>>>>> Did you include a file drill-module.conf in your
> > jar
> > > > > along
> > > > > >> > with
> > > > > >> > > > > >> source
> > > > > >> > > > > >>>>>> files?
> > > > > >> > > > > >>>>>> On Jul 19, 2015 12:20 PM, "Stefán Baxter" <
> > > > > >> > > > > stefan@activitystream.com <javascript:;>
> > > > > >>
> > > > > >> > > > > >>>
> > > > > >> > > > > >>>>>> wrote:
> > > > > >> > > > > >>>>>>
> > > > > >> > > > > >>>>>>> Hi,
> > > > > >> > > > > >>>>>>>
> > > > > >> > > > > >>>>>>> I'm trying to deploy a UDF that I have written
> > > > according
> > > > > >> to
> > > > > >> > the
> > > > > >> > > > > >>>>>>> documentation.
> > > > > >> > > > > >>>>>>>
> > > > > >> > > > > >>>>>>> I have also:
> > > > > >> > > > > >>>>>>>
> > > > > >> > > > > >>>>>>> 1. Copied the jar file
> > > > > >> > > > > >>>>>>> to jars/3rdparty
> > > > > >> > > > > >>>>>>>
> > > > > >> > > > > >>>>>>> 2. Changed the config
> "conf/drill-override.conf"
> > > to
> > > > > >> > include:
> > > > > >> > > > > >>>>>>> drill.logical.function.package +=
> > > > > >> > > > > >>
> > > > > >> > >
> > > > >
> > ["org.apache.drill.exec.expr.fn.impl","com.activitystream.drill.udfs"]
> > > > > >> > > > > >>>>>>>
> > > > > >> > > > > >>>>>>> 3. Rerun the Drillbit
> > > > > >> > > > > >>>>>>> bin/drillbit.sh start
> > > > > >> > > > > >>>>>>>
> > > > > >> > > > > >>>>>>> The log shows no sign of the the UDF being
> loaded
> > > but
> > > > > if I
> > > > > >> > use
> > > > > >> > > it
> > > > > >> > > > > >> in
> > > > > >> > > > > >>> a
> > > > > >> > > > > >>>>>> SQL
> > > > > >> > > > > >>>>>>> statement then this error is thrown:
> > > > > >> > > > > >>>>>>>
> > > > > >> > > > > >>>>>>> Error: PARSE ERROR: From line 1, column 184 to
> > line
> > > 1,
> > > > > >> column
> > > > > >> > > > 219:
> > > > > >> > > > > >> No
> > > > > >> > > > > >>>>>> match
> > > > > >> > > > > >>>>>>> found for function signature
> > asRoundTimeStamp(<ANY>,
> > > > > >> > > <CHARACTER>)
> > > > > >> > > > > >>>>>>>
> > > > > >> > > > > >>>>>>>
> > > > > >> > > > > >>>>>>> The relevant part of the UDF source:
> > > > > >> > > > > >>>>>>>
> > > > > >> > > > > >>>>>>> package com.activitystream.drill.udfs;
> > > > > >> > > > > >>>>>>>
> > > > > >> > > > > >>>>>>> public class ASUserDefinedFunctions {
> > > > > >> > > > > >>>>>>>
> > > > > >> > > > > >>>>>>> @FunctionTemplate(name = "asRoundTimeStamp",
> > > scope
> > > > =
> > > > > >> > > > > >>>>>>> FunctionTemplate.FunctionScope.SIMPLE, nulls =
> > > > > >> > > > > >>>>>>> FunctionTemplate.NullHandling.NULL_IF_NULL)
> > > > > >> > > > > >>>>>>>
> > > > > >> > > > > >>>>>>> public static class RoundTimeStamp implements
> > > > > >> > > DrillSimpleFunc
> > > > > >> > > > > >>>>>>>
> > > > > >> > > > > >>>>>>>
> > > > > >> > > > > >>>>>>> Can anyone please assist?
> > > > > >> > > > > >>>>>>>
> > > > > >> > > > > >>>>>>> Regards,
> > > > > >> > > > > >>>>>>> -Stefan
> > > > > >> > > > > >>
> > > > > >> > > > >
> > > > > >> > > >
> > > > > >> > >
> > > > > >> >
> > > > > >>
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: Drill not picking up a UDF
Posted by Andrew Brust <an...@bluebadgeinsights.com>.
I think you should include that text verbatim, in the docs. Very helpful to developing an instinct for writing UDFs that work. And if I can understand it (and I do), then I would think most good Java devs will too.
On 7/20/15, 11:15 AM, "Jacques Nadeau" <ja...@dremio.com> wrote:
>Hey Stefan,
>
>Can you propose some edits/updates to the documentation? The doc is
>maintained on github [1].
>
>The key thing to understand is that a UDF is slightly incomplete Java.
>This is because Drill actually rips apart the functionality of the UDF and
>recomposes it directly within expression evaluation code. This gives a
>substantial performance and memory benefit but also creates some
>challenges. As such, there are some key rules one should follow:
>
> - Don't use imports
> - Both class file and source file have to be on the classpath (yes, Drill
>uses the source of the function)
> - Any JAR files holding UDF resources must include drill-module.conf
>marker file so that Drill knows to include that JAR in consideration for
>UDF loading
> - ValueHolders should be treated as structs. As such, don't call any
>methods on ValueHolders
>
>If you fork the documentation and provide a pull request, doc people
>(Kristine and Bridget) will work to include your feedback so that others
>don't find as many challenges as you did.
>
>thanks,
>Jacques
>
>[1]
>https://github.com/apache/drill/tree/gh-pages/_docs/develop-custom-functions
>
>On Mon, Jul 20, 2015 at 7:19 AM, Stefán Baxter <st...@activitystream.com>
>wrote:
>
>> Hi,
>>
>> This is working now.
>>
>> This, unless someone corrects me, is the code needed to get a string
>> parameter for a UDF:
>>
>>
>> - input2.buffer.toString(input2.start,
>> input2.end-input2.start,java.nio.charset.Charset.defaultCharset())
>>
>> Regards,
>> -Stefan (not the happiest camper)
>>
>>
>> On Mon, Jul 20, 2015 at 2:07 PM, Stefán Baxter <st...@activitystream.com>
>> wrote:
>>
>> > Hi,
>> >
>> > After going through the log this is clear what is happening (once the
>> > Drill picked up the UDF a bit earlier this morning).
>> >
>> > I'm calling the VarCharHolder.toString() to get the text value for the
>> > parameter and that is throwing this exception.
>> >
>> > Two observations:
>> >
>> > 1. Calling a deprecated function, even though that its not optimal,
>> > usually does not cause such drastic results.
>> > 2. As far as I can see there is no easy way to get the string value of
>> > a property without carving out a piece of the buffer.
>> >
>> > What am I missing here?
>> >
>> > Regards,
>> > -Stefan
>> >
>> >
>> > On Mon, Jul 20, 2015 at 12:58 PM, Jacques Nadeau <ja...@apache.org>
>> > wrote:
>> >
>> >> Can you enable verbose errors at the session level? It may reveal more
>> >> about what is failing.
>> >> On Jul 20, 2015 5:32 AM, "Stefán Baxter" <st...@activitystream.com>
>> >> wrote:
>> >>
>> >> > Hi Jim,
>> >> >
>> >> > I have made those changes and I'm wondering if you can runs this be
>> >> using
>> >> > the two .jar files that the "mvn package" places in the target
>> >> directory?
>> >> >
>> >> > I have tried to have Drill pick those up but the error now is:
>> >> >
>> >> > Error: SYSTEM ERROR: UnsupportedOperationException
>> >> > Fragment 0:0
>> >> > [Error Id: da589dd4-4cfd-4659-8b93-219074ab8c72 on localhost:31010]
>> >> > (state=,code=0)
>> >> >
>> >> > It seems to indicate that it's picking up the functions but that they
>> >> can
>> >> > not be run.
>> >> >
>> >> > Regards,
>> >> > - Stefán
>> >> >
>> >> > On Mon, Jul 20, 2015 at 1:25 AM, Jim Bates <jb...@maprtech.com>
>> wrote:
>> >> >
>> >> > > I pulled out your udf class and threw it into my package. It worked
>> >> for
>> >> > me
>> >> > > with a few modifications.
>> >> > >
>> >> > > You can not have imported classes or method references in your eval
>> or
>> >> > > setup methods as the code will get pulled out and executed somewhere
>> >> else
>> >> > > and it won't be able to find it. With that in mind, Period will
>> need
>> >> to
>> >> > be
>> >> > > org.joda.time.Period, DateTime will need to be
>> org.joda.time.DateTime
>> >> and
>> >> > > roundTimeStamp will need to be
>> >> > > com.activitystream.drill.udfs.ASUserDefinedFunctions.roundTimeStamp.
>> >> > >
>> >> > >
>> >> > >
>> >> > > On Sun, Jul 19, 2015 at 7:02 PM, Stefán Baxter <
>> >> > stefan@activitystream.com>
>> >> > > wrote:
>> >> > >
>> >> > > > Hi,
>> >> > > >
>> >> > > > The project can be found here:
>> >> > > > https://github.com/acmeguy/asdrill
>> >> > > >
>> >> > > > Thank you,
>> >> > > > -Stefán
>> >> > > >
>> >> > > > On Sun, Jul 19, 2015 at 11:57 PM, Stefán Baxter <
>> >> > > stefan@activitystream.com
>> >>
>> >> > > > >
>> >> > > > wrote:
>> >> > > >
>> >> > > > > Hi,
>> >> > > > >
>> >> > > > > I'm more than happy to share the little that is there (I will
>> >> publish
>> >> > > it
>> >> > > > > on github and send link tomorrow).
>> >> > > > >
>> >> > > > > I ended up copying my UDF (singl-file.java) into the
>> >> > > > simple-drill-function
>> >> > > > > project where it got picked up.
>> >> > > > >
>> >> > > > > Then I discovered a whole new set of dependencies/limitations
>> >> > > > >
>> >> > > > > - The UDF are recompiled - any imports are invalid or at
>> least
>> >> > > > > overwritten
>> >> > > > > - import org.joda.time.Period; (means that Period class is
>> not
>> >> > > > > resolved on runtime)
>> >> > > > > - Error: SYSTEM ERROR: CompileException: Line 71, Column 26:
>> >> > Cannot
>> >> > > > > determine simple type name "Period"
>> >> > > > >
>> >> > > > > - Calling any "outside" functions, like I was calling a
>> static
>> >> > > > > function of the new class (same file), leads to an errors as
>> >> they
>> >> > > are
>> >> > > > not
>> >> > > > > not resolved
>> >> > > > > - Error: SYSTEM ERROR: CompileException: Line 70, Column
>> 35: A
>> >> > > method
>> >> > > > > named "roundTimeStamp" is not declared in any enclosing class
>> >> nor
>> >> > > any
>> >> > > > > supertype, nor through a static import
>> >> > > > >
>> >> > > > > Perhaps this was mentioned in the documentation but this is, at
>> >> the
>> >> > > very
>> >> > > > > least, not straight forward and super-inviting.
>> >> > > > >
>> >> > > > > Thank you for your assistance, we will keep trying :)
>> >> > > > >
>> >> > > > > Regards,
>> >> > > > > -Stefan
>> >> > > > >
>> >> > > > >
>> >> > > > > On Sun, Jul 19, 2015 at 11:34 PM, Tugdual Grall <
>> >> tugdual@gmail.com>
>> >> > > > wrote:
>> >> > > > >
>> >> > > > >> Hi Stefan,
>> >> > > > >>
>> >> > > > >> Do you think you can share your complete project ?
>> >> > > > >>
>> >> > > > >> This will help to debug it for you.
>> >> > > > >>
>> >> > > > >> T
>> >> > > > >>
>> >> > > > >> On Sunday, July 19, 2015, Stefán Baxter <
>> >> stefan@activitystream.com>
>> >> > > > >> wrote:
>> >> > > > >>
>> >> > > > >> > Hi Ted,
>> >> > > > >> >
>> >> > > > >> > I fetched this, built it and deployed it without problems.
>> >> > > > >> > I can not see any real difference other than this deploys two
>> >> .jar
>> >> > > (I
>> >> > > > >> tried
>> >> > > > >> > that as well earlier).
>> >> > > > >> >
>> >> > > > >> > I'm still trying to figure out why Drill is not picking up my
>> >> UDFs
>> >> > > > >> >
>> >> > > > >> > Regards,
>> >> > > > >> > -Stefán
>> >> > > > >> >
>> >> > > > >> > On Sun, Jul 19, 2015 at 10:45 PM, Ted Dunning <
>> >> > > ted.dunning@gmail.com
>> >> > > > >> > <javascript:;>> wrote:
>> >> > > > >> >
>> >> > > > >> > > Stefan,
>> >> > > > >> > >
>> >> > > > >> > > Have you seen this github project:
>> >> > > > >> > >
>> >> > > > >> > > https://github.com/mapr-demos/simple-drill-functions
>> >> > > > >> > >
>> >> > > > >> > > ?
>> >> > > > >> > >
>> >> > > > >> > >
>> >> > > > >> > > On Sun, Jul 19, 2015 at 2:14 PM, Stefán Baxter <
>> >> > > > >> > stefan@activitystream.com <javascript:;>>
>> >> > > > >> > > wrote:
>> >> > > > >> > >
>> >> > > > >> > > > Hi Jim,
>> >> > > > >> > > >
>> >> > > > >> > > > I'm still not able to make this work. Do you have a
>> sample
>> >> > .jar
>> >> > > > file
>> >> > > > >> > > with a
>> >> > > > >> > > > small example that you are running?
>> >> > > > >> > > >
>> >> > > > >> > > > Regards,
>> >> > > > >> > > > -Stefan
>> >> > > > >> > > >
>> >> > > > >> > > > On Sun, Jul 19, 2015 at 6:46 PM, Ted Dunning <
>> >> > > > ted.dunning@gmail.com
>> >> > > > >> > <javascript:;>>
>> >> > > > >> > > > wrote:
>> >> > > > >> > > >
>> >> > > > >> > > > >
>> >> > > > >> > > > > Sounds like a fine example, not because of
>> sophistication
>> >> > but
>> >> > > > >> because
>> >> > > > >> > > it
>> >> > > > >> > > > > deals with dates.
>> >> > > > >> > > > >
>> >> > > > >> > > > > Check the drill logs. It is likely that drill is
>> grumpy
>> >> > about
>> >> > > > >> > > something
>> >> > > > >> > > > > in your udf or packaging.
>> >> > > > >> > > > >
>> >> > > > >> > > > > Also, feel free to snitch the pom from the simple
>> >> examples
>> >> > in
>> >> > > > >> order
>> >> > > > >> > to
>> >> > > > >> > > > get
>> >> > > > >> > > > > the pieces assembled and packaged correctly.
>> >> > > > >> > > > >
>> >> > > > >> > > > > Sent from my iPhone
>> >> > > > >> > > > >
>> >> > > > >> > > > > > On Jul 19, 2015, at 11:25, Stefán Baxter <
>> >> > > > >> > stefan@activitystream.com <javascript:;>>
>> >> > > > >> > > > > wrote:
>> >> > > > >> > > > > >
>> >> > > > >> > > > > > Hi Jim,
>> >> > > > >> > > > > >
>> >> > > > >> > > > > > My UDF is currently so simple that I'm not sure you
>> >> need
>> >> > it
>> >> > > > (or
>> >> > > > >> > want
>> >> > > > >> > > > it).
>> >> > > > >> > > > > >
>> >> > > > >> > > > > > It basically just rounds a timestamp value with ISO
>> >> 8601
>> >> > > > periods
>> >> > > > >> > > > > > asRoundTimestamp(timestampvalue,'PT10M').
>> >> > > > >> > > > > >
>> >> > > > >> > > > > > I would be more than happy to contribute to your
>> >> project
>> >> > > > rather
>> >> > > > >> > than
>> >> > > > >> > > > > > building our own :).
>> >> > > > >> > > > > >
>> >> > > > >> > > > > > Is the repo public?
>> >> > > > >> > > > > >
>> >> > > > >> > > > > > Regards,
>> >> > > > >> > > > > > -Stefan
>> >> > > > >> > > > > >
>> >> > > > >> > > > > >
>> >> > > > >> > > > > >
>> >> > > > >> > > > > >> On Sun, Jul 19, 2015 at 6:18 PM, Jim Bates <
>> >> > > > >> jbates@maprtech.com
>> >> > > > >> > <javascript:;>>
>> >> > > > >> > > > wrote:
>> >> > > > >> > > > > >>
>> >> > > > >> > > > > >> Maven will typically create a jar for class and a
>> jar
>> >> for
>> >> > > > >> source
>> >> > > > >> > > when
>> >> > > > >> > > > > told
>> >> > > > >> > > > > >> to do so. I just include the source files in the
>> same
>> >> jar
>> >> > > as
>> >> > > > >> the
>> >> > > > >> > > class
>> >> > > > >> > > > > >> files. There is a github example drill udf project
>> we
>> >> are
>> >> > > > >> working
>> >> > > > >> > on
>> >> > > > >> > > > to
>> >> > > > >> > > > > >> include several examples to simplify the learning
>> >> curve.
>> >> > If
>> >> > > > >> your
>> >> > > > >> > > > > >> interested... I'd love to have you add your udf.
>> >> > > > >> > > > > >> On Jul 19, 2015 12:59 PM, "Stefán Baxter" <
>> >> > > > >> > > stefan@activitystream.com <javascript:;>>
>> >> > > > >> > > > > >> wrote:
>> >> > > > >> > > > > >>
>> >> > > > >> > > > > >>> Hi again,
>> >> > > > >> > > > > >>>
>> >> > > > >> > > > > >>> Going over the documentation once more I came
>> across
>> >> > this:
>> >> > > > >> > > > > >>>
>> >> > > > >> > > > > >>> - Add the sources and classes JAR files to
>> Drill’s
>> >> > > > >> classpath.
>> >> > > > >> > > > > >>>
>> >> > > > >> > > > > >>> I'm only including a standard .jar (with classes no
>> >> > > sources)
>> >> > > > >> and
>> >> > > > >> > I
>> >> > > > >> > > > > wonder
>> >> > > > >> > > > > >>> why this dependency is listed.
>> >> > > > >> > > > > >>>
>> >> > > > >> > > > > >>> Anyways, that it the next step.
>> >> > > > >> > > > > >>>
>> >> > > > >> > > > > >>> A sample .jar that works with the drill-config
>> >> > counterpart
>> >> > > > >> would
>> >> > > > >> > > be a
>> >> > > > >> > > > > >>> welcome addition to the documentation.
>> >> > > > >> > > > > >>>
>> >> > > > >> > > > > >>> Regards,
>> >> > > > >> > > > > >>> -Stefan
>> >> > > > >> > > > > >>>
>> >> > > > >> > > > > >>>
>> >> > > > >> > > > > >>> On Sun, Jul 19, 2015 at 5:53 PM, Stefán Baxter <
>> >> > > > >> > > > > >> stefan@activitystream.com <javascript:;>>
>> >> > > > >> > > > > >>> wrote:
>> >> > > > >> > > > > >>>
>> >> > > > >> > > > > >>>> Hi Jim,
>> >> > > > >> > > > > >>>>
>> >> > > > >> > > > > >>>> Now I have added the file to the jar (both root
>> and
>> >> > > > resources
>> >> > > > >> > > > folder)
>> >> > > > >> > > > > >> but
>> >> > > > >> > > > > >>>> that does not seem to change anything.
>> >> > > > >> > > > > >>>>
>> >> > > > >> > > > > >>>> Any additional ideas?
>> >> > > > >> > > > > >>>>
>> >> > > > >> > > > > >>>> Regards,
>> >> > > > >> > > > > >>>> -Stefan
>> >> > > > >> > > > > >>>>
>> >> > > > >> > > > > >>>> On Sun, Jul 19, 2015 at 5:40 PM, Stefán Baxter <
>> >> > > > >> > > > > >>> stefan@activitystream.com <javascript:;>>
>> >> > > > >> > > > > >>>> wrote:
>> >> > > > >> > > > > >>>>
>> >> > > > >> > > > > >>>>> NO!
>> >> > > > >> > > > > >>>>>
>> >> > > > >> > > > > >>>>> Thank you , will do that right now :)
>> >> > > > >> > > > > >>>>>
>> >> > > > >> > > > > >>>>> On Sun, Jul 19, 2015 at 5:38 PM, Jim Bates <
>> >> > > > >> > jbates@maprtech.com <javascript:;>>
>> >> > > > >> > > > > >> wrote:
>> >> > > > >> > > > > >>>>>
>> >> > > > >> > > > > >>>>>> Did you include a file drill-module.conf in your
>> >> jar
>> >> > > > along
>> >> > > > >> > with
>> >> > > > >> > > > > >> source
>> >> > > > >> > > > > >>>>>> files?
>> >> > > > >> > > > > >>>>>> On Jul 19, 2015 12:20 PM, "Stefán Baxter" <
>> >> > > > >> > > > > stefan@activitystream.com <javascript:;>
>> >>
>> >> > > > >>
>> >> > > > >> > > > > >>>
>> >> > > > >> > > > > >>>>>> wrote:
>> >> > > > >> > > > > >>>>>>
>> >> > > > >> > > > > >>>>>>> Hi,
>> >> > > > >> > > > > >>>>>>>
>> >> > > > >> > > > > >>>>>>> I'm trying to deploy a UDF that I have written
>> >> > > according
>> >> > > > >> to
>> >> > > > >> > the
>> >> > > > >> > > > > >>>>>>> documentation.
>> >> > > > >> > > > > >>>>>>>
>> >> > > > >> > > > > >>>>>>> I have also:
>> >> > > > >> > > > > >>>>>>>
>> >> > > > >> > > > > >>>>>>> 1. Copied the jar file
>> >> > > > >> > > > > >>>>>>> to jars/3rdparty
>> >> > > > >> > > > > >>>>>>>
>> >> > > > >> > > > > >>>>>>> 2. Changed the config
>> >> "conf/drill-override.conf"
>> >> > to
>> >> > > > >> > include:
>> >> > > > >> > > > > >>>>>>> drill.logical.function.package +=
>> >> > > > >> > > > > >>
>> >> > > > >> > >
>> >> > > >
>> >> ["org.apache.drill.exec.expr.fn.impl","com.activitystream.drill.udfs"]
>> >> > > > >> > > > > >>>>>>>
>> >> > > > >> > > > > >>>>>>> 3. Rerun the Drillbit
>> >> > > > >> > > > > >>>>>>> bin/drillbit.sh start
>> >> > > > >> > > > > >>>>>>>
>> >> > > > >> > > > > >>>>>>> The log shows no sign of the the UDF being
>> loaded
>> >> > but
>> >> > > > if I
>> >> > > > >> > use
>> >> > > > >> > > it
>> >> > > > >> > > > > >> in
>> >> > > > >> > > > > >>> a
>> >> > > > >> > > > > >>>>>> SQL
>> >> > > > >> > > > > >>>>>>> statement then this error is thrown:
>> >> > > > >> > > > > >>>>>>>
>> >> > > > >> > > > > >>>>>>> Error: PARSE ERROR: From line 1, column 184 to
>> >> line
>> >> > 1,
>> >> > > > >> column
>> >> > > > >> > > > 219:
>> >> > > > >> > > > > >> No
>> >> > > > >> > > > > >>>>>> match
>> >> > > > >> > > > > >>>>>>> found for function signature
>> >> asRoundTimeStamp(<ANY>,
>> >> > > > >> > > <CHARACTER>)
>> >> > > > >> > > > > >>>>>>>
>> >> > > > >> > > > > >>>>>>>
>> >> > > > >> > > > > >>>>>>> The relevant part of the UDF source:
>> >> > > > >> > > > > >>>>>>>
>> >> > > > >> > > > > >>>>>>> package com.activitystream.drill.udfs;
>> >> > > > >> > > > > >>>>>>>
>> >> > > > >> > > > > >>>>>>> public class ASUserDefinedFunctions {
>> >> > > > >> > > > > >>>>>>>
>> >> > > > >> > > > > >>>>>>> @FunctionTemplate(name = "asRoundTimeStamp",
>> >> > scope
>> >> > > =
>> >> > > > >> > > > > >>>>>>> FunctionTemplate.FunctionScope.SIMPLE, nulls =
>> >> > > > >> > > > > >>>>>>> FunctionTemplate.NullHandling.NULL_IF_NULL)
>> >> > > > >> > > > > >>>>>>>
>> >> > > > >> > > > > >>>>>>> public static class RoundTimeStamp
>> implements
>> >> > > > >> > > DrillSimpleFunc
>> >> > > > >> > > > > >>>>>>>
>> >> > > > >> > > > > >>>>>>>
>> >> > > > >> > > > > >>>>>>> Can anyone please assist?
>> >> > > > >> > > > > >>>>>>>
>> >> > > > >> > > > > >>>>>>> Regards,
>> >> > > > >> > > > > >>>>>>> -Stefan
>> >> > > > >> > > > > >>
>> >> > > > >> > > > >
>> >> > > > >> > > >
>> >> > > > >> > >
>> >> > > > >> >
>> >> > > > >>
>> >> > > > >
>> >> > > > >
>> >> > > >
>> >> > >
>> >> >
>> >>
>> >
>> >
>>
Re: Drill not picking up a UDF
Posted by Jacques Nadeau <ja...@dremio.com>.
Hey Stefan,
Can you propose some edits/updates to the documentation? The doc is
maintained on github [1].
The key thing to understand is that a UDF is slightly incomplete Java.
This is because Drill actually rips apart the functionality of the UDF and
recomposes it directly within expression evaluation code. This gives a
substantial performance and memory benefit but also creates some
challenges. As such, there are some key rules one should follow:
- Don't use imports
- Both class file and source file have to be on the classpath (yes, Drill
uses the source of the function)
- Any JAR files holding UDF resources must include drill-module.conf
marker file so that Drill knows to include that JAR in consideration for
UDF loading
- ValueHolders should be treated as structs. As such, don't call any
methods on ValueHolders
If you fork the documentation and provide a pull request, doc people
(Kristine and Bridget) will work to include your feedback so that others
don't find as many challenges as you did.
thanks,
Jacques
[1]
https://github.com/apache/drill/tree/gh-pages/_docs/develop-custom-functions
On Mon, Jul 20, 2015 at 7:19 AM, Stefán Baxter <st...@activitystream.com>
wrote:
> Hi,
>
> This is working now.
>
> This, unless someone corrects me, is the code needed to get a string
> parameter for a UDF:
>
>
> - input2.buffer.toString(input2.start,
> input2.end-input2.start,java.nio.charset.Charset.defaultCharset())
>
> Regards,
> -Stefan (not the happiest camper)
>
>
> On Mon, Jul 20, 2015 at 2:07 PM, Stefán Baxter <st...@activitystream.com>
> wrote:
>
> > Hi,
> >
> > After going through the log this is clear what is happening (once the
> > Drill picked up the UDF a bit earlier this morning).
> >
> > I'm calling the VarCharHolder.toString() to get the text value for the
> > parameter and that is throwing this exception.
> >
> > Two observations:
> >
> > 1. Calling a deprecated function, even though that its not optimal,
> > usually does not cause such drastic results.
> > 2. As far as I can see there is no easy way to get the string value of
> > a property without carving out a piece of the buffer.
> >
> > What am I missing here?
> >
> > Regards,
> > -Stefan
> >
> >
> > On Mon, Jul 20, 2015 at 12:58 PM, Jacques Nadeau <ja...@apache.org>
> > wrote:
> >
> >> Can you enable verbose errors at the session level? It may reveal more
> >> about what is failing.
> >> On Jul 20, 2015 5:32 AM, "Stefán Baxter" <st...@activitystream.com>
> >> wrote:
> >>
> >> > Hi Jim,
> >> >
> >> > I have made those changes and I'm wondering if you can runs this be
> >> using
> >> > the two .jar files that the "mvn package" places in the target
> >> directory?
> >> >
> >> > I have tried to have Drill pick those up but the error now is:
> >> >
> >> > Error: SYSTEM ERROR: UnsupportedOperationException
> >> > Fragment 0:0
> >> > [Error Id: da589dd4-4cfd-4659-8b93-219074ab8c72 on localhost:31010]
> >> > (state=,code=0)
> >> >
> >> > It seems to indicate that it's picking up the functions but that they
> >> can
> >> > not be run.
> >> >
> >> > Regards,
> >> > - Stefán
> >> >
> >> > On Mon, Jul 20, 2015 at 1:25 AM, Jim Bates <jb...@maprtech.com>
> wrote:
> >> >
> >> > > I pulled out your udf class and threw it into my package. It worked
> >> for
> >> > me
> >> > > with a few modifications.
> >> > >
> >> > > You can not have imported classes or method references in your eval
> or
> >> > > setup methods as the code will get pulled out and executed somewhere
> >> else
> >> > > and it won't be able to find it. With that in mind, Period will
> need
> >> to
> >> > be
> >> > > org.joda.time.Period, DateTime will need to be
> org.joda.time.DateTime
> >> and
> >> > > roundTimeStamp will need to be
> >> > > com.activitystream.drill.udfs.ASUserDefinedFunctions.roundTimeStamp.
> >> > >
> >> > >
> >> > >
> >> > > On Sun, Jul 19, 2015 at 7:02 PM, Stefán Baxter <
> >> > stefan@activitystream.com>
> >> > > wrote:
> >> > >
> >> > > > Hi,
> >> > > >
> >> > > > The project can be found here:
> >> > > > https://github.com/acmeguy/asdrill
> >> > > >
> >> > > > Thank you,
> >> > > > -Stefán
> >> > > >
> >> > > > On Sun, Jul 19, 2015 at 11:57 PM, Stefán Baxter <
> >> > > stefan@activitystream.com
> >>
> >> > > > >
> >> > > > wrote:
> >> > > >
> >> > > > > Hi,
> >> > > > >
> >> > > > > I'm more than happy to share the little that is there (I will
> >> publish
> >> > > it
> >> > > > > on github and send link tomorrow).
> >> > > > >
> >> > > > > I ended up copying my UDF (singl-file.java) into the
> >> > > > simple-drill-function
> >> > > > > project where it got picked up.
> >> > > > >
> >> > > > > Then I discovered a whole new set of dependencies/limitations
> >> > > > >
> >> > > > > - The UDF are recompiled - any imports are invalid or at
> least
> >> > > > > overwritten
> >> > > > > - import org.joda.time.Period; (means that Period class is
> not
> >> > > > > resolved on runtime)
> >> > > > > - Error: SYSTEM ERROR: CompileException: Line 71, Column 26:
> >> > Cannot
> >> > > > > determine simple type name "Period"
> >> > > > >
> >> > > > > - Calling any "outside" functions, like I was calling a
> static
> >> > > > > function of the new class (same file), leads to an errors as
> >> they
> >> > > are
> >> > > > not
> >> > > > > not resolved
> >> > > > > - Error: SYSTEM ERROR: CompileException: Line 70, Column
> 35: A
> >> > > method
> >> > > > > named "roundTimeStamp" is not declared in any enclosing class
> >> nor
> >> > > any
> >> > > > > supertype, nor through a static import
> >> > > > >
> >> > > > > Perhaps this was mentioned in the documentation but this is, at
> >> the
> >> > > very
> >> > > > > least, not straight forward and super-inviting.
> >> > > > >
> >> > > > > Thank you for your assistance, we will keep trying :)
> >> > > > >
> >> > > > > Regards,
> >> > > > > -Stefan
> >> > > > >
> >> > > > >
> >> > > > > On Sun, Jul 19, 2015 at 11:34 PM, Tugdual Grall <
> >> tugdual@gmail.com>
> >> > > > wrote:
> >> > > > >
> >> > > > >> Hi Stefan,
> >> > > > >>
> >> > > > >> Do you think you can share your complete project ?
> >> > > > >>
> >> > > > >> This will help to debug it for you.
> >> > > > >>
> >> > > > >> T
> >> > > > >>
> >> > > > >> On Sunday, July 19, 2015, Stefán Baxter <
> >> stefan@activitystream.com>
> >> > > > >> wrote:
> >> > > > >>
> >> > > > >> > Hi Ted,
> >> > > > >> >
> >> > > > >> > I fetched this, built it and deployed it without problems.
> >> > > > >> > I can not see any real difference other than this deploys two
> >> .jar
> >> > > (I
> >> > > > >> tried
> >> > > > >> > that as well earlier).
> >> > > > >> >
> >> > > > >> > I'm still trying to figure out why Drill is not picking up my
> >> UDFs
> >> > > > >> >
> >> > > > >> > Regards,
> >> > > > >> > -Stefán
> >> > > > >> >
> >> > > > >> > On Sun, Jul 19, 2015 at 10:45 PM, Ted Dunning <
> >> > > ted.dunning@gmail.com
> >> > > > >> > <javascript:;>> wrote:
> >> > > > >> >
> >> > > > >> > > Stefan,
> >> > > > >> > >
> >> > > > >> > > Have you seen this github project:
> >> > > > >> > >
> >> > > > >> > > https://github.com/mapr-demos/simple-drill-functions
> >> > > > >> > >
> >> > > > >> > > ?
> >> > > > >> > >
> >> > > > >> > >
> >> > > > >> > > On Sun, Jul 19, 2015 at 2:14 PM, Stefán Baxter <
> >> > > > >> > stefan@activitystream.com <javascript:;>>
> >> > > > >> > > wrote:
> >> > > > >> > >
> >> > > > >> > > > Hi Jim,
> >> > > > >> > > >
> >> > > > >> > > > I'm still not able to make this work. Do you have a
> sample
> >> > .jar
> >> > > > file
> >> > > > >> > > with a
> >> > > > >> > > > small example that you are running?
> >> > > > >> > > >
> >> > > > >> > > > Regards,
> >> > > > >> > > > -Stefan
> >> > > > >> > > >
> >> > > > >> > > > On Sun, Jul 19, 2015 at 6:46 PM, Ted Dunning <
> >> > > > ted.dunning@gmail.com
> >> > > > >> > <javascript:;>>
> >> > > > >> > > > wrote:
> >> > > > >> > > >
> >> > > > >> > > > >
> >> > > > >> > > > > Sounds like a fine example, not because of
> sophistication
> >> > but
> >> > > > >> because
> >> > > > >> > > it
> >> > > > >> > > > > deals with dates.
> >> > > > >> > > > >
> >> > > > >> > > > > Check the drill logs. It is likely that drill is
> grumpy
> >> > about
> >> > > > >> > > something
> >> > > > >> > > > > in your udf or packaging.
> >> > > > >> > > > >
> >> > > > >> > > > > Also, feel free to snitch the pom from the simple
> >> examples
> >> > in
> >> > > > >> order
> >> > > > >> > to
> >> > > > >> > > > get
> >> > > > >> > > > > the pieces assembled and packaged correctly.
> >> > > > >> > > > >
> >> > > > >> > > > > Sent from my iPhone
> >> > > > >> > > > >
> >> > > > >> > > > > > On Jul 19, 2015, at 11:25, Stefán Baxter <
> >> > > > >> > stefan@activitystream.com <javascript:;>>
> >> > > > >> > > > > wrote:
> >> > > > >> > > > > >
> >> > > > >> > > > > > Hi Jim,
> >> > > > >> > > > > >
> >> > > > >> > > > > > My UDF is currently so simple that I'm not sure you
> >> need
> >> > it
> >> > > > (or
> >> > > > >> > want
> >> > > > >> > > > it).
> >> > > > >> > > > > >
> >> > > > >> > > > > > It basically just rounds a timestamp value with ISO
> >> 8601
> >> > > > periods
> >> > > > >> > > > > > asRoundTimestamp(timestampvalue,'PT10M').
> >> > > > >> > > > > >
> >> > > > >> > > > > > I would be more than happy to contribute to your
> >> project
> >> > > > rather
> >> > > > >> > than
> >> > > > >> > > > > > building our own :).
> >> > > > >> > > > > >
> >> > > > >> > > > > > Is the repo public?
> >> > > > >> > > > > >
> >> > > > >> > > > > > Regards,
> >> > > > >> > > > > > -Stefan
> >> > > > >> > > > > >
> >> > > > >> > > > > >
> >> > > > >> > > > > >
> >> > > > >> > > > > >> On Sun, Jul 19, 2015 at 6:18 PM, Jim Bates <
> >> > > > >> jbates@maprtech.com
> >> > > > >> > <javascript:;>>
> >> > > > >> > > > wrote:
> >> > > > >> > > > > >>
> >> > > > >> > > > > >> Maven will typically create a jar for class and a
> jar
> >> for
> >> > > > >> source
> >> > > > >> > > when
> >> > > > >> > > > > told
> >> > > > >> > > > > >> to do so. I just include the source files in the
> same
> >> jar
> >> > > as
> >> > > > >> the
> >> > > > >> > > class
> >> > > > >> > > > > >> files. There is a github example drill udf project
> we
> >> are
> >> > > > >> working
> >> > > > >> > on
> >> > > > >> > > > to
> >> > > > >> > > > > >> include several examples to simplify the learning
> >> curve.
> >> > If
> >> > > > >> your
> >> > > > >> > > > > >> interested... I'd love to have you add your udf.
> >> > > > >> > > > > >> On Jul 19, 2015 12:59 PM, "Stefán Baxter" <
> >> > > > >> > > stefan@activitystream.com <javascript:;>>
> >> > > > >> > > > > >> wrote:
> >> > > > >> > > > > >>
> >> > > > >> > > > > >>> Hi again,
> >> > > > >> > > > > >>>
> >> > > > >> > > > > >>> Going over the documentation once more I came
> across
> >> > this:
> >> > > > >> > > > > >>>
> >> > > > >> > > > > >>> - Add the sources and classes JAR files to
> Drill’s
> >> > > > >> classpath.
> >> > > > >> > > > > >>>
> >> > > > >> > > > > >>> I'm only including a standard .jar (with classes no
> >> > > sources)
> >> > > > >> and
> >> > > > >> > I
> >> > > > >> > > > > wonder
> >> > > > >> > > > > >>> why this dependency is listed.
> >> > > > >> > > > > >>>
> >> > > > >> > > > > >>> Anyways, that it the next step.
> >> > > > >> > > > > >>>
> >> > > > >> > > > > >>> A sample .jar that works with the drill-config
> >> > counterpart
> >> > > > >> would
> >> > > > >> > > be a
> >> > > > >> > > > > >>> welcome addition to the documentation.
> >> > > > >> > > > > >>>
> >> > > > >> > > > > >>> Regards,
> >> > > > >> > > > > >>> -Stefan
> >> > > > >> > > > > >>>
> >> > > > >> > > > > >>>
> >> > > > >> > > > > >>> On Sun, Jul 19, 2015 at 5:53 PM, Stefán Baxter <
> >> > > > >> > > > > >> stefan@activitystream.com <javascript:;>>
> >> > > > >> > > > > >>> wrote:
> >> > > > >> > > > > >>>
> >> > > > >> > > > > >>>> Hi Jim,
> >> > > > >> > > > > >>>>
> >> > > > >> > > > > >>>> Now I have added the file to the jar (both root
> and
> >> > > > resources
> >> > > > >> > > > folder)
> >> > > > >> > > > > >> but
> >> > > > >> > > > > >>>> that does not seem to change anything.
> >> > > > >> > > > > >>>>
> >> > > > >> > > > > >>>> Any additional ideas?
> >> > > > >> > > > > >>>>
> >> > > > >> > > > > >>>> Regards,
> >> > > > >> > > > > >>>> -Stefan
> >> > > > >> > > > > >>>>
> >> > > > >> > > > > >>>> On Sun, Jul 19, 2015 at 5:40 PM, Stefán Baxter <
> >> > > > >> > > > > >>> stefan@activitystream.com <javascript:;>>
> >> > > > >> > > > > >>>> wrote:
> >> > > > >> > > > > >>>>
> >> > > > >> > > > > >>>>> NO!
> >> > > > >> > > > > >>>>>
> >> > > > >> > > > > >>>>> Thank you , will do that right now :)
> >> > > > >> > > > > >>>>>
> >> > > > >> > > > > >>>>> On Sun, Jul 19, 2015 at 5:38 PM, Jim Bates <
> >> > > > >> > jbates@maprtech.com <javascript:;>>
> >> > > > >> > > > > >> wrote:
> >> > > > >> > > > > >>>>>
> >> > > > >> > > > > >>>>>> Did you include a file drill-module.conf in your
> >> jar
> >> > > > along
> >> > > > >> > with
> >> > > > >> > > > > >> source
> >> > > > >> > > > > >>>>>> files?
> >> > > > >> > > > > >>>>>> On Jul 19, 2015 12:20 PM, "Stefán Baxter" <
> >> > > > >> > > > > stefan@activitystream.com <javascript:;>
> >>
> >> > > > >>
> >> > > > >> > > > > >>>
> >> > > > >> > > > > >>>>>> wrote:
> >> > > > >> > > > > >>>>>>
> >> > > > >> > > > > >>>>>>> Hi,
> >> > > > >> > > > > >>>>>>>
> >> > > > >> > > > > >>>>>>> I'm trying to deploy a UDF that I have written
> >> > > according
> >> > > > >> to
> >> > > > >> > the
> >> > > > >> > > > > >>>>>>> documentation.
> >> > > > >> > > > > >>>>>>>
> >> > > > >> > > > > >>>>>>> I have also:
> >> > > > >> > > > > >>>>>>>
> >> > > > >> > > > > >>>>>>> 1. Copied the jar file
> >> > > > >> > > > > >>>>>>> to jars/3rdparty
> >> > > > >> > > > > >>>>>>>
> >> > > > >> > > > > >>>>>>> 2. Changed the config
> >> "conf/drill-override.conf"
> >> > to
> >> > > > >> > include:
> >> > > > >> > > > > >>>>>>> drill.logical.function.package +=
> >> > > > >> > > > > >>
> >> > > > >> > >
> >> > > >
> >> ["org.apache.drill.exec.expr.fn.impl","com.activitystream.drill.udfs"]
> >> > > > >> > > > > >>>>>>>
> >> > > > >> > > > > >>>>>>> 3. Rerun the Drillbit
> >> > > > >> > > > > >>>>>>> bin/drillbit.sh start
> >> > > > >> > > > > >>>>>>>
> >> > > > >> > > > > >>>>>>> The log shows no sign of the the UDF being
> loaded
> >> > but
> >> > > > if I
> >> > > > >> > use
> >> > > > >> > > it
> >> > > > >> > > > > >> in
> >> > > > >> > > > > >>> a
> >> > > > >> > > > > >>>>>> SQL
> >> > > > >> > > > > >>>>>>> statement then this error is thrown:
> >> > > > >> > > > > >>>>>>>
> >> > > > >> > > > > >>>>>>> Error: PARSE ERROR: From line 1, column 184 to
> >> line
> >> > 1,
> >> > > > >> column
> >> > > > >> > > > 219:
> >> > > > >> > > > > >> No
> >> > > > >> > > > > >>>>>> match
> >> > > > >> > > > > >>>>>>> found for function signature
> >> asRoundTimeStamp(<ANY>,
> >> > > > >> > > <CHARACTER>)
> >> > > > >> > > > > >>>>>>>
> >> > > > >> > > > > >>>>>>>
> >> > > > >> > > > > >>>>>>> The relevant part of the UDF source:
> >> > > > >> > > > > >>>>>>>
> >> > > > >> > > > > >>>>>>> package com.activitystream.drill.udfs;
> >> > > > >> > > > > >>>>>>>
> >> > > > >> > > > > >>>>>>> public class ASUserDefinedFunctions {
> >> > > > >> > > > > >>>>>>>
> >> > > > >> > > > > >>>>>>> @FunctionTemplate(name = "asRoundTimeStamp",
> >> > scope
> >> > > =
> >> > > > >> > > > > >>>>>>> FunctionTemplate.FunctionScope.SIMPLE, nulls =
> >> > > > >> > > > > >>>>>>> FunctionTemplate.NullHandling.NULL_IF_NULL)
> >> > > > >> > > > > >>>>>>>
> >> > > > >> > > > > >>>>>>> public static class RoundTimeStamp
> implements
> >> > > > >> > > DrillSimpleFunc
> >> > > > >> > > > > >>>>>>>
> >> > > > >> > > > > >>>>>>>
> >> > > > >> > > > > >>>>>>> Can anyone please assist?
> >> > > > >> > > > > >>>>>>>
> >> > > > >> > > > > >>>>>>> Regards,
> >> > > > >> > > > > >>>>>>> -Stefan
> >> > > > >> > > > > >>
> >> > > > >> > > > >
> >> > > > >> > > >
> >> > > > >> > >
> >> > > > >> >
> >> > > > >>
> >> > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> >
> >
>
Re: Drill not picking up a UDF
Posted by Stefán Baxter <st...@activitystream.com>.
Hi,
This is working now.
This, unless someone corrects me, is the code needed to get a string
parameter for a UDF:
- input2.buffer.toString(input2.start,
input2.end-input2.start,java.nio.charset.Charset.defaultCharset())
Regards,
-Stefan (not the happiest camper)
On Mon, Jul 20, 2015 at 2:07 PM, Stefán Baxter <st...@activitystream.com>
wrote:
> Hi,
>
> After going through the log this is clear what is happening (once the
> Drill picked up the UDF a bit earlier this morning).
>
> I'm calling the VarCharHolder.toString() to get the text value for the
> parameter and that is throwing this exception.
>
> Two observations:
>
> 1. Calling a deprecated function, even though that its not optimal,
> usually does not cause such drastic results.
> 2. As far as I can see there is no easy way to get the string value of
> a property without carving out a piece of the buffer.
>
> What am I missing here?
>
> Regards,
> -Stefan
>
>
> On Mon, Jul 20, 2015 at 12:58 PM, Jacques Nadeau <ja...@apache.org>
> wrote:
>
>> Can you enable verbose errors at the session level? It may reveal more
>> about what is failing.
>> On Jul 20, 2015 5:32 AM, "Stefán Baxter" <st...@activitystream.com>
>> wrote:
>>
>> > Hi Jim,
>> >
>> > I have made those changes and I'm wondering if you can runs this be
>> using
>> > the two .jar files that the "mvn package" places in the target
>> directory?
>> >
>> > I have tried to have Drill pick those up but the error now is:
>> >
>> > Error: SYSTEM ERROR: UnsupportedOperationException
>> > Fragment 0:0
>> > [Error Id: da589dd4-4cfd-4659-8b93-219074ab8c72 on localhost:31010]
>> > (state=,code=0)
>> >
>> > It seems to indicate that it's picking up the functions but that they
>> can
>> > not be run.
>> >
>> > Regards,
>> > - Stefán
>> >
>> > On Mon, Jul 20, 2015 at 1:25 AM, Jim Bates <jb...@maprtech.com> wrote:
>> >
>> > > I pulled out your udf class and threw it into my package. It worked
>> for
>> > me
>> > > with a few modifications.
>> > >
>> > > You can not have imported classes or method references in your eval or
>> > > setup methods as the code will get pulled out and executed somewhere
>> else
>> > > and it won't be able to find it. With that in mind, Period will need
>> to
>> > be
>> > > org.joda.time.Period, DateTime will need to be org.joda.time.DateTime
>> and
>> > > roundTimeStamp will need to be
>> > > com.activitystream.drill.udfs.ASUserDefinedFunctions.roundTimeStamp.
>> > >
>> > >
>> > >
>> > > On Sun, Jul 19, 2015 at 7:02 PM, Stefán Baxter <
>> > stefan@activitystream.com>
>> > > wrote:
>> > >
>> > > > Hi,
>> > > >
>> > > > The project can be found here:
>> > > > https://github.com/acmeguy/asdrill
>> > > >
>> > > > Thank you,
>> > > > -Stefán
>> > > >
>> > > > On Sun, Jul 19, 2015 at 11:57 PM, Stefán Baxter <
>> > > stefan@activitystream.com
>>
>> > > > >
>> > > > wrote:
>> > > >
>> > > > > Hi,
>> > > > >
>> > > > > I'm more than happy to share the little that is there (I will
>> publish
>> > > it
>> > > > > on github and send link tomorrow).
>> > > > >
>> > > > > I ended up copying my UDF (singl-file.java) into the
>> > > > simple-drill-function
>> > > > > project where it got picked up.
>> > > > >
>> > > > > Then I discovered a whole new set of dependencies/limitations
>> > > > >
>> > > > > - The UDF are recompiled - any imports are invalid or at least
>> > > > > overwritten
>> > > > > - import org.joda.time.Period; (means that Period class is not
>> > > > > resolved on runtime)
>> > > > > - Error: SYSTEM ERROR: CompileException: Line 71, Column 26:
>> > Cannot
>> > > > > determine simple type name "Period"
>> > > > >
>> > > > > - Calling any "outside" functions, like I was calling a static
>> > > > > function of the new class (same file), leads to an errors as
>> they
>> > > are
>> > > > not
>> > > > > not resolved
>> > > > > - Error: SYSTEM ERROR: CompileException: Line 70, Column 35: A
>> > > method
>> > > > > named "roundTimeStamp" is not declared in any enclosing class
>> nor
>> > > any
>> > > > > supertype, nor through a static import
>> > > > >
>> > > > > Perhaps this was mentioned in the documentation but this is, at
>> the
>> > > very
>> > > > > least, not straight forward and super-inviting.
>> > > > >
>> > > > > Thank you for your assistance, we will keep trying :)
>> > > > >
>> > > > > Regards,
>> > > > > -Stefan
>> > > > >
>> > > > >
>> > > > > On Sun, Jul 19, 2015 at 11:34 PM, Tugdual Grall <
>> tugdual@gmail.com>
>> > > > wrote:
>> > > > >
>> > > > >> Hi Stefan,
>> > > > >>
>> > > > >> Do you think you can share your complete project ?
>> > > > >>
>> > > > >> This will help to debug it for you.
>> > > > >>
>> > > > >> T
>> > > > >>
>> > > > >> On Sunday, July 19, 2015, Stefán Baxter <
>> stefan@activitystream.com>
>> > > > >> wrote:
>> > > > >>
>> > > > >> > Hi Ted,
>> > > > >> >
>> > > > >> > I fetched this, built it and deployed it without problems.
>> > > > >> > I can not see any real difference other than this deploys two
>> .jar
>> > > (I
>> > > > >> tried
>> > > > >> > that as well earlier).
>> > > > >> >
>> > > > >> > I'm still trying to figure out why Drill is not picking up my
>> UDFs
>> > > > >> >
>> > > > >> > Regards,
>> > > > >> > -Stefán
>> > > > >> >
>> > > > >> > On Sun, Jul 19, 2015 at 10:45 PM, Ted Dunning <
>> > > ted.dunning@gmail.com
>> > > > >> > <javascript:;>> wrote:
>> > > > >> >
>> > > > >> > > Stefan,
>> > > > >> > >
>> > > > >> > > Have you seen this github project:
>> > > > >> > >
>> > > > >> > > https://github.com/mapr-demos/simple-drill-functions
>> > > > >> > >
>> > > > >> > > ?
>> > > > >> > >
>> > > > >> > >
>> > > > >> > > On Sun, Jul 19, 2015 at 2:14 PM, Stefán Baxter <
>> > > > >> > stefan@activitystream.com <javascript:;>>
>> > > > >> > > wrote:
>> > > > >> > >
>> > > > >> > > > Hi Jim,
>> > > > >> > > >
>> > > > >> > > > I'm still not able to make this work. Do you have a sample
>> > .jar
>> > > > file
>> > > > >> > > with a
>> > > > >> > > > small example that you are running?
>> > > > >> > > >
>> > > > >> > > > Regards,
>> > > > >> > > > -Stefan
>> > > > >> > > >
>> > > > >> > > > On Sun, Jul 19, 2015 at 6:46 PM, Ted Dunning <
>> > > > ted.dunning@gmail.com
>> > > > >> > <javascript:;>>
>> > > > >> > > > wrote:
>> > > > >> > > >
>> > > > >> > > > >
>> > > > >> > > > > Sounds like a fine example, not because of sophistication
>> > but
>> > > > >> because
>> > > > >> > > it
>> > > > >> > > > > deals with dates.
>> > > > >> > > > >
>> > > > >> > > > > Check the drill logs. It is likely that drill is grumpy
>> > about
>> > > > >> > > something
>> > > > >> > > > > in your udf or packaging.
>> > > > >> > > > >
>> > > > >> > > > > Also, feel free to snitch the pom from the simple
>> examples
>> > in
>> > > > >> order
>> > > > >> > to
>> > > > >> > > > get
>> > > > >> > > > > the pieces assembled and packaged correctly.
>> > > > >> > > > >
>> > > > >> > > > > Sent from my iPhone
>> > > > >> > > > >
>> > > > >> > > > > > On Jul 19, 2015, at 11:25, Stefán Baxter <
>> > > > >> > stefan@activitystream.com <javascript:;>>
>> > > > >> > > > > wrote:
>> > > > >> > > > > >
>> > > > >> > > > > > Hi Jim,
>> > > > >> > > > > >
>> > > > >> > > > > > My UDF is currently so simple that I'm not sure you
>> need
>> > it
>> > > > (or
>> > > > >> > want
>> > > > >> > > > it).
>> > > > >> > > > > >
>> > > > >> > > > > > It basically just rounds a timestamp value with ISO
>> 8601
>> > > > periods
>> > > > >> > > > > > asRoundTimestamp(timestampvalue,'PT10M').
>> > > > >> > > > > >
>> > > > >> > > > > > I would be more than happy to contribute to your
>> project
>> > > > rather
>> > > > >> > than
>> > > > >> > > > > > building our own :).
>> > > > >> > > > > >
>> > > > >> > > > > > Is the repo public?
>> > > > >> > > > > >
>> > > > >> > > > > > Regards,
>> > > > >> > > > > > -Stefan
>> > > > >> > > > > >
>> > > > >> > > > > >
>> > > > >> > > > > >
>> > > > >> > > > > >> On Sun, Jul 19, 2015 at 6:18 PM, Jim Bates <
>> > > > >> jbates@maprtech.com
>> > > > >> > <javascript:;>>
>> > > > >> > > > wrote:
>> > > > >> > > > > >>
>> > > > >> > > > > >> Maven will typically create a jar for class and a jar
>> for
>> > > > >> source
>> > > > >> > > when
>> > > > >> > > > > told
>> > > > >> > > > > >> to do so. I just include the source files in the same
>> jar
>> > > as
>> > > > >> the
>> > > > >> > > class
>> > > > >> > > > > >> files. There is a github example drill udf project we
>> are
>> > > > >> working
>> > > > >> > on
>> > > > >> > > > to
>> > > > >> > > > > >> include several examples to simplify the learning
>> curve.
>> > If
>> > > > >> your
>> > > > >> > > > > >> interested... I'd love to have you add your udf.
>> > > > >> > > > > >> On Jul 19, 2015 12:59 PM, "Stefán Baxter" <
>> > > > >> > > stefan@activitystream.com <javascript:;>>
>> > > > >> > > > > >> wrote:
>> > > > >> > > > > >>
>> > > > >> > > > > >>> Hi again,
>> > > > >> > > > > >>>
>> > > > >> > > > > >>> Going over the documentation once more I came across
>> > this:
>> > > > >> > > > > >>>
>> > > > >> > > > > >>> - Add the sources and classes JAR files to Drill’s
>> > > > >> classpath.
>> > > > >> > > > > >>>
>> > > > >> > > > > >>> I'm only including a standard .jar (with classes no
>> > > sources)
>> > > > >> and
>> > > > >> > I
>> > > > >> > > > > wonder
>> > > > >> > > > > >>> why this dependency is listed.
>> > > > >> > > > > >>>
>> > > > >> > > > > >>> Anyways, that it the next step.
>> > > > >> > > > > >>>
>> > > > >> > > > > >>> A sample .jar that works with the drill-config
>> > counterpart
>> > > > >> would
>> > > > >> > > be a
>> > > > >> > > > > >>> welcome addition to the documentation.
>> > > > >> > > > > >>>
>> > > > >> > > > > >>> Regards,
>> > > > >> > > > > >>> -Stefan
>> > > > >> > > > > >>>
>> > > > >> > > > > >>>
>> > > > >> > > > > >>> On Sun, Jul 19, 2015 at 5:53 PM, Stefán Baxter <
>> > > > >> > > > > >> stefan@activitystream.com <javascript:;>>
>> > > > >> > > > > >>> wrote:
>> > > > >> > > > > >>>
>> > > > >> > > > > >>>> Hi Jim,
>> > > > >> > > > > >>>>
>> > > > >> > > > > >>>> Now I have added the file to the jar (both root and
>> > > > resources
>> > > > >> > > > folder)
>> > > > >> > > > > >> but
>> > > > >> > > > > >>>> that does not seem to change anything.
>> > > > >> > > > > >>>>
>> > > > >> > > > > >>>> Any additional ideas?
>> > > > >> > > > > >>>>
>> > > > >> > > > > >>>> Regards,
>> > > > >> > > > > >>>> -Stefan
>> > > > >> > > > > >>>>
>> > > > >> > > > > >>>> On Sun, Jul 19, 2015 at 5:40 PM, Stefán Baxter <
>> > > > >> > > > > >>> stefan@activitystream.com <javascript:;>>
>> > > > >> > > > > >>>> wrote:
>> > > > >> > > > > >>>>
>> > > > >> > > > > >>>>> NO!
>> > > > >> > > > > >>>>>
>> > > > >> > > > > >>>>> Thank you , will do that right now :)
>> > > > >> > > > > >>>>>
>> > > > >> > > > > >>>>> On Sun, Jul 19, 2015 at 5:38 PM, Jim Bates <
>> > > > >> > jbates@maprtech.com <javascript:;>>
>> > > > >> > > > > >> wrote:
>> > > > >> > > > > >>>>>
>> > > > >> > > > > >>>>>> Did you include a file drill-module.conf in your
>> jar
>> > > > along
>> > > > >> > with
>> > > > >> > > > > >> source
>> > > > >> > > > > >>>>>> files?
>> > > > >> > > > > >>>>>> On Jul 19, 2015 12:20 PM, "Stefán Baxter" <
>> > > > >> > > > > stefan@activitystream.com <javascript:;>
>>
>> > > > >>
>> > > > >> > > > > >>>
>> > > > >> > > > > >>>>>> wrote:
>> > > > >> > > > > >>>>>>
>> > > > >> > > > > >>>>>>> Hi,
>> > > > >> > > > > >>>>>>>
>> > > > >> > > > > >>>>>>> I'm trying to deploy a UDF that I have written
>> > > according
>> > > > >> to
>> > > > >> > the
>> > > > >> > > > > >>>>>>> documentation.
>> > > > >> > > > > >>>>>>>
>> > > > >> > > > > >>>>>>> I have also:
>> > > > >> > > > > >>>>>>>
>> > > > >> > > > > >>>>>>> 1. Copied the jar file
>> > > > >> > > > > >>>>>>> to jars/3rdparty
>> > > > >> > > > > >>>>>>>
>> > > > >> > > > > >>>>>>> 2. Changed the config
>> "conf/drill-override.conf"
>> > to
>> > > > >> > include:
>> > > > >> > > > > >>>>>>> drill.logical.function.package +=
>> > > > >> > > > > >>
>> > > > >> > >
>> > > >
>> ["org.apache.drill.exec.expr.fn.impl","com.activitystream.drill.udfs"]
>> > > > >> > > > > >>>>>>>
>> > > > >> > > > > >>>>>>> 3. Rerun the Drillbit
>> > > > >> > > > > >>>>>>> bin/drillbit.sh start
>> > > > >> > > > > >>>>>>>
>> > > > >> > > > > >>>>>>> The log shows no sign of the the UDF being loaded
>> > but
>> > > > if I
>> > > > >> > use
>> > > > >> > > it
>> > > > >> > > > > >> in
>> > > > >> > > > > >>> a
>> > > > >> > > > > >>>>>> SQL
>> > > > >> > > > > >>>>>>> statement then this error is thrown:
>> > > > >> > > > > >>>>>>>
>> > > > >> > > > > >>>>>>> Error: PARSE ERROR: From line 1, column 184 to
>> line
>> > 1,
>> > > > >> column
>> > > > >> > > > 219:
>> > > > >> > > > > >> No
>> > > > >> > > > > >>>>>> match
>> > > > >> > > > > >>>>>>> found for function signature
>> asRoundTimeStamp(<ANY>,
>> > > > >> > > <CHARACTER>)
>> > > > >> > > > > >>>>>>>
>> > > > >> > > > > >>>>>>>
>> > > > >> > > > > >>>>>>> The relevant part of the UDF source:
>> > > > >> > > > > >>>>>>>
>> > > > >> > > > > >>>>>>> package com.activitystream.drill.udfs;
>> > > > >> > > > > >>>>>>>
>> > > > >> > > > > >>>>>>> public class ASUserDefinedFunctions {
>> > > > >> > > > > >>>>>>>
>> > > > >> > > > > >>>>>>> @FunctionTemplate(name = "asRoundTimeStamp",
>> > scope
>> > > =
>> > > > >> > > > > >>>>>>> FunctionTemplate.FunctionScope.SIMPLE, nulls =
>> > > > >> > > > > >>>>>>> FunctionTemplate.NullHandling.NULL_IF_NULL)
>> > > > >> > > > > >>>>>>>
>> > > > >> > > > > >>>>>>> public static class RoundTimeStamp implements
>> > > > >> > > DrillSimpleFunc
>> > > > >> > > > > >>>>>>>
>> > > > >> > > > > >>>>>>>
>> > > > >> > > > > >>>>>>> Can anyone please assist?
>> > > > >> > > > > >>>>>>>
>> > > > >> > > > > >>>>>>> Regards,
>> > > > >> > > > > >>>>>>> -Stefan
>> > > > >> > > > > >>
>> > > > >> > > > >
>> > > > >> > > >
>> > > > >> > >
>> > > > >> >
>> > > > >>
>> > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>
>
Re: Drill not picking up a UDF
Posted by Stefán Baxter <st...@activitystream.com>.
Hi,
After going through the log this is clear what is happening (once the Drill
picked up the UDF a bit earlier this morning).
I'm calling the VarCharHolder.toString() to get the text value for the
parameter and that is throwing this exception.
Two observations:
1. Calling a deprecated function, even though that its not optimal,
usually does not cause such drastic results.
2. As far as I can see there is no easy way to get the string value of a
property without carving out a piece of the buffer.
What am I missing here?
Regards,
-Stefan
On Mon, Jul 20, 2015 at 12:58 PM, Jacques Nadeau <ja...@apache.org> wrote:
> Can you enable verbose errors at the session level? It may reveal more
> about what is failing.
> On Jul 20, 2015 5:32 AM, "Stefán Baxter" <st...@activitystream.com>
> wrote:
>
> > Hi Jim,
> >
> > I have made those changes and I'm wondering if you can runs this be using
> > the two .jar files that the "mvn package" places in the target directory?
> >
> > I have tried to have Drill pick those up but the error now is:
> >
> > Error: SYSTEM ERROR: UnsupportedOperationException
> > Fragment 0:0
> > [Error Id: da589dd4-4cfd-4659-8b93-219074ab8c72 on localhost:31010]
> > (state=,code=0)
> >
> > It seems to indicate that it's picking up the functions but that they can
> > not be run.
> >
> > Regards,
> > - Stefán
> >
> > On Mon, Jul 20, 2015 at 1:25 AM, Jim Bates <jb...@maprtech.com> wrote:
> >
> > > I pulled out your udf class and threw it into my package. It worked for
> > me
> > > with a few modifications.
> > >
> > > You can not have imported classes or method references in your eval or
> > > setup methods as the code will get pulled out and executed somewhere
> else
> > > and it won't be able to find it. With that in mind, Period will need
> to
> > be
> > > org.joda.time.Period, DateTime will need to be org.joda.time.DateTime
> and
> > > roundTimeStamp will need to be
> > > com.activitystream.drill.udfs.ASUserDefinedFunctions.roundTimeStamp.
> > >
> > >
> > >
> > > On Sun, Jul 19, 2015 at 7:02 PM, Stefán Baxter <
> > stefan@activitystream.com>
> > > wrote:
> > >
> > > > Hi,
> > > >
> > > > The project can be found here:
> > > > https://github.com/acmeguy/asdrill
> > > >
> > > > Thank you,
> > > > -Stefán
> > > >
> > > > On Sun, Jul 19, 2015 at 11:57 PM, Stefán Baxter <
> > > stefan@activitystream.com
> > > > >
> > > > wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > I'm more than happy to share the little that is there (I will
> publish
> > > it
> > > > > on github and send link tomorrow).
> > > > >
> > > > > I ended up copying my UDF (singl-file.java) into the
> > > > simple-drill-function
> > > > > project where it got picked up.
> > > > >
> > > > > Then I discovered a whole new set of dependencies/limitations
> > > > >
> > > > > - The UDF are recompiled - any imports are invalid or at least
> > > > > overwritten
> > > > > - import org.joda.time.Period; (means that Period class is not
> > > > > resolved on runtime)
> > > > > - Error: SYSTEM ERROR: CompileException: Line 71, Column 26:
> > Cannot
> > > > > determine simple type name "Period"
> > > > >
> > > > > - Calling any "outside" functions, like I was calling a static
> > > > > function of the new class (same file), leads to an errors as
> they
> > > are
> > > > not
> > > > > not resolved
> > > > > - Error: SYSTEM ERROR: CompileException: Line 70, Column 35: A
> > > method
> > > > > named "roundTimeStamp" is not declared in any enclosing class
> nor
> > > any
> > > > > supertype, nor through a static import
> > > > >
> > > > > Perhaps this was mentioned in the documentation but this is, at the
> > > very
> > > > > least, not straight forward and super-inviting.
> > > > >
> > > > > Thank you for your assistance, we will keep trying :)
> > > > >
> > > > > Regards,
> > > > > -Stefan
> > > > >
> > > > >
> > > > > On Sun, Jul 19, 2015 at 11:34 PM, Tugdual Grall <tugdual@gmail.com
> >
> > > > wrote:
> > > > >
> > > > >> Hi Stefan,
> > > > >>
> > > > >> Do you think you can share your complete project ?
> > > > >>
> > > > >> This will help to debug it for you.
> > > > >>
> > > > >> T
> > > > >>
> > > > >> On Sunday, July 19, 2015, Stefán Baxter <
> stefan@activitystream.com>
> > > > >> wrote:
> > > > >>
> > > > >> > Hi Ted,
> > > > >> >
> > > > >> > I fetched this, built it and deployed it without problems.
> > > > >> > I can not see any real difference other than this deploys two
> .jar
> > > (I
> > > > >> tried
> > > > >> > that as well earlier).
> > > > >> >
> > > > >> > I'm still trying to figure out why Drill is not picking up my
> UDFs
> > > > >> >
> > > > >> > Regards,
> > > > >> > -Stefán
> > > > >> >
> > > > >> > On Sun, Jul 19, 2015 at 10:45 PM, Ted Dunning <
> > > ted.dunning@gmail.com
> > > > >> > <javascript:;>> wrote:
> > > > >> >
> > > > >> > > Stefan,
> > > > >> > >
> > > > >> > > Have you seen this github project:
> > > > >> > >
> > > > >> > > https://github.com/mapr-demos/simple-drill-functions
> > > > >> > >
> > > > >> > > ?
> > > > >> > >
> > > > >> > >
> > > > >> > > On Sun, Jul 19, 2015 at 2:14 PM, Stefán Baxter <
> > > > >> > stefan@activitystream.com <javascript:;>>
> > > > >> > > wrote:
> > > > >> > >
> > > > >> > > > Hi Jim,
> > > > >> > > >
> > > > >> > > > I'm still not able to make this work. Do you have a sample
> > .jar
> > > > file
> > > > >> > > with a
> > > > >> > > > small example that you are running?
> > > > >> > > >
> > > > >> > > > Regards,
> > > > >> > > > -Stefan
> > > > >> > > >
> > > > >> > > > On Sun, Jul 19, 2015 at 6:46 PM, Ted Dunning <
> > > > ted.dunning@gmail.com
> > > > >> > <javascript:;>>
> > > > >> > > > wrote:
> > > > >> > > >
> > > > >> > > > >
> > > > >> > > > > Sounds like a fine example, not because of sophistication
> > but
> > > > >> because
> > > > >> > > it
> > > > >> > > > > deals with dates.
> > > > >> > > > >
> > > > >> > > > > Check the drill logs. It is likely that drill is grumpy
> > about
> > > > >> > > something
> > > > >> > > > > in your udf or packaging.
> > > > >> > > > >
> > > > >> > > > > Also, feel free to snitch the pom from the simple examples
> > in
> > > > >> order
> > > > >> > to
> > > > >> > > > get
> > > > >> > > > > the pieces assembled and packaged correctly.
> > > > >> > > > >
> > > > >> > > > > Sent from my iPhone
> > > > >> > > > >
> > > > >> > > > > > On Jul 19, 2015, at 11:25, Stefán Baxter <
> > > > >> > stefan@activitystream.com <javascript:;>>
> > > > >> > > > > wrote:
> > > > >> > > > > >
> > > > >> > > > > > Hi Jim,
> > > > >> > > > > >
> > > > >> > > > > > My UDF is currently so simple that I'm not sure you need
> > it
> > > > (or
> > > > >> > want
> > > > >> > > > it).
> > > > >> > > > > >
> > > > >> > > > > > It basically just rounds a timestamp value with ISO 8601
> > > > periods
> > > > >> > > > > > asRoundTimestamp(timestampvalue,'PT10M').
> > > > >> > > > > >
> > > > >> > > > > > I would be more than happy to contribute to your project
> > > > rather
> > > > >> > than
> > > > >> > > > > > building our own :).
> > > > >> > > > > >
> > > > >> > > > > > Is the repo public?
> > > > >> > > > > >
> > > > >> > > > > > Regards,
> > > > >> > > > > > -Stefan
> > > > >> > > > > >
> > > > >> > > > > >
> > > > >> > > > > >
> > > > >> > > > > >> On Sun, Jul 19, 2015 at 6:18 PM, Jim Bates <
> > > > >> jbates@maprtech.com
> > > > >> > <javascript:;>>
> > > > >> > > > wrote:
> > > > >> > > > > >>
> > > > >> > > > > >> Maven will typically create a jar for class and a jar
> for
> > > > >> source
> > > > >> > > when
> > > > >> > > > > told
> > > > >> > > > > >> to do so. I just include the source files in the same
> jar
> > > as
> > > > >> the
> > > > >> > > class
> > > > >> > > > > >> files. There is a github example drill udf project we
> are
> > > > >> working
> > > > >> > on
> > > > >> > > > to
> > > > >> > > > > >> include several examples to simplify the learning
> curve.
> > If
> > > > >> your
> > > > >> > > > > >> interested... I'd love to have you add your udf.
> > > > >> > > > > >> On Jul 19, 2015 12:59 PM, "Stefán Baxter" <
> > > > >> > > stefan@activitystream.com <javascript:;>>
> > > > >> > > > > >> wrote:
> > > > >> > > > > >>
> > > > >> > > > > >>> Hi again,
> > > > >> > > > > >>>
> > > > >> > > > > >>> Going over the documentation once more I came across
> > this:
> > > > >> > > > > >>>
> > > > >> > > > > >>> - Add the sources and classes JAR files to Drill’s
> > > > >> classpath.
> > > > >> > > > > >>>
> > > > >> > > > > >>> I'm only including a standard .jar (with classes no
> > > sources)
> > > > >> and
> > > > >> > I
> > > > >> > > > > wonder
> > > > >> > > > > >>> why this dependency is listed.
> > > > >> > > > > >>>
> > > > >> > > > > >>> Anyways, that it the next step.
> > > > >> > > > > >>>
> > > > >> > > > > >>> A sample .jar that works with the drill-config
> > counterpart
> > > > >> would
> > > > >> > > be a
> > > > >> > > > > >>> welcome addition to the documentation.
> > > > >> > > > > >>>
> > > > >> > > > > >>> Regards,
> > > > >> > > > > >>> -Stefan
> > > > >> > > > > >>>
> > > > >> > > > > >>>
> > > > >> > > > > >>> On Sun, Jul 19, 2015 at 5:53 PM, Stefán Baxter <
> > > > >> > > > > >> stefan@activitystream.com <javascript:;>>
> > > > >> > > > > >>> wrote:
> > > > >> > > > > >>>
> > > > >> > > > > >>>> Hi Jim,
> > > > >> > > > > >>>>
> > > > >> > > > > >>>> Now I have added the file to the jar (both root and
> > > > resources
> > > > >> > > > folder)
> > > > >> > > > > >> but
> > > > >> > > > > >>>> that does not seem to change anything.
> > > > >> > > > > >>>>
> > > > >> > > > > >>>> Any additional ideas?
> > > > >> > > > > >>>>
> > > > >> > > > > >>>> Regards,
> > > > >> > > > > >>>> -Stefan
> > > > >> > > > > >>>>
> > > > >> > > > > >>>> On Sun, Jul 19, 2015 at 5:40 PM, Stefán Baxter <
> > > > >> > > > > >>> stefan@activitystream.com <javascript:;>>
> > > > >> > > > > >>>> wrote:
> > > > >> > > > > >>>>
> > > > >> > > > > >>>>> NO!
> > > > >> > > > > >>>>>
> > > > >> > > > > >>>>> Thank you , will do that right now :)
> > > > >> > > > > >>>>>
> > > > >> > > > > >>>>> On Sun, Jul 19, 2015 at 5:38 PM, Jim Bates <
> > > > >> > jbates@maprtech.com <javascript:;>>
> > > > >> > > > > >> wrote:
> > > > >> > > > > >>>>>
> > > > >> > > > > >>>>>> Did you include a file drill-module.conf in your
> jar
> > > > along
> > > > >> > with
> > > > >> > > > > >> source
> > > > >> > > > > >>>>>> files?
> > > > >> > > > > >>>>>> On Jul 19, 2015 12:20 PM, "Stefán Baxter" <
> > > > >> > > > > stefan@activitystream.com <javascript:;>
> > > > >>
> > > > >> > > > > >>>
> > > > >> > > > > >>>>>> wrote:
> > > > >> > > > > >>>>>>
> > > > >> > > > > >>>>>>> Hi,
> > > > >> > > > > >>>>>>>
> > > > >> > > > > >>>>>>> I'm trying to deploy a UDF that I have written
> > > according
> > > > >> to
> > > > >> > the
> > > > >> > > > > >>>>>>> documentation.
> > > > >> > > > > >>>>>>>
> > > > >> > > > > >>>>>>> I have also:
> > > > >> > > > > >>>>>>>
> > > > >> > > > > >>>>>>> 1. Copied the jar file
> > > > >> > > > > >>>>>>> to jars/3rdparty
> > > > >> > > > > >>>>>>>
> > > > >> > > > > >>>>>>> 2. Changed the config "conf/drill-override.conf"
> > to
> > > > >> > include:
> > > > >> > > > > >>>>>>> drill.logical.function.package +=
> > > > >> > > > > >>
> > > > >> > >
> > > >
> ["org.apache.drill.exec.expr.fn.impl","com.activitystream.drill.udfs"]
> > > > >> > > > > >>>>>>>
> > > > >> > > > > >>>>>>> 3. Rerun the Drillbit
> > > > >> > > > > >>>>>>> bin/drillbit.sh start
> > > > >> > > > > >>>>>>>
> > > > >> > > > > >>>>>>> The log shows no sign of the the UDF being loaded
> > but
> > > > if I
> > > > >> > use
> > > > >> > > it
> > > > >> > > > > >> in
> > > > >> > > > > >>> a
> > > > >> > > > > >>>>>> SQL
> > > > >> > > > > >>>>>>> statement then this error is thrown:
> > > > >> > > > > >>>>>>>
> > > > >> > > > > >>>>>>> Error: PARSE ERROR: From line 1, column 184 to
> line
> > 1,
> > > > >> column
> > > > >> > > > 219:
> > > > >> > > > > >> No
> > > > >> > > > > >>>>>> match
> > > > >> > > > > >>>>>>> found for function signature
> asRoundTimeStamp(<ANY>,
> > > > >> > > <CHARACTER>)
> > > > >> > > > > >>>>>>>
> > > > >> > > > > >>>>>>>
> > > > >> > > > > >>>>>>> The relevant part of the UDF source:
> > > > >> > > > > >>>>>>>
> > > > >> > > > > >>>>>>> package com.activitystream.drill.udfs;
> > > > >> > > > > >>>>>>>
> > > > >> > > > > >>>>>>> public class ASUserDefinedFunctions {
> > > > >> > > > > >>>>>>>
> > > > >> > > > > >>>>>>> @FunctionTemplate(name = "asRoundTimeStamp",
> > scope
> > > =
> > > > >> > > > > >>>>>>> FunctionTemplate.FunctionScope.SIMPLE, nulls =
> > > > >> > > > > >>>>>>> FunctionTemplate.NullHandling.NULL_IF_NULL)
> > > > >> > > > > >>>>>>>
> > > > >> > > > > >>>>>>> public static class RoundTimeStamp implements
> > > > >> > > DrillSimpleFunc
> > > > >> > > > > >>>>>>>
> > > > >> > > > > >>>>>>>
> > > > >> > > > > >>>>>>> Can anyone please assist?
> > > > >> > > > > >>>>>>>
> > > > >> > > > > >>>>>>> Regards,
> > > > >> > > > > >>>>>>> -Stefan
> > > > >> > > > > >>
> > > > >> > > > >
> > > > >> > > >
> > > > >> > >
> > > > >> >
> > > > >>
> > > > >
> > > > >
> > > >
> > >
> >
>
Re: Drill not picking up a UDF
Posted by Jacques Nadeau <ja...@apache.org>.
Can you enable verbose errors at the session level? It may reveal more
about what is failing.
On Jul 20, 2015 5:32 AM, "Stefán Baxter" <st...@activitystream.com> wrote:
> Hi Jim,
>
> I have made those changes and I'm wondering if you can runs this be using
> the two .jar files that the "mvn package" places in the target directory?
>
> I have tried to have Drill pick those up but the error now is:
>
> Error: SYSTEM ERROR: UnsupportedOperationException
> Fragment 0:0
> [Error Id: da589dd4-4cfd-4659-8b93-219074ab8c72 on localhost:31010]
> (state=,code=0)
>
> It seems to indicate that it's picking up the functions but that they can
> not be run.
>
> Regards,
> - Stefán
>
> On Mon, Jul 20, 2015 at 1:25 AM, Jim Bates <jb...@maprtech.com> wrote:
>
> > I pulled out your udf class and threw it into my package. It worked for
> me
> > with a few modifications.
> >
> > You can not have imported classes or method references in your eval or
> > setup methods as the code will get pulled out and executed somewhere else
> > and it won't be able to find it. With that in mind, Period will need to
> be
> > org.joda.time.Period, DateTime will need to be org.joda.time.DateTime and
> > roundTimeStamp will need to be
> > com.activitystream.drill.udfs.ASUserDefinedFunctions.roundTimeStamp.
> >
> >
> >
> > On Sun, Jul 19, 2015 at 7:02 PM, Stefán Baxter <
> stefan@activitystream.com>
> > wrote:
> >
> > > Hi,
> > >
> > > The project can be found here:
> > > https://github.com/acmeguy/asdrill
> > >
> > > Thank you,
> > > -Stefán
> > >
> > > On Sun, Jul 19, 2015 at 11:57 PM, Stefán Baxter <
> > stefan@activitystream.com
> > > >
> > > wrote:
> > >
> > > > Hi,
> > > >
> > > > I'm more than happy to share the little that is there (I will publish
> > it
> > > > on github and send link tomorrow).
> > > >
> > > > I ended up copying my UDF (singl-file.java) into the
> > > simple-drill-function
> > > > project where it got picked up.
> > > >
> > > > Then I discovered a whole new set of dependencies/limitations
> > > >
> > > > - The UDF are recompiled - any imports are invalid or at least
> > > > overwritten
> > > > - import org.joda.time.Period; (means that Period class is not
> > > > resolved on runtime)
> > > > - Error: SYSTEM ERROR: CompileException: Line 71, Column 26:
> Cannot
> > > > determine simple type name "Period"
> > > >
> > > > - Calling any "outside" functions, like I was calling a static
> > > > function of the new class (same file), leads to an errors as they
> > are
> > > not
> > > > not resolved
> > > > - Error: SYSTEM ERROR: CompileException: Line 70, Column 35: A
> > method
> > > > named "roundTimeStamp" is not declared in any enclosing class nor
> > any
> > > > supertype, nor through a static import
> > > >
> > > > Perhaps this was mentioned in the documentation but this is, at the
> > very
> > > > least, not straight forward and super-inviting.
> > > >
> > > > Thank you for your assistance, we will keep trying :)
> > > >
> > > > Regards,
> > > > -Stefan
> > > >
> > > >
> > > > On Sun, Jul 19, 2015 at 11:34 PM, Tugdual Grall <tu...@gmail.com>
> > > wrote:
> > > >
> > > >> Hi Stefan,
> > > >>
> > > >> Do you think you can share your complete project ?
> > > >>
> > > >> This will help to debug it for you.
> > > >>
> > > >> T
> > > >>
> > > >> On Sunday, July 19, 2015, Stefán Baxter <st...@activitystream.com>
> > > >> wrote:
> > > >>
> > > >> > Hi Ted,
> > > >> >
> > > >> > I fetched this, built it and deployed it without problems.
> > > >> > I can not see any real difference other than this deploys two .jar
> > (I
> > > >> tried
> > > >> > that as well earlier).
> > > >> >
> > > >> > I'm still trying to figure out why Drill is not picking up my UDFs
> > > >> >
> > > >> > Regards,
> > > >> > -Stefán
> > > >> >
> > > >> > On Sun, Jul 19, 2015 at 10:45 PM, Ted Dunning <
> > ted.dunning@gmail.com
> > > >> > <javascript:;>> wrote:
> > > >> >
> > > >> > > Stefan,
> > > >> > >
> > > >> > > Have you seen this github project:
> > > >> > >
> > > >> > > https://github.com/mapr-demos/simple-drill-functions
> > > >> > >
> > > >> > > ?
> > > >> > >
> > > >> > >
> > > >> > > On Sun, Jul 19, 2015 at 2:14 PM, Stefán Baxter <
> > > >> > stefan@activitystream.com <javascript:;>>
> > > >> > > wrote:
> > > >> > >
> > > >> > > > Hi Jim,
> > > >> > > >
> > > >> > > > I'm still not able to make this work. Do you have a sample
> .jar
> > > file
> > > >> > > with a
> > > >> > > > small example that you are running?
> > > >> > > >
> > > >> > > > Regards,
> > > >> > > > -Stefan
> > > >> > > >
> > > >> > > > On Sun, Jul 19, 2015 at 6:46 PM, Ted Dunning <
> > > ted.dunning@gmail.com
> > > >> > <javascript:;>>
> > > >> > > > wrote:
> > > >> > > >
> > > >> > > > >
> > > >> > > > > Sounds like a fine example, not because of sophistication
> but
> > > >> because
> > > >> > > it
> > > >> > > > > deals with dates.
> > > >> > > > >
> > > >> > > > > Check the drill logs. It is likely that drill is grumpy
> about
> > > >> > > something
> > > >> > > > > in your udf or packaging.
> > > >> > > > >
> > > >> > > > > Also, feel free to snitch the pom from the simple examples
> in
> > > >> order
> > > >> > to
> > > >> > > > get
> > > >> > > > > the pieces assembled and packaged correctly.
> > > >> > > > >
> > > >> > > > > Sent from my iPhone
> > > >> > > > >
> > > >> > > > > > On Jul 19, 2015, at 11:25, Stefán Baxter <
> > > >> > stefan@activitystream.com <javascript:;>>
> > > >> > > > > wrote:
> > > >> > > > > >
> > > >> > > > > > Hi Jim,
> > > >> > > > > >
> > > >> > > > > > My UDF is currently so simple that I'm not sure you need
> it
> > > (or
> > > >> > want
> > > >> > > > it).
> > > >> > > > > >
> > > >> > > > > > It basically just rounds a timestamp value with ISO 8601
> > > periods
> > > >> > > > > > asRoundTimestamp(timestampvalue,'PT10M').
> > > >> > > > > >
> > > >> > > > > > I would be more than happy to contribute to your project
> > > rather
> > > >> > than
> > > >> > > > > > building our own :).
> > > >> > > > > >
> > > >> > > > > > Is the repo public?
> > > >> > > > > >
> > > >> > > > > > Regards,
> > > >> > > > > > -Stefan
> > > >> > > > > >
> > > >> > > > > >
> > > >> > > > > >
> > > >> > > > > >> On Sun, Jul 19, 2015 at 6:18 PM, Jim Bates <
> > > >> jbates@maprtech.com
> > > >> > <javascript:;>>
> > > >> > > > wrote:
> > > >> > > > > >>
> > > >> > > > > >> Maven will typically create a jar for class and a jar for
> > > >> source
> > > >> > > when
> > > >> > > > > told
> > > >> > > > > >> to do so. I just include the source files in the same jar
> > as
> > > >> the
> > > >> > > class
> > > >> > > > > >> files. There is a github example drill udf project we are
> > > >> working
> > > >> > on
> > > >> > > > to
> > > >> > > > > >> include several examples to simplify the learning curve.
> If
> > > >> your
> > > >> > > > > >> interested... I'd love to have you add your udf.
> > > >> > > > > >> On Jul 19, 2015 12:59 PM, "Stefán Baxter" <
> > > >> > > stefan@activitystream.com <javascript:;>>
> > > >> > > > > >> wrote:
> > > >> > > > > >>
> > > >> > > > > >>> Hi again,
> > > >> > > > > >>>
> > > >> > > > > >>> Going over the documentation once more I came across
> this:
> > > >> > > > > >>>
> > > >> > > > > >>> - Add the sources and classes JAR files to Drill’s
> > > >> classpath.
> > > >> > > > > >>>
> > > >> > > > > >>> I'm only including a standard .jar (with classes no
> > sources)
> > > >> and
> > > >> > I
> > > >> > > > > wonder
> > > >> > > > > >>> why this dependency is listed.
> > > >> > > > > >>>
> > > >> > > > > >>> Anyways, that it the next step.
> > > >> > > > > >>>
> > > >> > > > > >>> A sample .jar that works with the drill-config
> counterpart
> > > >> would
> > > >> > > be a
> > > >> > > > > >>> welcome addition to the documentation.
> > > >> > > > > >>>
> > > >> > > > > >>> Regards,
> > > >> > > > > >>> -Stefan
> > > >> > > > > >>>
> > > >> > > > > >>>
> > > >> > > > > >>> On Sun, Jul 19, 2015 at 5:53 PM, Stefán Baxter <
> > > >> > > > > >> stefan@activitystream.com <javascript:;>>
> > > >> > > > > >>> wrote:
> > > >> > > > > >>>
> > > >> > > > > >>>> Hi Jim,
> > > >> > > > > >>>>
> > > >> > > > > >>>> Now I have added the file to the jar (both root and
> > > resources
> > > >> > > > folder)
> > > >> > > > > >> but
> > > >> > > > > >>>> that does not seem to change anything.
> > > >> > > > > >>>>
> > > >> > > > > >>>> Any additional ideas?
> > > >> > > > > >>>>
> > > >> > > > > >>>> Regards,
> > > >> > > > > >>>> -Stefan
> > > >> > > > > >>>>
> > > >> > > > > >>>> On Sun, Jul 19, 2015 at 5:40 PM, Stefán Baxter <
> > > >> > > > > >>> stefan@activitystream.com <javascript:;>>
> > > >> > > > > >>>> wrote:
> > > >> > > > > >>>>
> > > >> > > > > >>>>> NO!
> > > >> > > > > >>>>>
> > > >> > > > > >>>>> Thank you , will do that right now :)
> > > >> > > > > >>>>>
> > > >> > > > > >>>>> On Sun, Jul 19, 2015 at 5:38 PM, Jim Bates <
> > > >> > jbates@maprtech.com <javascript:;>>
> > > >> > > > > >> wrote:
> > > >> > > > > >>>>>
> > > >> > > > > >>>>>> Did you include a file drill-module.conf in your jar
> > > along
> > > >> > with
> > > >> > > > > >> source
> > > >> > > > > >>>>>> files?
> > > >> > > > > >>>>>> On Jul 19, 2015 12:20 PM, "Stefán Baxter" <
> > > >> > > > > stefan@activitystream.com <javascript:;>
> > > >>
> > > >> > > > > >>>
> > > >> > > > > >>>>>> wrote:
> > > >> > > > > >>>>>>
> > > >> > > > > >>>>>>> Hi,
> > > >> > > > > >>>>>>>
> > > >> > > > > >>>>>>> I'm trying to deploy a UDF that I have written
> > according
> > > >> to
> > > >> > the
> > > >> > > > > >>>>>>> documentation.
> > > >> > > > > >>>>>>>
> > > >> > > > > >>>>>>> I have also:
> > > >> > > > > >>>>>>>
> > > >> > > > > >>>>>>> 1. Copied the jar file
> > > >> > > > > >>>>>>> to jars/3rdparty
> > > >> > > > > >>>>>>>
> > > >> > > > > >>>>>>> 2. Changed the config "conf/drill-override.conf"
> to
> > > >> > include:
> > > >> > > > > >>>>>>> drill.logical.function.package +=
> > > >> > > > > >>
> > > >> > >
> > > ["org.apache.drill.exec.expr.fn.impl","com.activitystream.drill.udfs"]
> > > >> > > > > >>>>>>>
> > > >> > > > > >>>>>>> 3. Rerun the Drillbit
> > > >> > > > > >>>>>>> bin/drillbit.sh start
> > > >> > > > > >>>>>>>
> > > >> > > > > >>>>>>> The log shows no sign of the the UDF being loaded
> but
> > > if I
> > > >> > use
> > > >> > > it
> > > >> > > > > >> in
> > > >> > > > > >>> a
> > > >> > > > > >>>>>> SQL
> > > >> > > > > >>>>>>> statement then this error is thrown:
> > > >> > > > > >>>>>>>
> > > >> > > > > >>>>>>> Error: PARSE ERROR: From line 1, column 184 to line
> 1,
> > > >> column
> > > >> > > > 219:
> > > >> > > > > >> No
> > > >> > > > > >>>>>> match
> > > >> > > > > >>>>>>> found for function signature asRoundTimeStamp(<ANY>,
> > > >> > > <CHARACTER>)
> > > >> > > > > >>>>>>>
> > > >> > > > > >>>>>>>
> > > >> > > > > >>>>>>> The relevant part of the UDF source:
> > > >> > > > > >>>>>>>
> > > >> > > > > >>>>>>> package com.activitystream.drill.udfs;
> > > >> > > > > >>>>>>>
> > > >> > > > > >>>>>>> public class ASUserDefinedFunctions {
> > > >> > > > > >>>>>>>
> > > >> > > > > >>>>>>> @FunctionTemplate(name = "asRoundTimeStamp",
> scope
> > =
> > > >> > > > > >>>>>>> FunctionTemplate.FunctionScope.SIMPLE, nulls =
> > > >> > > > > >>>>>>> FunctionTemplate.NullHandling.NULL_IF_NULL)
> > > >> > > > > >>>>>>>
> > > >> > > > > >>>>>>> public static class RoundTimeStamp implements
> > > >> > > DrillSimpleFunc
> > > >> > > > > >>>>>>>
> > > >> > > > > >>>>>>>
> > > >> > > > > >>>>>>> Can anyone please assist?
> > > >> > > > > >>>>>>>
> > > >> > > > > >>>>>>> Regards,
> > > >> > > > > >>>>>>> -Stefan
> > > >> > > > > >>
> > > >> > > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > > >
> > > >
> > >
> >
>
Re: Drill not picking up a UDF
Posted by Stefán Baxter <st...@activitystream.com>.
Hi Jim,
I have made those changes and I'm wondering if you can runs this be using
the two .jar files that the "mvn package" places in the target directory?
I have tried to have Drill pick those up but the error now is:
Error: SYSTEM ERROR: UnsupportedOperationException
Fragment 0:0
[Error Id: da589dd4-4cfd-4659-8b93-219074ab8c72 on localhost:31010]
(state=,code=0)
It seems to indicate that it's picking up the functions but that they can
not be run.
Regards,
- Stefán
On Mon, Jul 20, 2015 at 1:25 AM, Jim Bates <jb...@maprtech.com> wrote:
> I pulled out your udf class and threw it into my package. It worked for me
> with a few modifications.
>
> You can not have imported classes or method references in your eval or
> setup methods as the code will get pulled out and executed somewhere else
> and it won't be able to find it. With that in mind, Period will need to be
> org.joda.time.Period, DateTime will need to be org.joda.time.DateTime and
> roundTimeStamp will need to be
> com.activitystream.drill.udfs.ASUserDefinedFunctions.roundTimeStamp.
>
>
>
> On Sun, Jul 19, 2015 at 7:02 PM, Stefán Baxter <st...@activitystream.com>
> wrote:
>
> > Hi,
> >
> > The project can be found here:
> > https://github.com/acmeguy/asdrill
> >
> > Thank you,
> > -Stefán
> >
> > On Sun, Jul 19, 2015 at 11:57 PM, Stefán Baxter <
> stefan@activitystream.com
> > >
> > wrote:
> >
> > > Hi,
> > >
> > > I'm more than happy to share the little that is there (I will publish
> it
> > > on github and send link tomorrow).
> > >
> > > I ended up copying my UDF (singl-file.java) into the
> > simple-drill-function
> > > project where it got picked up.
> > >
> > > Then I discovered a whole new set of dependencies/limitations
> > >
> > > - The UDF are recompiled - any imports are invalid or at least
> > > overwritten
> > > - import org.joda.time.Period; (means that Period class is not
> > > resolved on runtime)
> > > - Error: SYSTEM ERROR: CompileException: Line 71, Column 26: Cannot
> > > determine simple type name "Period"
> > >
> > > - Calling any "outside" functions, like I was calling a static
> > > function of the new class (same file), leads to an errors as they
> are
> > not
> > > not resolved
> > > - Error: SYSTEM ERROR: CompileException: Line 70, Column 35: A
> method
> > > named "roundTimeStamp" is not declared in any enclosing class nor
> any
> > > supertype, nor through a static import
> > >
> > > Perhaps this was mentioned in the documentation but this is, at the
> very
> > > least, not straight forward and super-inviting.
> > >
> > > Thank you for your assistance, we will keep trying :)
> > >
> > > Regards,
> > > -Stefan
> > >
> > >
> > > On Sun, Jul 19, 2015 at 11:34 PM, Tugdual Grall <tu...@gmail.com>
> > wrote:
> > >
> > >> Hi Stefan,
> > >>
> > >> Do you think you can share your complete project ?
> > >>
> > >> This will help to debug it for you.
> > >>
> > >> T
> > >>
> > >> On Sunday, July 19, 2015, Stefán Baxter <st...@activitystream.com>
> > >> wrote:
> > >>
> > >> > Hi Ted,
> > >> >
> > >> > I fetched this, built it and deployed it without problems.
> > >> > I can not see any real difference other than this deploys two .jar
> (I
> > >> tried
> > >> > that as well earlier).
> > >> >
> > >> > I'm still trying to figure out why Drill is not picking up my UDFs
> > >> >
> > >> > Regards,
> > >> > -Stefán
> > >> >
> > >> > On Sun, Jul 19, 2015 at 10:45 PM, Ted Dunning <
> ted.dunning@gmail.com
> > >> > <javascript:;>> wrote:
> > >> >
> > >> > > Stefan,
> > >> > >
> > >> > > Have you seen this github project:
> > >> > >
> > >> > > https://github.com/mapr-demos/simple-drill-functions
> > >> > >
> > >> > > ?
> > >> > >
> > >> > >
> > >> > > On Sun, Jul 19, 2015 at 2:14 PM, Stefán Baxter <
> > >> > stefan@activitystream.com <javascript:;>>
> > >> > > wrote:
> > >> > >
> > >> > > > Hi Jim,
> > >> > > >
> > >> > > > I'm still not able to make this work. Do you have a sample .jar
> > file
> > >> > > with a
> > >> > > > small example that you are running?
> > >> > > >
> > >> > > > Regards,
> > >> > > > -Stefan
> > >> > > >
> > >> > > > On Sun, Jul 19, 2015 at 6:46 PM, Ted Dunning <
> > ted.dunning@gmail.com
> > >> > <javascript:;>>
> > >> > > > wrote:
> > >> > > >
> > >> > > > >
> > >> > > > > Sounds like a fine example, not because of sophistication but
> > >> because
> > >> > > it
> > >> > > > > deals with dates.
> > >> > > > >
> > >> > > > > Check the drill logs. It is likely that drill is grumpy about
> > >> > > something
> > >> > > > > in your udf or packaging.
> > >> > > > >
> > >> > > > > Also, feel free to snitch the pom from the simple examples in
> > >> order
> > >> > to
> > >> > > > get
> > >> > > > > the pieces assembled and packaged correctly.
> > >> > > > >
> > >> > > > > Sent from my iPhone
> > >> > > > >
> > >> > > > > > On Jul 19, 2015, at 11:25, Stefán Baxter <
> > >> > stefan@activitystream.com <javascript:;>>
> > >> > > > > wrote:
> > >> > > > > >
> > >> > > > > > Hi Jim,
> > >> > > > > >
> > >> > > > > > My UDF is currently so simple that I'm not sure you need it
> > (or
> > >> > want
> > >> > > > it).
> > >> > > > > >
> > >> > > > > > It basically just rounds a timestamp value with ISO 8601
> > periods
> > >> > > > > > asRoundTimestamp(timestampvalue,'PT10M').
> > >> > > > > >
> > >> > > > > > I would be more than happy to contribute to your project
> > rather
> > >> > than
> > >> > > > > > building our own :).
> > >> > > > > >
> > >> > > > > > Is the repo public?
> > >> > > > > >
> > >> > > > > > Regards,
> > >> > > > > > -Stefan
> > >> > > > > >
> > >> > > > > >
> > >> > > > > >
> > >> > > > > >> On Sun, Jul 19, 2015 at 6:18 PM, Jim Bates <
> > >> jbates@maprtech.com
> > >> > <javascript:;>>
> > >> > > > wrote:
> > >> > > > > >>
> > >> > > > > >> Maven will typically create a jar for class and a jar for
> > >> source
> > >> > > when
> > >> > > > > told
> > >> > > > > >> to do so. I just include the source files in the same jar
> as
> > >> the
> > >> > > class
> > >> > > > > >> files. There is a github example drill udf project we are
> > >> working
> > >> > on
> > >> > > > to
> > >> > > > > >> include several examples to simplify the learning curve. If
> > >> your
> > >> > > > > >> interested... I'd love to have you add your udf.
> > >> > > > > >> On Jul 19, 2015 12:59 PM, "Stefán Baxter" <
> > >> > > stefan@activitystream.com <javascript:;>>
> > >> > > > > >> wrote:
> > >> > > > > >>
> > >> > > > > >>> Hi again,
> > >> > > > > >>>
> > >> > > > > >>> Going over the documentation once more I came across this:
> > >> > > > > >>>
> > >> > > > > >>> - Add the sources and classes JAR files to Drill’s
> > >> classpath.
> > >> > > > > >>>
> > >> > > > > >>> I'm only including a standard .jar (with classes no
> sources)
> > >> and
> > >> > I
> > >> > > > > wonder
> > >> > > > > >>> why this dependency is listed.
> > >> > > > > >>>
> > >> > > > > >>> Anyways, that it the next step.
> > >> > > > > >>>
> > >> > > > > >>> A sample .jar that works with the drill-config counterpart
> > >> would
> > >> > > be a
> > >> > > > > >>> welcome addition to the documentation.
> > >> > > > > >>>
> > >> > > > > >>> Regards,
> > >> > > > > >>> -Stefan
> > >> > > > > >>>
> > >> > > > > >>>
> > >> > > > > >>> On Sun, Jul 19, 2015 at 5:53 PM, Stefán Baxter <
> > >> > > > > >> stefan@activitystream.com <javascript:;>>
> > >> > > > > >>> wrote:
> > >> > > > > >>>
> > >> > > > > >>>> Hi Jim,
> > >> > > > > >>>>
> > >> > > > > >>>> Now I have added the file to the jar (both root and
> > resources
> > >> > > > folder)
> > >> > > > > >> but
> > >> > > > > >>>> that does not seem to change anything.
> > >> > > > > >>>>
> > >> > > > > >>>> Any additional ideas?
> > >> > > > > >>>>
> > >> > > > > >>>> Regards,
> > >> > > > > >>>> -Stefan
> > >> > > > > >>>>
> > >> > > > > >>>> On Sun, Jul 19, 2015 at 5:40 PM, Stefán Baxter <
> > >> > > > > >>> stefan@activitystream.com <javascript:;>>
> > >> > > > > >>>> wrote:
> > >> > > > > >>>>
> > >> > > > > >>>>> NO!
> > >> > > > > >>>>>
> > >> > > > > >>>>> Thank you , will do that right now :)
> > >> > > > > >>>>>
> > >> > > > > >>>>> On Sun, Jul 19, 2015 at 5:38 PM, Jim Bates <
> > >> > jbates@maprtech.com <javascript:;>>
> > >> > > > > >> wrote:
> > >> > > > > >>>>>
> > >> > > > > >>>>>> Did you include a file drill-module.conf in your jar
> > along
> > >> > with
> > >> > > > > >> source
> > >> > > > > >>>>>> files?
> > >> > > > > >>>>>> On Jul 19, 2015 12:20 PM, "Stefán Baxter" <
> > >> > > > > stefan@activitystream.com <javascript:;>
> > >>
> > >> > > > > >>>
> > >> > > > > >>>>>> wrote:
> > >> > > > > >>>>>>
> > >> > > > > >>>>>>> Hi,
> > >> > > > > >>>>>>>
> > >> > > > > >>>>>>> I'm trying to deploy a UDF that I have written
> according
> > >> to
> > >> > the
> > >> > > > > >>>>>>> documentation.
> > >> > > > > >>>>>>>
> > >> > > > > >>>>>>> I have also:
> > >> > > > > >>>>>>>
> > >> > > > > >>>>>>> 1. Copied the jar file
> > >> > > > > >>>>>>> to jars/3rdparty
> > >> > > > > >>>>>>>
> > >> > > > > >>>>>>> 2. Changed the config "conf/drill-override.conf" to
> > >> > include:
> > >> > > > > >>>>>>> drill.logical.function.package +=
> > >> > > > > >>
> > >> > >
> > ["org.apache.drill.exec.expr.fn.impl","com.activitystream.drill.udfs"]
> > >> > > > > >>>>>>>
> > >> > > > > >>>>>>> 3. Rerun the Drillbit
> > >> > > > > >>>>>>> bin/drillbit.sh start
> > >> > > > > >>>>>>>
> > >> > > > > >>>>>>> The log shows no sign of the the UDF being loaded but
> > if I
> > >> > use
> > >> > > it
> > >> > > > > >> in
> > >> > > > > >>> a
> > >> > > > > >>>>>> SQL
> > >> > > > > >>>>>>> statement then this error is thrown:
> > >> > > > > >>>>>>>
> > >> > > > > >>>>>>> Error: PARSE ERROR: From line 1, column 184 to line 1,
> > >> column
> > >> > > > 219:
> > >> > > > > >> No
> > >> > > > > >>>>>> match
> > >> > > > > >>>>>>> found for function signature asRoundTimeStamp(<ANY>,
> > >> > > <CHARACTER>)
> > >> > > > > >>>>>>>
> > >> > > > > >>>>>>>
> > >> > > > > >>>>>>> The relevant part of the UDF source:
> > >> > > > > >>>>>>>
> > >> > > > > >>>>>>> package com.activitystream.drill.udfs;
> > >> > > > > >>>>>>>
> > >> > > > > >>>>>>> public class ASUserDefinedFunctions {
> > >> > > > > >>>>>>>
> > >> > > > > >>>>>>> @FunctionTemplate(name = "asRoundTimeStamp", scope
> =
> > >> > > > > >>>>>>> FunctionTemplate.FunctionScope.SIMPLE, nulls =
> > >> > > > > >>>>>>> FunctionTemplate.NullHandling.NULL_IF_NULL)
> > >> > > > > >>>>>>>
> > >> > > > > >>>>>>> public static class RoundTimeStamp implements
> > >> > > DrillSimpleFunc
> > >> > > > > >>>>>>>
> > >> > > > > >>>>>>>
> > >> > > > > >>>>>>> Can anyone please assist?
> > >> > > > > >>>>>>>
> > >> > > > > >>>>>>> Regards,
> > >> > > > > >>>>>>> -Stefan
> > >> > > > > >>
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> > >
> > >
> >
>
Re: Drill not picking up a UDF
Posted by Jim Bates <jb...@maprtech.com>.
I pulled out your udf class and threw it into my package. It worked for me
with a few modifications.
You can not have imported classes or method references in your eval or
setup methods as the code will get pulled out and executed somewhere else
and it won't be able to find it. With that in mind, Period will need to be
org.joda.time.Period, DateTime will need to be org.joda.time.DateTime and
roundTimeStamp will need to be
com.activitystream.drill.udfs.ASUserDefinedFunctions.roundTimeStamp.
On Sun, Jul 19, 2015 at 7:02 PM, Stefán Baxter <st...@activitystream.com>
wrote:
> Hi,
>
> The project can be found here:
> https://github.com/acmeguy/asdrill
>
> Thank you,
> -Stefán
>
> On Sun, Jul 19, 2015 at 11:57 PM, Stefán Baxter <stefan@activitystream.com
> >
> wrote:
>
> > Hi,
> >
> > I'm more than happy to share the little that is there (I will publish it
> > on github and send link tomorrow).
> >
> > I ended up copying my UDF (singl-file.java) into the
> simple-drill-function
> > project where it got picked up.
> >
> > Then I discovered a whole new set of dependencies/limitations
> >
> > - The UDF are recompiled - any imports are invalid or at least
> > overwritten
> > - import org.joda.time.Period; (means that Period class is not
> > resolved on runtime)
> > - Error: SYSTEM ERROR: CompileException: Line 71, Column 26: Cannot
> > determine simple type name "Period"
> >
> > - Calling any "outside" functions, like I was calling a static
> > function of the new class (same file), leads to an errors as they are
> not
> > not resolved
> > - Error: SYSTEM ERROR: CompileException: Line 70, Column 35: A method
> > named "roundTimeStamp" is not declared in any enclosing class nor any
> > supertype, nor through a static import
> >
> > Perhaps this was mentioned in the documentation but this is, at the very
> > least, not straight forward and super-inviting.
> >
> > Thank you for your assistance, we will keep trying :)
> >
> > Regards,
> > -Stefan
> >
> >
> > On Sun, Jul 19, 2015 at 11:34 PM, Tugdual Grall <tu...@gmail.com>
> wrote:
> >
> >> Hi Stefan,
> >>
> >> Do you think you can share your complete project ?
> >>
> >> This will help to debug it for you.
> >>
> >> T
> >>
> >> On Sunday, July 19, 2015, Stefán Baxter <st...@activitystream.com>
> >> wrote:
> >>
> >> > Hi Ted,
> >> >
> >> > I fetched this, built it and deployed it without problems.
> >> > I can not see any real difference other than this deploys two .jar (I
> >> tried
> >> > that as well earlier).
> >> >
> >> > I'm still trying to figure out why Drill is not picking up my UDFs
> >> >
> >> > Regards,
> >> > -Stefán
> >> >
> >> > On Sun, Jul 19, 2015 at 10:45 PM, Ted Dunning <ted.dunning@gmail.com
> >> > <javascript:;>> wrote:
> >> >
> >> > > Stefan,
> >> > >
> >> > > Have you seen this github project:
> >> > >
> >> > > https://github.com/mapr-demos/simple-drill-functions
> >> > >
> >> > > ?
> >> > >
> >> > >
> >> > > On Sun, Jul 19, 2015 at 2:14 PM, Stefán Baxter <
> >> > stefan@activitystream.com <javascript:;>>
> >> > > wrote:
> >> > >
> >> > > > Hi Jim,
> >> > > >
> >> > > > I'm still not able to make this work. Do you have a sample .jar
> file
> >> > > with a
> >> > > > small example that you are running?
> >> > > >
> >> > > > Regards,
> >> > > > -Stefan
> >> > > >
> >> > > > On Sun, Jul 19, 2015 at 6:46 PM, Ted Dunning <
> ted.dunning@gmail.com
> >> > <javascript:;>>
> >> > > > wrote:
> >> > > >
> >> > > > >
> >> > > > > Sounds like a fine example, not because of sophistication but
> >> because
> >> > > it
> >> > > > > deals with dates.
> >> > > > >
> >> > > > > Check the drill logs. It is likely that drill is grumpy about
> >> > > something
> >> > > > > in your udf or packaging.
> >> > > > >
> >> > > > > Also, feel free to snitch the pom from the simple examples in
> >> order
> >> > to
> >> > > > get
> >> > > > > the pieces assembled and packaged correctly.
> >> > > > >
> >> > > > > Sent from my iPhone
> >> > > > >
> >> > > > > > On Jul 19, 2015, at 11:25, Stefán Baxter <
> >> > stefan@activitystream.com <javascript:;>>
> >> > > > > wrote:
> >> > > > > >
> >> > > > > > Hi Jim,
> >> > > > > >
> >> > > > > > My UDF is currently so simple that I'm not sure you need it
> (or
> >> > want
> >> > > > it).
> >> > > > > >
> >> > > > > > It basically just rounds a timestamp value with ISO 8601
> periods
> >> > > > > > asRoundTimestamp(timestampvalue,'PT10M').
> >> > > > > >
> >> > > > > > I would be more than happy to contribute to your project
> rather
> >> > than
> >> > > > > > building our own :).
> >> > > > > >
> >> > > > > > Is the repo public?
> >> > > > > >
> >> > > > > > Regards,
> >> > > > > > -Stefan
> >> > > > > >
> >> > > > > >
> >> > > > > >
> >> > > > > >> On Sun, Jul 19, 2015 at 6:18 PM, Jim Bates <
> >> jbates@maprtech.com
> >> > <javascript:;>>
> >> > > > wrote:
> >> > > > > >>
> >> > > > > >> Maven will typically create a jar for class and a jar for
> >> source
> >> > > when
> >> > > > > told
> >> > > > > >> to do so. I just include the source files in the same jar as
> >> the
> >> > > class
> >> > > > > >> files. There is a github example drill udf project we are
> >> working
> >> > on
> >> > > > to
> >> > > > > >> include several examples to simplify the learning curve. If
> >> your
> >> > > > > >> interested... I'd love to have you add your udf.
> >> > > > > >> On Jul 19, 2015 12:59 PM, "Stefán Baxter" <
> >> > > stefan@activitystream.com <javascript:;>>
> >> > > > > >> wrote:
> >> > > > > >>
> >> > > > > >>> Hi again,
> >> > > > > >>>
> >> > > > > >>> Going over the documentation once more I came across this:
> >> > > > > >>>
> >> > > > > >>> - Add the sources and classes JAR files to Drill’s
> >> classpath.
> >> > > > > >>>
> >> > > > > >>> I'm only including a standard .jar (with classes no sources)
> >> and
> >> > I
> >> > > > > wonder
> >> > > > > >>> why this dependency is listed.
> >> > > > > >>>
> >> > > > > >>> Anyways, that it the next step.
> >> > > > > >>>
> >> > > > > >>> A sample .jar that works with the drill-config counterpart
> >> would
> >> > > be a
> >> > > > > >>> welcome addition to the documentation.
> >> > > > > >>>
> >> > > > > >>> Regards,
> >> > > > > >>> -Stefan
> >> > > > > >>>
> >> > > > > >>>
> >> > > > > >>> On Sun, Jul 19, 2015 at 5:53 PM, Stefán Baxter <
> >> > > > > >> stefan@activitystream.com <javascript:;>>
> >> > > > > >>> wrote:
> >> > > > > >>>
> >> > > > > >>>> Hi Jim,
> >> > > > > >>>>
> >> > > > > >>>> Now I have added the file to the jar (both root and
> resources
> >> > > > folder)
> >> > > > > >> but
> >> > > > > >>>> that does not seem to change anything.
> >> > > > > >>>>
> >> > > > > >>>> Any additional ideas?
> >> > > > > >>>>
> >> > > > > >>>> Regards,
> >> > > > > >>>> -Stefan
> >> > > > > >>>>
> >> > > > > >>>> On Sun, Jul 19, 2015 at 5:40 PM, Stefán Baxter <
> >> > > > > >>> stefan@activitystream.com <javascript:;>>
> >> > > > > >>>> wrote:
> >> > > > > >>>>
> >> > > > > >>>>> NO!
> >> > > > > >>>>>
> >> > > > > >>>>> Thank you , will do that right now :)
> >> > > > > >>>>>
> >> > > > > >>>>> On Sun, Jul 19, 2015 at 5:38 PM, Jim Bates <
> >> > jbates@maprtech.com <javascript:;>>
> >> > > > > >> wrote:
> >> > > > > >>>>>
> >> > > > > >>>>>> Did you include a file drill-module.conf in your jar
> along
> >> > with
> >> > > > > >> source
> >> > > > > >>>>>> files?
> >> > > > > >>>>>> On Jul 19, 2015 12:20 PM, "Stefán Baxter" <
> >> > > > > stefan@activitystream.com <javascript:;>
> >>
> >> > > > > >>>
> >> > > > > >>>>>> wrote:
> >> > > > > >>>>>>
> >> > > > > >>>>>>> Hi,
> >> > > > > >>>>>>>
> >> > > > > >>>>>>> I'm trying to deploy a UDF that I have written according
> >> to
> >> > the
> >> > > > > >>>>>>> documentation.
> >> > > > > >>>>>>>
> >> > > > > >>>>>>> I have also:
> >> > > > > >>>>>>>
> >> > > > > >>>>>>> 1. Copied the jar file
> >> > > > > >>>>>>> to jars/3rdparty
> >> > > > > >>>>>>>
> >> > > > > >>>>>>> 2. Changed the config "conf/drill-override.conf" to
> >> > include:
> >> > > > > >>>>>>> drill.logical.function.package +=
> >> > > > > >>
> >> > >
> ["org.apache.drill.exec.expr.fn.impl","com.activitystream.drill.udfs"]
> >> > > > > >>>>>>>
> >> > > > > >>>>>>> 3. Rerun the Drillbit
> >> > > > > >>>>>>> bin/drillbit.sh start
> >> > > > > >>>>>>>
> >> > > > > >>>>>>> The log shows no sign of the the UDF being loaded but
> if I
> >> > use
> >> > > it
> >> > > > > >> in
> >> > > > > >>> a
> >> > > > > >>>>>> SQL
> >> > > > > >>>>>>> statement then this error is thrown:
> >> > > > > >>>>>>>
> >> > > > > >>>>>>> Error: PARSE ERROR: From line 1, column 184 to line 1,
> >> column
> >> > > > 219:
> >> > > > > >> No
> >> > > > > >>>>>> match
> >> > > > > >>>>>>> found for function signature asRoundTimeStamp(<ANY>,
> >> > > <CHARACTER>)
> >> > > > > >>>>>>>
> >> > > > > >>>>>>>
> >> > > > > >>>>>>> The relevant part of the UDF source:
> >> > > > > >>>>>>>
> >> > > > > >>>>>>> package com.activitystream.drill.udfs;
> >> > > > > >>>>>>>
> >> > > > > >>>>>>> public class ASUserDefinedFunctions {
> >> > > > > >>>>>>>
> >> > > > > >>>>>>> @FunctionTemplate(name = "asRoundTimeStamp", scope =
> >> > > > > >>>>>>> FunctionTemplate.FunctionScope.SIMPLE, nulls =
> >> > > > > >>>>>>> FunctionTemplate.NullHandling.NULL_IF_NULL)
> >> > > > > >>>>>>>
> >> > > > > >>>>>>> public static class RoundTimeStamp implements
> >> > > DrillSimpleFunc
> >> > > > > >>>>>>>
> >> > > > > >>>>>>>
> >> > > > > >>>>>>> Can anyone please assist?
> >> > > > > >>>>>>>
> >> > > > > >>>>>>> Regards,
> >> > > > > >>>>>>> -Stefan
> >> > > > > >>
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> >
> >
>
Re: Drill not picking up a UDF
Posted by Stefán Baxter <st...@activitystream.com>.
Hi,
The project can be found here:
https://github.com/acmeguy/asdrill
Thank you,
-Stefán
On Sun, Jul 19, 2015 at 11:57 PM, Stefán Baxter <st...@activitystream.com>
wrote:
> Hi,
>
> I'm more than happy to share the little that is there (I will publish it
> on github and send link tomorrow).
>
> I ended up copying my UDF (singl-file.java) into the simple-drill-function
> project where it got picked up.
>
> Then I discovered a whole new set of dependencies/limitations
>
> - The UDF are recompiled - any imports are invalid or at least
> overwritten
> - import org.joda.time.Period; (means that Period class is not
> resolved on runtime)
> - Error: SYSTEM ERROR: CompileException: Line 71, Column 26: Cannot
> determine simple type name "Period"
>
> - Calling any "outside" functions, like I was calling a static
> function of the new class (same file), leads to an errors as they are not
> not resolved
> - Error: SYSTEM ERROR: CompileException: Line 70, Column 35: A method
> named "roundTimeStamp" is not declared in any enclosing class nor any
> supertype, nor through a static import
>
> Perhaps this was mentioned in the documentation but this is, at the very
> least, not straight forward and super-inviting.
>
> Thank you for your assistance, we will keep trying :)
>
> Regards,
> -Stefan
>
>
> On Sun, Jul 19, 2015 at 11:34 PM, Tugdual Grall <tu...@gmail.com> wrote:
>
>> Hi Stefan,
>>
>> Do you think you can share your complete project ?
>>
>> This will help to debug it for you.
>>
>> T
>>
>> On Sunday, July 19, 2015, Stefán Baxter <st...@activitystream.com>
>> wrote:
>>
>> > Hi Ted,
>> >
>> > I fetched this, built it and deployed it without problems.
>> > I can not see any real difference other than this deploys two .jar (I
>> tried
>> > that as well earlier).
>> >
>> > I'm still trying to figure out why Drill is not picking up my UDFs
>> >
>> > Regards,
>> > -Stefán
>> >
>> > On Sun, Jul 19, 2015 at 10:45 PM, Ted Dunning <ted.dunning@gmail.com
>> > <javascript:;>> wrote:
>> >
>> > > Stefan,
>> > >
>> > > Have you seen this github project:
>> > >
>> > > https://github.com/mapr-demos/simple-drill-functions
>> > >
>> > > ?
>> > >
>> > >
>> > > On Sun, Jul 19, 2015 at 2:14 PM, Stefán Baxter <
>> > stefan@activitystream.com <javascript:;>>
>> > > wrote:
>> > >
>> > > > Hi Jim,
>> > > >
>> > > > I'm still not able to make this work. Do you have a sample .jar file
>> > > with a
>> > > > small example that you are running?
>> > > >
>> > > > Regards,
>> > > > -Stefan
>> > > >
>> > > > On Sun, Jul 19, 2015 at 6:46 PM, Ted Dunning <ted.dunning@gmail.com
>> > <javascript:;>>
>> > > > wrote:
>> > > >
>> > > > >
>> > > > > Sounds like a fine example, not because of sophistication but
>> because
>> > > it
>> > > > > deals with dates.
>> > > > >
>> > > > > Check the drill logs. It is likely that drill is grumpy about
>> > > something
>> > > > > in your udf or packaging.
>> > > > >
>> > > > > Also, feel free to snitch the pom from the simple examples in
>> order
>> > to
>> > > > get
>> > > > > the pieces assembled and packaged correctly.
>> > > > >
>> > > > > Sent from my iPhone
>> > > > >
>> > > > > > On Jul 19, 2015, at 11:25, Stefán Baxter <
>> > stefan@activitystream.com <javascript:;>>
>> > > > > wrote:
>> > > > > >
>> > > > > > Hi Jim,
>> > > > > >
>> > > > > > My UDF is currently so simple that I'm not sure you need it (or
>> > want
>> > > > it).
>> > > > > >
>> > > > > > It basically just rounds a timestamp value with ISO 8601 periods
>> > > > > > asRoundTimestamp(timestampvalue,'PT10M').
>> > > > > >
>> > > > > > I would be more than happy to contribute to your project rather
>> > than
>> > > > > > building our own :).
>> > > > > >
>> > > > > > Is the repo public?
>> > > > > >
>> > > > > > Regards,
>> > > > > > -Stefan
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > > >> On Sun, Jul 19, 2015 at 6:18 PM, Jim Bates <
>> jbates@maprtech.com
>> > <javascript:;>>
>> > > > wrote:
>> > > > > >>
>> > > > > >> Maven will typically create a jar for class and a jar for
>> source
>> > > when
>> > > > > told
>> > > > > >> to do so. I just include the source files in the same jar as
>> the
>> > > class
>> > > > > >> files. There is a github example drill udf project we are
>> working
>> > on
>> > > > to
>> > > > > >> include several examples to simplify the learning curve. If
>> your
>> > > > > >> interested... I'd love to have you add your udf.
>> > > > > >> On Jul 19, 2015 12:59 PM, "Stefán Baxter" <
>> > > stefan@activitystream.com <javascript:;>>
>> > > > > >> wrote:
>> > > > > >>
>> > > > > >>> Hi again,
>> > > > > >>>
>> > > > > >>> Going over the documentation once more I came across this:
>> > > > > >>>
>> > > > > >>> - Add the sources and classes JAR files to Drill’s
>> classpath.
>> > > > > >>>
>> > > > > >>> I'm only including a standard .jar (with classes no sources)
>> and
>> > I
>> > > > > wonder
>> > > > > >>> why this dependency is listed.
>> > > > > >>>
>> > > > > >>> Anyways, that it the next step.
>> > > > > >>>
>> > > > > >>> A sample .jar that works with the drill-config counterpart
>> would
>> > > be a
>> > > > > >>> welcome addition to the documentation.
>> > > > > >>>
>> > > > > >>> Regards,
>> > > > > >>> -Stefan
>> > > > > >>>
>> > > > > >>>
>> > > > > >>> On Sun, Jul 19, 2015 at 5:53 PM, Stefán Baxter <
>> > > > > >> stefan@activitystream.com <javascript:;>>
>> > > > > >>> wrote:
>> > > > > >>>
>> > > > > >>>> Hi Jim,
>> > > > > >>>>
>> > > > > >>>> Now I have added the file to the jar (both root and resources
>> > > > folder)
>> > > > > >> but
>> > > > > >>>> that does not seem to change anything.
>> > > > > >>>>
>> > > > > >>>> Any additional ideas?
>> > > > > >>>>
>> > > > > >>>> Regards,
>> > > > > >>>> -Stefan
>> > > > > >>>>
>> > > > > >>>> On Sun, Jul 19, 2015 at 5:40 PM, Stefán Baxter <
>> > > > > >>> stefan@activitystream.com <javascript:;>>
>> > > > > >>>> wrote:
>> > > > > >>>>
>> > > > > >>>>> NO!
>> > > > > >>>>>
>> > > > > >>>>> Thank you , will do that right now :)
>> > > > > >>>>>
>> > > > > >>>>> On Sun, Jul 19, 2015 at 5:38 PM, Jim Bates <
>> > jbates@maprtech.com <javascript:;>>
>> > > > > >> wrote:
>> > > > > >>>>>
>> > > > > >>>>>> Did you include a file drill-module.conf in your jar along
>> > with
>> > > > > >> source
>> > > > > >>>>>> files?
>> > > > > >>>>>> On Jul 19, 2015 12:20 PM, "Stefán Baxter" <
>> > > > > stefan@activitystream.com <javascript:;>
>>
>> > > > > >>>
>> > > > > >>>>>> wrote:
>> > > > > >>>>>>
>> > > > > >>>>>>> Hi,
>> > > > > >>>>>>>
>> > > > > >>>>>>> I'm trying to deploy a UDF that I have written according
>> to
>> > the
>> > > > > >>>>>>> documentation.
>> > > > > >>>>>>>
>> > > > > >>>>>>> I have also:
>> > > > > >>>>>>>
>> > > > > >>>>>>> 1. Copied the jar file
>> > > > > >>>>>>> to jars/3rdparty
>> > > > > >>>>>>>
>> > > > > >>>>>>> 2. Changed the config "conf/drill-override.conf" to
>> > include:
>> > > > > >>>>>>> drill.logical.function.package +=
>> > > > > >>
>> > > ["org.apache.drill.exec.expr.fn.impl","com.activitystream.drill.udfs"]
>> > > > > >>>>>>>
>> > > > > >>>>>>> 3. Rerun the Drillbit
>> > > > > >>>>>>> bin/drillbit.sh start
>> > > > > >>>>>>>
>> > > > > >>>>>>> The log shows no sign of the the UDF being loaded but if I
>> > use
>> > > it
>> > > > > >> in
>> > > > > >>> a
>> > > > > >>>>>> SQL
>> > > > > >>>>>>> statement then this error is thrown:
>> > > > > >>>>>>>
>> > > > > >>>>>>> Error: PARSE ERROR: From line 1, column 184 to line 1,
>> column
>> > > > 219:
>> > > > > >> No
>> > > > > >>>>>> match
>> > > > > >>>>>>> found for function signature asRoundTimeStamp(<ANY>,
>> > > <CHARACTER>)
>> > > > > >>>>>>>
>> > > > > >>>>>>>
>> > > > > >>>>>>> The relevant part of the UDF source:
>> > > > > >>>>>>>
>> > > > > >>>>>>> package com.activitystream.drill.udfs;
>> > > > > >>>>>>>
>> > > > > >>>>>>> public class ASUserDefinedFunctions {
>> > > > > >>>>>>>
>> > > > > >>>>>>> @FunctionTemplate(name = "asRoundTimeStamp", scope =
>> > > > > >>>>>>> FunctionTemplate.FunctionScope.SIMPLE, nulls =
>> > > > > >>>>>>> FunctionTemplate.NullHandling.NULL_IF_NULL)
>> > > > > >>>>>>>
>> > > > > >>>>>>> public static class RoundTimeStamp implements
>> > > DrillSimpleFunc
>> > > > > >>>>>>>
>> > > > > >>>>>>>
>> > > > > >>>>>>> Can anyone please assist?
>> > > > > >>>>>>>
>> > > > > >>>>>>> Regards,
>> > > > > >>>>>>> -Stefan
>> > > > > >>
>> > > > >
>> > > >
>> > >
>> >
>>
>
>
Re: Drill not picking up a UDF
Posted by Ted Dunning <te...@gmail.com>.
On Sun, Jul 19, 2015 at 4:57 PM, Stefán Baxter <st...@activitystream.com>
wrote:
> Perhaps this was mentioned in the documentation but this is, at the very
> least, not straight forward and super-inviting.
>
You are very correct on this. The purpose of the simple drill function is
as a start at making some of these obscure details much easier to
understand.
If you want to send a pull request on the README for that project, I will
either incorporate your suggestions or answer any questions that you insert.
Re: Drill not picking up a UDF
Posted by Stefán Baxter <st...@activitystream.com>.
Hi,
I'm more than happy to share the little that is there (I will publish it on
github and send link tomorrow).
I ended up copying my UDF (singl-file.java) into the simple-drill-function
project where it got picked up.
Then I discovered a whole new set of dependencies/limitations
- The UDF are recompiled - any imports are invalid or at least
overwritten
- import org.joda.time.Period; (means that Period class is not resolved
on runtime)
- Error: SYSTEM ERROR: CompileException: Line 71, Column 26: Cannot
determine simple type name "Period"
- Calling any "outside" functions, like I was calling a static function
of the new class (same file), leads to an errors as they are not not
resolved
- Error: SYSTEM ERROR: CompileException: Line 70, Column 35: A method
named "roundTimeStamp" is not declared in any enclosing class nor any
supertype, nor through a static import
Perhaps this was mentioned in the documentation but this is, at the very
least, not straight forward and super-inviting.
Thank you for your assistance, we will keep trying :)
Regards,
-Stefan
On Sun, Jul 19, 2015 at 11:34 PM, Tugdual Grall <tu...@gmail.com> wrote:
> Hi Stefan,
>
> Do you think you can share your complete project ?
>
> This will help to debug it for you.
>
> T
>
> On Sunday, July 19, 2015, Stefán Baxter <st...@activitystream.com> wrote:
>
> > Hi Ted,
> >
> > I fetched this, built it and deployed it without problems.
> > I can not see any real difference other than this deploys two .jar (I
> tried
> > that as well earlier).
> >
> > I'm still trying to figure out why Drill is not picking up my UDFs
> >
> > Regards,
> > -Stefán
> >
> > On Sun, Jul 19, 2015 at 10:45 PM, Ted Dunning <ted.dunning@gmail.com
> > <javascript:;>> wrote:
> >
> > > Stefan,
> > >
> > > Have you seen this github project:
> > >
> > > https://github.com/mapr-demos/simple-drill-functions
> > >
> > > ?
> > >
> > >
> > > On Sun, Jul 19, 2015 at 2:14 PM, Stefán Baxter <
> > stefan@activitystream.com <javascript:;>>
> > > wrote:
> > >
> > > > Hi Jim,
> > > >
> > > > I'm still not able to make this work. Do you have a sample .jar file
> > > with a
> > > > small example that you are running?
> > > >
> > > > Regards,
> > > > -Stefan
> > > >
> > > > On Sun, Jul 19, 2015 at 6:46 PM, Ted Dunning <ted.dunning@gmail.com
> > <javascript:;>>
> > > > wrote:
> > > >
> > > > >
> > > > > Sounds like a fine example, not because of sophistication but
> because
> > > it
> > > > > deals with dates.
> > > > >
> > > > > Check the drill logs. It is likely that drill is grumpy about
> > > something
> > > > > in your udf or packaging.
> > > > >
> > > > > Also, feel free to snitch the pom from the simple examples in order
> > to
> > > > get
> > > > > the pieces assembled and packaged correctly.
> > > > >
> > > > > Sent from my iPhone
> > > > >
> > > > > > On Jul 19, 2015, at 11:25, Stefán Baxter <
> > stefan@activitystream.com <javascript:;>>
> > > > > wrote:
> > > > > >
> > > > > > Hi Jim,
> > > > > >
> > > > > > My UDF is currently so simple that I'm not sure you need it (or
> > want
> > > > it).
> > > > > >
> > > > > > It basically just rounds a timestamp value with ISO 8601 periods
> > > > > > asRoundTimestamp(timestampvalue,'PT10M').
> > > > > >
> > > > > > I would be more than happy to contribute to your project rather
> > than
> > > > > > building our own :).
> > > > > >
> > > > > > Is the repo public?
> > > > > >
> > > > > > Regards,
> > > > > > -Stefan
> > > > > >
> > > > > >
> > > > > >
> > > > > >> On Sun, Jul 19, 2015 at 6:18 PM, Jim Bates <jbates@maprtech.com
> > <javascript:;>>
> > > > wrote:
> > > > > >>
> > > > > >> Maven will typically create a jar for class and a jar for source
> > > when
> > > > > told
> > > > > >> to do so. I just include the source files in the same jar as the
> > > class
> > > > > >> files. There is a github example drill udf project we are
> working
> > on
> > > > to
> > > > > >> include several examples to simplify the learning curve. If your
> > > > > >> interested... I'd love to have you add your udf.
> > > > > >> On Jul 19, 2015 12:59 PM, "Stefán Baxter" <
> > > stefan@activitystream.com <javascript:;>>
> > > > > >> wrote:
> > > > > >>
> > > > > >>> Hi again,
> > > > > >>>
> > > > > >>> Going over the documentation once more I came across this:
> > > > > >>>
> > > > > >>> - Add the sources and classes JAR files to Drill’s classpath.
> > > > > >>>
> > > > > >>> I'm only including a standard .jar (with classes no sources)
> and
> > I
> > > > > wonder
> > > > > >>> why this dependency is listed.
> > > > > >>>
> > > > > >>> Anyways, that it the next step.
> > > > > >>>
> > > > > >>> A sample .jar that works with the drill-config counterpart
> would
> > > be a
> > > > > >>> welcome addition to the documentation.
> > > > > >>>
> > > > > >>> Regards,
> > > > > >>> -Stefan
> > > > > >>>
> > > > > >>>
> > > > > >>> On Sun, Jul 19, 2015 at 5:53 PM, Stefán Baxter <
> > > > > >> stefan@activitystream.com <javascript:;>>
> > > > > >>> wrote:
> > > > > >>>
> > > > > >>>> Hi Jim,
> > > > > >>>>
> > > > > >>>> Now I have added the file to the jar (both root and resources
> > > > folder)
> > > > > >> but
> > > > > >>>> that does not seem to change anything.
> > > > > >>>>
> > > > > >>>> Any additional ideas?
> > > > > >>>>
> > > > > >>>> Regards,
> > > > > >>>> -Stefan
> > > > > >>>>
> > > > > >>>> On Sun, Jul 19, 2015 at 5:40 PM, Stefán Baxter <
> > > > > >>> stefan@activitystream.com <javascript:;>>
> > > > > >>>> wrote:
> > > > > >>>>
> > > > > >>>>> NO!
> > > > > >>>>>
> > > > > >>>>> Thank you , will do that right now :)
> > > > > >>>>>
> > > > > >>>>> On Sun, Jul 19, 2015 at 5:38 PM, Jim Bates <
> > jbates@maprtech.com <javascript:;>>
> > > > > >> wrote:
> > > > > >>>>>
> > > > > >>>>>> Did you include a file drill-module.conf in your jar along
> > with
> > > > > >> source
> > > > > >>>>>> files?
> > > > > >>>>>> On Jul 19, 2015 12:20 PM, "Stefán Baxter" <
> > > > > stefan@activitystream.com <javascript:;>
> > > > > >>>
> > > > > >>>>>> wrote:
> > > > > >>>>>>
> > > > > >>>>>>> Hi,
> > > > > >>>>>>>
> > > > > >>>>>>> I'm trying to deploy a UDF that I have written according to
> > the
> > > > > >>>>>>> documentation.
> > > > > >>>>>>>
> > > > > >>>>>>> I have also:
> > > > > >>>>>>>
> > > > > >>>>>>> 1. Copied the jar file
> > > > > >>>>>>> to jars/3rdparty
> > > > > >>>>>>>
> > > > > >>>>>>> 2. Changed the config "conf/drill-override.conf" to
> > include:
> > > > > >>>>>>> drill.logical.function.package +=
> > > > > >>
> > > ["org.apache.drill.exec.expr.fn.impl","com.activitystream.drill.udfs"]
> > > > > >>>>>>>
> > > > > >>>>>>> 3. Rerun the Drillbit
> > > > > >>>>>>> bin/drillbit.sh start
> > > > > >>>>>>>
> > > > > >>>>>>> The log shows no sign of the the UDF being loaded but if I
> > use
> > > it
> > > > > >> in
> > > > > >>> a
> > > > > >>>>>> SQL
> > > > > >>>>>>> statement then this error is thrown:
> > > > > >>>>>>>
> > > > > >>>>>>> Error: PARSE ERROR: From line 1, column 184 to line 1,
> column
> > > > 219:
> > > > > >> No
> > > > > >>>>>> match
> > > > > >>>>>>> found for function signature asRoundTimeStamp(<ANY>,
> > > <CHARACTER>)
> > > > > >>>>>>>
> > > > > >>>>>>>
> > > > > >>>>>>> The relevant part of the UDF source:
> > > > > >>>>>>>
> > > > > >>>>>>> package com.activitystream.drill.udfs;
> > > > > >>>>>>>
> > > > > >>>>>>> public class ASUserDefinedFunctions {
> > > > > >>>>>>>
> > > > > >>>>>>> @FunctionTemplate(name = "asRoundTimeStamp", scope =
> > > > > >>>>>>> FunctionTemplate.FunctionScope.SIMPLE, nulls =
> > > > > >>>>>>> FunctionTemplate.NullHandling.NULL_IF_NULL)
> > > > > >>>>>>>
> > > > > >>>>>>> public static class RoundTimeStamp implements
> > > DrillSimpleFunc
> > > > > >>>>>>>
> > > > > >>>>>>>
> > > > > >>>>>>> Can anyone please assist?
> > > > > >>>>>>>
> > > > > >>>>>>> Regards,
> > > > > >>>>>>> -Stefan
> > > > > >>
> > > > >
> > > >
> > >
> >
>
Re: Drill not picking up a UDF
Posted by Tugdual Grall <tu...@gmail.com>.
Hi Stefan,
Do you think you can share your complete project ?
This will help to debug it for you.
T
On Sunday, July 19, 2015, Stefán Baxter <st...@activitystream.com> wrote:
> Hi Ted,
>
> I fetched this, built it and deployed it without problems.
> I can not see any real difference other than this deploys two .jar (I tried
> that as well earlier).
>
> I'm still trying to figure out why Drill is not picking up my UDFs
>
> Regards,
> -Stefán
>
> On Sun, Jul 19, 2015 at 10:45 PM, Ted Dunning <ted.dunning@gmail.com
> <javascript:;>> wrote:
>
> > Stefan,
> >
> > Have you seen this github project:
> >
> > https://github.com/mapr-demos/simple-drill-functions
> >
> > ?
> >
> >
> > On Sun, Jul 19, 2015 at 2:14 PM, Stefán Baxter <
> stefan@activitystream.com <javascript:;>>
> > wrote:
> >
> > > Hi Jim,
> > >
> > > I'm still not able to make this work. Do you have a sample .jar file
> > with a
> > > small example that you are running?
> > >
> > > Regards,
> > > -Stefan
> > >
> > > On Sun, Jul 19, 2015 at 6:46 PM, Ted Dunning <ted.dunning@gmail.com
> <javascript:;>>
> > > wrote:
> > >
> > > >
> > > > Sounds like a fine example, not because of sophistication but because
> > it
> > > > deals with dates.
> > > >
> > > > Check the drill logs. It is likely that drill is grumpy about
> > something
> > > > in your udf or packaging.
> > > >
> > > > Also, feel free to snitch the pom from the simple examples in order
> to
> > > get
> > > > the pieces assembled and packaged correctly.
> > > >
> > > > Sent from my iPhone
> > > >
> > > > > On Jul 19, 2015, at 11:25, Stefán Baxter <
> stefan@activitystream.com <javascript:;>>
> > > > wrote:
> > > > >
> > > > > Hi Jim,
> > > > >
> > > > > My UDF is currently so simple that I'm not sure you need it (or
> want
> > > it).
> > > > >
> > > > > It basically just rounds a timestamp value with ISO 8601 periods
> > > > > asRoundTimestamp(timestampvalue,'PT10M').
> > > > >
> > > > > I would be more than happy to contribute to your project rather
> than
> > > > > building our own :).
> > > > >
> > > > > Is the repo public?
> > > > >
> > > > > Regards,
> > > > > -Stefan
> > > > >
> > > > >
> > > > >
> > > > >> On Sun, Jul 19, 2015 at 6:18 PM, Jim Bates <jbates@maprtech.com
> <javascript:;>>
> > > wrote:
> > > > >>
> > > > >> Maven will typically create a jar for class and a jar for source
> > when
> > > > told
> > > > >> to do so. I just include the source files in the same jar as the
> > class
> > > > >> files. There is a github example drill udf project we are working
> on
> > > to
> > > > >> include several examples to simplify the learning curve. If your
> > > > >> interested... I'd love to have you add your udf.
> > > > >> On Jul 19, 2015 12:59 PM, "Stefán Baxter" <
> > stefan@activitystream.com <javascript:;>>
> > > > >> wrote:
> > > > >>
> > > > >>> Hi again,
> > > > >>>
> > > > >>> Going over the documentation once more I came across this:
> > > > >>>
> > > > >>> - Add the sources and classes JAR files to Drill’s classpath.
> > > > >>>
> > > > >>> I'm only including a standard .jar (with classes no sources) and
> I
> > > > wonder
> > > > >>> why this dependency is listed.
> > > > >>>
> > > > >>> Anyways, that it the next step.
> > > > >>>
> > > > >>> A sample .jar that works with the drill-config counterpart would
> > be a
> > > > >>> welcome addition to the documentation.
> > > > >>>
> > > > >>> Regards,
> > > > >>> -Stefan
> > > > >>>
> > > > >>>
> > > > >>> On Sun, Jul 19, 2015 at 5:53 PM, Stefán Baxter <
> > > > >> stefan@activitystream.com <javascript:;>>
> > > > >>> wrote:
> > > > >>>
> > > > >>>> Hi Jim,
> > > > >>>>
> > > > >>>> Now I have added the file to the jar (both root and resources
> > > folder)
> > > > >> but
> > > > >>>> that does not seem to change anything.
> > > > >>>>
> > > > >>>> Any additional ideas?
> > > > >>>>
> > > > >>>> Regards,
> > > > >>>> -Stefan
> > > > >>>>
> > > > >>>> On Sun, Jul 19, 2015 at 5:40 PM, Stefán Baxter <
> > > > >>> stefan@activitystream.com <javascript:;>>
> > > > >>>> wrote:
> > > > >>>>
> > > > >>>>> NO!
> > > > >>>>>
> > > > >>>>> Thank you , will do that right now :)
> > > > >>>>>
> > > > >>>>> On Sun, Jul 19, 2015 at 5:38 PM, Jim Bates <
> jbates@maprtech.com <javascript:;>>
> > > > >> wrote:
> > > > >>>>>
> > > > >>>>>> Did you include a file drill-module.conf in your jar along
> with
> > > > >> source
> > > > >>>>>> files?
> > > > >>>>>> On Jul 19, 2015 12:20 PM, "Stefán Baxter" <
> > > > stefan@activitystream.com <javascript:;>
> > > > >>>
> > > > >>>>>> wrote:
> > > > >>>>>>
> > > > >>>>>>> Hi,
> > > > >>>>>>>
> > > > >>>>>>> I'm trying to deploy a UDF that I have written according to
> the
> > > > >>>>>>> documentation.
> > > > >>>>>>>
> > > > >>>>>>> I have also:
> > > > >>>>>>>
> > > > >>>>>>> 1. Copied the jar file
> > > > >>>>>>> to jars/3rdparty
> > > > >>>>>>>
> > > > >>>>>>> 2. Changed the config "conf/drill-override.conf" to
> include:
> > > > >>>>>>> drill.logical.function.package +=
> > > > >>
> > ["org.apache.drill.exec.expr.fn.impl","com.activitystream.drill.udfs"]
> > > > >>>>>>>
> > > > >>>>>>> 3. Rerun the Drillbit
> > > > >>>>>>> bin/drillbit.sh start
> > > > >>>>>>>
> > > > >>>>>>> The log shows no sign of the the UDF being loaded but if I
> use
> > it
> > > > >> in
> > > > >>> a
> > > > >>>>>> SQL
> > > > >>>>>>> statement then this error is thrown:
> > > > >>>>>>>
> > > > >>>>>>> Error: PARSE ERROR: From line 1, column 184 to line 1, column
> > > 219:
> > > > >> No
> > > > >>>>>> match
> > > > >>>>>>> found for function signature asRoundTimeStamp(<ANY>,
> > <CHARACTER>)
> > > > >>>>>>>
> > > > >>>>>>>
> > > > >>>>>>> The relevant part of the UDF source:
> > > > >>>>>>>
> > > > >>>>>>> package com.activitystream.drill.udfs;
> > > > >>>>>>>
> > > > >>>>>>> public class ASUserDefinedFunctions {
> > > > >>>>>>>
> > > > >>>>>>> @FunctionTemplate(name = "asRoundTimeStamp", scope =
> > > > >>>>>>> FunctionTemplate.FunctionScope.SIMPLE, nulls =
> > > > >>>>>>> FunctionTemplate.NullHandling.NULL_IF_NULL)
> > > > >>>>>>>
> > > > >>>>>>> public static class RoundTimeStamp implements
> > DrillSimpleFunc
> > > > >>>>>>>
> > > > >>>>>>>
> > > > >>>>>>> Can anyone please assist?
> > > > >>>>>>>
> > > > >>>>>>> Regards,
> > > > >>>>>>> -Stefan
> > > > >>
> > > >
> > >
> >
>
Re: Drill not picking up a UDF
Posted by Stefán Baxter <st...@activitystream.com>.
Hi Ted,
I fetched this, built it and deployed it without problems.
I can not see any real difference other than this deploys two .jar (I tried
that as well earlier).
I'm still trying to figure out why Drill is not picking up my UDFs
Regards,
-Stefán
On Sun, Jul 19, 2015 at 10:45 PM, Ted Dunning <te...@gmail.com> wrote:
> Stefan,
>
> Have you seen this github project:
>
> https://github.com/mapr-demos/simple-drill-functions
>
> ?
>
>
> On Sun, Jul 19, 2015 at 2:14 PM, Stefán Baxter <st...@activitystream.com>
> wrote:
>
> > Hi Jim,
> >
> > I'm still not able to make this work. Do you have a sample .jar file
> with a
> > small example that you are running?
> >
> > Regards,
> > -Stefan
> >
> > On Sun, Jul 19, 2015 at 6:46 PM, Ted Dunning <te...@gmail.com>
> > wrote:
> >
> > >
> > > Sounds like a fine example, not because of sophistication but because
> it
> > > deals with dates.
> > >
> > > Check the drill logs. It is likely that drill is grumpy about
> something
> > > in your udf or packaging.
> > >
> > > Also, feel free to snitch the pom from the simple examples in order to
> > get
> > > the pieces assembled and packaged correctly.
> > >
> > > Sent from my iPhone
> > >
> > > > On Jul 19, 2015, at 11:25, Stefán Baxter <st...@activitystream.com>
> > > wrote:
> > > >
> > > > Hi Jim,
> > > >
> > > > My UDF is currently so simple that I'm not sure you need it (or want
> > it).
> > > >
> > > > It basically just rounds a timestamp value with ISO 8601 periods
> > > > asRoundTimestamp(timestampvalue,'PT10M').
> > > >
> > > > I would be more than happy to contribute to your project rather than
> > > > building our own :).
> > > >
> > > > Is the repo public?
> > > >
> > > > Regards,
> > > > -Stefan
> > > >
> > > >
> > > >
> > > >> On Sun, Jul 19, 2015 at 6:18 PM, Jim Bates <jb...@maprtech.com>
> > wrote:
> > > >>
> > > >> Maven will typically create a jar for class and a jar for source
> when
> > > told
> > > >> to do so. I just include the source files in the same jar as the
> class
> > > >> files. There is a github example drill udf project we are working on
> > to
> > > >> include several examples to simplify the learning curve. If your
> > > >> interested... I'd love to have you add your udf.
> > > >> On Jul 19, 2015 12:59 PM, "Stefán Baxter" <
> stefan@activitystream.com>
> > > >> wrote:
> > > >>
> > > >>> Hi again,
> > > >>>
> > > >>> Going over the documentation once more I came across this:
> > > >>>
> > > >>> - Add the sources and classes JAR files to Drill’s classpath.
> > > >>>
> > > >>> I'm only including a standard .jar (with classes no sources) and I
> > > wonder
> > > >>> why this dependency is listed.
> > > >>>
> > > >>> Anyways, that it the next step.
> > > >>>
> > > >>> A sample .jar that works with the drill-config counterpart would
> be a
> > > >>> welcome addition to the documentation.
> > > >>>
> > > >>> Regards,
> > > >>> -Stefan
> > > >>>
> > > >>>
> > > >>> On Sun, Jul 19, 2015 at 5:53 PM, Stefán Baxter <
> > > >> stefan@activitystream.com>
> > > >>> wrote:
> > > >>>
> > > >>>> Hi Jim,
> > > >>>>
> > > >>>> Now I have added the file to the jar (both root and resources
> > folder)
> > > >> but
> > > >>>> that does not seem to change anything.
> > > >>>>
> > > >>>> Any additional ideas?
> > > >>>>
> > > >>>> Regards,
> > > >>>> -Stefan
> > > >>>>
> > > >>>> On Sun, Jul 19, 2015 at 5:40 PM, Stefán Baxter <
> > > >>> stefan@activitystream.com>
> > > >>>> wrote:
> > > >>>>
> > > >>>>> NO!
> > > >>>>>
> > > >>>>> Thank you , will do that right now :)
> > > >>>>>
> > > >>>>> On Sun, Jul 19, 2015 at 5:38 PM, Jim Bates <jb...@maprtech.com>
> > > >> wrote:
> > > >>>>>
> > > >>>>>> Did you include a file drill-module.conf in your jar along with
> > > >> source
> > > >>>>>> files?
> > > >>>>>> On Jul 19, 2015 12:20 PM, "Stefán Baxter" <
> > > stefan@activitystream.com
> > > >>>
> > > >>>>>> wrote:
> > > >>>>>>
> > > >>>>>>> Hi,
> > > >>>>>>>
> > > >>>>>>> I'm trying to deploy a UDF that I have written according to the
> > > >>>>>>> documentation.
> > > >>>>>>>
> > > >>>>>>> I have also:
> > > >>>>>>>
> > > >>>>>>> 1. Copied the jar file
> > > >>>>>>> to jars/3rdparty
> > > >>>>>>>
> > > >>>>>>> 2. Changed the config "conf/drill-override.conf" to include:
> > > >>>>>>> drill.logical.function.package +=
> > > >>
> ["org.apache.drill.exec.expr.fn.impl","com.activitystream.drill.udfs"]
> > > >>>>>>>
> > > >>>>>>> 3. Rerun the Drillbit
> > > >>>>>>> bin/drillbit.sh start
> > > >>>>>>>
> > > >>>>>>> The log shows no sign of the the UDF being loaded but if I use
> it
> > > >> in
> > > >>> a
> > > >>>>>> SQL
> > > >>>>>>> statement then this error is thrown:
> > > >>>>>>>
> > > >>>>>>> Error: PARSE ERROR: From line 1, column 184 to line 1, column
> > 219:
> > > >> No
> > > >>>>>> match
> > > >>>>>>> found for function signature asRoundTimeStamp(<ANY>,
> <CHARACTER>)
> > > >>>>>>>
> > > >>>>>>>
> > > >>>>>>> The relevant part of the UDF source:
> > > >>>>>>>
> > > >>>>>>> package com.activitystream.drill.udfs;
> > > >>>>>>>
> > > >>>>>>> public class ASUserDefinedFunctions {
> > > >>>>>>>
> > > >>>>>>> @FunctionTemplate(name = "asRoundTimeStamp", scope =
> > > >>>>>>> FunctionTemplate.FunctionScope.SIMPLE, nulls =
> > > >>>>>>> FunctionTemplate.NullHandling.NULL_IF_NULL)
> > > >>>>>>>
> > > >>>>>>> public static class RoundTimeStamp implements
> DrillSimpleFunc
> > > >>>>>>>
> > > >>>>>>>
> > > >>>>>>> Can anyone please assist?
> > > >>>>>>>
> > > >>>>>>> Regards,
> > > >>>>>>> -Stefan
> > > >>
> > >
> >
>
Re: Drill not picking up a UDF
Posted by Ted Dunning <te...@gmail.com>.
Stefan,
Have you seen this github project:
https://github.com/mapr-demos/simple-drill-functions
?
On Sun, Jul 19, 2015 at 2:14 PM, Stefán Baxter <st...@activitystream.com>
wrote:
> Hi Jim,
>
> I'm still not able to make this work. Do you have a sample .jar file with a
> small example that you are running?
>
> Regards,
> -Stefan
>
> On Sun, Jul 19, 2015 at 6:46 PM, Ted Dunning <te...@gmail.com>
> wrote:
>
> >
> > Sounds like a fine example, not because of sophistication but because it
> > deals with dates.
> >
> > Check the drill logs. It is likely that drill is grumpy about something
> > in your udf or packaging.
> >
> > Also, feel free to snitch the pom from the simple examples in order to
> get
> > the pieces assembled and packaged correctly.
> >
> > Sent from my iPhone
> >
> > > On Jul 19, 2015, at 11:25, Stefán Baxter <st...@activitystream.com>
> > wrote:
> > >
> > > Hi Jim,
> > >
> > > My UDF is currently so simple that I'm not sure you need it (or want
> it).
> > >
> > > It basically just rounds a timestamp value with ISO 8601 periods
> > > asRoundTimestamp(timestampvalue,'PT10M').
> > >
> > > I would be more than happy to contribute to your project rather than
> > > building our own :).
> > >
> > > Is the repo public?
> > >
> > > Regards,
> > > -Stefan
> > >
> > >
> > >
> > >> On Sun, Jul 19, 2015 at 6:18 PM, Jim Bates <jb...@maprtech.com>
> wrote:
> > >>
> > >> Maven will typically create a jar for class and a jar for source when
> > told
> > >> to do so. I just include the source files in the same jar as the class
> > >> files. There is a github example drill udf project we are working on
> to
> > >> include several examples to simplify the learning curve. If your
> > >> interested... I'd love to have you add your udf.
> > >> On Jul 19, 2015 12:59 PM, "Stefán Baxter" <st...@activitystream.com>
> > >> wrote:
> > >>
> > >>> Hi again,
> > >>>
> > >>> Going over the documentation once more I came across this:
> > >>>
> > >>> - Add the sources and classes JAR files to Drill’s classpath.
> > >>>
> > >>> I'm only including a standard .jar (with classes no sources) and I
> > wonder
> > >>> why this dependency is listed.
> > >>>
> > >>> Anyways, that it the next step.
> > >>>
> > >>> A sample .jar that works with the drill-config counterpart would be a
> > >>> welcome addition to the documentation.
> > >>>
> > >>> Regards,
> > >>> -Stefan
> > >>>
> > >>>
> > >>> On Sun, Jul 19, 2015 at 5:53 PM, Stefán Baxter <
> > >> stefan@activitystream.com>
> > >>> wrote:
> > >>>
> > >>>> Hi Jim,
> > >>>>
> > >>>> Now I have added the file to the jar (both root and resources
> folder)
> > >> but
> > >>>> that does not seem to change anything.
> > >>>>
> > >>>> Any additional ideas?
> > >>>>
> > >>>> Regards,
> > >>>> -Stefan
> > >>>>
> > >>>> On Sun, Jul 19, 2015 at 5:40 PM, Stefán Baxter <
> > >>> stefan@activitystream.com>
> > >>>> wrote:
> > >>>>
> > >>>>> NO!
> > >>>>>
> > >>>>> Thank you , will do that right now :)
> > >>>>>
> > >>>>> On Sun, Jul 19, 2015 at 5:38 PM, Jim Bates <jb...@maprtech.com>
> > >> wrote:
> > >>>>>
> > >>>>>> Did you include a file drill-module.conf in your jar along with
> > >> source
> > >>>>>> files?
> > >>>>>> On Jul 19, 2015 12:20 PM, "Stefán Baxter" <
> > stefan@activitystream.com
> > >>>
> > >>>>>> wrote:
> > >>>>>>
> > >>>>>>> Hi,
> > >>>>>>>
> > >>>>>>> I'm trying to deploy a UDF that I have written according to the
> > >>>>>>> documentation.
> > >>>>>>>
> > >>>>>>> I have also:
> > >>>>>>>
> > >>>>>>> 1. Copied the jar file
> > >>>>>>> to jars/3rdparty
> > >>>>>>>
> > >>>>>>> 2. Changed the config "conf/drill-override.conf" to include:
> > >>>>>>> drill.logical.function.package +=
> > >> ["org.apache.drill.exec.expr.fn.impl","com.activitystream.drill.udfs"]
> > >>>>>>>
> > >>>>>>> 3. Rerun the Drillbit
> > >>>>>>> bin/drillbit.sh start
> > >>>>>>>
> > >>>>>>> The log shows no sign of the the UDF being loaded but if I use it
> > >> in
> > >>> a
> > >>>>>> SQL
> > >>>>>>> statement then this error is thrown:
> > >>>>>>>
> > >>>>>>> Error: PARSE ERROR: From line 1, column 184 to line 1, column
> 219:
> > >> No
> > >>>>>> match
> > >>>>>>> found for function signature asRoundTimeStamp(<ANY>, <CHARACTER>)
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> The relevant part of the UDF source:
> > >>>>>>>
> > >>>>>>> package com.activitystream.drill.udfs;
> > >>>>>>>
> > >>>>>>> public class ASUserDefinedFunctions {
> > >>>>>>>
> > >>>>>>> @FunctionTemplate(name = "asRoundTimeStamp", scope =
> > >>>>>>> FunctionTemplate.FunctionScope.SIMPLE, nulls =
> > >>>>>>> FunctionTemplate.NullHandling.NULL_IF_NULL)
> > >>>>>>>
> > >>>>>>> public static class RoundTimeStamp implements DrillSimpleFunc
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> Can anyone please assist?
> > >>>>>>>
> > >>>>>>> Regards,
> > >>>>>>> -Stefan
> > >>
> >
>
Re: Drill not picking up a UDF
Posted by Stefán Baxter <st...@activitystream.com>.
Hi Jim,
I'm still not able to make this work. Do you have a sample .jar file with a
small example that you are running?
Regards,
-Stefan
On Sun, Jul 19, 2015 at 6:46 PM, Ted Dunning <te...@gmail.com> wrote:
>
> Sounds like a fine example, not because of sophistication but because it
> deals with dates.
>
> Check the drill logs. It is likely that drill is grumpy about something
> in your udf or packaging.
>
> Also, feel free to snitch the pom from the simple examples in order to get
> the pieces assembled and packaged correctly.
>
> Sent from my iPhone
>
> > On Jul 19, 2015, at 11:25, Stefán Baxter <st...@activitystream.com>
> wrote:
> >
> > Hi Jim,
> >
> > My UDF is currently so simple that I'm not sure you need it (or want it).
> >
> > It basically just rounds a timestamp value with ISO 8601 periods
> > asRoundTimestamp(timestampvalue,'PT10M').
> >
> > I would be more than happy to contribute to your project rather than
> > building our own :).
> >
> > Is the repo public?
> >
> > Regards,
> > -Stefan
> >
> >
> >
> >> On Sun, Jul 19, 2015 at 6:18 PM, Jim Bates <jb...@maprtech.com> wrote:
> >>
> >> Maven will typically create a jar for class and a jar for source when
> told
> >> to do so. I just include the source files in the same jar as the class
> >> files. There is a github example drill udf project we are working on to
> >> include several examples to simplify the learning curve. If your
> >> interested... I'd love to have you add your udf.
> >> On Jul 19, 2015 12:59 PM, "Stefán Baxter" <st...@activitystream.com>
> >> wrote:
> >>
> >>> Hi again,
> >>>
> >>> Going over the documentation once more I came across this:
> >>>
> >>> - Add the sources and classes JAR files to Drill’s classpath.
> >>>
> >>> I'm only including a standard .jar (with classes no sources) and I
> wonder
> >>> why this dependency is listed.
> >>>
> >>> Anyways, that it the next step.
> >>>
> >>> A sample .jar that works with the drill-config counterpart would be a
> >>> welcome addition to the documentation.
> >>>
> >>> Regards,
> >>> -Stefan
> >>>
> >>>
> >>> On Sun, Jul 19, 2015 at 5:53 PM, Stefán Baxter <
> >> stefan@activitystream.com>
> >>> wrote:
> >>>
> >>>> Hi Jim,
> >>>>
> >>>> Now I have added the file to the jar (both root and resources folder)
> >> but
> >>>> that does not seem to change anything.
> >>>>
> >>>> Any additional ideas?
> >>>>
> >>>> Regards,
> >>>> -Stefan
> >>>>
> >>>> On Sun, Jul 19, 2015 at 5:40 PM, Stefán Baxter <
> >>> stefan@activitystream.com>
> >>>> wrote:
> >>>>
> >>>>> NO!
> >>>>>
> >>>>> Thank you , will do that right now :)
> >>>>>
> >>>>> On Sun, Jul 19, 2015 at 5:38 PM, Jim Bates <jb...@maprtech.com>
> >> wrote:
> >>>>>
> >>>>>> Did you include a file drill-module.conf in your jar along with
> >> source
> >>>>>> files?
> >>>>>> On Jul 19, 2015 12:20 PM, "Stefán Baxter" <
> stefan@activitystream.com
> >>>
> >>>>>> wrote:
> >>>>>>
> >>>>>>> Hi,
> >>>>>>>
> >>>>>>> I'm trying to deploy a UDF that I have written according to the
> >>>>>>> documentation.
> >>>>>>>
> >>>>>>> I have also:
> >>>>>>>
> >>>>>>> 1. Copied the jar file
> >>>>>>> to jars/3rdparty
> >>>>>>>
> >>>>>>> 2. Changed the config "conf/drill-override.conf" to include:
> >>>>>>> drill.logical.function.package +=
> >> ["org.apache.drill.exec.expr.fn.impl","com.activitystream.drill.udfs"]
> >>>>>>>
> >>>>>>> 3. Rerun the Drillbit
> >>>>>>> bin/drillbit.sh start
> >>>>>>>
> >>>>>>> The log shows no sign of the the UDF being loaded but if I use it
> >> in
> >>> a
> >>>>>> SQL
> >>>>>>> statement then this error is thrown:
> >>>>>>>
> >>>>>>> Error: PARSE ERROR: From line 1, column 184 to line 1, column 219:
> >> No
> >>>>>> match
> >>>>>>> found for function signature asRoundTimeStamp(<ANY>, <CHARACTER>)
> >>>>>>>
> >>>>>>>
> >>>>>>> The relevant part of the UDF source:
> >>>>>>>
> >>>>>>> package com.activitystream.drill.udfs;
> >>>>>>>
> >>>>>>> public class ASUserDefinedFunctions {
> >>>>>>>
> >>>>>>> @FunctionTemplate(name = "asRoundTimeStamp", scope =
> >>>>>>> FunctionTemplate.FunctionScope.SIMPLE, nulls =
> >>>>>>> FunctionTemplate.NullHandling.NULL_IF_NULL)
> >>>>>>>
> >>>>>>> public static class RoundTimeStamp implements DrillSimpleFunc
> >>>>>>>
> >>>>>>>
> >>>>>>> Can anyone please assist?
> >>>>>>>
> >>>>>>> Regards,
> >>>>>>> -Stefan
> >>
>
Re: Drill not picking up a UDF
Posted by Ted Dunning <te...@gmail.com>.
Sounds like a fine example, not because of sophistication but because it deals with dates.
Check the drill logs. It is likely that drill is grumpy about something in your udf or packaging.
Also, feel free to snitch the pom from the simple examples in order to get the pieces assembled and packaged correctly.
Sent from my iPhone
> On Jul 19, 2015, at 11:25, Stefán Baxter <st...@activitystream.com> wrote:
>
> Hi Jim,
>
> My UDF is currently so simple that I'm not sure you need it (or want it).
>
> It basically just rounds a timestamp value with ISO 8601 periods
> asRoundTimestamp(timestampvalue,'PT10M').
>
> I would be more than happy to contribute to your project rather than
> building our own :).
>
> Is the repo public?
>
> Regards,
> -Stefan
>
>
>
>> On Sun, Jul 19, 2015 at 6:18 PM, Jim Bates <jb...@maprtech.com> wrote:
>>
>> Maven will typically create a jar for class and a jar for source when told
>> to do so. I just include the source files in the same jar as the class
>> files. There is a github example drill udf project we are working on to
>> include several examples to simplify the learning curve. If your
>> interested... I'd love to have you add your udf.
>> On Jul 19, 2015 12:59 PM, "Stefán Baxter" <st...@activitystream.com>
>> wrote:
>>
>>> Hi again,
>>>
>>> Going over the documentation once more I came across this:
>>>
>>> - Add the sources and classes JAR files to Drill’s classpath.
>>>
>>> I'm only including a standard .jar (with classes no sources) and I wonder
>>> why this dependency is listed.
>>>
>>> Anyways, that it the next step.
>>>
>>> A sample .jar that works with the drill-config counterpart would be a
>>> welcome addition to the documentation.
>>>
>>> Regards,
>>> -Stefan
>>>
>>>
>>> On Sun, Jul 19, 2015 at 5:53 PM, Stefán Baxter <
>> stefan@activitystream.com>
>>> wrote:
>>>
>>>> Hi Jim,
>>>>
>>>> Now I have added the file to the jar (both root and resources folder)
>> but
>>>> that does not seem to change anything.
>>>>
>>>> Any additional ideas?
>>>>
>>>> Regards,
>>>> -Stefan
>>>>
>>>> On Sun, Jul 19, 2015 at 5:40 PM, Stefán Baxter <
>>> stefan@activitystream.com>
>>>> wrote:
>>>>
>>>>> NO!
>>>>>
>>>>> Thank you , will do that right now :)
>>>>>
>>>>> On Sun, Jul 19, 2015 at 5:38 PM, Jim Bates <jb...@maprtech.com>
>> wrote:
>>>>>
>>>>>> Did you include a file drill-module.conf in your jar along with
>> source
>>>>>> files?
>>>>>> On Jul 19, 2015 12:20 PM, "Stefán Baxter" <stefan@activitystream.com
>>>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> I'm trying to deploy a UDF that I have written according to the
>>>>>>> documentation.
>>>>>>>
>>>>>>> I have also:
>>>>>>>
>>>>>>> 1. Copied the jar file
>>>>>>> to jars/3rdparty
>>>>>>>
>>>>>>> 2. Changed the config "conf/drill-override.conf" to include:
>>>>>>> drill.logical.function.package +=
>> ["org.apache.drill.exec.expr.fn.impl","com.activitystream.drill.udfs"]
>>>>>>>
>>>>>>> 3. Rerun the Drillbit
>>>>>>> bin/drillbit.sh start
>>>>>>>
>>>>>>> The log shows no sign of the the UDF being loaded but if I use it
>> in
>>> a
>>>>>> SQL
>>>>>>> statement then this error is thrown:
>>>>>>>
>>>>>>> Error: PARSE ERROR: From line 1, column 184 to line 1, column 219:
>> No
>>>>>> match
>>>>>>> found for function signature asRoundTimeStamp(<ANY>, <CHARACTER>)
>>>>>>>
>>>>>>>
>>>>>>> The relevant part of the UDF source:
>>>>>>>
>>>>>>> package com.activitystream.drill.udfs;
>>>>>>>
>>>>>>> public class ASUserDefinedFunctions {
>>>>>>>
>>>>>>> @FunctionTemplate(name = "asRoundTimeStamp", scope =
>>>>>>> FunctionTemplate.FunctionScope.SIMPLE, nulls =
>>>>>>> FunctionTemplate.NullHandling.NULL_IF_NULL)
>>>>>>>
>>>>>>> public static class RoundTimeStamp implements DrillSimpleFunc
>>>>>>>
>>>>>>>
>>>>>>> Can anyone please assist?
>>>>>>>
>>>>>>> Regards,
>>>>>>> -Stefan
>>
Re: Drill not picking up a UDF
Posted by Stefán Baxter <st...@activitystream.com>.
Hi Jim,
My UDF is currently so simple that I'm not sure you need it (or want it).
It basically just rounds a timestamp value with ISO 8601 periods
asRoundTimestamp(timestampvalue,'PT10M').
I would be more than happy to contribute to your project rather than
building our own :).
Is the repo public?
Regards,
-Stefan
On Sun, Jul 19, 2015 at 6:18 PM, Jim Bates <jb...@maprtech.com> wrote:
> Maven will typically create a jar for class and a jar for source when told
> to do so. I just include the source files in the same jar as the class
> files. There is a github example drill udf project we are working on to
> include several examples to simplify the learning curve. If your
> interested... I'd love to have you add your udf.
> On Jul 19, 2015 12:59 PM, "Stefán Baxter" <st...@activitystream.com>
> wrote:
>
> > Hi again,
> >
> > Going over the documentation once more I came across this:
> >
> > - Add the sources and classes JAR files to Drill’s classpath.
> >
> > I'm only including a standard .jar (with classes no sources) and I wonder
> > why this dependency is listed.
> >
> > Anyways, that it the next step.
> >
> > A sample .jar that works with the drill-config counterpart would be a
> > welcome addition to the documentation.
> >
> > Regards,
> > -Stefan
> >
> >
> > On Sun, Jul 19, 2015 at 5:53 PM, Stefán Baxter <
> stefan@activitystream.com>
> > wrote:
> >
> > > Hi Jim,
> > >
> > > Now I have added the file to the jar (both root and resources folder)
> but
> > > that does not seem to change anything.
> > >
> > > Any additional ideas?
> > >
> > > Regards,
> > > -Stefan
> > >
> > > On Sun, Jul 19, 2015 at 5:40 PM, Stefán Baxter <
> > stefan@activitystream.com>
> > > wrote:
> > >
> > >> NO!
> > >>
> > >> Thank you , will do that right now :)
> > >>
> > >> On Sun, Jul 19, 2015 at 5:38 PM, Jim Bates <jb...@maprtech.com>
> wrote:
> > >>
> > >>> Did you include a file drill-module.conf in your jar along with
> source
> > >>> files?
> > >>> On Jul 19, 2015 12:20 PM, "Stefán Baxter" <stefan@activitystream.com
> >
> > >>> wrote:
> > >>>
> > >>> > Hi,
> > >>> >
> > >>> > I'm trying to deploy a UDF that I have written according to the
> > >>> > documentation.
> > >>> >
> > >>> > I have also:
> > >>> >
> > >>> > 1. Copied the jar file
> > >>> > to jars/3rdparty
> > >>> >
> > >>> > 2. Changed the config "conf/drill-override.conf" to include:
> > >>> > drill.logical.function.package +=
> > >>> >
> > >>>
> ["org.apache.drill.exec.expr.fn.impl","com.activitystream.drill.udfs"]
> > >>> >
> > >>> > 3. Rerun the Drillbit
> > >>> > bin/drillbit.sh start
> > >>> >
> > >>> > The log shows no sign of the the UDF being loaded but if I use it
> in
> > a
> > >>> SQL
> > >>> > statement then this error is thrown:
> > >>> >
> > >>> > Error: PARSE ERROR: From line 1, column 184 to line 1, column 219:
> No
> > >>> match
> > >>> > found for function signature asRoundTimeStamp(<ANY>, <CHARACTER>)
> > >>> >
> > >>> >
> > >>> > The relevant part of the UDF source:
> > >>> >
> > >>> > package com.activitystream.drill.udfs;
> > >>> >
> > >>> > public class ASUserDefinedFunctions {
> > >>> >
> > >>> > @FunctionTemplate(name = "asRoundTimeStamp", scope =
> > >>> > FunctionTemplate.FunctionScope.SIMPLE, nulls =
> > >>> > FunctionTemplate.NullHandling.NULL_IF_NULL)
> > >>> >
> > >>> > public static class RoundTimeStamp implements DrillSimpleFunc
> > >>> >
> > >>> >
> > >>> > Can anyone please assist?
> > >>> >
> > >>> > Regards,
> > >>> > -Stefan
> > >>> >
> > >>>
> > >>
> > >>
> > >
> >
>
Re: Drill not picking up a UDF
Posted by Jim Bates <jb...@maprtech.com>.
Maven will typically create a jar for class and a jar for source when told
to do so. I just include the source files in the same jar as the class
files. There is a github example drill udf project we are working on to
include several examples to simplify the learning curve. If your
interested... I'd love to have you add your udf.
On Jul 19, 2015 12:59 PM, "Stefán Baxter" <st...@activitystream.com> wrote:
> Hi again,
>
> Going over the documentation once more I came across this:
>
> - Add the sources and classes JAR files to Drill’s classpath.
>
> I'm only including a standard .jar (with classes no sources) and I wonder
> why this dependency is listed.
>
> Anyways, that it the next step.
>
> A sample .jar that works with the drill-config counterpart would be a
> welcome addition to the documentation.
>
> Regards,
> -Stefan
>
>
> On Sun, Jul 19, 2015 at 5:53 PM, Stefán Baxter <st...@activitystream.com>
> wrote:
>
> > Hi Jim,
> >
> > Now I have added the file to the jar (both root and resources folder) but
> > that does not seem to change anything.
> >
> > Any additional ideas?
> >
> > Regards,
> > -Stefan
> >
> > On Sun, Jul 19, 2015 at 5:40 PM, Stefán Baxter <
> stefan@activitystream.com>
> > wrote:
> >
> >> NO!
> >>
> >> Thank you , will do that right now :)
> >>
> >> On Sun, Jul 19, 2015 at 5:38 PM, Jim Bates <jb...@maprtech.com> wrote:
> >>
> >>> Did you include a file drill-module.conf in your jar along with source
> >>> files?
> >>> On Jul 19, 2015 12:20 PM, "Stefán Baxter" <st...@activitystream.com>
> >>> wrote:
> >>>
> >>> > Hi,
> >>> >
> >>> > I'm trying to deploy a UDF that I have written according to the
> >>> > documentation.
> >>> >
> >>> > I have also:
> >>> >
> >>> > 1. Copied the jar file
> >>> > to jars/3rdparty
> >>> >
> >>> > 2. Changed the config "conf/drill-override.conf" to include:
> >>> > drill.logical.function.package +=
> >>> >
> >>> ["org.apache.drill.exec.expr.fn.impl","com.activitystream.drill.udfs"]
> >>> >
> >>> > 3. Rerun the Drillbit
> >>> > bin/drillbit.sh start
> >>> >
> >>> > The log shows no sign of the the UDF being loaded but if I use it in
> a
> >>> SQL
> >>> > statement then this error is thrown:
> >>> >
> >>> > Error: PARSE ERROR: From line 1, column 184 to line 1, column 219: No
> >>> match
> >>> > found for function signature asRoundTimeStamp(<ANY>, <CHARACTER>)
> >>> >
> >>> >
> >>> > The relevant part of the UDF source:
> >>> >
> >>> > package com.activitystream.drill.udfs;
> >>> >
> >>> > public class ASUserDefinedFunctions {
> >>> >
> >>> > @FunctionTemplate(name = "asRoundTimeStamp", scope =
> >>> > FunctionTemplate.FunctionScope.SIMPLE, nulls =
> >>> > FunctionTemplate.NullHandling.NULL_IF_NULL)
> >>> >
> >>> > public static class RoundTimeStamp implements DrillSimpleFunc
> >>> >
> >>> >
> >>> > Can anyone please assist?
> >>> >
> >>> > Regards,
> >>> > -Stefan
> >>> >
> >>>
> >>
> >>
> >
>
Re: Drill not picking up a UDF
Posted by Stefán Baxter <st...@activitystream.com>.
Hi,
Building it using the maven-source-plugin (source:jar) and copying that
.jar file over to "drill/jar/3rdparty" changes nothing :(.
The .jar file now includes the empty drill-module.conf missing earlier.
This is all getting a tiny bit frustrating and any additional assistance is
welcomed :).
Regards,
-Stefan
On Sun, Jul 19, 2015 at 5:59 PM, Stefán Baxter <st...@activitystream.com>
wrote:
> Hi again,
>
> Going over the documentation once more I came across this:
>
> - Add the sources and classes JAR files to Drill’s classpath.
>
> I'm only including a standard .jar (with classes no sources) and I wonder
> why this dependency is listed.
>
> Anyways, that it the next step.
>
> A sample .jar that works with the drill-config counterpart would be a
> welcome addition to the documentation.
>
> Regards,
> -Stefan
>
>
> On Sun, Jul 19, 2015 at 5:53 PM, Stefán Baxter <st...@activitystream.com>
> wrote:
>
>> Hi Jim,
>>
>> Now I have added the file to the jar (both root and resources folder) but
>> that does not seem to change anything.
>>
>> Any additional ideas?
>>
>> Regards,
>> -Stefan
>>
>> On Sun, Jul 19, 2015 at 5:40 PM, Stefán Baxter <stefan@activitystream.com
>> > wrote:
>>
>>> NO!
>>>
>>> Thank you , will do that right now :)
>>>
>>> On Sun, Jul 19, 2015 at 5:38 PM, Jim Bates <jb...@maprtech.com> wrote:
>>>
>>>> Did you include a file drill-module.conf in your jar along with source
>>>> files?
>>>> On Jul 19, 2015 12:20 PM, "Stefán Baxter" <st...@activitystream.com>
>>>> wrote:
>>>>
>>>> > Hi,
>>>> >
>>>> > I'm trying to deploy a UDF that I have written according to the
>>>> > documentation.
>>>> >
>>>> > I have also:
>>>> >
>>>> > 1. Copied the jar file
>>>> > to jars/3rdparty
>>>> >
>>>> > 2. Changed the config "conf/drill-override.conf" to include:
>>>> > drill.logical.function.package +=
>>>> >
>>>> ["org.apache.drill.exec.expr.fn.impl","com.activitystream.drill.udfs"]
>>>> >
>>>> > 3. Rerun the Drillbit
>>>> > bin/drillbit.sh start
>>>> >
>>>> > The log shows no sign of the the UDF being loaded but if I use it in
>>>> a SQL
>>>> > statement then this error is thrown:
>>>> >
>>>> > Error: PARSE ERROR: From line 1, column 184 to line 1, column 219: No
>>>> match
>>>> > found for function signature asRoundTimeStamp(<ANY>, <CHARACTER>)
>>>> >
>>>> >
>>>> > The relevant part of the UDF source:
>>>> >
>>>> > package com.activitystream.drill.udfs;
>>>> >
>>>> > public class ASUserDefinedFunctions {
>>>> >
>>>> > @FunctionTemplate(name = "asRoundTimeStamp", scope =
>>>> > FunctionTemplate.FunctionScope.SIMPLE, nulls =
>>>> > FunctionTemplate.NullHandling.NULL_IF_NULL)
>>>> >
>>>> > public static class RoundTimeStamp implements DrillSimpleFunc
>>>> >
>>>> >
>>>> > Can anyone please assist?
>>>> >
>>>> > Regards,
>>>> > -Stefan
>>>> >
>>>>
>>>
>>>
>>
>
Re: Drill not picking up a UDF
Posted by Stefán Baxter <st...@activitystream.com>.
Hi again,
Going over the documentation once more I came across this:
- Add the sources and classes JAR files to Drill’s classpath.
I'm only including a standard .jar (with classes no sources) and I wonder
why this dependency is listed.
Anyways, that it the next step.
A sample .jar that works with the drill-config counterpart would be a
welcome addition to the documentation.
Regards,
-Stefan
On Sun, Jul 19, 2015 at 5:53 PM, Stefán Baxter <st...@activitystream.com>
wrote:
> Hi Jim,
>
> Now I have added the file to the jar (both root and resources folder) but
> that does not seem to change anything.
>
> Any additional ideas?
>
> Regards,
> -Stefan
>
> On Sun, Jul 19, 2015 at 5:40 PM, Stefán Baxter <st...@activitystream.com>
> wrote:
>
>> NO!
>>
>> Thank you , will do that right now :)
>>
>> On Sun, Jul 19, 2015 at 5:38 PM, Jim Bates <jb...@maprtech.com> wrote:
>>
>>> Did you include a file drill-module.conf in your jar along with source
>>> files?
>>> On Jul 19, 2015 12:20 PM, "Stefán Baxter" <st...@activitystream.com>
>>> wrote:
>>>
>>> > Hi,
>>> >
>>> > I'm trying to deploy a UDF that I have written according to the
>>> > documentation.
>>> >
>>> > I have also:
>>> >
>>> > 1. Copied the jar file
>>> > to jars/3rdparty
>>> >
>>> > 2. Changed the config "conf/drill-override.conf" to include:
>>> > drill.logical.function.package +=
>>> >
>>> ["org.apache.drill.exec.expr.fn.impl","com.activitystream.drill.udfs"]
>>> >
>>> > 3. Rerun the Drillbit
>>> > bin/drillbit.sh start
>>> >
>>> > The log shows no sign of the the UDF being loaded but if I use it in a
>>> SQL
>>> > statement then this error is thrown:
>>> >
>>> > Error: PARSE ERROR: From line 1, column 184 to line 1, column 219: No
>>> match
>>> > found for function signature asRoundTimeStamp(<ANY>, <CHARACTER>)
>>> >
>>> >
>>> > The relevant part of the UDF source:
>>> >
>>> > package com.activitystream.drill.udfs;
>>> >
>>> > public class ASUserDefinedFunctions {
>>> >
>>> > @FunctionTemplate(name = "asRoundTimeStamp", scope =
>>> > FunctionTemplate.FunctionScope.SIMPLE, nulls =
>>> > FunctionTemplate.NullHandling.NULL_IF_NULL)
>>> >
>>> > public static class RoundTimeStamp implements DrillSimpleFunc
>>> >
>>> >
>>> > Can anyone please assist?
>>> >
>>> > Regards,
>>> > -Stefan
>>> >
>>>
>>
>>
>
Re: Drill not picking up a UDF
Posted by Stefán Baxter <st...@activitystream.com>.
Hi Jim,
Now I have added the file to the jar (both root and resources folder) but
that does not seem to change anything.
Any additional ideas?
Regards,
-Stefan
On Sun, Jul 19, 2015 at 5:40 PM, Stefán Baxter <st...@activitystream.com>
wrote:
> NO!
>
> Thank you , will do that right now :)
>
> On Sun, Jul 19, 2015 at 5:38 PM, Jim Bates <jb...@maprtech.com> wrote:
>
>> Did you include a file drill-module.conf in your jar along with source
>> files?
>> On Jul 19, 2015 12:20 PM, "Stefán Baxter" <st...@activitystream.com>
>> wrote:
>>
>> > Hi,
>> >
>> > I'm trying to deploy a UDF that I have written according to the
>> > documentation.
>> >
>> > I have also:
>> >
>> > 1. Copied the jar file
>> > to jars/3rdparty
>> >
>> > 2. Changed the config "conf/drill-override.conf" to include:
>> > drill.logical.function.package +=
>> >
>> ["org.apache.drill.exec.expr.fn.impl","com.activitystream.drill.udfs"]
>> >
>> > 3. Rerun the Drillbit
>> > bin/drillbit.sh start
>> >
>> > The log shows no sign of the the UDF being loaded but if I use it in a
>> SQL
>> > statement then this error is thrown:
>> >
>> > Error: PARSE ERROR: From line 1, column 184 to line 1, column 219: No
>> match
>> > found for function signature asRoundTimeStamp(<ANY>, <CHARACTER>)
>> >
>> >
>> > The relevant part of the UDF source:
>> >
>> > package com.activitystream.drill.udfs;
>> >
>> > public class ASUserDefinedFunctions {
>> >
>> > @FunctionTemplate(name = "asRoundTimeStamp", scope =
>> > FunctionTemplate.FunctionScope.SIMPLE, nulls =
>> > FunctionTemplate.NullHandling.NULL_IF_NULL)
>> >
>> > public static class RoundTimeStamp implements DrillSimpleFunc
>> >
>> >
>> > Can anyone please assist?
>> >
>> > Regards,
>> > -Stefan
>> >
>>
>
>
Re: Drill not picking up a UDF
Posted by Stefán Baxter <st...@activitystream.com>.
NO!
Thank you , will do that right now :)
On Sun, Jul 19, 2015 at 5:38 PM, Jim Bates <jb...@maprtech.com> wrote:
> Did you include a file drill-module.conf in your jar along with source
> files?
> On Jul 19, 2015 12:20 PM, "Stefán Baxter" <st...@activitystream.com>
> wrote:
>
> > Hi,
> >
> > I'm trying to deploy a UDF that I have written according to the
> > documentation.
> >
> > I have also:
> >
> > 1. Copied the jar file
> > to jars/3rdparty
> >
> > 2. Changed the config "conf/drill-override.conf" to include:
> > drill.logical.function.package +=
> > ["org.apache.drill.exec.expr.fn.impl","com.activitystream.drill.udfs"]
> >
> > 3. Rerun the Drillbit
> > bin/drillbit.sh start
> >
> > The log shows no sign of the the UDF being loaded but if I use it in a
> SQL
> > statement then this error is thrown:
> >
> > Error: PARSE ERROR: From line 1, column 184 to line 1, column 219: No
> match
> > found for function signature asRoundTimeStamp(<ANY>, <CHARACTER>)
> >
> >
> > The relevant part of the UDF source:
> >
> > package com.activitystream.drill.udfs;
> >
> > public class ASUserDefinedFunctions {
> >
> > @FunctionTemplate(name = "asRoundTimeStamp", scope =
> > FunctionTemplate.FunctionScope.SIMPLE, nulls =
> > FunctionTemplate.NullHandling.NULL_IF_NULL)
> >
> > public static class RoundTimeStamp implements DrillSimpleFunc
> >
> >
> > Can anyone please assist?
> >
> > Regards,
> > -Stefan
> >
>
Re: Drill not picking up a UDF
Posted by Jim Bates <jb...@maprtech.com>.
Did you include a file drill-module.conf in your jar along with source
files?
On Jul 19, 2015 12:20 PM, "Stefán Baxter" <st...@activitystream.com> wrote:
> Hi,
>
> I'm trying to deploy a UDF that I have written according to the
> documentation.
>
> I have also:
>
> 1. Copied the jar file
> to jars/3rdparty
>
> 2. Changed the config "conf/drill-override.conf" to include:
> drill.logical.function.package +=
> ["org.apache.drill.exec.expr.fn.impl","com.activitystream.drill.udfs"]
>
> 3. Rerun the Drillbit
> bin/drillbit.sh start
>
> The log shows no sign of the the UDF being loaded but if I use it in a SQL
> statement then this error is thrown:
>
> Error: PARSE ERROR: From line 1, column 184 to line 1, column 219: No match
> found for function signature asRoundTimeStamp(<ANY>, <CHARACTER>)
>
>
> The relevant part of the UDF source:
>
> package com.activitystream.drill.udfs;
>
> public class ASUserDefinedFunctions {
>
> @FunctionTemplate(name = "asRoundTimeStamp", scope =
> FunctionTemplate.FunctionScope.SIMPLE, nulls =
> FunctionTemplate.NullHandling.NULL_IF_NULL)
>
> public static class RoundTimeStamp implements DrillSimpleFunc
>
>
> Can anyone please assist?
>
> Regards,
> -Stefan
>