You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@drill.apache.org by Joseph Swingle <jo...@gmail.com> on 2016/11/09 16:02:39 UTC
Need help with UDF/Custom Functions (DRILL-4776)
Drill team,
Are Custom Functions simply broke in Version 1.8?
Any movement on DRILL-4776 would be appreciated.
Heck, willing to even fix all the doc, provided someone can provide me a
Custom Function that works in 1.8.
https://issues.apache.org/jira/browse/DRILL-4776
Re: Need help with UDF/Custom Functions (DRILL-4776)
Posted by Arina Yelchiyeva <ar...@gmail.com>.
Hi Joseph,
I assume in drill-module.conf you indicate path to your package where
function class is in (ex: drill.classpath.scanning.packages +=
"com.drill.udf.md5"),
and after adding your jars to the classpath ($DRILL/jars), you restart your
drillbit.
Also you need to update your code:
1. do not use imports, use fully qualified class name for HashFunction,
HashCode and StandardCharsets.
2. annotate class variable HashFunction with @Workspace annotation:
@Workspace
com.google.common.hash.HashFunction hashFunction;
Hope this helps.
Kind regards
Arina
On Sun, Nov 20, 2016 at 2:20 AM, Charles Givre <cg...@gmail.com> wrote:
> Hi Joseph,
> Can you share your pom.xml? Also what command are you using to build your
> UDF?
> Thanks,
> — Charles
>
>
> > On Nov 18, 2016, at 14:44, Joseph Swingle <jo...@gmail.com> wrote:
> >
> > Hi Arina,
> > I apologize for not getting back to you sooner, and if this doesn't get
> > formatted properly. I just want to MD5 a string in a query.
> > I have pasted the java code for the function below. I included
> DRILL-4776
> > in my initial question, because it pretty much summed up my frustration
> > trying to deploy this function.
> >
> > Here are the additional things I have done.
> >
> > 1. drill-module.conf. I have tried including in my jars. I have
> tried
> > including it empty in my jar, I have added to $DRILL/conf folder.
> > 2. I have both the classes and sources jars in $DRILL/jars. I have
> even
> > tried putting them in $DRILL/jars/3rdParty
> > 3. Source hasn't changed it is below.
> >
> > A simple query like this one: SELECT (`finish` - `start`) as
> > `taken`,`start`,finish, outcome, username, to_md5(queryText)
> > FROM drillprofiles.`default`.`drillbit_queries.json` limit 2;
> >
> > give following error:
> > Error: VALIDATION ERROR: From line 1, column 75 to line 1, column 91: No
> > match found for function signature to_md5(<ANY>)
> >
> >
> > @FunctionTemplate( name = "to_md5", scope = FunctionTemplate.
> FunctionScope.
> > SIMPLE, nulls = FunctionTemplate.NullHandling.NULL_IF_NULL)
> >
> > public class MD5MaskFunc implements DrillSimpleFunc {
> >
> > @Param
> >
> > NullableVarCharHolder input;
> >
> > @Output
> >
> > VarCharHolder out;
> >
> > @Inject
> >
> > DrillBuf buffer;
> >
> > HashFunction hashFunction;
> >
> > public void setup() {
> >
> > hashFunction = Hashing.md5();
> >
> > }
> >
> > public void eval() {
> >
> > String stringValue =
> > org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.
> toStringFromUTF8(
> > input.start, input.end, input.buffer);
> >
> > HashCode hashCode = hashFunction.hashString(stringValue,
> > StandardCharsets.UTF_8);//hashFunction.hashString(stringValue);
> >
> > String outputValue = hashCode.toString();
> >
> > // put the output value in the out buffer
> >
> > out.buffer = buffer;
> >
> > out.start = 0;
> >
> > out.end = outputValue.getBytes().length;
> >
> > buffer.setBytes(0, outputValue.getBytes());
> >
> > }
> >
> > }
> >
> >
> >
> > On Wed, Nov 9, 2016 at 11:09 AM, Arina Yelchiyeva <
> > arina.yelchiyeva@gmail.com> wrote:
> >
> >> Hi Joseph,
> >>
> >> custom functions work should work fine in 1.8. At least I am not aware
> of
> >> any issues.
> >> Can you specify what problems with custom UDFs do you have?
> >>
> >> Kind regards
> >> Arina
> >>
> >> On Wed, Nov 9, 2016 at 6:02 PM, Joseph Swingle <jo...@gmail.com>
> >> wrote:
> >>
> >>> Drill team,
> >>>
> >>> Are Custom Functions simply broke in Version 1.8?
> >>>
> >>> Any movement on DRILL-4776 would be appreciated.
> >>>
> >>> Heck, willing to even fix all the doc, provided someone can provide
> me
> >> a
> >>> Custom Function that works in 1.8.
> >>>
> >>> https://issues.apache.org/jira/browse/DRILL-4776
> >>>
> >>
>
>
Re: Need help with UDF/Custom Functions (DRILL-4776)
Posted by Joseph Swingle <jo...@gmail.com>.
Sure thing. I'll get on it.
Re: Need help with UDF/Custom Functions (DRILL-4776)
Posted by Charles Givre <cg...@gmail.com>.
Joe,
Glad you got it to work. Mine wouldn’t even build with the hyphen in the package name. Not sure why. Would you consider posting this function on github or submitting a pull-request to have it incorporated in the next version of Drill?
> On Nov 22, 2016, at 13:20, Joseph Swingle <jo...@gmail.com> wrote:
>
> Charles and Arina,
> Thank you for the help. I'm guessing I had 2 issues.
>
> 1. @Workspace annotation missing. Is this required?
> 2. the drill-module.conf. I had mine including the class, based on one
> of the examples I saw. The one you provided Charles, just had the java
> package.
>
> I also moved the jars into the $DRILL_HOME/jars folder.
>
> Having the hyphen in the package has no effect.
>
> Thanks for the help.
>
> Joe
>
> On Sun, Nov 20, 2016 at 11:37 PM, Charles Givre <cg...@gmail.com> wrote:
>
>> Hi Joseph,
>> I’m not sure what I did exactly, but this works. One thing I found was
>> that my system didn’t seem to like the ‘-‘ character in the package name.
>>
>>
>>
>>
>>
>>
>>
>> On Nov 20, 2016, at 15:16, Joseph Swingle <jo...@gmail.com> wrote:
>>
>> Thanks Charles.
>> executing mvn clean package
>>
>> POM is attached.
>>
>>
>> On Sat, Nov 19, 2016 at 7:20 PM, Charles Givre <cg...@gmail.com> wrote:
>>
>>> Hi Joseph,
>>> Can you share your pom.xml? Also what command are you using to build
>>> your UDF?
>>> Thanks,
>>> — Charles
>>>
>>>
>>>> On Nov 18, 2016, at 14:44, Joseph Swingle <jo...@gmail.com> wrote:
>>>>
>>>> Hi Arina,
>>>> I apologize for not getting back to you sooner, and if this doesn't get
>>>> formatted properly. I just want to MD5 a string in a query.
>>>> I have pasted the java code for the function below. I included
>>> DRILL-4776
>>>> in my initial question, because it pretty much summed up my frustration
>>>> trying to deploy this function.
>>>>
>>>> Here are the additional things I have done.
>>>>
>>>> 1. drill-module.conf. I have tried including in my jars. I have
>>> tried
>>>> including it empty in my jar, I have added to $DRILL/conf folder.
>>>> 2. I have both the classes and sources jars in $DRILL/jars. I have
>>> even
>>>> tried putting them in $DRILL/jars/3rdParty
>>>> 3. Source hasn't changed it is below.
>>>>
>>>> A simple query like this one: SELECT (`finish` - `start`) as
>>>> `taken`,`start`,finish, outcome, username, to_md5(queryText)
>>>> FROM drillprofiles.`default`.`drillbit_queries.json` limit 2;
>>>>
>>>> give following error:
>>>> Error: VALIDATION ERROR: From line 1, column 75 to line 1, column 91: No
>>>> match found for function signature to_md5(<ANY>)
>>>>
>>>>
>>>> @FunctionTemplate( name = "to_md5", scope =
>>> FunctionTemplate.FunctionScope.
>>>> SIMPLE, nulls = FunctionTemplate.NullHandling.NULL_IF_NULL)
>>>>
>>>> public class MD5MaskFunc implements DrillSimpleFunc {
>>>>
>>>> @Param
>>>>
>>>> NullableVarCharHolder input;
>>>>
>>>> @Output
>>>>
>>>> VarCharHolder out;
>>>>
>>>> @Inject
>>>>
>>>> DrillBuf buffer;
>>>>
>>>> HashFunction hashFunction;
>>>>
>>>> public void setup() {
>>>>
>>>> hashFunction = Hashing.md5();
>>>>
>>>> }
>>>>
>>>> public void eval() {
>>>>
>>>> String stringValue =
>>>> org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toS
>>> tringFromUTF8(
>>>> input.start, input.end, input.buffer);
>>>>
>>>> HashCode hashCode = hashFunction.hashString(stringValue,
>>>> StandardCharsets.UTF_8);//hashFunction.hashString(stringValue);
>>>>
>>>> String outputValue = hashCode.toString();
>>>>
>>>> // put the output value in the out buffer
>>>>
>>>> out.buffer = buffer;
>>>>
>>>> out.start = 0;
>>>>
>>>> out.end = outputValue.getBytes().length;
>>>>
>>>> buffer.setBytes(0, outputValue.getBytes());
>>>>
>>>> }
>>>>
>>>> }
>>>>
>>>>
>>>>
>>>> On Wed, Nov 9, 2016 at 11:09 AM, Arina Yelchiyeva <
>>>> arina.yelchiyeva@gmail.com> wrote:
>>>>
>>>>> Hi Joseph,
>>>>>
>>>>> custom functions work should work fine in 1.8. At least I am not aware
>>> of
>>>>> any issues.
>>>>> Can you specify what problems with custom UDFs do you have?
>>>>>
>>>>> Kind regards
>>>>> Arina
>>>>>
>>>>> On Wed, Nov 9, 2016 at 6:02 PM, Joseph Swingle <jo...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Drill team,
>>>>>>
>>>>>> Are Custom Functions simply broke in Version 1.8?
>>>>>>
>>>>>> Any movement on DRILL-4776 would be appreciated.
>>>>>>
>>>>>> Heck, willing to even fix all the doc, provided someone can provide
>>> me
>>>>> a
>>>>>> Custom Function that works in 1.8.
>>>>>>
>>>>>> https://issues.apache.org/jira/browse/DRILL-4776
>>>>>>
>>>>>
>>>
>>>
>> <pom.xml>
>>
>>
>>
>>
Re: Need help with UDF/Custom Functions (DRILL-4776)
Posted by Joseph Swingle <jo...@gmail.com>.
Charles and Arina,
Thank you for the help. I'm guessing I had 2 issues.
1. @Workspace annotation missing. Is this required?
2. the drill-module.conf. I had mine including the class, based on one
of the examples I saw. The one you provided Charles, just had the java
package.
I also moved the jars into the $DRILL_HOME/jars folder.
Having the hyphen in the package has no effect.
Thanks for the help.
Joe
On Sun, Nov 20, 2016 at 11:37 PM, Charles Givre <cg...@gmail.com> wrote:
> Hi Joseph,
> I’m not sure what I did exactly, but this works. One thing I found was
> that my system didn’t seem to like the ‘-‘ character in the package name.
>
>
>
>
>
>
>
> On Nov 20, 2016, at 15:16, Joseph Swingle <jo...@gmail.com> wrote:
>
> Thanks Charles.
> executing mvn clean package
>
> POM is attached.
>
>
> On Sat, Nov 19, 2016 at 7:20 PM, Charles Givre <cg...@gmail.com> wrote:
>
>> Hi Joseph,
>> Can you share your pom.xml? Also what command are you using to build
>> your UDF?
>> Thanks,
>> — Charles
>>
>>
>> > On Nov 18, 2016, at 14:44, Joseph Swingle <jo...@gmail.com> wrote:
>> >
>> > Hi Arina,
>> > I apologize for not getting back to you sooner, and if this doesn't get
>> > formatted properly. I just want to MD5 a string in a query.
>> > I have pasted the java code for the function below. I included
>> DRILL-4776
>> > in my initial question, because it pretty much summed up my frustration
>> > trying to deploy this function.
>> >
>> > Here are the additional things I have done.
>> >
>> > 1. drill-module.conf. I have tried including in my jars. I have
>> tried
>> > including it empty in my jar, I have added to $DRILL/conf folder.
>> > 2. I have both the classes and sources jars in $DRILL/jars. I have
>> even
>> > tried putting them in $DRILL/jars/3rdParty
>> > 3. Source hasn't changed it is below.
>> >
>> > A simple query like this one: SELECT (`finish` - `start`) as
>> > `taken`,`start`,finish, outcome, username, to_md5(queryText)
>> > FROM drillprofiles.`default`.`drillbit_queries.json` limit 2;
>> >
>> > give following error:
>> > Error: VALIDATION ERROR: From line 1, column 75 to line 1, column 91: No
>> > match found for function signature to_md5(<ANY>)
>> >
>> >
>> > @FunctionTemplate( name = "to_md5", scope =
>> FunctionTemplate.FunctionScope.
>> > SIMPLE, nulls = FunctionTemplate.NullHandling.NULL_IF_NULL)
>> >
>> > public class MD5MaskFunc implements DrillSimpleFunc {
>> >
>> > @Param
>> >
>> > NullableVarCharHolder input;
>> >
>> > @Output
>> >
>> > VarCharHolder out;
>> >
>> > @Inject
>> >
>> > DrillBuf buffer;
>> >
>> > HashFunction hashFunction;
>> >
>> > public void setup() {
>> >
>> > hashFunction = Hashing.md5();
>> >
>> > }
>> >
>> > public void eval() {
>> >
>> > String stringValue =
>> > org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toS
>> tringFromUTF8(
>> > input.start, input.end, input.buffer);
>> >
>> > HashCode hashCode = hashFunction.hashString(stringValue,
>> > StandardCharsets.UTF_8);//hashFunction.hashString(stringValue);
>> >
>> > String outputValue = hashCode.toString();
>> >
>> > // put the output value in the out buffer
>> >
>> > out.buffer = buffer;
>> >
>> > out.start = 0;
>> >
>> > out.end = outputValue.getBytes().length;
>> >
>> > buffer.setBytes(0, outputValue.getBytes());
>> >
>> > }
>> >
>> > }
>> >
>> >
>> >
>> > On Wed, Nov 9, 2016 at 11:09 AM, Arina Yelchiyeva <
>> > arina.yelchiyeva@gmail.com> wrote:
>> >
>> >> Hi Joseph,
>> >>
>> >> custom functions work should work fine in 1.8. At least I am not aware
>> of
>> >> any issues.
>> >> Can you specify what problems with custom UDFs do you have?
>> >>
>> >> Kind regards
>> >> Arina
>> >>
>> >> On Wed, Nov 9, 2016 at 6:02 PM, Joseph Swingle <jo...@gmail.com>
>> >> wrote:
>> >>
>> >>> Drill team,
>> >>>
>> >>> Are Custom Functions simply broke in Version 1.8?
>> >>>
>> >>> Any movement on DRILL-4776 would be appreciated.
>> >>>
>> >>> Heck, willing to even fix all the doc, provided someone can provide
>> me
>> >> a
>> >>> Custom Function that works in 1.8.
>> >>>
>> >>> https://issues.apache.org/jira/browse/DRILL-4776
>> >>>
>> >>
>>
>>
> <pom.xml>
>
>
>
>
Re: Need help with UDF/Custom Functions (DRILL-4776)
Posted by Charles Givre <cg...@gmail.com>.
Hi Joseph,
I’m not sure what I did exactly, but this works. One thing I found was that my system didn’t seem to like the ‘-‘ character in the package name.
> On Nov 20, 2016, at 15:16, Joseph Swingle <joeswingle@gmail.com <ma...@gmail.com>> wrote:
>
> Thanks Charles.
> executing mvn clean package
>
> POM is attached.
>
>
> On Sat, Nov 19, 2016 at 7:20 PM, Charles Givre <cgivre@gmail.com <ma...@gmail.com>> wrote:
> Hi Joseph,
> Can you share your pom.xml? Also what command are you using to build your UDF?
> Thanks,
> — Charles
>
>
> > On Nov 18, 2016, at 14:44, Joseph Swingle <joeswingle@gmail.com <ma...@gmail.com>> wrote:
> >
> > Hi Arina,
> > I apologize for not getting back to you sooner, and if this doesn't get
> > formatted properly. I just want to MD5 a string in a query.
> > I have pasted the java code for the function below. I included DRILL-4776
> > in my initial question, because it pretty much summed up my frustration
> > trying to deploy this function.
> >
> > Here are the additional things I have done.
> >
> > 1. drill-module.conf. I have tried including in my jars. I have tried
> > including it empty in my jar, I have added to $DRILL/conf folder.
> > 2. I have both the classes and sources jars in $DRILL/jars. I have even
> > tried putting them in $DRILL/jars/3rdParty
> > 3. Source hasn't changed it is below.
> >
> > A simple query like this one: SELECT (`finish` - `start`) as
> > `taken`,`start`,finish, outcome, username, to_md5(queryText)
> > FROM drillprofiles.`default`.`drillbit_queries.json` limit 2;
> >
> > give following error:
> > Error: VALIDATION ERROR: From line 1, column 75 to line 1, column 91: No
> > match found for function signature to_md5(<ANY>)
> >
> >
> > @FunctionTemplate( name = "to_md5", scope = FunctionTemplate.FunctionScope.
> > SIMPLE, nulls = FunctionTemplate.NullHandling.NULL_IF_NULL)
> >
> > public class MD5MaskFunc implements DrillSimpleFunc {
> >
> > @Param
> >
> > NullableVarCharHolder input;
> >
> > @Output
> >
> > VarCharHolder out;
> >
> > @Inject
> >
> > DrillBuf buffer;
> >
> > HashFunction hashFunction;
> >
> > public void setup() {
> >
> > hashFunction = Hashing.md5();
> >
> > }
> >
> > public void eval() {
> >
> > String stringValue =
> > org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(
> > input.start, input.end, input.buffer);
> >
> > HashCode hashCode = hashFunction.hashString(stringValue,
> > StandardCharsets.UTF_8);//hashFunction.hashString(stringValue);
> >
> > String outputValue = hashCode.toString();
> >
> > // put the output value in the out buffer
> >
> > out.buffer = buffer;
> >
> > out.start = 0;
> >
> > out.end = outputValue.getBytes().length;
> >
> > buffer.setBytes(0, outputValue.getBytes());
> >
> > }
> >
> > }
> >
> >
> >
> > On Wed, Nov 9, 2016 at 11:09 AM, Arina Yelchiyeva <
> > arina.yelchiyeva@gmail.com <ma...@gmail.com>> wrote:
> >
> >> Hi Joseph,
> >>
> >> custom functions work should work fine in 1.8. At least I am not aware of
> >> any issues.
> >> Can you specify what problems with custom UDFs do you have?
> >>
> >> Kind regards
> >> Arina
> >>
> >> On Wed, Nov 9, 2016 at 6:02 PM, Joseph Swingle <joeswingle@gmail.com <ma...@gmail.com>>
> >> wrote:
> >>
> >>> Drill team,
> >>>
> >>> Are Custom Functions simply broke in Version 1.8?
> >>>
> >>> Any movement on DRILL-4776 would be appreciated.
> >>>
> >>> Heck, willing to even fix all the doc, provided someone can provide me
> >> a
> >>> Custom Function that works in 1.8.
> >>>
> >>> https://issues.apache.org/jira/browse/DRILL-4776 <https://issues.apache.org/jira/browse/DRILL-4776>
> >>>
> >>
>
>
> <pom.xml>
Re: Need help with UDF/Custom Functions (DRILL-4776)
Posted by Joseph Swingle <jo...@gmail.com>.
Thanks Charles.
executing mvn clean package
POM is attached.
On Sat, Nov 19, 2016 at 7:20 PM, Charles Givre <cg...@gmail.com> wrote:
> Hi Joseph,
> Can you share your pom.xml? Also what command are you using to build your
> UDF?
> Thanks,
> — Charles
>
>
> > On Nov 18, 2016, at 14:44, Joseph Swingle <jo...@gmail.com> wrote:
> >
> > Hi Arina,
> > I apologize for not getting back to you sooner, and if this doesn't get
> > formatted properly. I just want to MD5 a string in a query.
> > I have pasted the java code for the function below. I included
> DRILL-4776
> > in my initial question, because it pretty much summed up my frustration
> > trying to deploy this function.
> >
> > Here are the additional things I have done.
> >
> > 1. drill-module.conf. I have tried including in my jars. I have
> tried
> > including it empty in my jar, I have added to $DRILL/conf folder.
> > 2. I have both the classes and sources jars in $DRILL/jars. I have
> even
> > tried putting them in $DRILL/jars/3rdParty
> > 3. Source hasn't changed it is below.
> >
> > A simple query like this one: SELECT (`finish` - `start`) as
> > `taken`,`start`,finish, outcome, username, to_md5(queryText)
> > FROM drillprofiles.`default`.`drillbit_queries.json` limit 2;
> >
> > give following error:
> > Error: VALIDATION ERROR: From line 1, column 75 to line 1, column 91: No
> > match found for function signature to_md5(<ANY>)
> >
> >
> > @FunctionTemplate( name = "to_md5", scope = FunctionTemplate.
> FunctionScope.
> > SIMPLE, nulls = FunctionTemplate.NullHandling.NULL_IF_NULL)
> >
> > public class MD5MaskFunc implements DrillSimpleFunc {
> >
> > @Param
> >
> > NullableVarCharHolder input;
> >
> > @Output
> >
> > VarCharHolder out;
> >
> > @Inject
> >
> > DrillBuf buffer;
> >
> > HashFunction hashFunction;
> >
> > public void setup() {
> >
> > hashFunction = Hashing.md5();
> >
> > }
> >
> > public void eval() {
> >
> > String stringValue =
> > org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.
> toStringFromUTF8(
> > input.start, input.end, input.buffer);
> >
> > HashCode hashCode = hashFunction.hashString(stringValue,
> > StandardCharsets.UTF_8);//hashFunction.hashString(stringValue);
> >
> > String outputValue = hashCode.toString();
> >
> > // put the output value in the out buffer
> >
> > out.buffer = buffer;
> >
> > out.start = 0;
> >
> > out.end = outputValue.getBytes().length;
> >
> > buffer.setBytes(0, outputValue.getBytes());
> >
> > }
> >
> > }
> >
> >
> >
> > On Wed, Nov 9, 2016 at 11:09 AM, Arina Yelchiyeva <
> > arina.yelchiyeva@gmail.com> wrote:
> >
> >> Hi Joseph,
> >>
> >> custom functions work should work fine in 1.8. At least I am not aware
> of
> >> any issues.
> >> Can you specify what problems with custom UDFs do you have?
> >>
> >> Kind regards
> >> Arina
> >>
> >> On Wed, Nov 9, 2016 at 6:02 PM, Joseph Swingle <jo...@gmail.com>
> >> wrote:
> >>
> >>> Drill team,
> >>>
> >>> Are Custom Functions simply broke in Version 1.8?
> >>>
> >>> Any movement on DRILL-4776 would be appreciated.
> >>>
> >>> Heck, willing to even fix all the doc, provided someone can provide
> me
> >> a
> >>> Custom Function that works in 1.8.
> >>>
> >>> https://issues.apache.org/jira/browse/DRILL-4776
> >>>
> >>
>
>
Re: Need help with UDF/Custom Functions (DRILL-4776)
Posted by Charles Givre <cg...@gmail.com>.
Hi Joseph,
Can you share your pom.xml? Also what command are you using to build your UDF?
Thanks,
— Charles
> On Nov 18, 2016, at 14:44, Joseph Swingle <jo...@gmail.com> wrote:
>
> Hi Arina,
> I apologize for not getting back to you sooner, and if this doesn't get
> formatted properly. I just want to MD5 a string in a query.
> I have pasted the java code for the function below. I included DRILL-4776
> in my initial question, because it pretty much summed up my frustration
> trying to deploy this function.
>
> Here are the additional things I have done.
>
> 1. drill-module.conf. I have tried including in my jars. I have tried
> including it empty in my jar, I have added to $DRILL/conf folder.
> 2. I have both the classes and sources jars in $DRILL/jars. I have even
> tried putting them in $DRILL/jars/3rdParty
> 3. Source hasn't changed it is below.
>
> A simple query like this one: SELECT (`finish` - `start`) as
> `taken`,`start`,finish, outcome, username, to_md5(queryText)
> FROM drillprofiles.`default`.`drillbit_queries.json` limit 2;
>
> give following error:
> Error: VALIDATION ERROR: From line 1, column 75 to line 1, column 91: No
> match found for function signature to_md5(<ANY>)
>
>
> @FunctionTemplate( name = "to_md5", scope = FunctionTemplate.FunctionScope.
> SIMPLE, nulls = FunctionTemplate.NullHandling.NULL_IF_NULL)
>
> public class MD5MaskFunc implements DrillSimpleFunc {
>
> @Param
>
> NullableVarCharHolder input;
>
> @Output
>
> VarCharHolder out;
>
> @Inject
>
> DrillBuf buffer;
>
> HashFunction hashFunction;
>
> public void setup() {
>
> hashFunction = Hashing.md5();
>
> }
>
> public void eval() {
>
> String stringValue =
> org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(
> input.start, input.end, input.buffer);
>
> HashCode hashCode = hashFunction.hashString(stringValue,
> StandardCharsets.UTF_8);//hashFunction.hashString(stringValue);
>
> String outputValue = hashCode.toString();
>
> // put the output value in the out buffer
>
> out.buffer = buffer;
>
> out.start = 0;
>
> out.end = outputValue.getBytes().length;
>
> buffer.setBytes(0, outputValue.getBytes());
>
> }
>
> }
>
>
>
> On Wed, Nov 9, 2016 at 11:09 AM, Arina Yelchiyeva <
> arina.yelchiyeva@gmail.com> wrote:
>
>> Hi Joseph,
>>
>> custom functions work should work fine in 1.8. At least I am not aware of
>> any issues.
>> Can you specify what problems with custom UDFs do you have?
>>
>> Kind regards
>> Arina
>>
>> On Wed, Nov 9, 2016 at 6:02 PM, Joseph Swingle <jo...@gmail.com>
>> wrote:
>>
>>> Drill team,
>>>
>>> Are Custom Functions simply broke in Version 1.8?
>>>
>>> Any movement on DRILL-4776 would be appreciated.
>>>
>>> Heck, willing to even fix all the doc, provided someone can provide me
>> a
>>> Custom Function that works in 1.8.
>>>
>>> https://issues.apache.org/jira/browse/DRILL-4776
>>>
>>
Re: Need help with UDF/Custom Functions (DRILL-4776)
Posted by Joseph Swingle <jo...@gmail.com>.
Hi Arina,
I apologize for not getting back to you sooner, and if this doesn't get
formatted properly. I just want to MD5 a string in a query.
I have pasted the java code for the function below. I included DRILL-4776
in my initial question, because it pretty much summed up my frustration
trying to deploy this function.
Here are the additional things I have done.
1. drill-module.conf. I have tried including in my jars. I have tried
including it empty in my jar, I have added to $DRILL/conf folder.
2. I have both the classes and sources jars in $DRILL/jars. I have even
tried putting them in $DRILL/jars/3rdParty
3. Source hasn't changed it is below.
A simple query like this one: SELECT (`finish` - `start`) as
`taken`,`start`,finish, outcome, username, to_md5(queryText)
FROM drillprofiles.`default`.`drillbit_queries.json` limit 2;
give following error:
Error: VALIDATION ERROR: From line 1, column 75 to line 1, column 91: No
match found for function signature to_md5(<ANY>)
@FunctionTemplate( name = "to_md5", scope = FunctionTemplate.FunctionScope.
SIMPLE, nulls = FunctionTemplate.NullHandling.NULL_IF_NULL)
public class MD5MaskFunc implements DrillSimpleFunc {
@Param
NullableVarCharHolder input;
@Output
VarCharHolder out;
@Inject
DrillBuf buffer;
HashFunction hashFunction;
public void setup() {
hashFunction = Hashing.md5();
}
public void eval() {
String stringValue =
org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(
input.start, input.end, input.buffer);
HashCode hashCode = hashFunction.hashString(stringValue,
StandardCharsets.UTF_8);//hashFunction.hashString(stringValue);
String outputValue = hashCode.toString();
// put the output value in the out buffer
out.buffer = buffer;
out.start = 0;
out.end = outputValue.getBytes().length;
buffer.setBytes(0, outputValue.getBytes());
}
}
On Wed, Nov 9, 2016 at 11:09 AM, Arina Yelchiyeva <
arina.yelchiyeva@gmail.com> wrote:
> Hi Joseph,
>
> custom functions work should work fine in 1.8. At least I am not aware of
> any issues.
> Can you specify what problems with custom UDFs do you have?
>
> Kind regards
> Arina
>
> On Wed, Nov 9, 2016 at 6:02 PM, Joseph Swingle <jo...@gmail.com>
> wrote:
>
> > Drill team,
> >
> > Are Custom Functions simply broke in Version 1.8?
> >
> > Any movement on DRILL-4776 would be appreciated.
> >
> > Heck, willing to even fix all the doc, provided someone can provide me
> a
> > Custom Function that works in 1.8.
> >
> > https://issues.apache.org/jira/browse/DRILL-4776
> >
>
Re: Need help with UDF/Custom Functions (DRILL-4776)
Posted by Arina Yelchiyeva <ar...@gmail.com>.
Hi Joseph,
custom functions work should work fine in 1.8. At least I am not aware of
any issues.
Can you specify what problems with custom UDFs do you have?
Kind regards
Arina
On Wed, Nov 9, 2016 at 6:02 PM, Joseph Swingle <jo...@gmail.com> wrote:
> Drill team,
>
> Are Custom Functions simply broke in Version 1.8?
>
> Any movement on DRILL-4776 would be appreciated.
>
> Heck, willing to even fix all the doc, provided someone can provide me a
> Custom Function that works in 1.8.
>
> https://issues.apache.org/jira/browse/DRILL-4776
>