You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@drill.apache.org by Charles Givre <cg...@gmail.com> on 2018/07/05 15:39:41 UTC
Re: Deprecation of BaseTestQuery FYI
Hi Timothy,
I am still having issues with the unit tests for this particular function. The code is below. This function takes the soundex values of two strings and if they are equal is supposed to return true, if not returns false. I do this by setting the value of a BitHolder to either -1 or 0.
The code for the function appears to work, however, when I think what is causing the error here was the sounds_like function itself.
If I run the query:
select *
from
WHERE soundex( ‘brian’ ) = soundex(`first_name` )
This returns results
However, if I run the query
SELECT *
FROM <Data>
WEHRE sounds_like( ‘brian’, `first_name` )
I get no results. However, if I run this query:
SELECT sounds_like( ‘brian’, `first_name` )
FROM cp..
I get trues and falses. I’m not sure why this isn’t working properly and in the interests of not holding up the works, I’m removing the sounds_like from the PR, but I’d like to understand how to fix this.
@FunctionTemplate(name = "sounds_like", scope = FunctionTemplate.FunctionScope.SIMPLE, nulls = FunctionTemplate.NullHandling.NULL_IF_NULL)
public static class SoundsLikeFunction implements DrillSimpleFunc {
@Param
VarCharHolder rawInput1;
@Param
VarCharHolder rawInput2;
@Output
BitHolder out;
@Override
public void setup() {
}
@Override
public void eval() {
String input1 = org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(rawInput1.start, rawInput1.end, rawInput1.buffer);
String input2 = org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(rawInput2.start, rawInput2.end, rawInput2.buffer);
String soundex1 = new org.apache.commons.codec.language.Soundex().soundex(input1);
String soundex2 = new org.apache.commons.codec.language.Soundex().soundex(input2);
boolean result = soundex1.equals(soundex2);
if (result) {
out.value = -1;
} else {
out.value = 0;
}
}
}
jdbc:drill:zk=local> SELECT typeof(sounds_like( 'Steven', 'Stephen' )) AS steve FROM (VALUES(1));
Error: SYSTEM ERROR: IllegalArgumentException: Can not set org.apache.drill.exec.vector.complex.reader.FieldReader field org.apache.drill.exec.expr.fn.impl.UnionFunctions$GetType.input to org.apache.drill.exec.expr.holders.BitHolder
[Error Id: f15a91b9-ae19-481a-b145-1024709547e9 on 10.1.9.176:31010] (state=,code=0)
0: jdbc:drill:zk=local> SELECT typeof(sounds_like( 'Steven', 'Stephen' )) FROM (VALUES(1));
Error: SYSTEM ERROR: IllegalArgumentException: Can not set org.apache.drill.exec.vector.complex.reader.FieldReader field org.apache.drill.exec.expr.fn.impl.UnionFunctions$GetType.input to org.apache.drill.exec.expr.holders.BitHolder
[Error Id: 870a2471-8ab7-4a0f-ab7f-9ff873ba767a on 10.1.9.176:31010] (state=,code=0)
0: jdbc:drill:zk=local> SELECT sounds_like( 'Steven', 'Stephen' ) FROM (VALUES(1));
+---------+
| EXPR$0 |
+---------+
| true |
+---------+
1 row selected (0.689 seconds)
0: jdbc:drill:zk=local> SELECT sounds_like( 'Steven', 'bob' ) FROM (VALUES(1));
+---------+
| EXPR$0 |
+---------+
| false |
+---------+
1 row selected (0.175 seconds)
> On Jun 28, 2018, at 19:46, Timothy Farkas <tf...@mapr.com> wrote:
>
> Hi Charles,
>
> So it is actually supported. Drill's boolean vector is BitVector.
> Internally bits are stored efficiently, but when you fetch a bit from the
> vector it becomes an int, -1 for true and 0 for false. So currently you can
> check this by using singletonInt and comparing against -1 and 0.
>
> Here is a test snippet that does it. We should probably add some
> convenience methods specifically for booleans so this is not so confusing.
>
> public class SimpleQueryTest extends ClusterTest {
> @Rule
> public final BaseDirTestWatcher baseDirTestWatcher = new BaseDirTestWatcher();
>
> @Before
> public void start() throws Exception {
> startCluster(new ClusterFixtureBuilder(baseDirTestWatcher));
> }
>
> @Test
> public void testBoolean() throws RpcException {
> RowSet rowSet = this.queryBuilder().sql("select position_id,
> cast(false as boolean) from cp.`employee.json`").rowSet();
> RowSetReader reader = rowSet.reader();
> reader.next();
> System.out.println(reader.column(1).scalar().getInt());
> rowSet.clear();
>
> rowSet = this.queryBuilder().sql("select position_id, cast(true as
> boolean) from cp.`employee.json`").rowSet();
> reader = rowSet.reader();
> reader.next();
> System.out.println(reader.column(1).scalar().getInt());
> rowSet.clear();
> }
> }
>
>
>
> On Thu, Jun 28, 2018 at 4:09 PM, Timothy Farkas <tf...@mapr.com> wrote:
>
>> Hi Charles,
>>
>> Currently the RowSetReader doesn't support booleans, but it can be added.
>> I'll try to add it now, and see if it could be done quickly. I'll update
>> you on my progress.
>>
>> Tim
>>
>> On Thu, Jun 28, 2018 at 2:38 PM, Charles Givre <cg...@gmail.com> wrote:
>>
>>> Hi Tim,
>>> Could post some sample code as to how to test a SQL query that returns a
>>> Boolean?
>>> —C
>>>
>>>> On Jun 28, 2018, at 17:30, Timothy Farkas <tf...@mapr.com> wrote:
>>>>
>>>> - We would have to add a boolean column reader to ColumnAccessors and
>>> wire
>>>> it in and add a getBoolean method to ScalarReader.
>>>>
>>>> - Your example should work as is, ClusterTest has a testBuilder method
>>>> that allows you to use the traditional test builder. Is there something
>>> not
>>>> working with the test builder?
>>>>
>>>> Tim
>>>>
>>>>
>>>> On Thu, Jun 28, 2018 at 12:39 PM, Arina Yelchiyeva <
>>>> arina.yelchiyeva@gmail.com> wrote:
>>>>
>>>>> Hi Tim,
>>>>>
>>>>> it looks like deprecating BaseTestQuery was a little bit pre-mature.
>>>>> For example, from in this PR - https://urldefense.proofpoint.
>>> com/v2/url?u=https-3A__urldefense.proofpoint&d=DwIFaQ&c=cskd
>>> kSMqhcnjZxdQVpwTXg&r=4eQVr8zB8ZBff-yxTimdOQ&m=D9i95wxPKk0aPH
>>> ip2Rj6Xju0J151UakaSe6OnMGTW5s&s=z5tYyOUI_Cyx1cwhMEMfnCb-
>>> BDFaZkRaKQKwt_zl2HA&e=.
>>>>> com/v2/url?u=https-3A__github.com_apache_drill_pull_1331&d=DwIBaQ&c=
>>>>> cskdkSMqhcnjZxdQVpwTXg&r=4eQVr8zB8ZBff-yxTimdOQ&m=
>>>>> zoNJPdWKxMX9-jbR2bftzwkX-CSrihIbpCirhHM1kA0&s=_uxhA_
>>>>> qiMBTjbit6DDw-DvZNRGesfeA5g-FQjkl7f10&e= -
>>>>> Charles is trying to re-work BaseTestQuery usage to ClusterTest.
>>>>> First, it did not contain getSigletonDouble method which Charles has
>>>>> implemented. Now he has troubles with implementing getSigletonBoolean
>>>>> method which might be due to reader limitations.
>>>>> Also I am not quite clear how we can verify columns names and multiple
>>>>> columns in the result.
>>>>> For example:
>>>>>
>>>>> testBuilder()
>>>>> .sqlQuery("select (mi || lname) as CONCATOperator, mi, lname,
>>>>> concat(mi, lname) as CONCAT from concatNull")
>>>>> .ordered()
>>>>> .baselineColumns("CONCATOperator", "mi", "lname", "CONCAT")
>>>>> .baselineValues("A.Nowmer", "A.", "Nowmer", "A.Nowmer")
>>>>> .baselineValues("I.Whelply", "I.", "Whelply", "I.Whelply")
>>>>> .baselineValues(null, null, "Derry", "Derry")
>>>>> .baselineValues("J.Spence", "J.", "Spence", "J.Spence")
>>>>> .build().run();
>>>>>
>>>>> Can you please suggest how this example can be re-written?
>>>>>
>>>>> Kind regards,
>>>>> Arina
>>>>>
>>>>> On Mon, Jun 25, 2018 at 11:10 PM Timothy Farkas <tf...@mapr.com>
>>> wrote:
>>>>>
>>>>>> Hi All,
>>>>>>
>>>>>> BaseTestQuery was deprecated a while ago. Keeping it short and sweet
>>> :),
>>>>> if
>>>>>> you want to use BaseTestQuery directly, don't. Use ClusterTest
>>> instead.
>>>>> If
>>>>>> you are using PlanTestBase for planner tests, continue to do so.
>>>>> Eventually
>>>>>> PlanTestBase will be changed to extend ClusterTest instead. There is a
>>>>> JIRA
>>>>>> to track that issue https://urldefense.proofpoint.
>>> com/v2/url?u=https-3A__urldefense.proofpoint&d=DwIFaQ&c=cskd
>>> kSMqhcnjZxdQVpwTXg&r=4eQVr8zB8ZBff-yxTimdOQ&m=D9i95wxPKk0aPH
>>> ip2Rj6Xju0J151UakaSe6OnMGTW5s&s=z5tYyOUI_Cyx1cwhMEMfnCb-
>>> BDFaZkRaKQKwt_zl2HA&e=.
>>>>> com/v2/url?u=https-3A__issues.apache.org_jira_browse_DRILL-
>>>>> 2D6536&d=DwIBaQ&c=cskdkSMqhcnjZxdQVpwTXg&r=4eQVr8zB8ZBff-yxTimdOQ&m=
>>>>> zoNJPdWKxMX9-jbR2bftzwkX-CSrihIbpCirhHM1kA0&s=
>>>>> BPAlenAq0k1kjAz7fUYXyEQMaJM1IPOgmdeySMMY84U&e=.
>>>>>>
>>>>>> Thanks,
>>>>>> Tim
>>>>>>
>>>>>
>>>
>>>
>>
Re: Deprecation of BaseTestQuery FYI
Posted by Charles Givre <cg...@gmail.com>.
Hi Paul,
That was what I thought as well. The function doesn't seem to work properly either setting positive as 1 or -1. I looked at other UDFs that I’ve written in the past that return Boolean values and the implementation is almost exactly the same, so I’m a bit at a loss on this one.
— C
> On Jul 5, 2018, at 16:01, Paul Rogers <pa...@yahoo.com.INVALID> wrote:
>
> Hi Tim,
>
> My understanding is that Booleans are stored, as you said, as a UInt1 vector. But, the values are 0 and (positive) 1. As you said, singletonInt will read this value. If the value is coming back as -1, then this is a bug. I'd be surprised if it is since I wrote lots of code that assumed the value is 1.
>
> Given this, it would be a simple addition to the test QueryBuilder to add a "singletonBoolean" that returns singletonInt() != 0.
>
> Charles,
> You should not set the BitHolder to -1, you should set it to either 0 or (positive) 1.
>
> Thanks,
> - Paul
>
>
>
> On Thursday, July 5, 2018, 8:39:58 AM PDT, Charles Givre <cg...@gmail.com> wrote:
>
> Hi Timothy,
> I am still having issues with the unit tests for this particular function. The code is below. This function takes the soundex values of two strings and if they are equal is supposed to return true, if not returns false. I do this by setting the value of a BitHolder to either -1 or 0.
>
> The code for the function appears to work, however, when I think what is causing the error here was the sounds_like function itself.
> If I run the query:
> select *
> from
> WHERE soundex( ‘brian’ ) = soundex(`first_name` )
>
> This returns results
>
> However, if I run the query
> SELECT *
> FROM <Data>
> WEHRE sounds_like( ‘brian’, `first_name` )
> I get no results. However, if I run this query:
>
> SELECT sounds_like( ‘brian’, `first_name` )
> FROM cp..
>
> I get trues and falses. I’m not sure why this isn’t working properly and in the interests of not holding up the works, I’m removing the sounds_like from the PR, but I’d like to understand how to fix this.
>
>
> @FunctionTemplate(name = "sounds_like", scope = FunctionTemplate.FunctionScope.SIMPLE, nulls = FunctionTemplate.NullHandling.NULL_IF_NULL)
> public static class SoundsLikeFunction implements DrillSimpleFunc {
>
> @Param
> VarCharHolder rawInput1;
>
> @Param
> VarCharHolder rawInput2;
>
> @Output
> BitHolder out;
>
> @Override
> public void setup() {
> }
>
> @Override
> public void eval() {
>
> String input1 = org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(rawInput1.start, rawInput1.end, rawInput1.buffer);
> String input2 = org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(rawInput2.start, rawInput2.end, rawInput2.buffer);
>
> String soundex1 = new org.apache.commons.codec.language.Soundex().soundex(input1);
> String soundex2 = new org.apache.commons.codec.language.Soundex().soundex(input2);
>
> boolean result = soundex1.equals(soundex2);
> if (result) {
> out.value = -1;
> } else {
> out.value = 0;
> }
> }
> }
>
>
>
>
> jdbc:drill:zk=local> SELECT typeof(sounds_like( 'Steven', 'Stephen' )) AS steve FROM (VALUES(1));
> Error: SYSTEM ERROR: IllegalArgumentException: Can not set org.apache.drill.exec.vector.complex.reader.FieldReader field org.apache.drill.exec.expr.fn.impl.UnionFunctions$GetType.input to org.apache.drill.exec.expr.holders.BitHolder
>
>
> [Error Id: f15a91b9-ae19-481a-b145-1024709547e9 on 10.1.9.176:31010] (state=,code=0)
> 0: jdbc:drill:zk=local> SELECT typeof(sounds_like( 'Steven', 'Stephen' )) FROM (VALUES(1));
> Error: SYSTEM ERROR: IllegalArgumentException: Can not set org.apache.drill.exec.vector.complex.reader.FieldReader field org.apache.drill.exec.expr.fn.impl.UnionFunctions$GetType.input to org.apache.drill.exec.expr.holders.BitHolder
>
>
> [Error Id: 870a2471-8ab7-4a0f-ab7f-9ff873ba767a on 10.1.9.176:31010] (state=,code=0)
> 0: jdbc:drill:zk=local> SELECT sounds_like( 'Steven', 'Stephen' ) FROM (VALUES(1));
> +---------+
> | EXPR$0 |
> +---------+
> | true |
> +---------+
> 1 row selected (0.689 seconds)
> 0: jdbc:drill:zk=local> SELECT sounds_like( 'Steven', 'bob' ) FROM (VALUES(1));
> +---------+
> | EXPR$0 |
> +---------+
> | false |
> +---------+
> 1 row selected (0.175 seconds)
>
>
>
>
>> On Jun 28, 2018, at 19:46, Timothy Farkas <tf...@mapr.com> wrote:
>>
>> Hi Charles,
>>
>> So it is actually supported. Drill's boolean vector is BitVector.
>> Internally bits are stored efficiently, but when you fetch a bit from the
>> vector it becomes an int, -1 for true and 0 for false. So currently you can
>> check this by using singletonInt and comparing against -1 and 0.
>>
>> Here is a test snippet that does it. We should probably add some
>> convenience methods specifically for booleans so this is not so confusing.
>>
>> public class SimpleQueryTest extends ClusterTest {
>> @Rule
>> public final BaseDirTestWatcher baseDirTestWatcher = new BaseDirTestWatcher();
>>
>> @Before
>> public void start() throws Exception {
>> startCluster(new ClusterFixtureBuilder(baseDirTestWatcher));
>> }
>>
>> @Test
>> public void testBoolean() throws RpcException {
>> RowSet rowSet = this.queryBuilder().sql("select position_id,
>> cast(false as boolean) from cp.`employee.json`").rowSet();
>> RowSetReader reader = rowSet.reader();
>> reader.next();
>> System.out.println(reader.column(1).scalar().getInt());
>> rowSet.clear();
>>
>> rowSet = this.queryBuilder().sql("select position_id, cast(true as
>> boolean) from cp.`employee.json`").rowSet();
>> reader = rowSet.reader();
>> reader.next();
>> System.out.println(reader.column(1).scalar().getInt());
>> rowSet.clear();
>> }
>> }
>>
>>
>>
>> On Thu, Jun 28, 2018 at 4:09 PM, Timothy Farkas <tf...@mapr.com> wrote:
>>
>>> Hi Charles,
>>>
>>> Currently the RowSetReader doesn't support booleans, but it can be added.
>>> I'll try to add it now, and see if it could be done quickly. I'll update
>>> you on my progress.
>>>
>>> Tim
>>>
>>> On Thu, Jun 28, 2018 at 2:38 PM, Charles Givre <cg...@gmail.com> wrote:
>>>
>>>> Hi Tim,
>>>> Could post some sample code as to how to test a SQL query that returns a
>>>> Boolean?
>>>> —C
>>>>
>>>>> On Jun 28, 2018, at 17:30, Timothy Farkas <tf...@mapr.com> wrote:
>>>>>
>>>>> - We would have to add a boolean column reader to ColumnAccessors and
>>>> wire
>>>>> it in and add a getBoolean method to ScalarReader.
>>>>>
>>>>> - Your example should work as is, ClusterTest has a testBuilder method
>>>>> that allows you to use the traditional test builder. Is there something
>>>> not
>>>>> working with the test builder?
>>>>>
>>>>> Tim
>>>>>
>>>>>
>>>>> On Thu, Jun 28, 2018 at 12:39 PM, Arina Yelchiyeva <
>>>>> arina.yelchiyeva@gmail.com> wrote:
>>>>>
>>>>>> Hi Tim,
>>>>>>
>>>>>> it looks like deprecating BaseTestQuery was a little bit pre-mature.
>>>>>> For example, from in this PR - https://urldefense.proofpoint.
>>>> com/v2/url?u=https-3A__urldefense.proofpoint&d=DwIFaQ&c=cskd
>>>> kSMqhcnjZxdQVpwTXg&r=4eQVr8zB8ZBff-yxTimdOQ&m=D9i95wxPKk0aPH
>>>> ip2Rj6Xju0J151UakaSe6OnMGTW5s&s=z5tYyOUI_Cyx1cwhMEMfnCb-
>>>> BDFaZkRaKQKwt_zl2HA&e=.
>>>>>> com/v2/url?u=https-3A__github.com_apache_drill_pull_1331&d=DwIBaQ&c=
>>>>>> cskdkSMqhcnjZxdQVpwTXg&r=4eQVr8zB8ZBff-yxTimdOQ&m=
>>>>>> zoNJPdWKxMX9-jbR2bftzwkX-CSrihIbpCirhHM1kA0&s=_uxhA_
>>>>>> qiMBTjbit6DDw-DvZNRGesfeA5g-FQjkl7f10&e= -
>>>>>> Charles is trying to re-work BaseTestQuery usage to ClusterTest.
>>>>>> First, it did not contain getSigletonDouble method which Charles has
>>>>>> implemented. Now he has troubles with implementing getSigletonBoolean
>>>>>> method which might be due to reader limitations.
>>>>>> Also I am not quite clear how we can verify columns names and multiple
>>>>>> columns in the result.
>>>>>> For example:
>>>>>>
>>>>>> testBuilder()
>>>>>> .sqlQuery("select (mi || lname) as CONCATOperator, mi, lname,
>>>>>> concat(mi, lname) as CONCAT from concatNull")
>>>>>> .ordered()
>>>>>> .baselineColumns("CONCATOperator", "mi", "lname", "CONCAT")
>>>>>> .baselineValues("A.Nowmer", "A.", "Nowmer", "A.Nowmer")
>>>>>> .baselineValues("I.Whelply", "I.", "Whelply", "I.Whelply")
>>>>>> .baselineValues(null, null, "Derry", "Derry")
>>>>>> .baselineValues("J.Spence", "J.", "Spence", "J.Spence")
>>>>>> .build().run();
>>>>>>
>>>>>> Can you please suggest how this example can be re-written?
>>>>>>
>>>>>> Kind regards,
>>>>>> Arina
>>>>>>
>>>>>> On Mon, Jun 25, 2018 at 11:10 PM Timothy Farkas <tf...@mapr.com>
>>>> wrote:
>>>>>>
>>>>>>> Hi All,
>>>>>>>
>>>>>>> BaseTestQuery was deprecated a while ago. Keeping it short and sweet
>>>> :),
>>>>>> if
>>>>>>> you want to use BaseTestQuery directly, don't. Use ClusterTest
>>>> instead.
>>>>>> If
>>>>>>> you are using PlanTestBase for planner tests, continue to do so.
>>>>>> Eventually
>>>>>>> PlanTestBase will be changed to extend ClusterTest instead. There is a
>>>>>> JIRA
>>>>>>> to track that issue https://urldefense.proofpoint.
>>>> com/v2/url?u=https-3A__urldefense.proofpoint&d=DwIFaQ&c=cskd
>>>> kSMqhcnjZxdQVpwTXg&r=4eQVr8zB8ZBff-yxTimdOQ&m=D9i95wxPKk0aPH
>>>> ip2Rj6Xju0J151UakaSe6OnMGTW5s&s=z5tYyOUI_Cyx1cwhMEMfnCb-
>>>> BDFaZkRaKQKwt_zl2HA&e=.
>>>>>> com/v2/url?u=https-3A__issues.apache.org_jira_browse_DRILL-
>>>>>> 2D6536&d=DwIBaQ&c=cskdkSMqhcnjZxdQVpwTXg&r=4eQVr8zB8ZBff-yxTimdOQ&m=
>>>>>> zoNJPdWKxMX9-jbR2bftzwkX-CSrihIbpCirhHM1kA0&s=
>>>>>> BPAlenAq0k1kjAz7fUYXyEQMaJM1IPOgmdeySMMY84U&e=.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Tim
>>>>>>>
>>>>>>
>>>>
>>>>
>>>
Re: Deprecation of BaseTestQuery FYI
Posted by Paul Rogers <pa...@yahoo.com.INVALID>.
Hi Tim,
My understanding is that Booleans are stored, as you said, as a UInt1 vector. But, the values are 0 and (positive) 1. As you said, singletonInt will read this value. If the value is coming back as -1, then this is a bug. I'd be surprised if it is since I wrote lots of code that assumed the value is 1.
Given this, it would be a simple addition to the test QueryBuilder to add a "singletonBoolean" that returns singletonInt() != 0.
Charles,
You should not set the BitHolder to -1, you should set it to either 0 or (positive) 1.
Thanks,
- Paul
On Thursday, July 5, 2018, 8:39:58 AM PDT, Charles Givre <cg...@gmail.com> wrote:
Hi Timothy,
I am still having issues with the unit tests for this particular function. The code is below. This function takes the soundex values of two strings and if they are equal is supposed to return true, if not returns false. I do this by setting the value of a BitHolder to either -1 or 0.
The code for the function appears to work, however, when I think what is causing the error here was the sounds_like function itself.
If I run the query:
select *
from
WHERE soundex( ‘brian’ ) = soundex(`first_name` )
This returns results
However, if I run the query
SELECT *
FROM <Data>
WEHRE sounds_like( ‘brian’, `first_name` )
I get no results. However, if I run this query:
SELECT sounds_like( ‘brian’, `first_name` )
FROM cp..
I get trues and falses. I’m not sure why this isn’t working properly and in the interests of not holding up the works, I’m removing the sounds_like from the PR, but I’d like to understand how to fix this.
@FunctionTemplate(name = "sounds_like", scope = FunctionTemplate.FunctionScope.SIMPLE, nulls = FunctionTemplate.NullHandling.NULL_IF_NULL)
public static class SoundsLikeFunction implements DrillSimpleFunc {
@Param
VarCharHolder rawInput1;
@Param
VarCharHolder rawInput2;
@Output
BitHolder out;
@Override
public void setup() {
}
@Override
public void eval() {
String input1 = org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(rawInput1.start, rawInput1.end, rawInput1.buffer);
String input2 = org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(rawInput2.start, rawInput2.end, rawInput2.buffer);
String soundex1 = new org.apache.commons.codec.language.Soundex().soundex(input1);
String soundex2 = new org.apache.commons.codec.language.Soundex().soundex(input2);
boolean result = soundex1.equals(soundex2);
if (result) {
out.value = -1;
} else {
out.value = 0;
}
}
}
jdbc:drill:zk=local> SELECT typeof(sounds_like( 'Steven', 'Stephen' )) AS steve FROM (VALUES(1));
Error: SYSTEM ERROR: IllegalArgumentException: Can not set org.apache.drill.exec.vector.complex.reader.FieldReader field org.apache.drill.exec.expr.fn.impl.UnionFunctions$GetType.input to org.apache.drill.exec.expr.holders.BitHolder
[Error Id: f15a91b9-ae19-481a-b145-1024709547e9 on 10.1.9.176:31010] (state=,code=0)
0: jdbc:drill:zk=local> SELECT typeof(sounds_like( 'Steven', 'Stephen' )) FROM (VALUES(1));
Error: SYSTEM ERROR: IllegalArgumentException: Can not set org.apache.drill.exec.vector.complex.reader.FieldReader field org.apache.drill.exec.expr.fn.impl.UnionFunctions$GetType.input to org.apache.drill.exec.expr.holders.BitHolder
[Error Id: 870a2471-8ab7-4a0f-ab7f-9ff873ba767a on 10.1.9.176:31010] (state=,code=0)
0: jdbc:drill:zk=local> SELECT sounds_like( 'Steven', 'Stephen' ) FROM (VALUES(1));
+---------+
| EXPR$0 |
+---------+
| true |
+---------+
1 row selected (0.689 seconds)
0: jdbc:drill:zk=local> SELECT sounds_like( 'Steven', 'bob' ) FROM (VALUES(1));
+---------+
| EXPR$0 |
+---------+
| false |
+---------+
1 row selected (0.175 seconds)
> On Jun 28, 2018, at 19:46, Timothy Farkas <tf...@mapr.com> wrote:
>
> Hi Charles,
>
> So it is actually supported. Drill's boolean vector is BitVector.
> Internally bits are stored efficiently, but when you fetch a bit from the
> vector it becomes an int, -1 for true and 0 for false. So currently you can
> check this by using singletonInt and comparing against -1 and 0.
>
> Here is a test snippet that does it. We should probably add some
> convenience methods specifically for booleans so this is not so confusing.
>
> public class SimpleQueryTest extends ClusterTest {
> @Rule
> public final BaseDirTestWatcher baseDirTestWatcher = new BaseDirTestWatcher();
>
> @Before
> public void start() throws Exception {
> startCluster(new ClusterFixtureBuilder(baseDirTestWatcher));
> }
>
> @Test
> public void testBoolean() throws RpcException {
> RowSet rowSet = this.queryBuilder().sql("select position_id,
> cast(false as boolean) from cp.`employee.json`").rowSet();
> RowSetReader reader = rowSet.reader();
> reader.next();
> System.out.println(reader.column(1).scalar().getInt());
> rowSet.clear();
>
> rowSet = this.queryBuilder().sql("select position_id, cast(true as
> boolean) from cp.`employee.json`").rowSet();
> reader = rowSet.reader();
> reader.next();
> System.out.println(reader.column(1).scalar().getInt());
> rowSet.clear();
> }
> }
>
>
>
> On Thu, Jun 28, 2018 at 4:09 PM, Timothy Farkas <tf...@mapr.com> wrote:
>
>> Hi Charles,
>>
>> Currently the RowSetReader doesn't support booleans, but it can be added.
>> I'll try to add it now, and see if it could be done quickly. I'll update
>> you on my progress.
>>
>> Tim
>>
>> On Thu, Jun 28, 2018 at 2:38 PM, Charles Givre <cg...@gmail.com> wrote:
>>
>>> Hi Tim,
>>> Could post some sample code as to how to test a SQL query that returns a
>>> Boolean?
>>> —C
>>>
>>>> On Jun 28, 2018, at 17:30, Timothy Farkas <tf...@mapr.com> wrote:
>>>>
>>>> - We would have to add a boolean column reader to ColumnAccessors and
>>> wire
>>>> it in and add a getBoolean method to ScalarReader.
>>>>
>>>> - Your example should work as is, ClusterTest has a testBuilder method
>>>> that allows you to use the traditional test builder. Is there something
>>> not
>>>> working with the test builder?
>>>>
>>>> Tim
>>>>
>>>>
>>>> On Thu, Jun 28, 2018 at 12:39 PM, Arina Yelchiyeva <
>>>> arina.yelchiyeva@gmail.com> wrote:
>>>>
>>>>> Hi Tim,
>>>>>
>>>>> it looks like deprecating BaseTestQuery was a little bit pre-mature.
>>>>> For example, from in this PR - https://urldefense.proofpoint.
>>> com/v2/url?u=https-3A__urldefense.proofpoint&d=DwIFaQ&c=cskd
>>> kSMqhcnjZxdQVpwTXg&r=4eQVr8zB8ZBff-yxTimdOQ&m=D9i95wxPKk0aPH
>>> ip2Rj6Xju0J151UakaSe6OnMGTW5s&s=z5tYyOUI_Cyx1cwhMEMfnCb-
>>> BDFaZkRaKQKwt_zl2HA&e=.
>>>>> com/v2/url?u=https-3A__github.com_apache_drill_pull_1331&d=DwIBaQ&c=
>>>>> cskdkSMqhcnjZxdQVpwTXg&r=4eQVr8zB8ZBff-yxTimdOQ&m=
>>>>> zoNJPdWKxMX9-jbR2bftzwkX-CSrihIbpCirhHM1kA0&s=_uxhA_
>>>>> qiMBTjbit6DDw-DvZNRGesfeA5g-FQjkl7f10&e= -
>>>>> Charles is trying to re-work BaseTestQuery usage to ClusterTest.
>>>>> First, it did not contain getSigletonDouble method which Charles has
>>>>> implemented. Now he has troubles with implementing getSigletonBoolean
>>>>> method which might be due to reader limitations.
>>>>> Also I am not quite clear how we can verify columns names and multiple
>>>>> columns in the result.
>>>>> For example:
>>>>>
>>>>> testBuilder()
>>>>> .sqlQuery("select (mi || lname) as CONCATOperator, mi, lname,
>>>>> concat(mi, lname) as CONCAT from concatNull")
>>>>> .ordered()
>>>>> .baselineColumns("CONCATOperator", "mi", "lname", "CONCAT")
>>>>> .baselineValues("A.Nowmer", "A.", "Nowmer", "A.Nowmer")
>>>>> .baselineValues("I.Whelply", "I.", "Whelply", "I.Whelply")
>>>>> .baselineValues(null, null, "Derry", "Derry")
>>>>> .baselineValues("J.Spence", "J.", "Spence", "J.Spence")
>>>>> .build().run();
>>>>>
>>>>> Can you please suggest how this example can be re-written?
>>>>>
>>>>> Kind regards,
>>>>> Arina
>>>>>
>>>>> On Mon, Jun 25, 2018 at 11:10 PM Timothy Farkas <tf...@mapr.com>
>>> wrote:
>>>>>
>>>>>> Hi All,
>>>>>>
>>>>>> BaseTestQuery was deprecated a while ago. Keeping it short and sweet
>>> :),
>>>>> if
>>>>>> you want to use BaseTestQuery directly, don't. Use ClusterTest
>>> instead.
>>>>> If
>>>>>> you are using PlanTestBase for planner tests, continue to do so.
>>>>> Eventually
>>>>>> PlanTestBase will be changed to extend ClusterTest instead. There is a
>>>>> JIRA
>>>>>> to track that issue https://urldefense.proofpoint.
>>> com/v2/url?u=https-3A__urldefense.proofpoint&d=DwIFaQ&c=cskd
>>> kSMqhcnjZxdQVpwTXg&r=4eQVr8zB8ZBff-yxTimdOQ&m=D9i95wxPKk0aPH
>>> ip2Rj6Xju0J151UakaSe6OnMGTW5s&s=z5tYyOUI_Cyx1cwhMEMfnCb-
>>> BDFaZkRaKQKwt_zl2HA&e=.
>>>>> com/v2/url?u=https-3A__issues.apache.org_jira_browse_DRILL-
>>>>> 2D6536&d=DwIBaQ&c=cskdkSMqhcnjZxdQVpwTXg&r=4eQVr8zB8ZBff-yxTimdOQ&m=
>>>>> zoNJPdWKxMX9-jbR2bftzwkX-CSrihIbpCirhHM1kA0&s=
>>>>> BPAlenAq0k1kjAz7fUYXyEQMaJM1IPOgmdeySMMY84U&e=.
>>>>>>
>>>>>> Thanks,
>>>>>> Tim
>>>>>>
>>>>>
>>>
>>>
>>