You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@drill.apache.org by Yash Sharma <ya...@gmail.com> on 2014/05/17 19:53:28 UTC
Query regarding JUnit UTF string assertion
Hi All,
While implementing the toascii(inputStr, encodingType) I am receiving this
assertion error. I have tried using different styles for passing the
assertion string in test case but this always fails with below error. Have
tried StackOverflow but not able to get it working.
Any tips would be appreciated. This is blocking me from putting my patch on
review board -
Peace,
Yash
-------------------------------------------------------------------------------------------
maven failure
-------------------------------------------------------------------------------------------
âpple âpple
Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 12.001 sec
<<< FAILURE! - in org.apache.drill.exec.physical.impl.TestStringFunctions
testNewStringFuncs(org.apache.drill.exec.physical.impl.TestStringFunctions)
Time elapsed: 4.04 sec <<< FAILURE!
org.junit.ComparisonFailure: column 8 does not match expected:<âpple[]>
but was:<âpple[]>
-------------------------------------------------------------------------------------------
string function implementation
-------------------------------------------------------------------------------------------
@FunctionTemplate(name = "toascii", scope = FunctionScope.SIMPLE, nulls =
NullHandling.NULL_IF_NULL)
public static class AsciiEndode implements DrillSimpleFunc {
@Param VarCharHolder in;
@Param VarCharHolder enc;
@Output VarCharHolder out;
public void setup(RecordBatch incoming) {
}
public void eval() {
byte[] bytea = new byte[in.end - in.start +1];
int index =0;
for(int i = in.start; i<in.end; i++, index++){
bytea[index]=in.buffer.getByte(i);
}
byte[] bytes = new String(bytea,
java.nio.charset.Charset.forName(enc.toString())).getBytes(java.nio.charset.Charset.forName("UTF-8"));
out.buffer = io.netty.buffer.Unpooled.wrappedBuffer(new byte [8000]);
out.start = 0;
out.buffer.setBytes(out.start, bytes);
out.end = bytes.length;
}
}
-------------------------------------------------------------------------------------------
test case
-------------------------------------------------------------------------------------------
@Test
public void testNewStringFuncs(@Injectable final DrillbitContext
bitContext,
@Injectable UserServer.UserClientConnection
connection) throws Throwable{
Object [] expected = new Object[] {97, 65, -32, "A", "trim", "Peace
Peace Peace ", "हकुना मताता हकुना मताता ", "katcit", "\u00C3\u00A2pple"};
runTest(bitContext, connection, expected,
"functions/string/testStringFuncs.json");
}
-------------------------------------------------------------------------------------------
call in physical plan
-------------------------------------------------------------------------------------------
{ ref : "ref9", expr : " toascii('âpple','ISO-8859-1') "}
Re: Query regarding JUnit UTF string assertion
Posted by Yash Sharma <ya...@gmail.com>.
Finally Cracked It...
Was the issue with the end index of VarCharHolder output - which was having
one extra blank value at end.
Thanks Tim. Would be putting Review Board request in morning.
Peace Finally,
Yash
On Sun, May 18, 2014 at 2:00 AM, Yash Sharma <ya...@gmail.com> wrote:
> Also to add -
> assertArrayEqauls might not be necessary since the output VarCharHolder
> and expected argument are both compared as strings. This works for my other
> test cases (substr/reverse etc).
> I doubt its something related to how encoded strings are being read.
>
> Peace,
> Yash
>
>
> On Sun, May 18, 2014 at 1:54 AM, Yash Sharma <ya...@gmail.com> wrote:
>
>> I am re-using the runtest method of the TestStringFunctions.java for my
>> test case.
>> It compares each output argument with the expected arg - all treated as
>> strings.
>>
>> Peace,
>> Yash
>>
>> Assert logic: public void runTest
>>
>> while(exec.next()){
>> Object [] res = getRunResult(exec);
>> assertEquals("return count does not match", expectedResults.length,
>> res.length);
>>
>>
>> for (int i = 0; i<res.length; i++) {
>> System.out.println(res[i]+"\t\t"+expectedResults[i]);
>> assertEquals(String.format("column %s does not match", i),
>> expectedResults[i], res[i]);
>> }
>> }
>>
>>
>>
>>
>> On Sun, May 18, 2014 at 12:34 AM, Timothy Chen <tn...@gmail.com> wrote:
>>
>>> I can''t tell how you're comparing the strings as the I don't see
>>> where the assertion happens.
>>>
>>> But if you're just doing a assertEquals with two byte arrays it won't
>>> work, you'll have to do assertArrayEqauls instead.
>>>
>>> Tim
>>>
>>> On Sat, May 17, 2014 at 10:53 AM, Yash Sharma <ya...@gmail.com> wrote:
>>> > Hi All,
>>> > While implementing the toascii(inputStr, encodingType) I am receiving
>>> this
>>> > assertion error. I have tried using different styles for passing the
>>> > assertion string in test case but this always fails with below error.
>>> Have
>>> > tried StackOverflow but not able to get it working.
>>> >
>>> > Any tips would be appreciated. This is blocking me from putting my
>>> patch on
>>> > review board -
>>> >
>>> > Peace,
>>> > Yash
>>> >
>>> >
>>> >
>>> -------------------------------------------------------------------------------------------
>>> > maven failure
>>> >
>>> -------------------------------------------------------------------------------------------
>>> > âpple âpple
>>> > Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 12.001
>>> sec
>>> > <<< FAILURE! - in
>>> org.apache.drill.exec.physical.impl.TestStringFunctions
>>> >
>>> testNewStringFuncs(org.apache.drill.exec.physical.impl.TestStringFunctions)
>>> > Time elapsed: 4.04 sec <<< FAILURE!
>>> > org.junit.ComparisonFailure: column 8 does not match
>>> expected:<âpple[]>
>>> > but was:<âpple[]>
>>> >
>>> >
>>> >
>>> >
>>> -------------------------------------------------------------------------------------------
>>> > string function implementation
>>> >
>>> -------------------------------------------------------------------------------------------
>>> > @FunctionTemplate(name = "toascii", scope = FunctionScope.SIMPLE,
>>> nulls =
>>> > NullHandling.NULL_IF_NULL)
>>> > public static class AsciiEndode implements DrillSimpleFunc {
>>> >
>>> > @Param VarCharHolder in;
>>> > @Param VarCharHolder enc;
>>> > @Output VarCharHolder out;
>>> >
>>> > public void setup(RecordBatch incoming) {
>>> > }
>>> >
>>> > public void eval() {
>>> >
>>> > byte[] bytea = new byte[in.end - in.start +1];
>>> > int index =0;
>>> > for(int i = in.start; i<in.end; i++, index++){
>>> > bytea[index]=in.buffer.getByte(i);
>>> > }
>>> > byte[] bytes = new String(bytea,
>>> >
>>> java.nio.charset.Charset.forName(enc.toString())).getBytes(java.nio.charset.Charset.forName("UTF-8"));
>>> >
>>> > out.buffer = io.netty.buffer.Unpooled.wrappedBuffer(new byte [8000]);
>>> > out.start = 0;
>>> > out.buffer.setBytes(out.start, bytes);
>>> > out.end = bytes.length;
>>> > }
>>> > }
>>> >
>>> >
>>> >
>>> -------------------------------------------------------------------------------------------
>>> > test case
>>> >
>>> -------------------------------------------------------------------------------------------
>>> >
>>> > @Test
>>> > public void testNewStringFuncs(@Injectable final DrillbitContext
>>> > bitContext,
>>> > @Injectable UserServer.UserClientConnection
>>> > connection) throws Throwable{
>>> > Object [] expected = new Object[] {97, 65, -32, "A", "trim", "Peace
>>> > Peace Peace ", "हकुना मताता हकुना मताता ", "katcit",
>>> "\u00C3\u00A2pple"};
>>> >
>>> > runTest(bitContext, connection, expected,
>>> > "functions/string/testStringFuncs.json");
>>> > }
>>> >
>>> >
>>> -------------------------------------------------------------------------------------------
>>> > call in physical plan
>>> >
>>> -------------------------------------------------------------------------------------------
>>> > { ref : "ref9", expr : " toascii('âpple','ISO-8859-1') "}
>>>
>>
>>
>
Re: Query regarding JUnit UTF string assertion
Posted by Yash Sharma <ya...@gmail.com>.
Also to add -
assertArrayEqauls might not be necessary since the output VarCharHolder
and expected argument are both compared as strings. This works for my other
test cases (substr/reverse etc).
I doubt its something related to how encoded strings are being read.
Peace,
Yash
On Sun, May 18, 2014 at 1:54 AM, Yash Sharma <ya...@gmail.com> wrote:
> I am re-using the runtest method of the TestStringFunctions.java for my
> test case.
> It compares each output argument with the expected arg - all treated as
> strings.
>
> Peace,
> Yash
>
> Assert logic: public void runTest
>
> while(exec.next()){
> Object [] res = getRunResult(exec);
> assertEquals("return count does not match", expectedResults.length,
> res.length);
>
>
> for (int i = 0; i<res.length; i++) {
> System.out.println(res[i]+"\t\t"+expectedResults[i]);
> assertEquals(String.format("column %s does not match", i),
> expectedResults[i], res[i]);
> }
> }
>
>
>
>
> On Sun, May 18, 2014 at 12:34 AM, Timothy Chen <tn...@gmail.com> wrote:
>
>> I can''t tell how you're comparing the strings as the I don't see
>> where the assertion happens.
>>
>> But if you're just doing a assertEquals with two byte arrays it won't
>> work, you'll have to do assertArrayEqauls instead.
>>
>> Tim
>>
>> On Sat, May 17, 2014 at 10:53 AM, Yash Sharma <ya...@gmail.com> wrote:
>> > Hi All,
>> > While implementing the toascii(inputStr, encodingType) I am receiving
>> this
>> > assertion error. I have tried using different styles for passing the
>> > assertion string in test case but this always fails with below error.
>> Have
>> > tried StackOverflow but not able to get it working.
>> >
>> > Any tips would be appreciated. This is blocking me from putting my
>> patch on
>> > review board -
>> >
>> > Peace,
>> > Yash
>> >
>> >
>> >
>> -------------------------------------------------------------------------------------------
>> > maven failure
>> >
>> -------------------------------------------------------------------------------------------
>> > âpple âpple
>> > Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 12.001
>> sec
>> > <<< FAILURE! - in
>> org.apache.drill.exec.physical.impl.TestStringFunctions
>> >
>> testNewStringFuncs(org.apache.drill.exec.physical.impl.TestStringFunctions)
>> > Time elapsed: 4.04 sec <<< FAILURE!
>> > org.junit.ComparisonFailure: column 8 does not match expected:<âpple[]>
>> > but was:<âpple[]>
>> >
>> >
>> >
>> >
>> -------------------------------------------------------------------------------------------
>> > string function implementation
>> >
>> -------------------------------------------------------------------------------------------
>> > @FunctionTemplate(name = "toascii", scope = FunctionScope.SIMPLE, nulls
>> =
>> > NullHandling.NULL_IF_NULL)
>> > public static class AsciiEndode implements DrillSimpleFunc {
>> >
>> > @Param VarCharHolder in;
>> > @Param VarCharHolder enc;
>> > @Output VarCharHolder out;
>> >
>> > public void setup(RecordBatch incoming) {
>> > }
>> >
>> > public void eval() {
>> >
>> > byte[] bytea = new byte[in.end - in.start +1];
>> > int index =0;
>> > for(int i = in.start; i<in.end; i++, index++){
>> > bytea[index]=in.buffer.getByte(i);
>> > }
>> > byte[] bytes = new String(bytea,
>> >
>> java.nio.charset.Charset.forName(enc.toString())).getBytes(java.nio.charset.Charset.forName("UTF-8"));
>> >
>> > out.buffer = io.netty.buffer.Unpooled.wrappedBuffer(new byte [8000]);
>> > out.start = 0;
>> > out.buffer.setBytes(out.start, bytes);
>> > out.end = bytes.length;
>> > }
>> > }
>> >
>> >
>> >
>> -------------------------------------------------------------------------------------------
>> > test case
>> >
>> -------------------------------------------------------------------------------------------
>> >
>> > @Test
>> > public void testNewStringFuncs(@Injectable final DrillbitContext
>> > bitContext,
>> > @Injectable UserServer.UserClientConnection
>> > connection) throws Throwable{
>> > Object [] expected = new Object[] {97, 65, -32, "A", "trim", "Peace
>> > Peace Peace ", "हकुना मताता हकुना मताता ", "katcit",
>> "\u00C3\u00A2pple"};
>> >
>> > runTest(bitContext, connection, expected,
>> > "functions/string/testStringFuncs.json");
>> > }
>> >
>> >
>> -------------------------------------------------------------------------------------------
>> > call in physical plan
>> >
>> -------------------------------------------------------------------------------------------
>> > { ref : "ref9", expr : " toascii('âpple','ISO-8859-1') "}
>>
>
>
Re: Query regarding JUnit UTF string assertion
Posted by Yash Sharma <ya...@gmail.com>.
I am re-using the runtest method of the TestStringFunctions.java for my
test case.
It compares each output argument with the expected arg - all treated as
strings.
Peace,
Yash
Assert logic: public void runTest
while(exec.next()){
Object [] res = getRunResult(exec);
assertEquals("return count does not match", expectedResults.length,
res.length);
for (int i = 0; i<res.length; i++) {
System.out.println(res[i]+"\t\t"+expectedResults[i]);
assertEquals(String.format("column %s does not match", i),
expectedResults[i], res[i]);
}
}
On Sun, May 18, 2014 at 12:34 AM, Timothy Chen <tn...@gmail.com> wrote:
> I can''t tell how you're comparing the strings as the I don't see
> where the assertion happens.
>
> But if you're just doing a assertEquals with two byte arrays it won't
> work, you'll have to do assertArrayEqauls instead.
>
> Tim
>
> On Sat, May 17, 2014 at 10:53 AM, Yash Sharma <ya...@gmail.com> wrote:
> > Hi All,
> > While implementing the toascii(inputStr, encodingType) I am receiving
> this
> > assertion error. I have tried using different styles for passing the
> > assertion string in test case but this always fails with below error.
> Have
> > tried StackOverflow but not able to get it working.
> >
> > Any tips would be appreciated. This is blocking me from putting my patch
> on
> > review board -
> >
> > Peace,
> > Yash
> >
> >
> >
> -------------------------------------------------------------------------------------------
> > maven failure
> >
> -------------------------------------------------------------------------------------------
> > âpple âpple
> > Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 12.001
> sec
> > <<< FAILURE! - in org.apache.drill.exec.physical.impl.TestStringFunctions
> >
> testNewStringFuncs(org.apache.drill.exec.physical.impl.TestStringFunctions)
> > Time elapsed: 4.04 sec <<< FAILURE!
> > org.junit.ComparisonFailure: column 8 does not match expected:<âpple[]>
> > but was:<âpple[]>
> >
> >
> >
> >
> -------------------------------------------------------------------------------------------
> > string function implementation
> >
> -------------------------------------------------------------------------------------------
> > @FunctionTemplate(name = "toascii", scope = FunctionScope.SIMPLE, nulls =
> > NullHandling.NULL_IF_NULL)
> > public static class AsciiEndode implements DrillSimpleFunc {
> >
> > @Param VarCharHolder in;
> > @Param VarCharHolder enc;
> > @Output VarCharHolder out;
> >
> > public void setup(RecordBatch incoming) {
> > }
> >
> > public void eval() {
> >
> > byte[] bytea = new byte[in.end - in.start +1];
> > int index =0;
> > for(int i = in.start; i<in.end; i++, index++){
> > bytea[index]=in.buffer.getByte(i);
> > }
> > byte[] bytes = new String(bytea,
> >
> java.nio.charset.Charset.forName(enc.toString())).getBytes(java.nio.charset.Charset.forName("UTF-8"));
> >
> > out.buffer = io.netty.buffer.Unpooled.wrappedBuffer(new byte [8000]);
> > out.start = 0;
> > out.buffer.setBytes(out.start, bytes);
> > out.end = bytes.length;
> > }
> > }
> >
> >
> >
> -------------------------------------------------------------------------------------------
> > test case
> >
> -------------------------------------------------------------------------------------------
> >
> > @Test
> > public void testNewStringFuncs(@Injectable final DrillbitContext
> > bitContext,
> > @Injectable UserServer.UserClientConnection
> > connection) throws Throwable{
> > Object [] expected = new Object[] {97, 65, -32, "A", "trim", "Peace
> > Peace Peace ", "हकुना मताता हकुना मताता ", "katcit", "\u00C3\u00A2pple"};
> >
> > runTest(bitContext, connection, expected,
> > "functions/string/testStringFuncs.json");
> > }
> >
> >
> -------------------------------------------------------------------------------------------
> > call in physical plan
> >
> -------------------------------------------------------------------------------------------
> > { ref : "ref9", expr : " toascii('âpple','ISO-8859-1') "}
>
Re: Query regarding JUnit UTF string assertion
Posted by Timothy Chen <tn...@gmail.com>.
I can''t tell how you're comparing the strings as the I don't see
where the assertion happens.
But if you're just doing a assertEquals with two byte arrays it won't
work, you'll have to do assertArrayEqauls instead.
Tim
On Sat, May 17, 2014 at 10:53 AM, Yash Sharma <ya...@gmail.com> wrote:
> Hi All,
> While implementing the toascii(inputStr, encodingType) I am receiving this
> assertion error. I have tried using different styles for passing the
> assertion string in test case but this always fails with below error. Have
> tried StackOverflow but not able to get it working.
>
> Any tips would be appreciated. This is blocking me from putting my patch on
> review board -
>
> Peace,
> Yash
>
>
> -------------------------------------------------------------------------------------------
> maven failure
> -------------------------------------------------------------------------------------------
> âpple âpple
> Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 12.001 sec
> <<< FAILURE! - in org.apache.drill.exec.physical.impl.TestStringFunctions
> testNewStringFuncs(org.apache.drill.exec.physical.impl.TestStringFunctions)
> Time elapsed: 4.04 sec <<< FAILURE!
> org.junit.ComparisonFailure: column 8 does not match expected:<âpple[]>
> but was:<âpple[]>
>
>
>
> -------------------------------------------------------------------------------------------
> string function implementation
> -------------------------------------------------------------------------------------------
> @FunctionTemplate(name = "toascii", scope = FunctionScope.SIMPLE, nulls =
> NullHandling.NULL_IF_NULL)
> public static class AsciiEndode implements DrillSimpleFunc {
>
> @Param VarCharHolder in;
> @Param VarCharHolder enc;
> @Output VarCharHolder out;
>
> public void setup(RecordBatch incoming) {
> }
>
> public void eval() {
>
> byte[] bytea = new byte[in.end - in.start +1];
> int index =0;
> for(int i = in.start; i<in.end; i++, index++){
> bytea[index]=in.buffer.getByte(i);
> }
> byte[] bytes = new String(bytea,
> java.nio.charset.Charset.forName(enc.toString())).getBytes(java.nio.charset.Charset.forName("UTF-8"));
>
> out.buffer = io.netty.buffer.Unpooled.wrappedBuffer(new byte [8000]);
> out.start = 0;
> out.buffer.setBytes(out.start, bytes);
> out.end = bytes.length;
> }
> }
>
>
> -------------------------------------------------------------------------------------------
> test case
> -------------------------------------------------------------------------------------------
>
> @Test
> public void testNewStringFuncs(@Injectable final DrillbitContext
> bitContext,
> @Injectable UserServer.UserClientConnection
> connection) throws Throwable{
> Object [] expected = new Object[] {97, 65, -32, "A", "trim", "Peace
> Peace Peace ", "हकुना मताता हकुना मताता ", "katcit", "\u00C3\u00A2pple"};
>
> runTest(bitContext, connection, expected,
> "functions/string/testStringFuncs.json");
> }
>
> -------------------------------------------------------------------------------------------
> call in physical plan
> -------------------------------------------------------------------------------------------
> { ref : "ref9", expr : " toascii('âpple','ISO-8859-1') "}