[jira] [Commented] (DRILL-3954) HBase tests use only 1 region, don't detect bug(s) in dummy-column NullableIntVector creation/resolution

["Daniel Barclay (Drill) (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/DRILL-3954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14984247#comment-14984247 ] 

Daniel Barclay (Drill) commented on DRILL-3954:
-----------------------------------------------

One problem revealed by increasing the numbers of regions beyond one is that one of the HBase tests violates the condition whose check yields the error message saying that column names "must be singular names" (which actually should say something like "simple names" (a standard term for the intended concept)).

(Additionally, the fact that changing the number of regions and thereby (presumably) fragments indicates that Drill's check for that column-name requirement is hacked-\-checking at the point of serialization or fragment splitting or whatever code imposes that requirement--rather than checking earlier when column names should be checked for all relevant requirements (so that, for example, the number of regions a table has doesn't affect whether the column name is checked).) 

> HBase tests use only 1 region, don't detect bug(s) in dummy-column NullableIntVector creation/resolution
> --------------------------------------------------------------------------------------------------------
>
>                 Key: DRILL-3954
>                 URL: https://issues.apache.org/jira/browse/DRILL-3954
>             Project: Apache Drill
>          Issue Type: Bug
>          Components: Storage - HBase
>            Reporter: Daniel Barclay (Drill)
>
> Currently, the HBase tests (e.g., {{TestHBaseFilterPushDown}}) use only one region.
> That causes them to miss detecting a bug in creating and/or resolving dummy fields ({{NullableIntVectors}} for referenced but non-existent fields) somewhere between reading from HBase and wherever dummy fields are supposed to be get resolved  (or maybe two separate bugs).
> Reproduction:
> In HBaseTestsSuite, change the line:
> {noformat}
>             UTIL.startMiniHBaseCluster(1, 1);
> {noformat}
> to:
> {noformat}
>             UTIL.startMiniHBaseCluster(1, 3);
> {noformat}
> and change the line:
> {noformat}
>     TestTableGenerator.generateHBaseDataset1(admin, TEST_TABLE_1, 1);
> {noformat}
> to:
> {noformat}
>     TestTableGenerator.generateHBaseDataset1(admin, TEST_TABLE_1, 3);
> {noformat}
> .
> Run unit test class {{TestHBaseFilterPushDown}}.
> Depending on which region gets processed first (it's non-deteministic), test methods {{testFilterPushDownOrRowKeyEqualRangePred}} and 
> {{testFilterPushDownMultiColumns}} get exceptions like this:
> {noformat}
> java.lang.IndexOutOfBoundsException: index: 0, length: 1 (expected: range(0, 0))
> 	at io.netty.buffer.DrillBuf.checkIndexD(DrillBuf.java:189)
> 	at io.netty.buffer.DrillBuf.chk(DrillBuf.java:211)
> 	at io.netty.buffer.DrillBuf.getByte(DrillBuf.java:746)
> 	at org.apache.drill.exec.vector.UInt1Vector$Accessor.get(UInt1Vector.java:364)
> 	at org.apache.drill.exec.vector.NullableVarBinaryVector$Accessor.isSet(NullableVarBinaryVector.java:391)
> 	at org.apache.drill.exec.vector.NullableVarBinaryVector$Accessor.isNull(NullableVarBinaryVector.java:387)
> 	at org.apache.drill.exec.vector.NullableVarBinaryVector$Accessor.getObject(NullableVarBinaryVector.java:411)
> 	at org.apache.drill.exec.vector.NullableVarBinaryVector$Accessor.getObject(NullableVarBinaryVector.java:1)
> 	at org.apache.drill.exec.vector.complex.MapVector$Accessor.getObject(MapVector.java:313)
> 	at org.apache.drill.exec.util.VectorUtil.showVectorAccessibleContent(VectorUtil.java:166)
> 	at org.apache.drill.BaseTestQuery.printResult(BaseTestQuery.java:487)
> 	at org.apache.drill.hbase.BaseHBaseTest.printResultAndVerifyRowCount(BaseHBaseTest.java:95)
> 	at org.apache.drill.hbase.BaseHBaseTest.runHBaseSQLVerifyCount(BaseHBaseTest.java:91)
> 	at org.apache.drill.hbase.TestHBaseFilterPushDown.testFilterPushDownMultiColumns(TestHBaseFilterPushDown.java:592)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at java.lang.reflect.Method.invoke(Method.java:606)
> 	at java.lang.reflect.Method.invoke(Method.java:606)
> {noformat}
> See DRILL-3955.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)