You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@drill.apache.org by "Volodymyr Vysotskyi (JIRA)" <ji...@apache.org> on 2017/07/04 17:05:01 UTC
[jira] [Commented] (DRILL-4755) StringIndexOutOfBoundsException
seen with CONVERT_FROM function
[ https://issues.apache.org/jira/browse/DRILL-4755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16073915#comment-16073915 ]
Volodymyr Vysotskyi commented on DRILL-4755:
--------------------------------------------
The query
{code:sql}
select CONVERT_FROM(binary_string(columns[0]),'INTEGER') from `binStrDuplcs.csv`;
{code}
sometimes returns correct error:
{noformat}
Error: UNSUPPORTED_OPERATION ERROR: CONVERT_FROM does not support conversion from type 'INTEGER'.
{noformat}
It depends on the order of functions in the list returned by [getOperatorList()|https://github.com/apache/drill/blob/3e8b01d5b0d3013e3811913f0fd6028b22c1ac3f/exec/java-exec/src/main/java/org/apache/drill/exec/planner/sql/DrillOperatorTable.java#L147] method. Drill returns correct error when method [getBestMatch()|https://github.com/apache/drill/blob/3e8b01d5b0d3013e3811913f0fd6028b22c1ac3f/exec/java-exec/src/main/java/org/apache/drill/exec/util/ApproximateStringMatcher.java#L73] returns function name that is longer than the name of the function used in the query, but when it is not, [this line|https://github.com/apache/drill/blob/3e8b01d5b0d3013e3811913f0fd6028b22c1ac3f/exec/java-exec/src/main/java/org/apache/drill/exec/planner/logical/PreProcessLogicalRel.java#L245] causes an error.
The other cause of this issue is that method [getOperatorList()|https://github.com/apache/drill/blob/3e8b01d5b0d3013e3811913f0fd6028b22c1ac3f/exec/java-exec/src/main/java/org/apache/drill/exec/planner/sql/DrillOperatorTable.java#L147] returns a list only with calciteOperators, but drillOperatorsWithInference or drillOperatorsWithoutInference are not added to the list.
Message with the suggested correct type for the convert functions should be displayed (it was added in DRILL-2862) when correct error is returned, but it is not displayed due to this cause. Full error message for this query should be:
{noformat}
Error: UNSUPPORTED_OPERATION ERROR: CONVERT_FROM does not support conversion from type 'INTEGER'.
Did you mean INT_BE?
{noformat}
> StringIndexOutOfBoundsException seen with CONVERT_FROM function
> ---------------------------------------------------------------
>
> Key: DRILL-4755
> URL: https://issues.apache.org/jira/browse/DRILL-4755
> Project: Apache Drill
> Issue Type: Bug
> Components: Query Planning & Optimization
> Affects Versions: 1.7.0
> Reporter: Khurram Faraaz
> Assignee: Volodymyr Vysotskyi
> Attachments: date_data.csv
>
>
> StringIndexOutOfBoundsException seen with CONVERT_FROM function
> Drill git commit ID : 6286c0a4
> {noformat}
> 0: jdbc:drill:schema=dfs.tmp> select CONVERT_FROM(binary_string(columns[0]),'INTEGER') from `binStrDuplcs.csv`;
> Error: SYSTEM ERROR: StringIndexOutOfBoundsException: String index out of range: -5
> [Error Id: 7e8fd49d-c6ac-40ac-849e-cf59145ed5dd on centos-01.qa.lab:31010]
> (org.apache.drill.exec.work.foreman.ForemanException) Unexpected exception during fragment initialization: String index out of range: -5
> org.apache.drill.exec.work.foreman.Foreman.run():271
> java.util.concurrent.ThreadPoolExecutor.runWorker():1142
> java.util.concurrent.ThreadPoolExecutor$Worker.run():617
> java.lang.Thread.run():745
> Caused By (java.lang.StringIndexOutOfBoundsException) String index out of range: -5
> java.lang.String.substring():1931
> org.apache.drill.exec.planner.logical.PreProcessLogicalRel.getConvertFunctionException():244
> org.apache.drill.exec.planner.logical.PreProcessLogicalRel.visit():148
> org.apache.calcite.rel.logical.LogicalProject.accept():132
> org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.preprocessNode():634
> org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.validateAndConvert():196
> org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan():164
> org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan():94
> org.apache.drill.exec.work.foreman.Foreman.runSQL():978
> org.apache.drill.exec.work.foreman.Foreman.run():257
> java.util.concurrent.ThreadPoolExecutor.runWorker():1142
> java.util.concurrent.ThreadPoolExecutor$Worker.run():617
> java.lang.Thread.run():745 (state=,code=0)
> {noformat}
> Stack trace from drillbit.log
> {noformat}
> 2016-06-27 05:31:02,515 [288f45e9-44ad-bc5e-3247-3e0ec4dc68e0:foreman] INFO o.a.drill.exec.work.foreman.Foreman - Query text for query id 288f45e9-44ad-bc5e-3247-3e0ec4dc68e0: select CONVERT_FROM(binary_string(columns[0]),'INTEGER') from `binStrDuplcs.csv`
> 2016-06-27 05:31:02,558 [288f45e9-44ad-bc5e-3247-3e0ec4dc68e0:foreman] ERROR o.a.drill.exec.work.foreman.Foreman - SYSTEM ERROR: StringIndexOutOfBoundsException: String index out of range: -5
> [Error Id: 7e8fd49d-c6ac-40ac-849e-cf59145ed5dd on centos-01.qa.lab:31010]
> org.apache.drill.common.exceptions.UserException: SYSTEM ERROR: StringIndexOutOfBoundsException: String index out of range: -5
> [Error Id: 7e8fd49d-c6ac-40ac-849e-cf59145ed5dd on centos-01.qa.lab:31010]
> at org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:543) ~[drill-common-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
> at org.apache.drill.exec.work.foreman.Foreman$ForemanResult.close(Foreman.java:791) [drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
> at org.apache.drill.exec.work.foreman.Foreman.moveToState(Foreman.java:901) [drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
> at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:271) [drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_91]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_91]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_91]
> Caused by: org.apache.drill.exec.work.foreman.ForemanException: Unexpected exception during fragment initialization: String index out of range: -5
> ... 4 common frames omitted
> Caused by: java.lang.StringIndexOutOfBoundsException: String index out of range: -5
> at java.lang.String.substring(String.java:1931) ~[na:1.8.0_91]
> at org.apache.drill.exec.planner.logical.PreProcessLogicalRel.getConvertFunctionException(PreProcessLogicalRel.java:244) ~[drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
> at org.apache.drill.exec.planner.logical.PreProcessLogicalRel.visit(PreProcessLogicalRel.java:148) ~[drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
> at org.apache.calcite.rel.logical.LogicalProject.accept(LogicalProject.java:132) ~[calcite-core-1.4.0-drill-r11.jar:1.4.0-drill-r11]
> at org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.preprocessNode(DefaultSqlHandler.java:634) ~[drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
> at org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.validateAndConvert(DefaultSqlHandler.java:196) ~[drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
> at org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan(DefaultSqlHandler.java:164) ~[drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
> at org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan(DrillSqlWorker.java:94) ~[drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
> at org.apache.drill.exec.work.foreman.Foreman.runSQL(Foreman.java:978) [drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
> at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:257) [drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
> ... 3 common frames omitted
> {noformat}
> Data used in test
> {noformat}
> [root@centos-01 ~]# cat binStrDuplcs.csv
> '\\x99\\x8c\\x2f\\x77'
> '\\x99\\x8c\\x2f\\x77'
> '\\x99\\x8c\\x2f\\x77'
> '\\x99\\x8c\\x2f\\x77'
> '\\x99\\x8c\\x2f\\x77'
> '\\x99\\x8c\\x2f\\x77'
> '\\x99\\x8c\\x2f\\x77'
> '\\x99\\x8c\\x2f\\x77'
> '\\x99\\x8c\\x2f\\x77'
> '\\x99\\x8c\\x2f\\x77'
> '\\x99\\x8c\\x2f\\x77'
> '\\x99\\x8c\\x2f\\x77'
> '\\x99\\x8c\\x2f\\x77'
> '\\x99\\x8c\\x2f\\x77'
> '\\x99\\x8c\\x2f\\x77'
> '\\x99\\x8c\\x2f\\x77'
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)