You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@drill.apache.org by "Khurram Faraaz (JIRA)" <ji...@apache.org> on 2017/03/28 17:48:41 UTC
[jira] [Commented] (DRILL-5288) JSON data not read as string when
store.json.all_text_mode=true
[ https://issues.apache.org/jira/browse/DRILL-5288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15945623#comment-15945623 ]
Khurram Faraaz commented on DRILL-5288:
---------------------------------------
Fixed the JSON (removed duplicate keyname key) and reset all config options to default. SELECT returns expected results.
Modifying the subject of this JIRA to, "Duplicate key names must be caught and an error reported to user"
{noformat}
[root@centos-01 ~]# cat f2_m.json
{"key1":"string", "key2":123, "key3":[1,2,3], "key4":true, "key5":false, "key6":null, "key7":{"key9":"b"}, "key8":"2011-08-21"}
{noformat}
{noformat}
0: jdbc:drill:schema=dfs.tmp> reset all;
+-------+---------------+
| ok | summary |
+-------+---------------+
| true | ALL updated. |
+-------+---------------+
1 row selected (0.148 seconds)
0: jdbc:drill:schema=dfs.tmp> select * from `f2_m.json`;
+---------+-------+----------+-------+--------+---------------+-------------+
| key1 | key2 | key3 | key4 | key5 | key7 | key8 |
+---------+-------+----------+-------+--------+---------------+-------------+
| string | 123 | [1,2,3] | true | false | {"key9":"b"} | 2011-08-21 |
+---------+-------+----------+-------+--------+---------------+-------------+
1 row selected (0.179 seconds)
{noformat}
> JSON data not read as string when store.json.all_text_mode=true
> ---------------------------------------------------------------
>
> Key: DRILL-5288
> URL: https://issues.apache.org/jira/browse/DRILL-5288
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Flow
> Reporter: Khurram Faraaz
>
> Setting all text mode to true (at system/session level) does not help in this case. Data from the JSON input file is not returned as string.
> Drill 1.10.0 git commit id: 300e9349
> Data used in test
> {noformat}
> [root@centos-01 ~]# cat f2.json
> {"key":"string", "key":123, "key":[1,2,3], "key":true, "key":false, "key":null, "key":{"key2":"b"}, "key":"2011-08-21"}
> [root@centos-01 ~]#
> {noformat}
> {noformat}
> 0: jdbc:drill:schema=dfs.tmp> alter session set `store.json.all_text_mode`=true;
> +-------+------------------------------------+
> | ok | summary |
> +-------+------------------------------------+
> | true | store.json.all_text_mode updated. |
> +-------+------------------------------------+
> 1 row selected (0.176 seconds)
> 0: jdbc:drill:schema=dfs.tmp> select key from `f2.json`;
> Error: SYSTEM ERROR: IllegalStateException: You tried to start when you are using a ValueWriter of type NullableVarCharWriterImpl.
> Fragment 0:0
> [Error Id: 3cb5b806-53b6-49bf-ab5e-59c666259463 on centos-01.qa.lab:31010] (state=,code=0)
> {noformat}
> stack trace from drillbit.log
> {noformat}
> [Error Id: 3cb5b806-53b6-49bf-ab5e-59c666259463 on centos-01.qa.lab:31010]
> org.apache.drill.common.exceptions.UserException: SYSTEM ERROR: IllegalStateException: You tried to start when you are using a ValueWriter of type NullableVarCharWriterImpl.
> Fragment 0:0
> [Error Id: 3cb5b806-53b6-49bf-ab5e-59c666259463 on centos-01.qa.lab:31010]
> at org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:544) ~[drill-common-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT]
> at org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:293) [drill-java-exec-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT]
> at org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:160) [drill-java-exec-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT]
> at org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:262) [drill-java-exec-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT]
> at org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38) [drill-common-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT]
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_91]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_91]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_91]
> Caused by: java.lang.IllegalStateException: You tried to start when you are using a ValueWriter of type NullableVarCharWriterImpl.
> at org.apache.drill.exec.vector.complex.impl.AbstractFieldWriter.startList(AbstractFieldWriter.java:108) ~[vector-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT]
> at org.apache.drill.exec.vector.complex.impl.NullableVarCharWriterImpl.startList(NullableVarCharWriterImpl.java:88) ~[vector-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT]
> at org.apache.drill.exec.vector.complex.fn.JsonReader.writeDataAllText(JsonReader.java:621) ~[drill-java-exec-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT]
> at org.apache.drill.exec.vector.complex.fn.JsonReader.writeDataAllText(JsonReader.java:466) ~[drill-java-exec-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT]
> at org.apache.drill.exec.vector.complex.fn.JsonReader.writeDataSwitch(JsonReader.java:319) ~[drill-java-exec-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT]
> at org.apache.drill.exec.vector.complex.fn.JsonReader.writeToVector(JsonReader.java:262) ~[drill-java-exec-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT]
> at org.apache.drill.exec.vector.complex.fn.JsonReader.write(JsonReader.java:217) ~[drill-java-exec-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT]
> at org.apache.drill.exec.store.easy.json.JSONRecordReader.next(JSONRecordReader.java:206) ~[drill-java-exec-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT]
> at org.apache.drill.exec.physical.impl.ScanBatch.next(ScanBatch.java:179) ~[drill-java-exec-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT]
> at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119) ~[drill-java-exec-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT]
> at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:109) ~[drill-java-exec-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT]
> at org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:51) ~[drill-java-exec-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT]
> at org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:135) ~[drill-java-exec-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT]
> at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:162) ~[drill-java-exec-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT]
> at org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:104) ~[drill-java-exec-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT]
> at org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.innerNext(ScreenCreator.java:81) ~[drill-java-exec-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT]
> at org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:94) ~[drill-java-exec-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT]
> at org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:232) ~[drill-java-exec-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT]
> at org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:226) ~[drill-java-exec-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT]
> at java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_91]
> at javax.security.auth.Subject.doAs(Subject.java:422) ~[na:1.8.0_91]
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1595) ~[hadoop-common-2.7.0-mapr-1607.jar:na]
> at org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:226) [drill-java-exec-1.10.0-SNAPSHOT.jar:1.10.0-SNAPSHOT]
> ... 4 common frames omitted
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)