You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@drill.apache.org by "Rahul Challapalli (JIRA)" <ji...@apache.org> on 2015/04/29 17:37:06 UTC
[jira] [Reopened] (DRILL-1673) Flatten function can not work well
with nested arrays.
[ https://issues.apache.org/jira/browse/DRILL-1673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Rahul Challapalli reopened DRILL-1673:
--------------------------------------
I am still seeing an issue with this :
{code}
0: jdbc:drill:schema=dfs_eea> select t.`fixed_column` as fixed_column, flatten(t.`list_column`) from `json_kvgenflatten/DRILL-1673.json` t;
Query failed: SYSTEM ERROR: null
Fragment 0:0
[371d098d-f679-49ab-8f83-48dd71b604ef on qa-node191.qa.lab:31010]
Error: exception while executing query: Failure while executing query. (state=,code=0)
{code}
I attached the new error log
> Flatten function can not work well with nested arrays.
> ------------------------------------------------------
>
> Key: DRILL-1673
> URL: https://issues.apache.org/jira/browse/DRILL-1673
> Project: Apache Drill
> Issue Type: Bug
> Components: Functions - Drill
> Affects Versions: 0.7.0
> Environment: 0.7.0
> Reporter: Hao Zhu
> Assignee: Mehant Baid
> Priority: Blocker
> Fix For: 0.7.0
>
> Attachments: DRILL-1673.patch, error.log
>
>
> Flatten function failed to scan nested arrays , for example something like ""[[ ]]"".
> The only difference of JSON files between below 2 tests is
> "num":[1,2,3]
> VS
> "num":[[1,2,3]]
> ==============Test 1 (Works well):==============
> file:
> {code}
> {"fixed_column":"abc", "list_column":[{"id1":"1","name":"zhu", "num": [1,2,3]}, {"id1":"2","name":"hao", "num": [4,5,6]} ]}
> {code}
> SQL:
> {code}
> 0: jdbc:drill:zk=local> select t.`fixed_column` as fixed_column, flatten(t.`list_column`) from dfs.root.`/Users/hzu/Documents/sharefolder/hp/n2.json` as t;
> +--------------+------------+
> | fixed_column | EXPR$1 |
> +--------------+------------+
> | abc | {"id1":"1","name":"zhu","num":[1,2,3]} |
> | abc | {"id1":"2","name":"hao","num":[4,5,6]} |
> +--------------+------------+
> 2 rows selected (0.154 seconds)
> {code}
> ==============Test 2 (Failed):==============
> file:
> {code}
> {"fixed_column":"abc", "list_column":[{"id1":"1","name":"zhu", "num": [[1,2,3]]}, {"id1":"2","name":"hao", "num": [[4,5,6]]} ]}
> {code}
> SQL:
> {code}
> 0: jdbc:drill:zk=local> select t.`fixed_column` as fixed_column, flatten(t.`list_column`) from dfs.root.`/Users/hzu/Documents/sharefolder/hp/n3.json` as t;
> +--------------+------------+
> | fixed_column | EXPR$1 |
> +--------------+------------+
> Query failed: Failure while running fragment.[ df28347b-fac1-497d-b9c5-a313ba77aa4d on 10.250.0.115:31010 ]
> (java.lang.UnsupportedOperationException)
> org.apache.drill.exec.vector.complex.RepeatedListVector$RepeatedListTransferPair.splitAndTransfer():339
> org.apache.drill.exec.vector.complex.RepeatedMapVector$SingleMapTransferPair.splitAndTransfer():305
> org.apache.drill.exec.test.generated.FlattenerGen22.flattenRecords():93
> org.apache.drill.exec.physical.impl.flatten.FlattenRecordBatch.doWork():152
> org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext():89
> org.apache.drill.exec.physical.impl.flatten.FlattenRecordBatch.innerNext():118
> org.apache.drill.exec.record.AbstractRecordBatch.next():106
> org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next():124
> org.apache.drill.exec.record.AbstractRecordBatch.next():86
> org.apache.drill.exec.record.AbstractRecordBatch.next():76
> org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext():52
> org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext():129
> org.apache.drill.exec.record.AbstractRecordBatch.next():106
> org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next():124
> org.apache.drill.exec.physical.impl.BaseRootExec.next():67
> org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.innerNext():122
> org.apache.drill.exec.physical.impl.BaseRootExec.next():57
> org.apache.drill.exec.work.fragment.FragmentExecutor.run():105
> org.apache.drill.exec.work.WorkManager$RunnableWrapper.run():249
> .......():0
> java.lang.RuntimeException: java.sql.SQLException: Failure while executing query.
> at sqlline.SqlLine$IncrementalRows.hasNext(SqlLine.java:2514)
> at sqlline.SqlLine$TableOutputFormat.print(SqlLine.java:2148)
> at sqlline.SqlLine.print(SqlLine.java:1809)
> at sqlline.SqlLine$Commands.execute(SqlLine.java:3766)
> at sqlline.SqlLine$Commands.sql(SqlLine.java:3663)
> at sqlline.SqlLine.dispatch(SqlLine.java:889)
> at sqlline.SqlLine.begin(SqlLine.java:763)
> at sqlline.SqlLine.start(SqlLine.java:498)
> at sqlline.SqlLine.main(SqlLine.java:460)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)