[jira] [Commented] (DRILL-1728) Better error messages on Drill JSON read error

["Jason Altekruse (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/DRILL-1728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14223913#comment-14223913 ] 

Jason Altekruse commented on DRILL-1728:
----------------------------------------

I think this fix could happen in two places. The message is coming out of the Drill Field/Map Writer interfaces that allow us to easily read and write complex data. I think that these need to produce user consumable messages, as there are other parts of the Drill code (such as the complex parquet reader, covertFromJSON, kvgen, etc.) that use this interface and will fail in the same manner currently.

On top of this fix, it would be useful if the various consumers of this interface, like the json reader would provide additional context information on top of the low level (but soon to be more readable) error. For the JSON reader it would be nice if this could include a filename and line number and field in the JSON document where the error occurred. For components later in execution this becomes less straight forward, as the data streaming in from several different incoming operators will come in a random order. It might be possible to include whatever information we have about the current or last successfully handled record in the error message to help users identify the issue in the dataset.

There is also the outstanding patch for just inserting null in the dataset when there is an error. [~mehant] I could not find the issue number for this, could you post it here? I think this needs to be merged, and if it does not return empty list or empty map appropriately when we have an error operating on a complex type, it should be enhanced to give the same flexibility if the users need to interact with imperfect complex data.

> Better error messages on Drill JSON read error
> ----------------------------------------------
>
>                 Key: DRILL-1728
>                 URL: https://issues.apache.org/jira/browse/DRILL-1728
>             Project: Apache Drill
>          Issue Type: Bug
>            Reporter: Tomer Shiran
>            Assignee: Jason Altekruse
>             Fix For: 0.7.0
>
>
> {code}
> 0: jdbc:drill:zk=localhost:2181> SELECT * FROM dfs.root.`Users/tshiran/Development/demo/data/yelp/business.json` WHERE true and REPEATED_CONTAINS(categories, 'Australian');
> +-------------+--------------+------------+------------+------------+------------+--------------+------------+------------+------------+------------+------------+------------+------------+---------------+
> | business_id | full_address |   hours    |    open    | categories |    city    | review_count |    name    | longitude  |   state    |   stars    |  latitude  | attributes |    type    | neighborhoods |
> +-------------+--------------+------------+------------+------------+------------+--------------+------------+------------+------------+------------+------------+------------+------------+---------------+
> Query failed: Query stopeed., You tried to start when you are using a ValueWriter of type NullableBitWriterImpl. [ e5bafa1e-6226-443d-80fd-51e18f330899 on 172.17.3.132:31010 ]
> java.lang.RuntimeException: java.sql.SQLException: Failure while executing query.
> 	at sqlline.SqlLine$IncrementalRows.hasNext(SqlLine.java:2514)
> 	at sqlline.SqlLine$TableOutputFormat.print(SqlLine.java:2148)
> 	at sqlline.SqlLine.print(SqlLine.java:1809)
> 	at sqlline.SqlLine$Commands.execute(SqlLine.java:3766)
> 	at sqlline.SqlLine$Commands.sql(SqlLine.java:3663)
> 	at sqlline.SqlLine.dispatch(SqlLine.java:889)
> 	at sqlline.SqlLine.begin(SqlLine.java:763)
> 	at sqlline.SqlLine.start(SqlLine.java:498)
> 	at sqlline.SqlLine.main(SqlLine.java:460)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)