[GitHub] [arrow-rs] jiangzhx commented on issue #4105: add specific error log for empty JSON array

["jiangzhx (via GitHub)" <gi...@apache.org>]

jiangzhx commented on issue #4105:
URL: https://github.com/apache/arrow-rs/issues/4105#issuecomment-1517307815

   > Btw I was able to fix this error by removing all empty arrays from my json source data.
   > 
   > But it would have been nice if it was possible to pass some option to ignore un-inferable fields like this so I don't get errors on 'SELECT * ...'
   
   In fact, you do not need to remove empty arrays. This situation only occurs when a file has just one line, like `{"items": []}`.
   
   If the file contains more than one line, such as:
   
   ```
   {"items": []}
   {"items": [1,2]}
   ```
   
   then arrow_json::reader::infer_json_schema will use the second line to infer the data type of the items. Therefore, you should not encounter errors when using "SELECT *".
   
   SELECT *  should return 
   ```
   +--------+
   | items  |
   +--------+
   | []     |
   | [1, 2] |
   +--------+
   ```
   
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org