[GitHub] [druid] andrewm-aero opened a new issue #11789: Feature Request: Support Ingesting JSON Row Arrays

[GitBox <gi...@apache.org>]

andrewm-aero opened a new issue #11789:
URL: https://github.com/apache/druid/issues/11789


   ### Description
   
   Currently, it appears that only JSON data in a line-oriented format will work, i.e.
   ```json
   {"foo":"bar"}
   {"baz":"qux"}
   ```
   
   it would be useful to be able to load data from an array of rows, i.e.
   
   ```json
   [{"foo":"bar"},{"baz":"qux"}]
   ```
   
   Looking at
   https://github.com/apache/druid/blob/master/core/src/main/java/org/apache/druid/data/input/impl/JsonInputFormat.java
   and
   https://github.com/apache/druid/blob/druid-0.22.0/core/src/main/java/org/apache/druid/data/input/impl/JsonReader.java
   
   It seems a new optional boolean field could be added to control this behavior (false by default to preserve existing behavior), and instead of calling
   ```java
   final MappingIterator<JsonNode> delegate = mapper.readValues(parser, JsonNode.class);
   ```
   call
   ```java
   final Iterator<JsonNode> delegate = mapper.readTree(parser).elements()
   ```
   if this field is set to true.
   
   ### Motivation
   
   My team is looking at using druid to periodically scrape results from a bespoke HTTP API endpoint whose implementation we do not control that returns data in a JSON array format, and it would be useful to be able to point druid directly at that endpoint using the existing HTTP input source option, rather than needing to copy and pre-process that data to a separate temporary store beforehand.
   
   I investigated the possibility of using the JavaScript parser to load the object from a string as a workaround, however, it appears this only supports returning one row at a time.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org