[GitHub] [drill] arina-ielchiieva opened a new pull request #1901: DRILL-7388: Kafka improvements

[GitBox <gi...@apache.org>]

arina-ielchiieva opened a new pull request #1901: DRILL-7388: Kafka improvements
URL: https://github.com/apache/drill/pull/1901
 
 
   Jira - [DRILL-7388](https://issues.apache.org/jira/browse/DRILL-7388).
   
   1. Upgraded Kafka libraries to 2.3.1 (DRILL-6739).
   2. Added new options to support the same features as native JSON reader:
     a. store.kafka.reader.skip_invalid_records, default: false (DRILL-6723);
     b. store.kafka.reader.allow_nan_inf, default: true;
     c. store.kafka.reader.allow_escape_any_char, default: false.
   3. Fixed issue when Kafka topic contains only one message (DRILL-7388).
   4. Replaced Gson parser with Jackson to parse JSON in the same manner as Drill native Json reader.
   5. Performance improvements: Kafka consumers will be closed async, fixed issue with resource leak (DRILL-7290), moved to debug unnecessary info logging.
   6. Updated bootstrap-storage-plugins.json to reflect actual Kafka connection properties.
   7. Added unit tests.
   8. Refactoring and code clean up.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services