[jira] [Created] (DRILL-5279) JsonRecordReader inefficient: allocates temp direct memory buffers

["Paul Rogers (JIRA)" <ji...@apache.org>]

Paul Rogers created DRILL-5279:
----------------------------------

             Summary: JsonRecordReader inefficient: allocates temp direct memory buffers
                 Key: DRILL-5279
                 URL: https://issues.apache.org/jira/browse/DRILL-5279
             Project: Apache Drill
          Issue Type: Improvement
    Affects Versions: 1.10.0
            Reporter: Paul Rogers
            Priority: Minor


The {{JsonRecordReader}} class uses the {{VarCharWriter}} class to add a string value to a VarChar vector. The {{VarCharWriter}} method to add a string has this signature:

{code}
void writeVarChar(int start, int end, DrillBuf buffer);
{code}

This means that the {{JsonRecordReader}} has to convert the JSON string value to a direct memory buffer, then write it into the buffer.

Direct memory buffers are expensive to create and should be done only to hold large amounts of semi-persistent data (such as value vectors.)

Extend the {{VarCharWriter}} to accept a Java String and/or heap buffer to avoid having to incur the overhead of the direct memory temporary buffer.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)