[jira] [Created] (DRILL-6794) Document the JDBC properties required to retrieve result sets in batches while querying large tables

["Rahul Raj (JIRA)" <ji...@apache.org>]

Rahul Raj created DRILL-6794:
--------------------------------

             Summary: Document the JDBC properties required to retrieve result sets in batches while querying large tables
                 Key: DRILL-6794
                 URL: https://issues.apache.org/jira/browse/DRILL-6794
             Project: Apache Drill
          Issue Type: Bug
          Components: Documentation
    Affects Versions: 1.14.0
            Reporter: Rahul Raj


Document the JDBC properties required to retrieve result sets in batches while querying large tables

Querying large tables using JDBC plugin causes OOM as most JDBC drivers cache the entire result set at the client by default.

To avoid this additional parameters needs to be specified with the JDBC connection string so that the driver fetches records in batches and reloads when exhausted.

For postgres driver set autocommit mode to false - jdbc:postgresql://url:port/schema?defaultAutoCommit=false

Links

[1] https://issues.apache.org/jira/browse/DRILL-4177

[2] https://jdbc.postgresql.org/documentation/93/query.html#fetchsize-example

[3] https://www.postgresql.org/docs/9.3/static/ecpg-sql-set-autocommit.html

[4] https://jdbc.postgresql.org/documentation/head/ds-cpds.htm



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)