[jira] [Commented] (DRILL-6794) Document the JDBC properties required to retrieve result sets in batches while querying large tables

["Arina Ielchiieva (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/DRILL-6794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17029124#comment-17029124 ] 

Arina Ielchiieva commented on DRILL-6794:
-----------------------------------------

Since Drill 1.18 after DRILL-7467, the following plugin configuration should be used to support batches:
{
  "type": "jdbc",
  "driver": "org.postgresql.Driver",
  "url": "jdbc:postgresql://localhost:5959/my_db?defaultRowFetchSize=2",
  "username": "my_user",
  "password": "my_pass",
  "caseInsensitiveTableNames": false,
  "sourceParameters": {
    "defaultAutoCommit": false
  },
  "enabled": true
}

> Document the JDBC properties required to retrieve result sets in batches while querying large tables
> ----------------------------------------------------------------------------------------------------
>
>                 Key: DRILL-6794
>                 URL: https://issues.apache.org/jira/browse/DRILL-6794
>             Project: Apache Drill
>          Issue Type: Bug
>          Components: Documentation
>    Affects Versions: 1.14.0
>            Reporter: Rahul Raj
>            Assignee: Bridget Bevens
>            Priority: Major
>              Labels: doc-impacting
>
> Document the JDBC properties required to retrieve result sets in batches while querying large tables
> Querying large tables using JDBC plugin causes OOM as most JDBC drivers cache the entire result set at the client by default.
> To avoid this additional parameters needs to be specified with the JDBC connection string so that the driver fetches records in batches and reloads when exhausted.
> For postgres driver set autocommit mode to false - jdbc:postgresql://url:port/schema?defaultAutoCommit=false
> Links
> [1] https://issues.apache.org/jira/browse/DRILL-4177
> [2] https://jdbc.postgresql.org/documentation/93/query.html#fetchsize-example
> [3] https://www.postgresql.org/docs/9.3/static/ecpg-sql-set-autocommit.html
> [4] https://jdbc.postgresql.org/documentation/head/ds-cpds.htm



--
This message was sent by Atlassian Jira
(v8.3.4#803005)