[jira] [Comment Edited] (CALCITE-2322) Add fetch size support to connection url and JDBC statement

["Kevin Minder (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CALCITE-2322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16505070#comment-16505070 ] 

Kevin Minder edited comment on CALCITE-2322 at 6/7/18 9:12 PM:
---------------------------------------------------------------

Sorry, I'm still uncertain.  I specifically chose "fetch_size" because it aligned with the style of all of the other URL parameters in BuiltInConnectionProperty (i.e. avatica_user, avatica_password, httpclient_factory, httpclient_impl, truststore_password, hostname_verification).  Can you confirm that you want a URL like below?

jdbc:phoenix:thin:url=...;avatica_user=u;avatica_password=p;httpclient_impl=c;truststore_password=t;*fetchSize=1000*
{code:java}
/** Avatica-based authentication user name */
AVATICA_USER("avatica_user", Type.STRING, null, false),

/** Avatica-based authentication password */
AVATICA_PASSWORD("avatica_password", Type.STRING, null, false),

/** Factory for constructing http clients. */
HTTP_CLIENT_FACTORY("httpclient_factory", Type.PLUGIN,
    AvaticaHttpClientFactoryImpl.class.getName(), false),

/** HttpClient implementation class name. */
HTTP_CLIENT_IMPL("httpclient_impl", Type.STRING, null, false),

/** Principal to use to perform Kerberos login. */
PRINCIPAL("principal", Type.STRING, null, false),

/** Keytab to use to perform Kerberos login. */
KEYTAB("keytab", Type.STRING, null, false),

/** Truststore for SSL/TLS communication */
TRUSTSTORE("truststore", Type.STRING, null, false),

/** Password for the truststore */
TRUSTSTORE_PASSWORD("truststore_password", Type.STRING, null, false),

HOSTNAME_VERIFICATION("hostname_verification", Type.ENUM, HostnameVerification.STRICT,
    HostnameVerification.class, false),

/** Fetch size limit, default is 100 rows. */
FETCH_SIZE("fetch_size", Type.NUMBER, AvaticaStatement.DEFAULT_FETCH_SIZE, false);{code}
Most of the ones you mention (i.e. escapeProcessing, fetchDirection, maxRows, poolable, queryTimeout) don't even appear to be settable via the connection URL.


was (Author: kminder):
Sorry, I'm still uncertain.  I specifically chose "fetch_size" because it aligned with the style of all of the other URL parameters in BuiltInConnectionProperty (i.e. avatica_user, avatica_password, httpclient_factory, httpclient_impl, truststore_password, hostname_verification).  Can you confirm that you want a URL like below?

jdbc:phoenix:thin:url=...;avatica_user=u;avatica_password=p;httpclient_impl=c;truststore_password=t;fetchSize=1000
{code:java}
/** Avatica-based authentication user name */
AVATICA_USER("avatica_user", Type.STRING, null, false),

/** Avatica-based authentication password */
AVATICA_PASSWORD("avatica_password", Type.STRING, null, false),

/** Factory for constructing http clients. */
HTTP_CLIENT_FACTORY("httpclient_factory", Type.PLUGIN,
    AvaticaHttpClientFactoryImpl.class.getName(), false),

/** HttpClient implementation class name. */
HTTP_CLIENT_IMPL("httpclient_impl", Type.STRING, null, false),

/** Principal to use to perform Kerberos login. */
PRINCIPAL("principal", Type.STRING, null, false),

/** Keytab to use to perform Kerberos login. */
KEYTAB("keytab", Type.STRING, null, false),

/** Truststore for SSL/TLS communication */
TRUSTSTORE("truststore", Type.STRING, null, false),

/** Password for the truststore */
TRUSTSTORE_PASSWORD("truststore_password", Type.STRING, null, false),

HOSTNAME_VERIFICATION("hostname_verification", Type.ENUM, HostnameVerification.STRICT,
    HostnameVerification.class, false),

/** Fetch size limit, default is 100 rows. */
FETCH_SIZE("fetch_size", Type.NUMBER, AvaticaStatement.DEFAULT_FETCH_SIZE, false);{code}
Most of the ones you mention (i.e. escapeProcessing, fetchDirection, maxRows, poolable, queryTimeout) don't even appear to be settable via the connection URL.

> Add fetch size support to connection url and JDBC statement
> -----------------------------------------------------------
>
>                 Key: CALCITE-2322
>                 URL: https://issues.apache.org/jira/browse/CALCITE-2322
>             Project: Calcite
>          Issue Type: Improvement
>          Components: core
>    Affects Versions: 1.11.0
>            Reporter: Kevin Minder
>            Assignee: Julian Hyde
>            Priority: Major
>
> Currently the remote driver defaults to hard coded fetch size of 100 rows.  When a connection is operating in HTTP mode having such a small fetch size can add enormous overhead.  This is especially true if TLS connections are used and made worse if each connection flows throw multiple proxies.  Consider that 100K rows returned 100 rows at a time will make 1K HTTP POST requests.  One might say that nobody should ever do that but some tools like Spotfire may end up doing this.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)