[jira] [Commented] (IMPALA-9818) Add fetch size as option to impala shell

["ASF subversion and git services (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/IMPALA-9818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17132559#comment-17132559 ] 

ASF subversion and git services commented on IMPALA-9818:
---------------------------------------------------------

Commit 3088ca8580ed2f8c87235cbee3f3b3a90c292218 in impala's branch refs/heads/master from Sahil Takiar
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=3088ca8 ]

IMPALA-9818: Add fetch size as option to impala shell

Adds the option --fetch_size to the Impala shell. This new option allows
users to specify the fetch size used when issuing fetch RPCs to the
Impala Coordinator (e.g. TFetchResultsReq and BeeswaxService.fetch).
This parameter applies for all client protocols: beeswax, hs2, hs2-http.
The default --fetch_size is set to 10240 (10x the default batch size).

The new --fetch_size parameter is most effective when result spooling is
enabled. When result spooling is disabled, Impala can only return a
single row batch per fetch RPC (so 1024 rows by default). When result
spooling is enabled, Impala can return up to 100 row batches per fetch
request.

Removes some logic in the the impala_client.py file that attempts to
simulate a fetch_size. The code would issue multiple fetch requests to
fullfill the given fetch_size. This logic is no longer needed now that
result spooling is available.

Testing:
* Ran core tests
* Added new tests in test_shell_client.py and test_shell_commandline.py

Change-Id: I8dc7962aada6b38795241d067a99bd94fabca57b
Reviewed-on: http://gerrit.cloudera.org:8080/16041
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Sahil Takiar <st...@cloudera.com>


> Add fetch size as option to impala shell
> ----------------------------------------
>
>                 Key: IMPALA-9818
>                 URL: https://issues.apache.org/jira/browse/IMPALA-9818
>             Project: IMPALA
>          Issue Type: Improvement
>          Components: Clients
>            Reporter: Sahil Takiar
>            Assignee: Sahil Takiar
>            Priority: Major
>
> The impala shell should have an option to control the fetch size (e.g. the number of rows fetched at a time). Currently the value is hard-coded to 1024. Other clients (e.g. JDBC) have similar options (e.g. Statement#setFetchSize).
> When result spooling is enabled, setting a higher fetch size can improve performance for clients with a high RTT to/from the Impala coordinator.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org