[jira] [Commented] (IMPALA-10202) Enable file handle cache for ABFS files

["ASF subversion and git services (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/IMPALA-10202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17206209#comment-17206209 ] 

ASF subversion and git services commented on IMPALA-10202:
----------------------------------------------------------

Commit d09294a4a6da142bf1bed7cf9b769c07d6d7baa0 in impala's branch refs/heads/master from Sahil Takiar
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=d09294a ]

IMPALA-10202: Enable file handle cache for ABFS files

Like IMPALA-8428, but for ABFS, instead of S3A. Adds support for adding
ABFS file handles to the file handle cache. Support for ABFSInputStream
unbuffer operations was added in HADOOP-16859.

Ran a full table scan of a 1GB store_sales table on ABFS, made
sure the file handles were cached (validated via the runtime
profile); did this multiple times, against several different
copies of the store_sales table, in order to increase the number
of file handles cached by an impalad.

Tested:
* Tested against a ABFS storage account I have access to

Change-Id: I64f12f832980f4e0207af78368402dd09e370fc3
Reviewed-on: http://gerrit.cloudera.org:8080/16532
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>


> Enable file handle cache for ABFS files
> ---------------------------------------
>
>                 Key: IMPALA-10202
>                 URL: https://issues.apache.org/jira/browse/IMPALA-10202
>             Project: IMPALA
>          Issue Type: Improvement
>            Reporter: Sahil Takiar
>            Assignee: Sahil Takiar
>            Priority: Major
>
> We should enable the file handle cache for ABFS, we have already seen it benefit jobs that read data from S3A.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org