[jira] [Resolved] (IMPALA-5431) Calling FileSystem.Exists() twice in a row for the same partition adds unnecessary latency to metadata loading

["bharath v (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/IMPALA-5431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

bharath v resolved IMPALA-5431.
-------------------------------
       Resolution: Fixed
    Fix Version/s: Impala 2.10.0

MPALA-5431: Remove redundant path exists checks during table load

There are multiple places that do an exists() check on a path and then
perform some subsequent action on it. This pattern results in two
RPCs to the NN (one for the exists() check and one for the subsequent
action). We can avoid the exists() check in these cases since most HDFS
methods on paths throw a FileNotFoundException if the path does not
exist. This can save an RPC to NN and improve the metadata loading time.

Testing: Enough tests already cover this code path. This patch
passed core and exhaustive tests.

Metadata benchmark shows decent increase in perf numbers, for ex:

100K-PARTITIONS-1M-FILES-CUSTOM-05-QUERY-AFTER-INV -20.51%
80-PARTITIONS-250K-FILES-S3-03-RECOVER -20.58%
80-PARTITIONS-250K-FILES-11-DROP-PARTITION -22.13%
80-PARTITIONS-250K-FILES-S3-08-ADD-PARTITION -22.38%
80-PARTITIONS-250K-FILES-S3-12-DROP -23.69%
100K-PARTITIONS-1M-FILES-CUSTOM-11-REFRESH-PARTITION -23.91%
100K-PARTITIONS-1M-FILES-CUSTOM-10-REFRESH-AFTER-ADD-PARTITION -26.04%
100K-PARTITIONS-1M-FILES-CUSTOM-07-REFRESH -26.38%
80-PARTITIONS-250K-FILES-S3-02-CREATE -36.47%
100K-PARTITIONS-1M-FILES-CUSTOM-12-QUERY-PARTITIONS -58.72%
80-PARTITIONS-250K-FILES-S3-01-DROP -95.33%
80-PARTITIONS-250K-FILES-01-DROP -95.93%

Change-Id: Id10ecf64ea2eda2d0f9299c0aa371933eca22281
Reviewed-on: http://gerrit.cloudera.org:8080/7095
Reviewed-by: Bharath Vissapragada <bh...@cloudera.com>
Tested-by: Impala Public Jenkins
---
M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java
M fe/src/main/java/org/apache/impala/common/FileSystemUtil.java
2 files changed, 64 insertions(+), 39 deletions(-)

> Calling FileSystem.Exists() twice in a row for the same partition adds unnecessary latency to metadata loading
> --------------------------------------------------------------------------------------------------------------
>
>                 Key: IMPALA-5431
>                 URL: https://issues.apache.org/jira/browse/IMPALA-5431
>             Project: IMPALA
>          Issue Type: Bug
>          Components: Catalog
>    Affects Versions: Impala 2.9.0
>            Reporter: Mostafa Mokhtar
>            Assignee: bharath v
>              Labels: catalog, ramp-up, s3, scalability
>             Fix For: Impala 2.10.0
>
>         Attachments: After removing redundant fs.exists().jfr, Baseline.jfr
>
>
> FileSystem.exists() is called in loadPartitionFileMetadata then again in refreshFileMetadata which seems redundant. 
> When dealing with a large number of partitions this can become a bottleneck.
> {code}
>  private void loadPartitionFileMetadata(StorageDescriptor storageDescriptor,
>       HdfsPartition partition) throws Exception {
>     Preconditions.checkNotNull(storageDescriptor);
>     Preconditions.checkNotNull(partition);
>     Path partDirPath = new Path(storageDescriptor.getLocation());
>     FileSystem fs = partDirPath.getFileSystem(CONF);
>    if (!fs.exists(partDirPath)) return;
>     refreshFileMetadata(partition);
>   }
> {code}
> {code}
> private void refreshFileMetadata(HdfsPartition partition) throws CatalogException {
>     Path partDir = partition.getLocationPath();
>     Preconditions.checkNotNull(partDir);
>     try {
>       FileSystem fs = partDir.getFileSystem(CONF);
>       if (!fs.exists(partDir)) {
>         partition.setFileDescriptors(new ArrayList<FileDescriptor>());
>         return;
>       }
>       if (!FileSystemUtil.supportsStorageIds(fs)) {
>         synthesizeBlockMetadata(fs, partition);
>         return;
>       }
>       // Index the partition file descriptors by their file names for O(1) look ups.
>       ImmutableMap<String, FileDescriptor> fileDescsByName = Maps.uniqueIndex(
>           partition.getFileDescriptors(), new Function<FileDescriptor, String>() {
>             public String apply(FileDescriptor desc) {
>               return desc.getFileName();
>             }
>           });
> {code}
> Before and after Java profiles attached, the number of socket reads goes down from 1,639 to 1,046.  For a table with 80 partitions and 250K files this gave a 15-20% speedup. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)