[jira] [Resolved] (CARBONDATA-429) Eliminate unnecessary file name check in dictionary cache

["Liang Chen (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/CARBONDATA-429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Liang Chen resolved CARBONDATA-429.
-----------------------------------
    Resolution: Fixed
      Assignee: Jacky Li  (was: Ashok Kumar)

> Eliminate unnecessary file name check in dictionary cache
> ---------------------------------------------------------
>
>                 Key: CARBONDATA-429
>                 URL: https://issues.apache.org/jira/browse/CARBONDATA-429
>             Project: CarbonData
>          Issue Type: Sub-task
>          Components: core
>    Affects Versions: 0.1.1-incubating
>            Reporter: Jacky Li
>            Assignee: Jacky Li
>             Fix For: 1.0.0-incubating
>
>          Time Spent: 5h 50m
>  Remaining Estimate: 0h
>
> 1.there are currently many file name check for each column's dictionary cache, which cause unnecessary calls to HDFS getFileStatus.
> 2.  in checkAndLoadDictionaryData, we get meta file's mtime from hdfs each time we invoke cache.get to check if the local cache is valid or not.  The local dictionary cache may be invalid after parallel data load.  This will in turn increase number of calls to getFileStatus as well.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)