[jira] [Resolved] (TAJO-2063) Refactor FileTablespace::commitOutputData.

["Jaehwa Jung (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/TAJO-2063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jaehwa Jung resolved TAJO-2063.
-------------------------------
    Resolution: Won't Fix

> Refactor FileTablespace::commitOutputData.
> ------------------------------------------
>
>                 Key: TAJO-2063
>                 URL: https://issues.apache.org/jira/browse/TAJO-2063
>             Project: Tajo
>          Issue Type: Improvement
>          Components: Storage
>            Reporter: Jaehwa Jung
>            Assignee: Jaehwa Jung
>
> {{FileTablespace::commitOutputData}} has some problems as following:
> First, it is too long and complexed because it handles various cases in a single method. We need to refactor this method into several small and well-defined methods. 
> Second, {{FileSystem::listStatus}} is widely used while committing output data. It especially causes a lot of overhead to list partitioned directories in S3. Also it occurs in HDFS too with large partitioned tables. We need to minimize its usage.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)