[jira] [Commented] (TAJO-1952) Implement PartitionFileFragment

["Tajo QA (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/TAJO-1952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15082658#comment-15082658 ] 

Tajo QA commented on TAJO-1952:
-------------------------------

{color:red}*-1 overall.*{color}  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12780492/TAJO-1952_2.patch
  against master revision release-0.9.0-rc0-619-g62cc6f6.

    {color:green}+1 @author.{color}  The patch does not contain any @author tags.

    {color:green}+1 tests included.{color}  The patch appears to include 3 new or modified test files.

    {color:green}+1 javac.{color}  The applied patch does not increase the total number of javac compiler warnings.

    {color:green}+1 javadoc.{color}  The applied patch does not increase the total number of javadoc warnings.

    {color:green}+1 checkstyle.{color}  The patch generated 0 code style errors.

    {color:red}-1 findbugs.{color}  The patch appears to cause Findbugs (version 2.0.3) to fail.

    {color:green}+1 release audit.{color}  The applied patch does not increase the total number of release audit warnings.

    {color:green}+1 core tests.{color}  The patch passed unit tests in tajo-core tajo-core-tests tajo-plan tajo-storage/tajo-storage-hdfs.

Test results: https://builds.apache.org/job/PreCommit-TAJO-Build/900//testReport/
Findbugs results: https://builds.apache.org/job/PreCommit-TAJO-Build/900//findbugsResult
Console output: https://builds.apache.org/job/PreCommit-TAJO-Build/900//console

This message is automatically generated.

> Implement PartitionFileFragment
> -------------------------------
>
>                 Key: TAJO-1952
>                 URL: https://issues.apache.org/jira/browse/TAJO-1952
>             Project: Tajo
>          Issue Type: Improvement
>          Components: Planner/Optimizer, Storage
>            Reporter: Jaehwa Jung
>            Assignee: Jaehwa Jung
>             Fix For: 0.12.0
>
>         Attachments: TAJO-1952.patch, TAJO-1952_2.patch
>
>
> Currently, PartitionedTableScanNode contains the list of partitions and it seems to me that the list has some problems as following:
> 1. Duplicate Informs: Task contains Fragment which specify target directory or target file for scanning. A path of partition lists already would write to Fragment. 
> 2. Network Resource: When scanning lost of partition, it will occupy network resource, for example, several hundred kilobytes or more. It looks like an unnecessary resource because Fragment already has the path of partitions.
> I want to improve above problems by implementing new Fragment called PartitionedFileFragment. Currently, I'm planning the implementation as following:
> * PartitionedFileFragment will borrow FileFragment and it contains the partition path and the partition key values.  
> * Remove the path array of partitions from PartitionedTableScanNode. 
> * Implement a method for getting filtered partition directories in FileTableSpace.
> * Implement a method for making PartitionedFileFragment array.
> * Before making splits, call above method and use it for making splits.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)