[jira] [Updated] (ASTERIXDB-1341) Defer the file path decision into NC

["Till Westmann (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/ASTERIXDB-1341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Till Westmann updated ASTERIXDB-1341:
-------------------------------------
    Assignee: Yingyi Bu

> Defer the file path decision into NC
> ------------------------------------
>
>                 Key: ASTERIXDB-1341
>                 URL: https://issues.apache.org/jira/browse/ASTERIXDB-1341
>             Project: Apache AsterixDB
>          Issue Type: Improvement
>            Reporter: Yingyi Bu
>            Assignee: Yingyi Bu
>
> Currently, the decision of storage file paths is made within the compiler, e.g., in StoragePathUtil.java. It would be nice to defer the decision to NCs at runtime. In this way, the index search, bulkload, insert/delete will have a fixed degree-of-parallelism but in each NC they take file paths from Dataset/Index lifecycle manager.  
> This will bring in the following benefits:
> 1. the degree of parallelism can be different from the number of file paths that storage-related operators work with;
> 2. it avoids shipping all-file-paths (within the JobSpecification) to every node for a query.  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)