You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@accumulo.apache.org by "Keith Turner (Updated) (JIRA)" <ji...@apache.org> on 2011/11/18 16:18:51 UTC

[jira] [Updated] (ACCUMULO-31) pattern match for bulk-import in-progress markers is not scalable

     [ https://issues.apache.org/jira/browse/ACCUMULO-31?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Keith Turner updated ACCUMULO-31:
---------------------------------

    Affects Version/s:     (was: 1.3.5)
        Fix Version/s: 1.3.5
    
> pattern match for bulk-import in-progress markers is not scalable
> -----------------------------------------------------------------
>
>                 Key: ACCUMULO-31
>                 URL: https://issues.apache.org/jira/browse/ACCUMULO-31
>             Project: Accumulo
>          Issue Type: Improvement
>          Components: gc
>            Reporter: Eric Newton
>            Assignee: Eric Newton
>             Fix For: 1.3.5
>
>   Original Estimate: 2h
>  Remaining Estimate: 2h
>
> When you ask HDFS for tables/*/bulk_*/processing_proc_*, the NameNode seems to be shipping back all the file names, and the client is performing the pattern match.  This results in very bad NameNode performance during accumulo garbage collection.
> This is fixed in the trunk: markers are put in the METADATA table.
> However, the fix is needed in very large accumulo installations presently using 1.3.x.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira