You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@accumulo.apache.org by "Eric Newton (Resolved) (JIRA)" <ji...@apache.org> on 2011/10/13 21:51:11 UTC

[jira] [Resolved] (ACCUMULO-31) pattern match for bulk-import in-progress markers is not scalable

     [ https://issues.apache.org/jira/browse/ACCUMULO-31?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Eric Newton resolved ACCUMULO-31.
---------------------------------

    Resolution: Fixed
    
> pattern match for bulk-import in-progress markers is not scalable
> -----------------------------------------------------------------
>
>                 Key: ACCUMULO-31
>                 URL: https://issues.apache.org/jira/browse/ACCUMULO-31
>             Project: Accumulo
>          Issue Type: Improvement
>          Components: gc
>    Affects Versions: 1.3.5
>            Reporter: Eric Newton
>            Assignee: Eric Newton
>   Original Estimate: 2h
>  Remaining Estimate: 2h
>
> When you ask HDFS for tables/*/bulk_*/processing_proc_*, the NameNode seems to be shipping back all the file names, and the client is performing the pattern match.  This results in very bad NameNode performance during accumulo garbage collection.
> This is fixed in the trunk: markers are put in the METADATA table.
> However, the fix is needed in very large accumulo installations presently using 1.3.x.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira