You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@accumulo.apache.org by "Keith Turner (Updated) (JIRA)" <ji...@apache.org> on 2011/11/18 16:18:51 UTC
[jira] [Updated] (ACCUMULO-31) pattern match for bulk-import
in-progress markers is not scalable
[ https://issues.apache.org/jira/browse/ACCUMULO-31?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Keith Turner updated ACCUMULO-31:
---------------------------------
Affects Version/s: (was: 1.3.5)
Fix Version/s: 1.3.5
> pattern match for bulk-import in-progress markers is not scalable
> -----------------------------------------------------------------
>
> Key: ACCUMULO-31
> URL: https://issues.apache.org/jira/browse/ACCUMULO-31
> Project: Accumulo
> Issue Type: Improvement
> Components: gc
> Reporter: Eric Newton
> Assignee: Eric Newton
> Fix For: 1.3.5
>
> Original Estimate: 2h
> Remaining Estimate: 2h
>
> When you ask HDFS for tables/*/bulk_*/processing_proc_*, the NameNode seems to be shipping back all the file names, and the client is performing the pattern match. This results in very bad NameNode performance during accumulo garbage collection.
> This is fixed in the trunk: markers are put in the METADATA table.
> However, the fix is needed in very large accumulo installations presently using 1.3.x.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira