You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@accumulo.apache.org by "Keith Turner (JIRA)" <ji...@apache.org> on 2013/02/12 23:41:14 UTC
[jira] [Commented] (ACCUMULO-1062) large numbers of threads make
in-memory updates slow
[ https://issues.apache.org/jira/browse/ACCUMULO-1062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13577122#comment-13577122 ]
Keith Turner commented on ACCUMULO-1062:
----------------------------------------
Eric and I were poking at this issue using the test program. We found the following code in InMemoryMap is causing the performance problem.
{code:java}
synchronized (this) {
// Can not update mutationCount while writes that started before
// are in progress, this would cause partial mutations to be seen.
// Also, can not continue until mutation count is updated, because
// a read may not see a successful write. Therefore writes must
// wait for writes that started before to finish.
while (kvCount.get() != kv - 1) {
try {
wait();
} catch (InterruptedException ex) {
// ignored
}
}
kvCount.set(kv + numKVs - 1);
notifyAll();
}
{code}
> large numbers of threads make in-memory updates slow
> ----------------------------------------------------
>
> Key: ACCUMULO-1062
> URL: https://issues.apache.org/jira/browse/ACCUMULO-1062
> Project: Accumulo
> Issue Type: Bug
> Components: tserver
> Affects Versions: 1.4.2
> Environment: testing on large clusters
> Reporter: Eric Newton
> Assignee: Keith Turner
> Priority: Critical
> Fix For: 1.5.0
>
> Attachments: SlamTest.java
>
>
> Using hundreds of ingest programs, each running several batch writers, ingest would eventually become slow. Testing (attached) proved that performance fell off as the number of threads increased.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira