[jira] [Resolved] (ACCUMULO-3259) Randomwalk Bulk.xml create queued threads without bound.

["Josh Elser (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/ACCUMULO-3259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Josh Elser resolved ACCUMULO-3259.
----------------------------------
    Resolution: Fixed

> Randomwalk Bulk.xml create queued threads without bound.
> --------------------------------------------------------
>
>                 Key: ACCUMULO-3259
>                 URL: https://issues.apache.org/jira/browse/ACCUMULO-3259
>             Project: Accumulo
>          Issue Type: Improvement
>          Components: test
>    Affects Versions: 1.5.2, 1.6.1
>            Reporter: Josh Elser
>            Assignee: Josh Elser
>             Fix For: 1.6.2, 1.7.0
>
>          Time Spent: 20m
>  Remaining Estimate: 0h
>
> While looking at ACCUMULO-3257, I saw that Bulk.xml really goes crazy creating new threads of bulk-imported related work without respect to the actual available resources.
> For example, on a single node after about 5 minutes, I can see a backlog of about 600+ tasks.
> This doesn't really hurt things server-side because the threadpool running these tasks is bounded, but it unnecessarily increases the load on the test runner. We can estimate the "throughput" of the system based on the number of tservers available, skip the queueing of new tasks when we have an excessive number already queued (relative to the number of tservers available), and then wait a bit before we continue to the next node in Bulk.xml.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)