[jira] [Created] (QPID-5947) [linearstore] Improvements to queue scalability

["Kim van der Riet (JIRA)" <ji...@apache.org>]

Kim van der Riet created QPID-5947:
--------------------------------------

             Summary: [linearstore] Improvements to queue scalability
                 Key: QPID-5947
                 URL: https://issues.apache.org/jira/browse/QPID-5947
             Project: Qpid
          Issue Type: Improvement
          Components: C++ Broker
            Reporter: Kim van der Riet
            Assignee: Kim van der Riet


The linearstore consumes kernel resources in such a way as to limit the number of possible queues that may be handled on a given hardware system. The current kernel-limited resources are:

#. File descriptors. Currently 1 is used per queue.
# AIO contexts.

Currently both of these have per-process and total default limits which can be modified through configuration of the kernel, but there remains an upper limit which cannot be exceeded. If we intend to be able to persist queues in the millions, changes to the way these resources are used will have to be made.

In addition, a large number of queues will have a negative impact on the time taken to recover them. While this is expected behaviour, there are improvements (such as multi-threaded recovery) that can be made which will help overcome these issues.

This bug is intended as a long-term container for individual changes intended to address these and other queue scalability issues. As these issues are identified and tackled, they should be added as a sub-task of this issue.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org