[jira] [Created] (QPID-7666) [linearstore] Enhancements to the write buffer default and per-queue options

["Kim van der Riet (JIRA)" <ji...@apache.org>]

Kim van der Riet created QPID-7666:
--------------------------------------

             Summary: [linearstore] Enhancements to the write buffer default and per-queue options
                 Key: QPID-7666
                 URL: https://issues.apache.org/jira/browse/QPID-7666
             Project: Qpid
          Issue Type: Improvement
          Components: C++ Broker
    Affects Versions: qpid-cpp-1.37.0
            Reporter: Kim van der Riet
            Assignee: Kim van der Riet


In applications where large numbers of persistent queues are created on a single broker, control over the default and per-queue store options are important so that resources are not wasted or become exhausted prematurely.

There are two resources of concern:
# Memory allocated per queue for buffering;
# Disk resources consumed for journal files.

Currently, the default setting for memory allocation is 1024kiB/queue (ie 32 pages of 32kiB). The only default setting available is the page size (using the {{wcache-page-size}} option), and the number of pages is set internally according to the following table:

||{{wcache-page-size}}||{{wcache-num-pages}}||total mmap||
||(kiB)||(set interanlly)||(kiB)||
|  4   |  64 |    256 |
|  8   |  64 |    512|
| 16   | 32 |    512|
| 32*  | 32 |   1024|
| 64   | 16 |   1024|
|128   |  8 |   1024|

\* default

This gives in effect only 3 per-queue memory allocation options: 256kiB, 512kiB and 1024kiB.

Currently, the values of {{wcache-page-size}} is limited to the powers of 2 between 4 and 128.

It is suggested that the {{wcache-num-pages}} value be set as as an option rather than internally, which would give the user much greater control over the per-queue memory footprint and the buffer geometry (which can be optimized for the disk hardware and queue load characteristics).

In addition, the upper value limit on {{wcache-page-size}} should be removed (as it is rather arbitrary) and the default value should be revised.

Further, it would be valuable to add these parameters to {{qpid-config}} so that if a queue is created using this tool, the default values for the store parameters can be overridden on a per-queue basis.

Disk resource controls are currently in place for the EFP partition ({{efp-partition}}) and the size of the EFP files ({{efp-file-size}}), both as a default and in {{qpid-config}}. The default values should also be revised.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org