You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@qpid.apache.org by "Kim van der Riet (JIRA)" <ji...@apache.org> on 2017/02/08 21:25:41 UTC
[jira] [Resolved] (QPID-7666) [linearstore] Enhancements to the
write buffer default and per-queue options
[ https://issues.apache.org/jira/browse/QPID-7666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kim van der Riet resolved QPID-7666.
------------------------------------
Resolution: Fixed
* Added option {{--wcache-num-pages}}, removed internal table/formulas for determining this value and connected to option.
* Set default for {{--wcache-num-pages}} to 16.
* Changed default for {{--wcache-page-size}} from 32 to 16.
* Added wiring to set the values of {{wcache-page-size}} and {{wcache-num-pages}} from qpid-config by looking for the values _*"qpid.wcache-page-size"*_ and _*"qpid.wcache-num-pages"*_ in the queue create arguments. This will allow {{qpid-config}} to use these as keys to set these values on queue creation.
* Added options {{\-\-wcache-page-size}} and {{\-\-wcache-num-pages}} to qpid-config as queue create options. This allows these parameters to be set when a queue is created using qpid-config.
* Tidy-up of some linearstore messages. In particular, when the broker is stopped, the linearstore no longer uses message
{noformat}
[Store] info Linear Store: Journal "<queue-name>": Destroyed
{noformat}
which may be confusing (and cause some unnecessary panic to users), but rather
{noformat}
[Store] info Linear Store: Journal "<queue-name>": Stopped
{noformat}
* When a queue is created using parameters *other* than the defaults, these are now reported at _info_ level logging.
> [linearstore] Enhancements to the write buffer default and per-queue options
> ----------------------------------------------------------------------------
>
> Key: QPID-7666
> URL: https://issues.apache.org/jira/browse/QPID-7666
> Project: Qpid
> Issue Type: Improvement
> Components: C++ Broker
> Affects Versions: qpid-cpp-1.37.0
> Reporter: Kim van der Riet
> Assignee: Kim van der Riet
>
> In applications where large numbers of persistent queues are created on a single broker, control over the default and per-queue store options are important so that resources are not wasted or become exhausted prematurely.
> There are two resources of concern:
> # Memory allocated per queue for buffering;
> # Disk resources consumed for journal files.
> Currently, the default setting for memory allocation is 1024kiB/queue (ie 32 pages of 32kiB). The only default setting available is the page size (using the {{wcache-page-size}} option), and the number of pages is set internally according to the following table:
> ||{{wcache-page-size}}||{{wcache-num-pages}}||total mmap||
> ||(kiB)||(set interanlly)||(kiB)||
> | 4 | 64 | 256 |
> | 8 | 64 | 512|
> | 16 | 32 | 512|
> | 32* | 32 | 1024|
> | 64 | 16 | 1024|
> |128 | 8 | 1024|
> \* default
> This gives in effect only 3 per-queue memory allocation options: 256kiB, 512kiB and 1024kiB.
> Currently, the values of {{wcache-page-size}} is limited to the powers of 2 between 4 and 128.
> It is suggested that the {{wcache-num-pages}} value be set as as an option rather than internally, which would give the user much greater control over the per-queue memory footprint and the buffer geometry (which can be optimized for the disk hardware and queue load characteristics).
> In addition, the upper value limit on {{wcache-page-size}} should be removed (as it is rather arbitrary) and the default value should be revised.
> Further, it would be valuable to add these parameters to {{qpid-config}} so that if a queue is created using this tool, the default values for the store parameters can be overridden on a per-queue basis.
> Disk resource controls are currently in place for the EFP partition ({{efp-partition}}) and the size of the EFP files ({{efp-file-size}}), both as a default and in {{qpid-config}}. The default values should also be revised.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org