You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@qpid.apache.org by "Alan Conway (JIRA)" <ji...@apache.org> on 2017/08/09 15:45:00 UTC
[jira] [Commented] (PROTON-584) Proton-c transport reserves large
buffers for brief use
[ https://issues.apache.org/jira/browse/PROTON-584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16120099#comment-16120099 ]
Alan Conway commented on PROTON-584:
------------------------------------
Not necessary but need to think carefully about how/when to reduce the buffer size. Re-allocating a buffer for each message is expensive, so keeping a buffer that is "big enough for most messages" is a performance win. The buffer also can hold more than one frame, and reading/writing multiple frames in a single read is an important performance win. The current scheme works well for streams of messages that don't have "exceptionallly large" outlier messages - so the trick will be to shrink buffers that have grown because of one "exceptionally large" message without penalizing the "normal sized" message stream by with very small reads or frequent re-allocations.
What are the stats for your message traffic? How much do "exceptional" messages diverge from "normal" messages in size? Would a statically configured min/normal-max settings work (never reduce below min, if you go over normal-max shrink back to normal-max ASAP)? Adaptive schemes based on past traffic/buffer-size/max-read might also be worth considering.
> Proton-c transport reserves large buffers for brief use
> -------------------------------------------------------
>
> Key: PROTON-584
> URL: https://issues.apache.org/jira/browse/PROTON-584
> Project: Qpid Proton
> Issue Type: Improvement
> Components: proton-c
> Affects Versions: 0.8
> Reporter: Cliff Jansen
> Assignee: Alan Conway
> Labels: perf
> Fix For: proton-c-0.19.0
>
>
> When processing transfer frames for incoming messages, Proton requires a temporary buffer holding the whole transfer frame briefly in contiguous space in the transport before moving it to the engine proper which holds the content in a separate memory area.
> pn_transport_capacity grows the buffer in the non-ssl case (transport->input_buf), and openssl.c largely duplicates the code (including the comment about "no limit") for ssl->inbuf for ssl connections.
> Either way, a large message will trigger reserving a similarly large buffer for the rest of the life of the connection.
> Is it necessary for the transport to buffer the whole message body and hang on to that memory?
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org