[jira] [Commented] (PROTON-1988) [cpp] streamable message support

["Alan Conway (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/PROTON-1988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16736123#comment-16736123 ] 

Alan Conway commented on PROTON-1988:
-------------------------------------

Partial sketch of an API, to get discussion started - nothing final here.

Receiving:
{code:java}
on_delivery_start(delivery& d, streaming_message& m) { new message begins }
on_delivery_more(delivery& d, streaming_message& m) { more arrives }
// Above are only called if on_message() is NOT implemented.


{code}
Sending:

 
{code:java}
on_sendable(sender s) {
  streaming_message sm;
  tracker = s.start_sending(sm);
  // maybe send initial data here...
}
on_tracker_sendable(tracker& t, streaming_message& m) { send more data }


{code}
Pass thru: pass streaming_message from on_delivery_start() to sender::start_sending, implement on_tracker_sendable() to send to credit. No need to implement on_delivery_more() since you can't do anything till it's possible to send.

API extensions:

delivery: read incoming data, check message complete/aborted status.

tracker: write outgoing data, abort message in progress.

streaming_messge: append data (from delivery or app), pop data to write (to tracker or app). Buffer incoming/outgoing data. Methods to construct/access message properties.

Need to think about buffer ownership and copies for send/receive ops.

 check incoming delivery for more data, end of message, aborted message - on delivery not streamable_message as it only applies to inbound
 * allow sender to abort message before completion.
 * For use case 1. (no streaming of headers) the API could be identical to existing proton::message, but without  body() setters/getters and with extra streaming methods.
 * For use case 2 we could possibly use the same API with a "progress indicator" that says which message properties are available and decodes on demand (dispatch does this) but we might want a different API here, maybe more SAX-like.

Flow control: proton should obey session byte flow control i.e. no calls to tracker_sendable unless we have session space. We should provide indication to user of how much session space there is. We may allow the user to exceed session space temporarily if they provide the buffers. The user can implement whatever other per-message limits they want.

> [cpp] streamable message support
> --------------------------------
>
>                 Key: PROTON-1988
>                 URL: https://issues.apache.org/jira/browse/PROTON-1988
>             Project: Qpid Proton
>          Issue Type: Improvement
>          Components: cpp-binding
>    Affects Versions: proton-c-0.26.0
>            Reporter: Alan Conway
>            Priority: Major
>
> C++ API support for streaming large messages in chunks  (too large to fit entire message in memory)
> The C link-level API does support streaming messages in chunks, but the C message encoder/decoder and the proton::message type do. Provide an alternative (or extended) C++ message type with a byte-stream API to send/receive the body data in chunks, without every requiring the entire body to be held in memory.
> Use cases:
>  # Only the message body need be streamed. Properties are encoded/decode as a unit before sending/receiving any of the body. Most client/endpoint applications would work this way.
>  # The entire message, headers, properties and all, need to be streamed, with support for partial decoding of an incoming message and efficient pass thru (incoming to outgoing without decode/re-encode). Only advanced intermediaries are likely to need this (e.g. this is what dispatch router does.)
>  
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org