Re: [DISCUSS] RPC upgrade

[Walt Karas <wk...@oath.com.INVALID>]

For inter-process comm on a single server, why not KISS and use
message queues and POD structs?

On Mon, Aug 27, 2018 at 2:43 PM, Alan Carroll
<so...@oath.com.invalid> wrote:
> Because most people probably dropped off the previous discussion on this
> list, I want to call out that, as far as I can tell, the consensus for
> upgrading the internal RPC mechanism is to use a JSON based message
> exchange compliant with JSON-RPC 2.0 (https://www.jsonrpc.org/). We can use
> YAMLCPP to parse messages and I think we can easily (externally) tweak
> YAMLCPP to output JSON. This will be very interoperable with external tools
> along with an easy way to pass relatively complex data between
> traffic_manager / traffic_server and external tools like traffic_ctl in
> both directions.

Re: [DISCUSS] RPC upgrade

[Alan Carroll <so...@oath.com.INVALID>]

Yes, you could add a length field, and then a ID field (to dispatch), etc.
It gets non-trivial quickly especially if you want it available to third
parties (e.g., someone wants to write the equivalent of traffic_ctl). For
plugins, we already have a plugin message capability which is very crude,
it would be useful if it could be extended to use the same format as the
internal RPC. This would make it possible to query plugin state, something
done now by dumping to traffic.out. It would be much cleaner and powerful
to be able to directly query. E.g. asking the CARP plugins things about it
hotlist, or when the Layer 7 Routing stuff is running, interacting with
some of load balancing or parent control plugins (e.g., telling a parent
selector "this parent is offline"). Overall, JSON-RPC seems easy to do,
because we have an internal parser now. It's a standard so third party
interoperability is easy and we don't have to design much of anything. It's
a lot more flexible than a POD based exchange because adding or omitting
data per message is easy.

On Tue, Sep 4, 2018 at 11:11 AM Walt Karas <wk...@oath.com.invalid> wrote:

> All messages could start with a byte length.  Don't see the plugin
> issue unless you mean we need Lua plugins to use IPC.
>
>

Re: [DISCUSS] RPC upgrade

[Walt Karas <wk...@oath.com.INVALID>]

All messages could start with a byte length.  Don't see the plugin
issue unless you mean we need Lua plugins to use IPC.


On Tue, Sep 4, 2018 at 11:02 AM, Alan Carroll
<so...@oath.com.invalid> wrote:
> That's what we have now and it's not worked out well. The problem is there
> is a central dispatcher and it needs to be able to parse the messages at
> least to the extent of knowing the size of the messages. Otherwise the
> essential problem remains that the dispatcher can't be generic and any
> update in set of messages or content requires modifying that code. This in
> turn makes it effectively impossible for plugins to participate in any
> meaningful way.
>
> On Tue, Sep 4, 2018 at 9:41 AM Walt Karas <wk...@oath.com.invalid> wrote:
>
>> For inter-process comm on a single server, why not KISS and use
>> message queues and POD structs?
>>
>> On Mon, Aug 27, 2018 at 2:43 PM, Alan Carroll
>> <so...@oath.com.invalid> wrote:
>> > Because most people probably dropped off the previous discussion on this
>> > list, I want to call out that, as far as I can tell, the consensus for
>> > upgrading the internal RPC mechanism is to use a JSON based message
>> > exchange compliant with JSON-RPC 2.0 (https://www.jsonrpc.org/). We can
>> use
>> > YAMLCPP to parse messages and I think we can easily (externally) tweak
>> > YAMLCPP to output JSON. This will be very interoperable with external
>> tools
>> > along with an easy way to pass relatively complex data between
>> > traffic_manager / traffic_server and external tools like traffic_ctl in
>> > both directions.
>>
>
>
> --
> *Beware the fisherman who's casting out his line in to a dried up riverbed.*
> *Oh don't try to tell him 'cause he won't believe. Throw some bread to the
> ducks instead.*
> *It's easier that way. *- Genesis : Duke : VI 25-28

Re: [DISCUSS] RPC upgrade

[Alan Carroll <so...@oath.com.INVALID>]

That's what we have now and it's not worked out well. The problem is there
is a central dispatcher and it needs to be able to parse the messages at
least to the extent of knowing the size of the messages. Otherwise the
essential problem remains that the dispatcher can't be generic and any
update in set of messages or content requires modifying that code. This in
turn makes it effectively impossible for plugins to participate in any
meaningful way.

On Tue, Sep 4, 2018 at 9:41 AM Walt Karas <wk...@oath.com.invalid> wrote:

> For inter-process comm on a single server, why not KISS and use
> message queues and POD structs?
>
> On Mon, Aug 27, 2018 at 2:43 PM, Alan Carroll
> <so...@oath.com.invalid> wrote:
> > Because most people probably dropped off the previous discussion on this
> > list, I want to call out that, as far as I can tell, the consensus for
> > upgrading the internal RPC mechanism is to use a JSON based message
> > exchange compliant with JSON-RPC 2.0 (https://www.jsonrpc.org/). We can
> use
> > YAMLCPP to parse messages and I think we can easily (externally) tweak
> > YAMLCPP to output JSON. This will be very interoperable with external
> tools
> > along with an easy way to pass relatively complex data between
> > traffic_manager / traffic_server and external tools like traffic_ctl in
> > both directions.
>


-- 
*Beware the fisherman who's casting out his line in to a dried up riverbed.*
*Oh don't try to tell him 'cause he won't believe. Throw some bread to the
ducks instead.*
*It's easier that way. *- Genesis : Duke : VI 25-28