[jira] [Commented] (KUDU-1973) Coalesce RPCs destined for the same server

["Todd Lipcon (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KUDU-1973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15967161#comment-15967161 ] 

Todd Lipcon commented on KUDU-1973:
-----------------------------------

I think this should be more specific to the consensus system, not to krpc in general.

The thing here is that all of the tablets have separate heartbeat timers, so each heartbeat (empty UpdateConsensus) RPC ends up causing a wakeup and context switch on the server side, etc. But there isn't really any particular reason not to coalesce them so they all get sent and arrive at the same time -- so long as it arrives before the election timer expires, it's fine to shift a heartbeat later by hundreds of milliseconds, for example. As such, we could bundle a bunch of empty UpdateConsensus RPCs destined for the same node into a single RPC and avoid the extra wakeups.

> Coalesce RPCs destined for the same server
> ------------------------------------------
>
>                 Key: KUDU-1973
>                 URL: https://issues.apache.org/jira/browse/KUDU-1973
>             Project: Kudu
>          Issue Type: Sub-task
>          Components: rpc, tserver
>    Affects Versions: 1.4.0
>            Reporter: Adar Dembo
>              Labels: data-scalability
>
> The krpc subsystem ensures that only one _connection_ exists between any pair of nodes, but it doesn't coalesce the _RPCs_ themselves. In clusters with dense nodes (especially with a lot of tablets), there's often a great number of RPCs sent between pairs of nodes.
> We should explore ways of coalescing those RPCs. I don't know whether that would happen within the krpc system itself (i.e. in a payload-agnostic way), or whether we'd only coalesce RPCs known to be "hot" (like UpdateConsensus).



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)