[jira] [Created] (IGNITE-17871) Use network serialization for RAFT commands

["Ivan Bessonov (Jira)" <ji...@apache.org>]

Ivan Bessonov created IGNITE-17871:
--------------------------------------

             Summary: Use network serialization for RAFT commands
                 Key: IGNITE-17871
                 URL: https://issues.apache.org/jira/browse/IGNITE-17871
             Project: Ignite
          Issue Type: Improvement
            Reporter: Ivan Bessonov


h3. Problem

Currently, there are two places where {{Command}} instances are being serialized:
 * ActionRequest - here the command property is marked is Marshallable, meaning that it will be serialized using a User Object Serialization approach
 * Listener - here command is explicitly serialized using JDKMarshaller, for further handling by RAFT. This is the data that will be written to the Log and deserialized on followers / learners

What are the problems?

For ActionRequest message, command is expected to be its largest part. And although writing serialized UOS byte array into a netty socket is faster, then optimized marshalling, it feels like overall throughput will be smaller. And the reason is that there's an extra step of converting command into a byte array, that happens in caller thread.

For serialization in listeners, using JDKMarshaller is both slow and inefficient in terms of space. Obvious example - network serialization of SnapshotMeta object, for instance, can be condensed to 8 bytes (assuming we optimize "writeShort" and change its message type). JDKMarshaller produces 232 bytes. Of course, here most of fields are nulls and real payload will be bigger, but JDKMarshaller will always lead to more data simply because it has to store schema meta-information.
h3. Solution

Making Command an implementation of NetworkMessage will solve both of these problems. ActionRequest will not have its "prepareMarshal" phase, listeners will have fast and space-efficient serialization algorithm.

Of course, there must be drawbacks. I'll try to explain what I see at the moment.
 * Currently, there's no explicit support for List properties, only Collection. It is easy to fix
 * CMG commands use classes like ClusterNode and IgniteProductVersion. We should introduce message alternatives
 * I saw some enums being used, they are not natively supported at the moment. There are two options:
 ** implement native support. I consider this a dangerous path
 ** store explicit ordinal where it's necessary
 * ByteBuffer support would be really nice to have natively. Should be fast to implement also

One important note: there should be no Marshallable properties in commands, because we can't persist them. Information about classes' ids is stored in sessions and can change between sessions. The way to achieve it is to pass a "null" UOS context into serializator.

Now about serializator: we can have thread-local buffers to write data to. When write is complete, data is copied as a byte[]. Reading will be done directly from the byte[].

Possible optimization for ByteBuffers - we can implement them as slices of the byte[] payload instead of copying sub-arrays. Will save some time and memory.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)