[jira] [Updated] (IGNITE-28939) Turn blob fields of internal types into normal message fields

["Ignite TC Bot (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/IGNITE-28939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ignite TC Bot updated IGNITE-28939:
-----------------------------------
    Ignite Flags: Release Notes Required  (was: Docs Required,Release Notes Required)

> Turn blob fields of internal types into normal message fields
> -------------------------------------------------------------
>
>                 Key: IGNITE-28939
>                 URL: https://issues.apache.org/jira/browse/IGNITE-28939
>             Project: Ignite
>          Issue Type: Sub-task
>          Components: messaging
>            Reporter: Anton Vinogradov
>            Assignee: Anton Vinogradov
>            Priority: Major
>              Labels: IEP-132, compatibility, wire-format
>             Fix For: 2.19
>
>
> *Wire format change. Do this before 2.19 is released - after that it would break rolling upgrade.*
> h3. Goal
> Remove the question "which marshaller" for fields that hold internal classes.
> h3. Why
> Some fields are sent as blobs only because an internal class was never made a {{Message}}. Binary gives no schema for them anyway: {{BinaryUtils#isCustomJavaSerialization}} checks the whole class hierarchy, so {{Externalizable}} classes fall back to {{OptimizedMarshaller}} inside binary. One of these fields is also a mismatch: {{BinaryMetadata}} uses binary in {{BinaryMetadataVersionInfo}} and jdk in {{MetadataUpdateProposedMessage}}.
> h3. How
> Convert to normal message fields:
> * {{BinaryMetadataVersionInfo#metadata}} and {{MetadataUpdateProposedMessage#metadata}} - {{BinaryMetadata}} has simple fields: type id, type name, field map, schemas, flags;
> * {{ChangeGlobalStateMessage#baselineTopology}} - {{BaselineTopology}};
> * {{GridJobSiblingsResponse#siblings}} - the implementation is the internal {{GridJobSiblingImpl}};
> * {{StoredCacheData#qryEntities}} and {{DynamicCacheChangeRequest#schema}} - {{QueryEntityMessage}} already exists.
> Fields that really hold user classes stay blobs: node attributes, service affinity key, query entity default values, entry processors, invoke arguments, query filters, task results, message topic, cache configuration, plugin data and {{Throwable}}.
> h3. On-disk compatibility
> Three of these types also live on disk, so the conversion must not change how they are stored:
> * {{BinaryMetadata}} - written by {{BinaryMetadataFileStore#writeMetadata}}; the format comes from its
> {{Externalizable}} methods;
> * {{BaselineTopology}} - kept in the metastore under {{METASTORE_CURR_BLT_KEY}}; {{MetaStorage}} marshals values with jdk;
> * {{QueryEntity}} inside {{StoredCacheData}} - {{cache_data.dat}} files and snapshots, jdk again.
> Rules that keep the stored format byte-identical:
> # keep the explicit {{serialVersionUID}}. All of these classes declare one, so adding the {{Message}} interface does
> not change the computed id. Without an explicit id, adding an interface changes it and old files stop loading.
> # make new wire companion fields {{transient}}, the way {{ccfgBytes}} and {{qryEntitiesBytes}} are done in
> {{StoredCacheData}}. Non-transient fields are what the disk format is made of.
> # do not add or remove non-transient fields, and do not touch {{writeExternal}}/{{readExternal}}. For
> {{BinaryMetadata}} the {{Externalizable}} methods *are* the on-disk format: adding {{Message}} on top is fine,
> dropping {{Externalizable}} is not.
> Precedent: {{StoredCacheData}}, {{CacheConfigurationEnrichment}} and {{SnapshotMetadata}} are already both
> {{Serializable}} and {{Message}}, and that coexists with the disk today.
> Verification for this ticket: read a {{cache_data.dat}} and a binary metadata file written by the previous build,
> and restore a snapshot taken by it.
> h3. Expected result
> Blob fields: 24 now, about 19 after. The {{BinaryMetadata}} mismatch is gone. These fields no longer depend on a marshaller, and usually become smaller on the wire.
> h3. How to verify
> Binary metadata tests, node join tests, cluster state change tests, dynamic cache start with query entities, job siblings tests. Compare message size before and after.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)