(fury-site) 02/02: 🔄 synced local 'docs/specification/' with remote 'docs/specification/'

[ch...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

chaokunyang pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/fury-site.git

commit c765f1ea8834249de741a4bd15ce1b8e4ea443a5
Author: chaokunyang <sh...@gmail.com>
AuthorDate: Mon Dec 30 13:52:47 2024 +0000

    🔄 synced local 'docs/specification/' with remote 'docs/specification/'
---
 docs/specification/xlang_serialization_spec.md | 66 +++++++++++++++++---------
 1 file changed, 43 insertions(+), 23 deletions(-)

diff --git a/docs/specification/xlang_serialization_spec.md b/docs/specification/xlang_serialization_spec.md
index 7fd991ce..c47710f5 100644
--- a/docs/specification/xlang_serialization_spec.md
+++ b/docs/specification/xlang_serialization_spec.md
@@ -39,28 +39,41 @@ also introduce more complexities compared to static serialization frameworks. So
 - string: a text string encoded using Latin1/UTF16/UTF-8 encoding.
 - enum: a data type consisting of a set of named values. Rust enum with non-predefined field values are not supported as
   an enum.
+- named_enum: an enum whose value will be serialized as the registered name.
+- struct: a morphic(final) type serialized by Fury Struct serializer. i.e. it doesn't have subclasses. Suppose we're
+  deserializing `List<SomeClass>`, we can save dynamic serializer dispatch since `SomeClass` is morphic(final).
+- polymorphic_struct: a type which is not morphic(not final). i.e. it has subclasses. Suppose we're deserializing
+  `List<SomeClass>`, we must dispatch serializer dynamically since `SomeClass` is morphic(final).
+- compatible_struct: a morphic(final) type serialized by Fury compatible Struct serializer.
+- polymorphic_compatible_struct: a non-morphic(non-final) type serialized by Fury compatible Struct serializer.
+- named_struct: a `struct` whose type mapping will be encoded as a name.
+- named_polymorphic_struct: a `polymorphic_struct` whose type mapping will be encoded as a name.
+- named_compatible_struct: a `compatible_struct` whose type mapping will be encoded as a name.
+- named_polymorphic_compatible_struct: a `polymorphic_compatible_struct` whose type mapping will be encoded as a name.
+- ext: a type which will be serialized by a customized serializer.
+- polymorphic_ext: an `ext` type which is not morphic(not final).
+- named_ext: an `ext` type whose type mapping will be encoded as a name.
+- named_polymorphic_ext: an `polymorphic_ext` type whose type mapping will be encoded as a name.
 - list: a sequence of objects.
 - set: an unordered set of unique elements.
 - map: a map of key-value pairs. Mutable types such as `list/map/set/array/tensor/arrow` are not allowed as key of map.
-- time types:
-  - duration: an absolute length of time, independent of any calendar/timezone, as a count of nanoseconds.
-  - timestamp: a point in time, independent of any calendar/timezone, as a count of nanoseconds. The count is relative
-      to an epoch at UTC midnight on January 1, 1970.
+- duration: an absolute length of time, independent of any calendar/timezone, as a count of nanoseconds.
+- timestamp: a point in time, independent of any calendar/timezone, as a count of nanoseconds. The count is relative
+  to an epoch at UTC midnight on January 1, 1970.
+- local_date: a naive date without timezone. The count is days relative to an epoch at UTC midnight on Jan 1, 1970.
 - decimal: exact decimal value represented as an integer value in two's complement.
 - binary: an variable-length array of bytes.
-- array type: only allow numeric components. Other arrays will be taken as List. The implementation should support the
+- array: only allow numeric components. Other arrays will be taken as List. The implementation should support the
   interoperability between array and list.
-  - array: multidimensional array which every sub-array can have different sizes but all have same type.
-  - bool_array: one dimensional int16 array.
-  - int8_array: one dimensional int8 array.
-  - int16_array: one dimensional int16 array.
-  - int32_array: one dimensional int32 array.
-  - int64_array: one dimensional int64 array.
-  - float16_array: one dimensional half_float_16 array.
-  - float32_array: one dimensional float32 array.
-  - float64_array: one dimensional float64 array.
-- tensor: a multidimensional dense array of fixed-size values such as a NumPy ndarray.
-- sparse tensor: a multidimensional array whose elements are almost all zeros.
+- array: multidimensional array which every sub-array can have different sizes but all have same type.
+- bool_array: one dimensional int16 array.
+- int8_array: one dimensional int8 array.
+- int16_array: one dimensional int16 array.
+- int32_array: one dimensional int32 array.
+- int64_array: one dimensional int64 array.
+- float16_array: one dimensional half_float_16 array.
+- float32_array: one dimensional float32 array.
+- float64_array: one dimensional float64 array.
 - arrow record batch: an arrow [record batch](https://arrow.apache.org/docs/cpp/tables.html#record-batches) object.
 - arrow table: an arrow [table](https://arrow.apache.org/docs/cpp/tables.html#tables) object.
 
@@ -68,6 +81,15 @@ Note:
 
 - Unsigned int/long are not added here, since not every language support those types.
 
+### Polymorphisms
+
+For polymorphism, if one non-final class is registered, and only one subclass is registered, then we can take all
+elements in List/Map have same type, thus reduce runtime check cost.
+
+Collection/Array polymorphism are not fully supported, since some languages such as golang have only one collection
+type. If users want to get exactly the type he passed, he must pass that type when deserializing or annotate that type
+to the field of struct.
+
 ### Type disambiguation
 
 Due to differences between type systems of languages, those types can't be mapped one-to-one between languages. When
@@ -117,8 +139,8 @@ Such information can be provided in other languages too:
 
 ### Type ID
 
-All internal data types are expressed using an ID in range `-64~-1`. Users can use `0~32703` for representing their
-types. At runtime, all type ids are added by `64`, and then encoded as an unsigned varint.
+All internal data types are expressed using an ID in range `0~64`. Users can use `0~4096` for representing their
+types.
 
 ### Type mapping
 
@@ -298,19 +320,17 @@ Meta header is a 64 bits number value encoded in little endian order.
 - type id: the registered id for the current type, which will be written as an unsigned varint.
 - field info:
   - header(8
-      bits): `3 bits size + 2 bits field name encoding + polymorphism flag + nullability flag + ref tracking flag`.
+      bits): `4 bits size + 2 bits field name encoding + nullability flag + ref tracking flag`.
       Users can use annotation to provide those info.
     - 2 bits field name encoding:
       - encoding: `UTF8/ALL_TO_LOWER_SPECIAL/LOWER_UPPER_DIGIT_SPECIAL/TAG_ID`
       - If tag id is used, i.e. field name is written by an unsigned varint tag id. 2 bits encoding will be `11`.
     - size of field name:
-      - The `3 bits size: 0~7`  will be used to indicate length `1~7`, the value `7` indicates to read more bytes,
-              the encoding will encode `size - 7` as a varint next.
+      - The `4 bits size: 0~14`  will be used to indicate length `1~15`, the value `15` indicates to read more bytes,
+              the encoding will encode `size - 15` as a varint next.
       - If encoding is `TAG_ID`, then num_bytes of field name will be used to store tag id.
     - ref tracking: when set to 1, ref tracking will be enabled for this field.
     - nullability: when set to 1, this field can be null.
-    - polymorphism: when set to 1, the actual type of field will be the declared field type even the type if
-          not `final`.
   - field name: If tag id is set, tag id will be used instead. Otherwise meta string encoding `[length]` and data will
       be written instead.
   - type id:


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fury.apache.org
For additional commands, e-mail: commits-help@fury.apache.org