You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@fory.apache.org by ch...@apache.org on 2025/12/10 14:53:38 UTC
(fory-site) branch update_guide_structure created (now ba6426c57)
This is an automated email from the ASF dual-hosted git repository.
chaokunyang pushed a change to branch update_guide_structure
in repository https://gitbox.apache.org/repos/asf/fory-site.git
at ba6426c57 refine docs
This branch includes the following new commits:
new 55e2f92f0 remvoe stale doc
new ba6426c57 refine docs
The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org
For additional commands, e-mail: commits-help@fory.apache.org
(fory-site) 01/02: remvoe stale doc
Posted by ch...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
chaokunyang pushed a commit to branch update_guide_structure
in repository https://gitbox.apache.org/repos/asf/fory-site.git
commit 55e2f92f0eba09b666781ecaf124a024ca2bc8df
Author: chaokunyang <sh...@gmail.com>
AuthorDate: Wed Dec 10 21:41:21 2025 +0800
remvoe stale doc
---
docs/docs/guide/java_serialization_guide.md | 1802 ---------------------------
docs/docs/guide/python_guide.md | 1394 ---------------------
docs/docs/guide/row_format_guide.md | 205 ---
docs/docs/guide/rust_guide.md | 729 -----------
4 files changed, 4130 deletions(-)
diff --git a/docs/docs/guide/java_serialization_guide.md b/docs/docs/guide/java_serialization_guide.md
deleted file mode 100644
index 6b7bef3f5..000000000
--- a/docs/docs/guide/java_serialization_guide.md
+++ /dev/null
@@ -1,1802 +0,0 @@
----
-title: Java Serialization Guide
-sidebar_position: 0
-id: java_serialization
-license: |
- Licensed to the Apache Software Foundation (ASF) under one or more
- contributor license agreements. See the NOTICE file distributed with
- this work for additional information regarding copyright ownership.
- The ASF licenses this file to You under the Apache License, Version 2.0
- (the "License"); you may not use this file except in compliance with
- the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
----
-
-When only java object serialization needed, this mode will have better performance compared to cross-language object
-graph serialization.
-
-## Quick Start
-
-Note that fory creation is not cheap, the **fory instances should be reused between serializations** instead of creating
-it everytime.
-You should keep fory to a static global variable, or instance variable of some singleton object or limited objects.
-
-Fory for single-thread usage:
-
-```java
-import java.util.List;
-import java.util.Arrays;
-
-import org.apache.fory.*;
-import org.apache.fory.config.*;
-
-public class Example {
- public static void main(String[] args) {
- SomeClass object = new SomeClass();
- // Note that Fory instances should be reused between
- // multiple serializations of different objects.
- Fory fory = Fory.builder().withLanguage(Language.JAVA)
- .requireClassRegistration(true)
- .build();
- // Registering types can reduce class name serialization overhead, but not mandatory.
- // If class registration enabled, all custom types must be registered.
- // Registration order must be consistent if id is not specified
- fory.register(SomeClass.class);
- byte[] bytes = fory.serialize(object);
- System.out.println(fory.deserialize(bytes));
- }
-}
-```
-
-Fory for multiple-thread usage:
-
-```java
-import java.util.List;
-import java.util.Arrays;
-
-import org.apache.fory.*;
-import org.apache.fory.config.*;
-
-public class Example {
- public static void main(String[] args) {
- SomeClass object = new SomeClass();
- // Note that Fory instances should be reused between
- // multiple serializations of different objects.
- ThreadSafeFory fory = new ThreadLocalFory(classLoader -> {
- Fory f = Fory.builder().withLanguage(Language.JAVA)
- .withClassLoader(classLoader).build();
- f.register(SomeClass.class);
- return f;
- });
- byte[] bytes = fory.serialize(object);
- System.out.println(fory.deserialize(bytes));
- }
-}
-```
-
-Fory instances reuse example:
-
-```java
-import java.util.List;
-import java.util.Arrays;
-
-import org.apache.fory.*;
-import org.apache.fory.config.*;
-
-public class Example {
- // reuse fory.
- private static final ThreadSafeFory fory = new ThreadLocalFory(classLoader -> {
- Fory f = Fory.builder().withLanguage(Language.JAVA)
- .withClassLoader(classLoader).build();
- f.register(SomeClass.class);
- return f;
- });
-
- public static void main(String[] args) {
- SomeClass object = new SomeClass();
- byte[] bytes = fory.serialize(object);
- System.out.println(fory.deserialize(bytes));
- }
-}
-```
-
-## ForyBuilder options
-
-| Option Name | Description [...]
-| ----------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- [...]
-| `timeRefIgnored` | Whether to ignore reference tracking of all time types registered in `TimeSerializers` and subclasses of those types when ref tracking is enabled. If ignored, ref tracking of every time type can be enabled by invoking `Fory#registerSerializer(Class, Serializer)`. For example, `fory.registerSerializer(Date.class, new DateSerializer(fory, true))`. Note that enabling ref tracking should happen before serializer codegen of any types which contain time [...]
-| `compressInt` | Enables or disables int compression for smaller size. [...]
-| `compressLong` | Enables or disables long compression for smaller size. [...]
-| `compressIntArray` | Enables or disables SIMD-accelerated compression for int arrays when values can fit in smaller data types. Requires Java 16+. [...]
-| `compressLongArray` | Enables or disables SIMD-accelerated compression for long arrays when values can fit in smaller data types. Requires Java 16+. [...]
-| `compressString` | Enables or disables string compression for smaller size. [...]
-| `classLoader` | The classloader should not be updated; Fory caches class metadata. Use `LoaderBinding` or `ThreadSafeFory` for classloader updates. [...]
-| `compatibleMode` | Type forward/backward compatibility config. Also Related to `checkClassVersion` config. `SCHEMA_CONSISTENT`: Class schema must be consistent between serialization peer and deserialization peer. `COMPATIBLE`: Class schema can be different between serialization peer and deserialization peer. They can add/delete fields independently. [See more](#class-inconsistency-and-class-version-check). [...]
-| `checkClassVersion` | Determines whether to check the consistency of the class schema. If enabled, Fory checks, writes, and checks consistency using the `classVersionHash`. It will be automatically disabled when `CompatibleMode#COMPATIBLE` is enabled. Disabling is not recommended unless you can ensure the class won't evolve. [...]
-| `checkJdkClassSerializable` | Enables or disables checking of `Serializable` interface for classes under `java.*`. If a class under `java.*` is not `Serializable`, Fory will throw an `UnsupportedOperationException`. [...]
-| `registerGuavaTypes` | Whether to pre-register Guava types such as `RegularImmutableMap`/`RegularImmutableList`. These types are not public API, but seem pretty stable. [...]
-| `requireClassRegistration` | Disabling may allow unknown classes to be deserialized, potentially causing security risks. [...]
-| `requireClassRegistration` | Set max depth for deserialization, when depth exceeds, an exception will be thrown. This can be used to refuse deserialization DDOS attack. [...]
-| `suppressClassRegistrationWarnings` | Whether to suppress class registration warnings. The warnings can be used for security audit, but may be annoying, this suppression will be enabled by default. [...]
-| `metaShareEnabled` | Enables or disables meta share mode. [...]
-| `scopedMetaShareEnabled` | Scoped meta share focuses on a single serialization process. Metadata created or identified during this process is exclusive to it and is not shared with by other serializations. [...]
-| `metaCompressor` | Set a compressor for meta compression. Note that the passed MetaCompressor should be thread-safe. By default, a `Deflater` based compressor `DeflaterMetaCompressor` will be used. Users can pass other compressor such as `zstd` for better compression rate. [...]
-| `deserializeNonexistentClass` | Enables or disables deserialization/skipping of data for non-existent classes. [...]
-| `codeGenEnabled` | Disabling may result in faster initial serialization but slower subsequent serializations. [...]
-| `asyncCompilationEnabled` | If enabled, serialization uses interpreter mode first and switches to JIT serialization after async serializer JIT for a class is finished. [...]
-| `scalaOptimizationEnabled` | Enables or disables Scala-specific serialization optimization. [...]
-| `copyRef` | When disabled, the copy performance will be better. But fory deep copy will ignore circular and shared reference. Same reference of an object graph will be copied into different objects in one `Fory#copy`. [...]
-| `serializeEnumByName` | When Enabled, fory serialize enum by name instead of ordinal. [...]
-
-## Advanced Usage
-
-### Fory creation
-
-Single thread fory:
-
-```java
-Fory fory = Fory.builder()
- .withLanguage(Language.JAVA)
- // enable reference tracking for shared/circular reference.
- // Disable it will have better performance if no duplicate reference.
- .withRefTracking(false)
- .withCompatibleMode(CompatibleMode.SCHEMA_CONSISTENT)
- // enable type forward/backward compatibility
- // disable it for small size and better performance.
- // .withCompatibleMode(CompatibleMode.COMPATIBLE)
- // enable async multi-threaded compilation.
- .withAsyncCompilation(true)
- .build();
-byte[] bytes = fory.serialize(object);
-System.out.println(fory.deserialize(bytes));
-```
-
-Thread-safe fory:
-
-```java
-ThreadSafeFory fory = Fory.builder()
- .withLanguage(Language.JAVA)
- // enable reference tracking for shared/circular reference.
- // Disable it will have better performance if no duplicate reference.
- .withRefTracking(false)
- // compress int for smaller size
- // .withIntCompressed(true)
- // compress long for smaller size
- // .withLongCompressed(true)
- .withCompatibleMode(CompatibleMode.SCHEMA_CONSISTENT)
- // enable type forward/backward compatibility
- // disable it for small size and better performance.
- // .withCompatibleMode(CompatibleMode.COMPATIBLE)
- // enable async multi-threaded compilation.
- .withAsyncCompilation(true)
- .buildThreadSafeFory();
-byte[] bytes = fory.serialize(object);
-System.out.println(fory.deserialize(bytes));
-```
-
-Note that calling `buildThreadSafeFory()` on `ForyBuilder` will create an instance of `ThreadLocalFory`.
-This may not be appropriate in environments where virtual threads are used, as each thread will create its own fory instance, a relatively expensive operation.
-An alternative for virtual threads is to use `buildThreadSafeForyPool`.
-
-### Handling Class Schema Evolution in Serialization
-
-In many systems, the schema of a class used for serialization may change over time. For instance, fields within a class
-may be added or removed. When serialization and deserialization processes use different versions of jars, the schema of
-the class being deserialized may differ from the one used during serialization.
-
-By default, Fory serializes objects using the `CompatibleMode.SCHEMA_CONSISTENT` mode. This mode assumes that the
-deserialization process uses the same class schema as the serialization process, minimizing payload overhead.
-However, if there is a schema inconsistency, deserialization will fail.
-
-If the schema is expected to change, to make deserialization succeed, i.e. schema forward/backward compatibility.
-Users must configure Fory to use `CompatibleMode.COMPATIBLE`. This can be done using the
-`ForyBuilder#withCompatibleMode(CompatibleMode.COMPATIBLE)` method.
-In this compatible mode, deserialization can handle schema changes such as missing or extra fields, allowing it to
-succeed even when the serialization and deserialization processes have different class schemas.
-
-Here is an example of creating Fory to support schema evolution:
-
-```java
-Fory fory = Fory.builder()
- .withCompatibleMode(CompatibleMode.COMPATIBLE)
- .build();
-
-byte[] bytes = fory.serialize(object);
-System.out.println(fory.deserialize(bytes));
-```
-
-This compatible mode involves serializing class metadata into the serialized output. Despite Fory's use of
-sophisticated compression techniques to minimize overhead, there is still some additional space cost associated with
-class metadata.
-
-To further reduce metadata costs, Fory introduces a class metadata sharing mechanism, which allows the metadata to be
-sent to the deserialization process only once. For more details, please refer to the [Meta Sharing](https://fory.apache.org/docs/specification/fory_java_serialization_spec#meta-share) specification.
-
-### Compression
-
-`ForyBuilder#withIntCompressed`/`ForyBuilder#withLongCompressed` can be used to compress int/long for smaller size.
-Normally compress int is enough.
-
-Both compression are enabled by default, if the serialized is not important, for example, you use flatbuffers for
-serialization before, which doesn't compress anything, then you should disable compression. If your data are all
-numbers,
-the compression may bring 80% performance regression.
-
-For int compression, fory use 1~5 bytes for encoding. First bit in every byte indicate whether has next byte. if first
-bit is set, then next byte will be read util first bit of next byte is unset.
-
-For long compression, fory support two encoding:
-
-- Fory SLI(Small long as int) Encoding (**used by default**):
- - If long is in `[-1073741824, 1073741823]`, encode as 4 bytes int: `| little-endian: ((int) value) << 1 |`
- - Otherwise write as 9 bytes: `| 0b1 | little-endian 8bytes long |`
-- Fory PVL(Progressive Variable-length Long) Encoding:
- - First bit in every byte indicate whether has next byte. if first bit is set, then next byte will be read util
- first bit of next byte is unset.
- - Negative number will be converted to positive number by `(v << 1) ^ (v >> 63)` to reduce cost of small negative
- numbers.
-
-If a number are `long` type, it can't be represented by smaller bytes mostly, the compression won't get good enough
-result,
-not worthy compared to performance cost. Maybe you should try to disable long compression if you find it didn't bring
-much
-space savings.
-
-### Array Compression
-
-Fory supports SIMD-accelerated compression for primitive arrays (`int[]` and `long[]`) when array values can fit in smaller data types. This feature is available on Java 16+ and uses the Vector API for optimal performance.
-
-#### How Array Compression Works
-
-Array compression analyzes arrays to determine if values can be stored using fewer bytes:
-
-- **`int[]` → `byte[]`**: When all values are in range [-128, 127] (75% size reduction)
-- **`int[]` → `short[]`**: When all values are in range [-32768, 32767] (50% size reduction)
-- **`long[]` → `int[]`**: When all values fit in integer range (50% size reduction)
-
-#### Configuration and Registration
-
-To enable array compression you must explicitly register the serializers:
-
-```java
-Fory fory = Fory.builder()
- .withLanguage(Language.JAVA)
- // Enable int array compression
- .withIntArrayCompressed(true)
- // Enable long array compression
- .withLongArrayCompressed(true)
- .build();
-
-// You must explicitly register compressed array serializers
-CompressedArraySerializers.registerSerializers(fory);
-```
-
-**Note**: The `fory-simd` module must be included in your dependencies for compressed array serializers to be available.
-
-For Maven:
-
-```xml
-<dependency>
- <groupId>org.apache.fory</groupId>
- <artifactId>fory-simd</artifactId>
- <version>0.13.2</version>
-</dependency>
-```
-
-### Object deep copy
-
-Deep copy example:
-
-```java
-Fory fory = Fory.builder().withRefCopy(true).build();
-SomeClass a = xxx;
-SomeClass copied = fory.copy(a);
-```
-
-Make fory deep copy ignore circular and shared reference, this deep copy mode will ignore circular and shared reference.
-Same reference of an object graph will be copied into different objects in one `Fory#copy`.
-
-```java
-Fory fory = Fory.builder().withRefCopy(false).build();
-SomeClass a = xxx;
-SomeClass copied = fory.copy(a);
-```
-
-### Cross-Language Serialization
-
-Apache Fory™ supports seamless data exchange between Java and other languages (Python, Rust, Go, JavaScript, etc.) through the xlang serialization format. This enables multi-language microservices, polyglot data pipelines, and cross-platform data sharing.
-
-#### Enable Cross-Language Mode
-
-To serialize data for consumption by other languages, use `Language.XLANG` mode:
-
-```java
-import org.apache.fory.*;
-import org.apache.fory.config.*;
-
-// Create Fory instance with XLANG mode
-Fory fory = Fory.builder()
- .withLanguage(Language.XLANG)
- .withRefTracking(true) // Enable reference tracking for complex graphs
- .build();
-```
-
-#### Register Types for Cross-Language Compatibility
-
-Types must be registered with **consistent IDs or names** across all languages. Fory supports two registration methods:
-
-**1. Register by ID (Recommended for Performance)**
-
-```java
-public record Person(String name, int age) {}
-
-// Register with numeric ID - faster and more compact
-fory.register(Person.class, 1);
-
-Person person = new Person("Alice", 30);
-byte[] bytes = fory.serialize(person);
-// bytes can be deserialized by Python, Rust, Go, etc.
-```
-
-**Benefits**: Faster serialization, smaller binary size
-**Trade-offs**: Requires coordination to avoid ID conflicts across teams/services
-
-**2. Register by Name (Recommended for Flexibility)**
-
-```java
-public record Person(String name, int age) {}
-
-// Register with string name - more flexible
-fory.register(Person.class, "example.Person");
-
-Person person = new Person("Alice", 30);
-byte[] bytes = fory.serialize(person);
-// bytes can be deserialized by Python, Rust, Go, etc.
-```
-
-**Benefits**: Less prone to conflicts, easier management across teams, no coordination needed
-**Trade-offs**: Slightly larger binary size due to string encoding
-
-#### Cross-Language Example: Java ↔ Python
-
-**Java (Serializer)**
-
-```java
-import org.apache.fory.*;
-import org.apache.fory.config.*;
-
-public record Person(String name, int age) {}
-
-public class Example {
- public static void main(String[] args) {
- Fory fory = Fory.builder()
- .withLanguage(Language.XLANG)
- .withRefTracking(true)
- .build();
-
- // Register with consistent name
- fory.register(Person.class, "example.Person");
-
- Person person = new Person("Bob", 25);
- byte[] bytes = fory.serialize(person);
-
- // Send bytes to Python service via network/file/queue
- }
-}
-```
-
-**Python (Deserializer)**
-
-```python
-import pyfory
-from dataclasses import dataclass
-
-@dataclass
-class Person:
- name: str
- age: pyfory.int32
-
-# Create Fory in xlang mode
-fory = pyfory.Fory(ref_tracking=True)
-
-# Register with the SAME name as Java
-fory.register_type(Person, typename="example.Person")
-
-# Deserialize bytes from Java
-person = fory.deserialize(bytes_from_java)
-print(f"{person.name}, {person.age}") # Output: Bob, 25
-```
-
-#### Handling Circular and Shared References
-
-Cross-language mode supports circular and shared references when reference tracking is enabled:
-
-```java
-public class Node {
- public String value;
- public Node next;
- public Node parent;
-}
-
-Fory fory = Fory.builder()
- .withLanguage(Language.XLANG)
- .withRefTracking(true) // Required for circular references
- .build();
-
-fory.register(Node.class, "example.Node");
-
-// Create circular reference
-Node node1 = new Node();
-node1.value = "A";
-Node node2 = new Node();
-node2.value = "B";
-node1.next = node2;
-node2.parent = node1; // Circular reference
-
-byte[] bytes = fory.serialize(node1);
-// Python/Rust/Go can correctly deserialize this with circular references preserved
-```
-
-#### Type Mapping Considerations
-
-Not all Java types have equivalents in other languages. When using xlang mode:
-
-- Use **primitive types** (`int`, `long`, `double`, `String`) for maximum compatibility
-- Use **standard collections** (`List`, `Map`, `Set`) instead of language-specific ones
-- Avoid **Java-specific types** like `Optional`, `BigDecimal` (unless the target language supports them)
-- See [Type Mapping Guide](https://fory.apache.org/docs/specification/xlang_type_mapping) for complete compatibility matrix
-
-**Compatible Types**:
-
-```java
-public record UserData(
- String name, // ✅ Compatible
- int age, // ✅ Compatible
- List<String> tags, // ✅ Compatible
- Map<String, Integer> scores // ✅ Compatible
-) {}
-```
-
-**Problematic Types**:
-
-```java
-public record UserData(
- Optional<String> name, // ❌ Not cross-language compatible
- BigDecimal balance, // ❌ Limited support
- EnumSet<Status> statuses // ❌ Java-specific collection
-) {}
-```
-
-#### Performance Considerations
-
-Cross-language mode has additional overhead compared to Java-only mode:
-
-- **Type metadata encoding**: Adds extra bytes per type
-- **Type resolution**: Requires name/ID lookup during deserialization
-
-**For best performance**:
-
-- Use **ID-based registration** when possible (smaller encoding)
-- **Disable reference tracking** if you don't need circular references (`withRefTracking(false)`)
-- **Use Java mode** (`Language.JAVA`) when only Java serialization is needed
-
-#### Cross-Language Best Practices
-
-1. **Consistent Registration**: Ensure all services register types with identical IDs/names
-2. **Version Compatibility**: Use compatible mode for schema evolution across services
-
-#### Troubleshooting Cross-Language Serialization
-
-**"Type not registered" errors**:
-
-- Verify type is registered with same ID/name on both sides
-- Check if type name has typos or case differences
-
-**"Type mismatch" errors**:
-
-- Ensure field types are compatible across languages
-- Review [Type Mapping Guide](https://fory.apache.org/docs/next/specification/xlang_type_mapping)
-
-**Data corruption or unexpected values**:
-
-- Verify both sides use `Language.XLANG` mode
-- Ensure both sides have compatible Fory versions
-
-**See Also**:
-
-- [Cross-Language Serialization Specification](https://fory.apache.org/docs/next/specification/fory_xlang_serialization_spec)
-- [Type Mapping Reference](https://fory.apache.org/docs/next/specification/xlang_type_mapping)
-- [Python Cross-Language Guide](python_guide.md#cross-language-serialization)
-- [Rust Cross-Language Guide](rust_guide.md#-cross-language-serialization)
-
-### Implement a customized serializer
-
-In some cases, you may want to implement a serializer for your type, especially some class customize serialization by
-JDK `writeObject/writeReplace/readObject/readResolve`, which is very inefficient. For example, if you don't want
-following `Foo#writeObject` got invoked, you can take following `FooSerializer` as an example:
-
-```java
-class Foo {
- public long f1;
-
- private void writeObject(ObjectOutputStream s) throws IOException {
- System.out.println(f1);
- s.defaultWriteObject();
- }
-}
-
-class FooSerializer extends Serializer<Foo> {
- public FooSerializer(Fory fory) {
- super(fory, Foo.class);
- }
-
- @Override
- public void write(MemoryBuffer buffer, Foo value) {
- buffer.writeInt64(value.f1);
- }
-
- @Override
- public Foo read(MemoryBuffer buffer) {
- Foo foo = new Foo();
- foo.f1 = buffer.readInt64();
- return foo;
- }
-}
-```
-
-Register serializer:
-
-```java
-Fory fory = getFory();
-fory.registerSerializer(Foo.class, new FooSerializer(fory));
-```
-
-### Implement Collection Serializer
-
-Similar to maps, when implementing a serializer for a custom Collection type, you must extend `CollectionSerializer` or `CollectionLikeSerializer`.
-The key difference between these two is that `CollectionLikeSerializer` can serialize a class which has a collection-like structure but is not a java Collection subtype.
-
-For collection serializer, this is a special parameter `supportCodegenHook` needs be configured:
-
-- When `true`:
- - Enables optimized access to collection elements and JIT compilation for better performance
- - Direct serialization invocation and inline for map key-value items without dynamic serializer dispatch cost.
- - Better performance for standard collection types
- - Recommended for most collections
-
-- When `false`:
- - Uses interfaced-based element access and dynamic serializer dispatch for elements, which have higher cost
- - More flexible for custom collection types
- - Required when collection has special serialization needs
- - Handles complex collection implementations
-
-#### Implement Collection Serializer with JIT support
-
-When implementing a Collection serializer with JIT support, you can leverage Fory's existing binary format and collection serialization infrastructure. The key is to properly implement the `onCollectionWrite` and `newCollection` methods to handle metadata while letting Fory handle the element serialization.
-
-Here's an example:
-
-```java
-public class CustomCollectionSerializer<T extends Collection> extends CollectionSerializer<T> {
- public CustomCollectionSerializer(Fory fory, Class<T> cls) {
- // supportCodegenHook controls whether to use JIT compilation
- super(fory, cls, true);
- }
-
- @Override
- public Collection onCollectionWrite(MemoryBuffer buffer, T value) {
- // Write collection size
- buffer.writeVarUint32Small7(value.size());
- // Write any additional collection metadata
- return value;
- }
-
- @Override
- public Collection newCollection(MemoryBuffer buffer) {
- // Create new collection instance
- Collection collection = super.newCollection(buffer);
- // Read and set collection size
- int numElements = getAndClearNumElements();
- setNumElements(numElements);
- return collection;
- }
-}
-```
-
-Note that please invoke `setNumElements` when implementing `newCollection` to let fory know how many elements to deserialize.
-
-#### Implement a totally-customzied Collection Serializer without JIT
-
-Sometimes you need to serialize a collection type that uses primitive arrays or has special requirements.
-In such cases, you can implement a serializer with JIT disabled and directly override the `write` and `read` methods.
-
-This approach:
-
-- Gives you full control over serialization
-- Works well with primitive arrays
-- Bypasses collection iteration overhead
-- Allows direct memory access
-
-Here's an example of a custom integer list backed by a primitive array:
-
-```java
-class IntList extends AbstractCollection<Integer> {
- private final int[] elements;
- private final int size;
-
- public IntList(int size) {
- this.elements = new int[size];
- this.size = size;
- }
-
- public IntList(int[] elements, int size) {
- this.elements = elements;
- this.size = size;
- }
-
- @Override
- public Iterator<Integer> iterator() {
- return new Iterator<Integer>() {
- private int index = 0;
-
- @Override
- public boolean hasNext() {
- return index < size;
- }
-
- @Override
- public Integer next() {
- if (!hasNext()) {
- throw new NoSuchElementException();
- }
- return elements[index++];
- }
- };
- }
-
- @Override
- public int size() {
- return size;
- }
-
- public int get(int index) {
- if (index >= size) {
- throw new IndexOutOfBoundsException();
- }
- return elements[index];
- }
-
- public void set(int index, int value) {
- if (index >= size) {
- throw new IndexOutOfBoundsException();
- }
- elements[index] = value;
- }
-
- public int[] getElements() {
- return elements;
- }
-}
-
-class IntListSerializer extends CollectionLikeSerializer<IntList> {
- public IntListSerializer(Fory fory) {
- // Disable JIT since we're handling serialization directly
- super(fory, IntList.class, false);
- }
-
- @Override
- public void write(MemoryBuffer buffer, IntList value) {
- // Write size
- buffer.writeVarUint32Small7(value.size());
-
- // Write elements directly as primitive ints
- int[] elements = value.getElements();
- for (int i = 0; i < value.size(); i++) {
- buffer.writeVarInt32(elements[i]);
- }
- }
-
- @Override
- public IntList read(MemoryBuffer buffer) {
- // Read size
- int size = buffer.readVarUint32Small7();
-
- // Create array and read elements
- int[] elements = new int[size];
- for (int i = 0; i < size; i++) {
- elements[i] = buffer.readVarInt32();
- }
-
- return new IntList(elements, size);
- }
-
- // These methods are not used when JIT is disabled
- @Override
- public Collection onCollectionWrite(MemoryBuffer buffer, IntList value) {
- throw new UnsupportedOperationException();
- }
-
- @Override
- public Collection newCollection(MemoryBuffer buffer) {
- throw new UnsupportedOperationException();
- }
-
- @Override
- public IntList onCollectionRead(Collection collection) {
- throw new UnsupportedOperationException();
- }
-}
-```
-
-Key Points:
-
-1. **Primitive Array Storage**:
- - Uses `int[]` for direct storage
- - Avoids boxing/unboxing overhead
- - Provides efficient memory layout
- - Enables direct array access
-
-2. **Direct Serialization**:
- - Write size first
- - Write primitive values directly
- - No iteration overhead
- - No boxing/unboxing during serialization
-
-3. **Direct Deserialization**:
- - Read size first
- - Create primitive array
- - Read values directly into array
- - Create list with populated array
-
-4. **Disabled JIT**:
- - Set `supportCodegenHook=false`
- - Override `write`/`read` methods
- - Skip collection view pattern
- - Full control over serialization format
-
-When to Use: this approach is best when:
-
-- Working with primitive types
-- Need maximum performance
-- Want to minimize memory overhead
-- Have special serialization requirements
-
-Usage Example:
-
-```java
-// Create and populate list
-IntList list = new IntList(3);
-list.set(0, 1);
-list.set(1, 2);
-list.set(2, 3);
-
-// Serialize
-byte[] bytes = fory.serialize(list);
-
-// Deserialize
-IntList newList = (IntList) fory.deserialize(bytes);
-```
-
-This implementation is particularly efficient for scenarios where:
-
-- You're working exclusively with integers
-- Performance is critical
-- Memory efficiency is important
-- Serialization overhead needs to be minimized
-
-Remember that while this approach gives up some of Fory's optimizations, it can provide better performance for specific use cases involving primitive types and direct array access.
-
-#### Implement Serializer for Collection-like Types
-
-Sometimes you may want to implement a serializer for a type that behaves like a collection but isn't a standard Java Collection. This section demonstrates how to implement a serializer for such types.
-
-The key principles for collection-like type serialization are:
-
-1. Extend `CollectionLikeSerializer` for custom collection-like types
-2. Enable JIT optimization with `supportCodegenHook`
-3. Provide efficient element access through views
-4. Maintain proper size tracking
-
-Here's an example:
-
-```java
-class CustomCollectionLike {
- private final Object[] elements;
- private final int size;
-
- public CustomCollectionLike(int size) {
- this.elements = new Object[size];
- this.size = size;
- }
-
- // Constructor for wrapping existing array
- public CustomCollectionLike(Object[] elements, int size) {
- this.elements = elements;
- this.size = size;
- }
-
- public Object get(int index) {
- if (index >= size) {
- throw new IndexOutOfBoundsException();
- }
- return elements[index];
- }
-
- public int size() {
- return size;
- }
-
- public Object[] getElements() {
- return elements;
- }
-}
-
-// A view class that extends AbstractCollection for simpler implementation
-class CollectionView extends AbstractCollection<Object> {
- private final Object[] elements;
- private final int size;
- private int writeIndex;
-
- // Constructor for serialization (wrapping existing array)
- public CollectionView(CustomCollectionLike collection) {
- this.elements = collection.getElements();
- this.size = collection.size();
- }
-
- // Constructor for deserialization
- public CollectionView(int size) {
- this.size = size;
- this.elements = new Object[size];
- }
-
- @Override
- public Iterator<Object> iterator() {
- return new Iterator<Object>() {
- private int index = 0;
-
- @Override
- public boolean hasNext() {
- return index < size;
- }
-
- @Override
- public Object next() {
- if (!hasNext()) {
- throw new NoSuchElementException();
- }
- return elements[index++];
- }
- };
- }
-
- @Override
- public boolean add(Object element) {
- if (writeIndex >= size) {
- throw new IllegalStateException("Collection is full");
- }
- elements[writeIndex++] = element;
- return true;
- }
-
- @Override
- public int size() {
- return size;
- }
-
- public Object[] getElements() {
- return elements;
- }
-}
-
-class CustomCollectionSerializer extends CollectionLikeSerializer<CustomCollectionLike> {
- public CustomCollectionSerializer(Fory fory) {
- super(fory, CustomCollectionLike.class, true);
- }
-
- @Override
- public Collection onCollectionWrite(MemoryBuffer buffer, CustomCollectionLike value) {
- buffer.writeVarUint32Small7(value.size());
- return new CollectionView(value);
- }
-
- @Override
- public Collection newCollection(MemoryBuffer buffer) {
- int numElements = buffer.readVarUint32Small7();
- setNumElements(numElements);
- return new CollectionView(numElements);
- }
-
- @Override
- public CustomCollectionLike onCollectionRead(Collection collection) {
- CollectionView view = (CollectionView) collection;
- return new CustomCollectionLike(view.getElements(), view.size());
- }
-}
-```
-
-Key takeways:
-
-1. **Collection Structure**:
- - Array-based storage for elements
- - Fixed size after creation
- - Direct element access
- - Size tracking
-
-2. **View Implementation**:
- - Extends `AbstractCollection` for simplicity
- - Provides iterator for element access
- - Implements `add()` for deserialization
- - Shares array reference with original type
-
-3. **Serializer Features**:
- - Uses `supportCodegenHook=true` for JIT optimization
- - Shares array references when possible
- - Maintains proper size tracking
- - Uses view pattern for serialization
-
-4. **Performance Aspects**:
- - Direct array access
- - Minimal object creation
- - Array sharing between instances
- - Efficient iteration
-
-Note that this implementation provides better performance at the cost of flexibility. Consider your specific use case when choosing this approach.
-
-### Implement Map Serializer
-
-When implementing a serializer for a custom Map type, you must extend `MapSerializer` or `MapLikeSerializer`. The key difference between these two is that `MapLikeSerializer` can serialize a class which has a map-like structure but is not a java Map subtype.
-
-Similar to collection serializer, this is a special parameter `supportCodegenHook` needs be configured:
-
-- When `true`:
- - Enables optimized access to map elements and JIT compilation for better performance
- - Direct serialization invocation and inline for map key-value items without dynamic serializer dispatch cost.
- - Better performance for standard map types
- - Recommended for most maps
-
-- When `false`:
- - Uses interfaced-based element access and dynamic serializer dispatch for elements, which have higher cost
- - More flexible for custom map types
- - Required when map has special serialization needs
- - Handles complex map implementations
-
-#### Implement Map Serializer with JIT support
-
-When implementing a Map serializer with JIT support, you can leverage Fory's existing chunk-based binary format and map serialization infrastructure. The key is to properly implement the `onMapWrite` and `newMap` methods to handle metadata while letting Fory handle the map key-value serialization.
-
-Here's an example of implementing a custom map serializer:
-
-```java
-public class CustomMapSerializer<T extends Map> extends MapSerializer<T> {
- public CustomMapSerializer(Fory fory, Class<T> cls) {
- // supportCodegenHook is a critical parameter that determines serialization behavior
- super(fory, cls, true);
- }
-
- @Override
- public Map onMapWrite(MemoryBuffer buffer, T value) {
- // Write map size
- buffer.writeVarUint32Small7(value.size());
- // Write any additional map metadata here
- return value;
- }
-
- @Override
- public Map newMap(MemoryBuffer buffer) {
- // Read map size
- int numElements = buffer.readVarUint32Small7();
- setNumElements(numElements);
- // Create and return new map instance
- T map = (T) new HashMap(numElements);
- fory.getRefResolver().reference(map);
- return map;
- }
-}
-```
-
-Note that please invoke `setNumElements` when implementing `newMap` to let fory know how many elements to deserialize.
-
-#### Implement a totally-customzied Map Serializer without JIT
-
-Sometimes you may need complete control over the serialization process, or your map type might have special requirements that don't fit the standard patterns. In such cases, you can implement a serializer with `supportCodegenHook=false` and directly override the `write` and `read` methods.
-
-This approach:
-
-- Gives you full control over serialization
-- Allows custom binary format
-- Bypasses the standard map serialization pattern
-- May be simpler for special cases
-
-Here's an example:
-
-```java
-class FixedValueMap extends AbstractMap<String, Integer> {
- private final Set<String> keys;
- private final int fixedValue;
-
- public FixedValueMap(Set<String> keys, int fixedValue) {
- this.keys = keys;
- this.fixedValue = fixedValue;
- }
-
- @Override
- public Set<Entry<String, Integer>> entrySet() {
- Set<Entry<String, Integer>> entries = new HashSet<>();
- for (String key : keys) {
- entries.add(new SimpleEntry<>(key, fixedValue));
- }
- return entries;
- }
-
- @Override
- public Integer get(Object key) {
- return keys.contains(key) ? fixedValue : null;
- }
-
- public Set<String> getKeys() {
- return keys;
- }
-
- public int getFixedValue() {
- return fixedValue;
- }
-}
-
-class FixedValueMapSerializer extends MapLikeSerializer<FixedValueMap> {
- public FixedValueMapSerializer(Fory fory) {
- // Disable codegen since we're handling serialization directly
- super(fory, FixedValueMap.class, false);
- }
-
- @Override
- public void write(MemoryBuffer buffer, FixedValueMap value) {
- // Write the fixed value
- buffer.writeInt32(value.getFixedValue());
- // Write the number of keys
- buffer.writeVarUint32Small7(value.getKeys().size());
- // Write each key
- for (String key : value.getKeys()) {
- buffer.writeString(key);
- }
- }
-
- @Override
- public FixedValueMap read(MemoryBuffer buffer) {
- // Read the fixed value
- int fixedValue = buffer.readInt32();
- // Read the number of keys
- int size = buffer.readVarUint32Small7();
- Set<String> keys = new HashSet<>(size);
- for (int i = 0; i < size; i++) {
- keys.add(buffer.readString());
- }
- return new FixedValueMap(keys, fixedValue);
- }
-
- // These methods are not used when supportCodegenHook is false
- @Override
- public Map onMapWrite(MemoryBuffer buffer, FixedValueMap value) {
- throw new UnsupportedOperationException();
- }
-
- @Override
- public FixedValueMap onMapRead(Map map) {
- throw new UnsupportedOperationException();
- }
-
- @Override
- public FixedValueMap onMapCopy(Map map) {
- throw new UnsupportedOperationException();
- }
-}
-```
-
-Key Points:
-
-1. **Disable Codegen**:
- - Set `supportCodegenHook=false` in constructor
- - Fory will use your `write`/`read` methods directly
- - No JIT optimization will be applied
- - Full control over serialization format
-
-2. **Write Method**:
- - Handle all serialization manually
- - Write custom fields first
- - Write map entries in your preferred format
- - Control the exact binary layout
-
-3. **Read Method**:
- - Handle all deserialization manually
- - Read in same order as written
- - Create and populate map instance
- - Restore custom fields
-
-4. **Unused Methods**:
- - `onMapWrite`, `onMapRead`, `onMapCopy` are not used
- - Can throw `UnsupportedOperationException`
- - Only `write` and `read` are important
-
-When to Use: this approach is best when
-
-- Map has custom fields or metadata
-- Special serialization format is needed
-- Complete control over binary format is required
-- Standard map patterns don't fit
-
-Trade-offs
-
-1. **Advantages**:
- - Complete control over serialization
- - Custom binary format possible
- - Simpler implementation for special cases
- - Direct handling of custom fields
-
-2. **Disadvantages**:
- - No JIT optimization
- - Potentially lower performance
- - Manual handling of all serialization
- - More code to maintain
-
-Remember that disabling codegen means giving up some performance optimizations that Fory provides. Only use this approach when the standard map serialization pattern doesn't meet your needs.
-
-#### Implement Serializer for Map-like Types
-
-Sometimes you may want to implement a serializer for a type that behaves like a map but isn't a standard Java map. This section demonstrates how to implement a serializer for such types.
-
-The key principles for map-like type serialization are:
-
-1. Extend `MapLikeSerializer` for custom collection-like types
-2. Enable JIT optimization with `supportCodegenHook`
-3. Provide efficient element access through views
-4. Maintain proper size tracking
-
-Here's a complete example:
-
-```java
-// It's better to make it to implements the java.util.Map interface, in this way we don't have to implement such serializers by ourself.
-class CustomMapLike {
- private final Object[] keyArray;
- private final Object[] valueArray;
- private final int size;
-
- // Constructor for creating new instance
- public CustomMapLike(int initialCapacity) {
- this.keyArray = new Object[initialCapacity];
- this.valueArray = new Object[initialCapacity];
- this.size = 0;
- }
-
- // Constructor for wrapping existing arrays
- public CustomMapLike(Object[] keyArray, Object[] valueArray, int size) {
- this.keyArray = keyArray;
- this.valueArray = valueArray;
- this.size = size;
- }
-
- public Integer get(String key) {
- for (int i = 0; i < size; i++) {
- if (key.equals(keyArray[i])) {
- return (Integer) valueArray[i];
- }
- }
- return null;
- }
-
- public int size() {
- return size;
- }
-
- public Object[] getKeyArray() {
- return keyArray;
- }
-
- public Object[] getValueArray() {
- return valueArray;
- }
-}
-
-class MapView extends AbstractMap<Object, Object> {
- private final Object[] keyArray;
- private final Object[] valueArray;
- private final int size;
- private int writeIndex;
-
- // Constructor for serialization (wrapping existing CustomMapLike)
- public MapView(CustomMapLike mapLike) {
- this.size = mapLike.size();
- this.keyArray = mapLike.getKeyArray();
- this.valueArray = mapLike.getValueArray();
- }
-
- // Constructor for deserialization
- public MapView(int size) {
- this.size = size;
- this.keyArray = new Object[size];
- this.valueArray = new Object[size];
- }
-
- @Override
- public Set<Entry<Object, Object>> entrySet() {
- return new AbstractSet<Entry<Object, Object>>() {
- @Override
- public Iterator<Entry<Object, Object>> iterator() {
- return new Iterator<Entry<Object, Object>>() {
- private int index = 0;
-
- @Override
- public boolean hasNext() {
- return index < size;
- }
-
- @Override
- public Entry<Object, Object> next() {
- if (!hasNext()) {
- throw new NoSuchElementException();
- }
- final int currentIndex = index++;
- return new SimpleEntry<>(
- keyArray[currentIndex],
- valueArray[currentIndex]
- );
- }
- };
- }
-
- @Override
- public int size() {
- return size;
- }
- };
- }
-
- @Override
- public Object put(Object key, Object value) {
- if (writeIndex >= size) {
- throw new IllegalStateException("Map is full");
- }
- keyArray[writeIndex] = key;
- valueArray[writeIndex] = value;
- writeIndex++;
- return null;
- }
-
- public Object[] getKeyArray() {
- return keyArray;
- }
-
- public Object[] getValueArray() {
- return valueArray;
- }
-
- public int size() {
- return size;
- }
-}
-
-class CustomMapLikeSerializer extends MapLikeSerializer<CustomMapLike> {
- public CustomMapLikeSerializer(Fory fory) {
- super(fory, CustomMapLike.class, true);
- }
-
- @Override
- public Map onMapWrite(MemoryBuffer buffer, CustomMapLike value) {
- buffer.writeVarUint32Small7(value.size());
- // Return a zero-copy view using the same underlying arrays
- return new MapView(value);
- }
-
- @Override
- public Map newMap(MemoryBuffer buffer) {
- int numElements = buffer.readVarUint32Small7();
- setNumElements(numElements);
- // Create a view with new arrays for deserialization
- return new MapView(numElements);
- }
-
- @Override
- public CustomMapLike onMapRead(Map map) {
- MapView view = (MapView) map;
- // Just pass the arrays directly - no copying needed
- return new CustomMapLike(view.getKeyArray(), view.getValueArray(), view.size());
- }
-
- @Override
- public CustomMapLike onMapCopy(Map map) {
- MapView view = (MapView) map;
- // Just pass the arrays directly - no copying needed
- return new CustomMapLike(view.getKeyArray(), view.getValueArray(), view.size());
- }
-}
-```
-
-### Register Custom Serializers
-
-After implementing your custom serializer, register it with Fory:
-
-```java
-Fory fory = Fory.builder()
- .withLanguage(Language.JAVA)
- .build();
-
-// Register map serializer
-fory.registerSerializer(CustomMap.class, new CustomMapSerializer<>(fory, CustomMap.class));
-
-// Register collection serializer
-fory.registerSerializer(CustomCollection.class, new CustomCollectionSerializer<>(fory, CustomCollection.class));
-```
-
-Note that when implementing custom map or collection serializers:
-
-1. Always extend the appropriate base class (`MapSerializer`/`MapLikeSerializer` for maps, `CollectionSerializer`/`CollectionLikeSerializer` for collections)
-2. Consider the impact of `supportCodegenHook` on performance and functionality
-3. Properly handle reference tracking if needed
-4. Implement proper size management using `setNumElements` and `getAndClearNumElements` when `supportCodegenHook` is `true`
-
-Besides registering serializes, one can also implement `java.io.Externalizable` for a class to customize serialization logic, such type will be serialized by fory `ExternalizableSerializer`.
-
-### Memory Allocation Customization
-
-Fory provides a `MemoryAllocator` interface that allows you to customize how memory buffers are allocated and grown during serialization operations. This can be useful for performance optimization, memory pooling, or debugging memory usage.
-
-#### MemoryAllocator Interface
-
-The `MemoryAllocator` interface defines two key methods:
-
-```java
-public interface MemoryAllocator {
- /**
- * Allocates a new MemoryBuffer with the specified initial capacity.
- */
- MemoryBuffer allocate(int initialCapacity);
-
- /**
- * Grows an existing buffer to accommodate the new capacity.
- * The implementation must grow the buffer in-place by modifying
- * the existing buffer instance.
- */
- MemoryBuffer grow(MemoryBuffer buffer, int newCapacity);
-}
-```
-
-#### Using Custom Memory Allocators
-
-You can set a global memory allocator that will be used by all `MemoryBuffer` instances:
-
-```java
-// Create a custom allocator
-MemoryAllocator customAllocator = new MemoryAllocator() {
- @Override
- public MemoryBuffer allocate(int initialCapacity) {
- // Add extra capacity for debugging or pooling
- return MemoryBuffer.fromByteArray(new byte[initialCapacity + 100]);
- }
-
- @Override
- public MemoryBuffer grow(MemoryBuffer buffer, int newCapacity) {
- if (newCapacity <= buffer.size()) {
- return buffer;
- }
-
- // Custom growth strategy - add 100% extra capacity
- int newSize = (int) (newCapacity * 2);
- byte[] data = new byte[newSize];
- buffer.copyToUnsafe(0, data, Platform.BYTE_ARRAY_OFFSET, buffer.size());
- buffer.initHeapBuffer(data, 0, data.length);
- return buffer;
- }
-};
-
-// Set the custom allocator globally
-MemoryBuffer.setGlobalAllocator(customAllocator);
-
-// All subsequent MemoryBuffer allocations will use your custom allocator
-Fory fory = Fory.builder().withLanguage(Language.JAVA).build();
-byte[] bytes = fory.serialize(someObject); // Uses custom allocator
-```
-
-#### Default Memory Allocator Behavior
-
-The default allocator uses the following growth strategy:
-
-- For buffers smaller than `BUFFER_GROW_STEP_THRESHOLD` (100MB): multiply capacity by 2
-- For larger buffers: multiply capacity by 1.5 (capped at `Integer.MAX_VALUE - 8`)
-
-This provides a balance between avoiding frequent reallocations and preventing excessive memory usage.
-
-#### Use Cases
-
-Custom memory allocators are useful for:
-
-- **Memory Pooling**: Reuse allocated buffers to reduce GC pressure
-- **Performance Tuning**: Use different growth strategies based on your workload
-- **Debugging**: Add logging or tracking to monitor memory usage
-- **Off-heap Memory**: Integrate with off-heap memory management systems
-
-### Security
-
-#### Class Registration
-
-`ForyBuilder#requireClassRegistration` can be used to disable class registration, this will allow to deserialize objects
-unknown types,
-more flexible but **may be insecure if the classes contains malicious code**.
-
-**Do not disable class registration unless you can ensure your environment is secure**.
-Malicious code in `init/equals/hashCode` can be executed when deserializing unknown/untrusted types when this option
-disabled.
-
-Class registration can not only reduce security risks, but also avoid classname serialization cost.
-
-You can register class with API `Fory#register`.
-
-Note that class registration order is important, serialization and deserialization peer
-should have same registration order.
-
-```java
-Fory fory = xxx;
-fory.register(SomeClass.class);
-fory.register(SomeClass1.class, 1);
-```
-
-If you invoke `ForyBuilder#requireClassRegistration(false)` to disable class registration check,
-you can set `org.apache.fory.resolver.ClassChecker` by `ClassResolver#setClassChecker` to control which classes are
-allowed
-for serialization. For example, you can allow classes started with `org.example.*` by:
-
-```java
-Fory fory = xxx;
-fory.getClassResolver().setClassChecker(
- (classResolver, className) -> className.startsWith("org.example."));
-```
-
-```java
-AllowListChecker checker = new AllowListChecker(AllowListChecker.CheckLevel.STRICT);
-ThreadSafeFory fory = new ThreadLocalFory(classLoader -> {
- Fory f = Fory.builder().requireClassRegistration(true).withClassLoader(classLoader).build();
- f.getClassResolver().setClassChecker(checker);
- checker.addListener(f.getClassResolver());
- return f;
-});
-checker.allowClass("org.example.*");
-```
-
-Fory also provided a `org.apache.fory.resolver.AllowListChecker` which is allowed/disallowed list based checker to
-simplify
-the customization of class check mechanism. You can use this checker or implement more sophisticated checker by
-yourself.
-
-#### Limit max deserization depth
-
-Fory also provides a `ForyBuilder#withMaxDepth` to limit max deserialization depth. The default max depth is 50.
-
-If max depth is reached, Fory will throw `ForyException`. This can be used to prevent malicious data from causing stack overflow or other issues.
-
-### Register class by name
-
-Register class by id will have better performance and smaller space overhead. But in some cases, management for a bunch
-of type id is complex. In such cases, registering class by name using API
-`register(Class<?> cls, String namespace, String typeName)` is recommended.
-
-```java
-fory.register(Foo.class, "demo", "Foo");
-```
-
-If there are no duplicate name for type, `namespace` can be left as empty to reduce serialized size.
-
-**Do not use this API to register class since it will increase serialized size a lot compared to register
-class by id**
-
-### Zero-Copy Serialization
-
-```java
-import org.apache.fory.*;
-import org.apache.fory.config.*;
-import org.apache.fory.serializer.BufferObject;
-import org.apache.fory.memory.MemoryBuffer;
-
-import java.util.*;
-import java.util.stream.Collectors;
-
-public class ZeroCopyExample {
- // Note that fory instance should be reused instead of creation every time.
- static Fory fory = Fory.builder()
- .withLanguage(Language.JAVA)
- .build();
-
- // mvn exec:java -Dexec.mainClass="io.ray.fory.examples.ZeroCopyExample"
- public static void main(String[] args) {
- List<Object> list = Arrays.asList("str", new byte[1000], new int[100], new double[100]);
- Collection<BufferObject> bufferObjects = new ArrayList<>();
- byte[] bytes = fory.serialize(list, e -> !bufferObjects.add(e));
- List<MemoryBuffer> buffers = bufferObjects.stream()
- .map(BufferObject::toBuffer).collect(Collectors.toList());
- System.out.println(fory.deserialize(bytes, buffers));
- }
-}
-```
-
-### Meta Sharing
-
-Fory supports share type metadata (class name, field name, final field type information, etc.) between multiple
-serializations in a context (ex. TCP connection), and this information will be sent to the peer during the first
-serialization in the context. Based on this metadata, the peer can rebuild the same deserializer, which avoids
-transmitting metadata for subsequent serializations and reduces network traffic pressure and supports type
-forward/backward compatibility automatically.
-
-```java
-// Fory.builder()
-// .withLanguage(Language.JAVA)
-// .withRefTracking(false)
-// // share meta across serialization.
-// .withMetaContextShare(true)
-// Not thread-safe fory.
-MetaContext context = xxx;
-fory.getSerializationContext().setMetaContext(context);
-byte[] bytes = fory.serialize(o);
-// Not thread-safe fory.
-MetaContext context = xxx;
-fory.getSerializationContext().setMetaContext(context);
-fory.deserialize(bytes);
-
-// Thread-safe fory
-fory.setClassLoader(beanA.getClass().getClassLoader());
-byte[] serialized = fory.execute(
- f -> {
- f.getSerializationContext().setMetaContext(context);
- return f.serialize(beanA);
- }
-);
-// thread-safe fory
-fory.setClassLoader(beanA.getClass().getClassLoader());
-Object newObj = fory.execute(
- f -> {
- f.getSerializationContext().setMetaContext(context);
- return f.deserialize(serialized);
- }
-);
-```
-
-Note that `MetaContext` is not thread-safe and cannot be reused across instances of fory or multiple threads.
-In cases of multi-threading, a separate `MetaContext` must be created for each fory instance.
-
-### Deserialize non-existent classes
-
-Fory support deserializing non-existent classes, this feature can be enabled
-by `ForyBuilder#deserializeNonexistentClass(true)`. When enabled, and metadata sharing enabled, Fory will store
-the deserialized data of this type in a lazy subclass of Map. By using the lazy map implemented by Fory, the rebalance
-cost of filling map during deserialization can be avoided, which further improves performance. If this data is sent to
-another process and the class exists in this process, the data will be deserialized into the object of this type without
-losing any information.
-
-If metadata sharing is not enabled, the new class data will be skipped and an `NonexistentSkipClass` stub object will be
-returned.
-
-### Copy/Map object from one type to another type
-
-Fory support mapping object from one type to another type.
-
-> Notes:
->
-> 1. This mapping will execute a deep copy, all mapped fields are serialized into binary and
-> deserialized from that binary to map into another type.
-> 2. All struct types must be registered with same ID, otherwise Fory can not mapping to correct struct type.
-> Be careful when you use `Fory#register(Class)`, because fory will allocate an auto-grown ID which might be
-> inconsistent if you register classes with different order between Fory instance.
-
-```java
-public class StructMappingExample {
- static class Struct1 {
- int f1;
- String f2;
-
- public Struct1(int f1, String f2) {
- this.f1 = f1;
- this.f2 = f2;
- }
- }
-
- static class Struct2 {
- int f1;
- String f2;
- double f3;
- }
-
- static ThreadSafeFory fory1 = Fory.builder()
- .withCompatibleMode(CompatibleMode.COMPATIBLE).buildThreadSafeFory();
- static ThreadSafeFory fory2 = Fory.builder()
- .withCompatibleMode(CompatibleMode.COMPATIBLE).buildThreadSafeFory();
-
- static {
- fory1.register(Struct1.class);
- fory2.register(Struct2.class);
- }
-
- public static void main(String[] args) {
- Struct1 struct1 = new Struct1(10, "abc");
- Struct2 struct2 = (Struct2) fory2.deserialize(fory1.serialize(struct1));
- Assert.assertEquals(struct2.f1, struct1.f1);
- Assert.assertEquals(struct2.f2, struct1.f2);
- struct1 = (Struct1) fory1.deserialize(fory2.serialize(struct2));
- Assert.assertEquals(struct1.f1, struct2.f1);
- Assert.assertEquals(struct1.f2, struct2.f2);
- }
-}
-```
-
-### Logging
-
-#### ForyLogger
-
-By default Fory uses a custom logger `ForyLogger` for internal needs.
-It builds resulting logged data into single string and sends it directly to `System.out`.
-The result line layout is similar to (in Log4j notation):
-
-```
-%d{yyyy-MM-dd hh:mm:ss} %p %C:%L [%t] - %m%n
-```
-
-The layout can't be changed.
-
-Example output:
-
-```
-2025-11-07 08:49:59 INFO CompileUnit:55 [main] - Generate code for org.apache.fory.builder.SerializedLambdaForyCodec_0 took 35 ms.
-2025-11-07 08:50:00 INFO JaninoUtils:121 [main] - Compile [SerializedLambdaForyCodec_0] take 144 ms
-```
-
-#### Slf4jLogger
-
-If more sophisticated logger is required, configure Fory to use Slf4j via `LoggerFactory.useSlf4jLogging()`.
-For example enabling Slf4j before creating Fory:
-
-```java
-public static final ThreadSafeFory FORY;
-
-static {
- LoggerFactory.useSlf4jLogging(true);
- FORY = Fory.builder()
- .buildThreadSafeFory();
-}
-```
-
-**Note:** Enabling Slf4j via `useSlf4jLogging` will be ignored when the application runs in a GraalVM native image.
-
-#### Suppress Fory logs
-
-Both `ForyLogger` and `Slf4jLogger` allow controling log outpul level or suppressing logs entirely.
-Configure logger level via `LoggerFactory.setLogLevel()`, for example:
-
-```java
-static {
- // to log only WARN and high
- LoggerFactory.setLogLevel(LogLevel.WARN_LEVEL);
-
- // to disable at all
- LoggerFactory.disableLogging();
-}
-```
-
-**Note:** Selected logging level is applied before Slfj4 implementation's logger level. So if you set `WARN_LEVEL` (as in
-the example above) then you will not see INFO messages from Fory even if INFO is enabled in Logback.
-
-## Migration
-
-### JDK migration
-
-If you use jdk serialization before, and you can't upgrade your client and server at the same time, which is common for
-online application. Fory provided an util method `org.apache.fory.serializer.JavaSerializer.serializedByJDK` to check
-whether
-the binary are generated by jdk serialization, you use following pattern to make exiting serialization protocol-aware,
-then upgrade serialization to fory in an async rolling-up way:
-
-```java
-if (JavaSerializer.serializedByJDK(bytes)) {
- ObjectInputStream objectInputStream=xxx;
- return objectInputStream.readObject();
-} else {
- return fory.deserialize(bytes);
-}
-```
-
-### Upgrade fory
-
-Currently binary compatibility is ensured for minor versions only. For example, if you are using fory`v0.2.0`, binary
-compatibility will
-be provided if you upgrade to fory `v0.2.1`. But if upgrade to fory `v0.4.1`, no binary compatibility are ensured.
-Most of the time there is no need to upgrade fory to newer major version, the current version is fast and compact
-enough,
-and we provide some minor fix for recent older versions.
-
-But if you do want to upgrade fory for better performance and smaller size, you need to write fory version as header to
-serialized data
-using code like following to keep binary compatibility:
-
-```java
-MemoryBuffer buffer = xxx;
-buffer.writeVarInt32(2);
-fory.serialize(buffer, obj);
-```
-
-Then for deserialization, you need:
-
-```java
-MemoryBuffer buffer = xxx;
-int foryVersion = buffer.readVarInt32();
-Fory fory = getFory(foryVersion);
-fory.deserialize(buffer);
-```
-
-`getFory` is a method to load corresponding fory, you can shade and relocate different version of fory to different
-package, and load fory by version.
-
-If you upgrade fory by minor version, or you won't have data serialized by older fory, you can upgrade fory directly,
-no need to `versioning` the data.
-
-## Trouble shooting
-
-### Class inconsistency and class version check
-
-If you create fory without setting `CompatibleMode` to `org.apache.fory.config.CompatibleMode.COMPATIBLE`, and you got a
-strange
-serialization error, it may be caused by class inconsistency between serialization peer and deserialization peer.
-
-In such cases, you can invoke `ForyBuilder#withClassVersionCheck` to create fory to validate it, if deserialization
-throws `org.apache.fory.exception.ClassNotCompatibleException`, it shows class are inconsistent, and you should create
-fory with
-`ForyBuilder#withCompaibleMode(CompatibleMode.COMPATIBLE)`.
-
-`CompatibleMode.COMPATIBLE` has more performance and space cost, do not set it by default if your classes are always
-consistent between serialization and deserialization.
-
-### Deserialize POJO into another type
-
-Fory allows you to serialize one POJO and deserialize it into a different POJO. The different POJO means the schema inconsistency. Users must to configure Fory with
-`CompatibleMode` set to `org.apache.fory.config.CompatibleMode.COMPATIBLE`.
-
-```java
-public class DeserializeIntoType {
- static class Struct1 {
- int f1;
- String f2;
-
- public Struct1(int f1, String f2) {
- this.f1 = f1;
- this.f2 = f2;
- }
- }
-
- static class Struct2 {
- int f1;
- String f2;
- double f3;
- }
-
- static ThreadSafeFory fory = Fory.builder()
- .withCompatibleMode(CompatibleMode.COMPATIBLE).buildThreadSafeFory();
-
- public static void main(String[] args) {
- Struct1 struct1 = new Struct1(10, "abc");
- byte[] data = fory.serializeJavaObject(struct1);
- Struct2 struct2 = (Struct2) fory.deserializeJavaObject(bytes, Struct2.class);
- }
-}
-```
-
-### Use wrong API for deserialization
-
-If you serialize an object by invoking `Fory#serialize`, you should invoke `Fory#deserialize` for deserialization
-instead of
-`Fory#deserializeJavaObject`.
-
-If you serialize an object by invoking `Fory#serializeJavaObject`, you should invoke `Fory#deserializeJavaObject` for
-deserialization instead of `Fory#deserializeJavaObjectAndClass`/`Fory#deserialize`.
-
-If you serialize an object by invoking `Fory#serializeJavaObjectAndClass`, you should
-invoke `Fory#deserializeJavaObjectAndClass` for deserialization instead
-of `Fory#deserializeJavaObject`/`Fory#deserialize`.
diff --git a/docs/docs/guide/python_guide.md b/docs/docs/guide/python_guide.md
deleted file mode 100644
index 801ee70ff..000000000
--- a/docs/docs/guide/python_guide.md
+++ /dev/null
@@ -1,1394 +0,0 @@
----
-title: Python Serialization
-sidebar_position: 1
-id: python_serialization
-license: |
- Licensed to the Apache Software Foundation (ASF) under one or more
- contributor license agreements. See the NOTICE file distributed with
- this work for additional information regarding copyright ownership.
- The ASF licenses this file to You under the Apache License, Version 2.0
- (the "License"); you may not use this file except in compliance with
- the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
----
-# Apache Fory™ Python
-
-[](https://github.com/apache/fory/actions/workflows/ci.yml)
-[](https://pypi.org/project/pyfory/)
-[](https://pypi.org/project/pyfory/)
-[](https://opensource.org/licenses/Apache-2.0)
-[](https://join.slack.com/t/fory-project/shared_invite/zt-36g0qouzm-kcQSvV_dtfbtBKHRwT5gsw)
-[](https://x.com/ApacheFory)
-
-**Apache Fory™** is a blazing fast multi-language serialization framework powered by **JIT compilation** and **zero-copy** techniques, providing up to **ultra-fast performance** while maintaining ease of use and safety.
-
-`pyfory` provides the Python implementation of Apache Fory™, offering both high-performance object serialization and advanced row-format capabilities for data processing tasks.
-
-## 🚀 Key Features
-
-### 🔧 **Flexible Serialization Modes**
-
-- **Python native Mode**: Full Python compatibility, drop-in replacement for pickle/cloudpickle
-- **Cross-Language Mode**: Optimized for multi-language data exchange
-- **Row Format**: Zero-copy row format for analytics workloads
-
-### 🎯 Versatile Serialization Features
-
-- **Shared/circular reference support** for complex object graphs in both Python-native and cross-language modes
-- **Polymorphism support** for customized types with automatic type dispatching
-- **Schema evolution** support for backward/forward compatibility when using dataclasses in cross-language mode
-- **Out-of-band buffer support** for zero-copy serialization of large data structures like NumPy arrays and Pandas DataFrames, compatible with pickle protocol 5
-
-### ⚡ **Blazing Fast Performance**
-
-- **Extremely fast performance** compared to other serialization frameworks
-- **Runtime code generation** and **Cython-accelerated** core implementation for optimal performance
-
-### 📦 Compact Data Size
-
-- **Compact object graph protocol** with minimal space overhead—up to 3× size reduction compared to pickle/cloudpickle
-- **Meta packing and sharing** to minimize type forward/backward compatibility space overhead
-
-### 🛡️ **Security & Safety**
-
-- **Strict mode** prevents deserialization of untrusted types by type registration and checks.
-- **Reference tracking** for handling circular references safely
-
-## 📦 Installation
-
-### Basic Installation
-
-Install pyfory using pip:
-
-```bash
-pip install pyfory
-```
-
-### Optional Dependencies
-
-```bash
-# Install with row format support (requires Apache Arrow)
-pip install pyfory[format]
-
-# Install from source for development
-git clone https://github.com/apache/fory.git
-cd fory/python
-pip install -e ".[dev,format]"
-```
-
-### Requirements
-
-- **Python**: 3.8 or higher
-- **OS**: Linux, macOS, Windows
-
-## 🐍 Python Native Serialization
-
-`pyfory` provides a Python-native serialization mode that offers the same functionality as pickle/cloudpickle, but with **significantly better performance, smaller data size, and enhanced security features**.
-
-The binary protocol and API are similar to Fory's xlang mode, but Python-native mode can serialize any Python object—including global functions, local functions, lambdas, local classes and types with customized serialization using `__getstate__/__reduce__/__reduce_ex__`, which are not allowed in xlang mode.
-
-To use Python-native mode, create `Fory` with `xlang=False`. This mode is optimized for pure Python applications:
-
-```python
-import pyfory
-fory = pyfory.Fory(xlang=False, ref=False, strict=True)
-```
-
-### Basic Object Serialization
-
-Serialize and deserialize Python objects with a simple API. This example shows serializing a dictionary with mixed types:
-
-```python
-import pyfory
-
-# Create Fory instance
-fory = pyfory.Fory(xlang=True)
-
-# Serialize any Python object
-data = fory.dumps({"name": "Alice", "age": 30, "scores": [95, 87, 92]})
-
-# Deserialize back to Python object
-obj = fory.loads(data)
-print(obj) # {'name': 'Alice', 'age': 30, 'scores': [95, 87, 92]}
-```
-
-**Note**: `dumps()`/`loads()` are aliases for `serialize()`/`deserialize()`. Both APIs are identical, use whichever feels more intuitive.
-
-### Custom Class Serialization
-
-Fory automatically handles dataclasses and custom types. Register your class once, then serialize instances seamlessly:
-
-```python
-import pyfory
-from dataclasses import dataclass
-from typing import List, Dict
-
-@dataclass
-class Person:
- name: str
- age: int
- scores: List[int]
- metadata: Dict[str, str]
-
-# Python mode - supports all Python types including dataclasses
-fory = pyfory.Fory(xlang=False, ref=True)
-fory.register(Person)
-person = Person("Bob", 25, [88, 92, 85], {"team": "engineering"})
-data = fory.serialize(person)
-result = fory.deserialize(data)
-print(result) # Person(name='Bob', age=25, ...)
-```
-
-### Drop-in Replacement for Pickle/Cloudpickle
-
-`pyfory` can serialize any Python object with the following configuration:
-
-- **For circular references**: Set `ref=True` to enable reference tracking
-- **For functions/classes**: Set `strict=False` to allow deserialization of dynamic types
-
-**⚠️ Security Warning**: When `strict=False`, Fory will deserialize arbitrary types, which can pose security risks if data comes from untrusted sources. Only use `strict=False` in controlled environments where you trust the data source completely. If you do need to use `strict=False`, please configure a `DeserializationPolicy` when creating fory using `policy=your_policy` to controlling deserialization behavior.
-
-#### Common Usage
-
-Serialize common Python objects including dicts, lists, and custom classes without any registration:
-
-```python
-import pyfory
-
-# Create Fory instance
-fory = pyfory.Fory(xlang=False, ref=True, strict=False)
-
-# serialize common Python objects
-data = fory.dumps({"name": "Alice", "age": 30, "scores": [95, 87, 92]})
-print(fory.loads(data))
-
-# serialize custom objects
-from dataclasses import dataclass
-
-@dataclass
-class Person:
- name: str
- age: int
-
-person = Person("Bob", 25)
-data = fory.dumps(person)
-print(fory.loads(data)) # Person(name='Bob', age=25)
-```
-
-#### Serialize Global Functions
-
-Capture and get functions defined at module level. Fory deserialize and return same function object:
-
-```python
-import pyfory
-
-# Create Fory instance
-fory = pyfory.Fory(xlang=False, ref=True, strict=False)
-
-# serialize global functions
-def my_global_function(x):
- return 10 * x
-
-data = fory.dumps(my_global_function)
-print(fory.loads(data)(10)) # 100
-```
-
-#### Serialize Local Functions/Lambdas
-
-Serialize functions with closures and lambda expressions. Fory captures the closure variables automatically:
-
-```python
-import pyfory
-
-# Create Fory instance
-fory = pyfory.Fory(xlang=False, ref=True, strict=False)
-
-# serialize local functions with closures
-def my_function():
- local_var = 10
- def local_func(x):
- return x * local_var
- return local_func
-
-data = fory.dumps(my_function())
-print(fory.loads(data)(10)) # 100
-
-# serialize lambdas
-data = fory.dumps(lambda x: 10 * x)
-print(fory.loads(data)(10)) # 100
-```
-
-#### Serialize Global Classes/Methods
-
-Serialize class objects, instance methods, class methods, and static methods. All method types are supported:
-
-```python
-from dataclasses import dataclass
-import pyfory
-fory = pyfory.Fory(xlang=False, ref=True, strict=False)
-
-# serialize global class
-@dataclass
-class Person:
- name: str
- age: int
-
- def f(self, x):
- return self.age * x
-
- @classmethod
- def g(cls, x):
- return 10 * x
-
- @staticmethod
- def h(x):
- return 10 * x
-
-print(fory.loads(fory.dumps(Person))("Bob", 25)) # Person(name='Bob', age=25)
-# serialize global class instance method
-print(fory.loads(fory.dumps(Person("Bob", 20).f))(10)) # 200
-# serialize global class class method
-print(fory.loads(fory.dumps(Person.g))(10)) # 100
-# serialize global class static method
-print(fory.loads(fory.dumps(Person.h))(10)) # 100
-```
-
-#### Serialize Local Classes/Methods
-
-Serialize classes defined inside functions along with their methods. Useful for dynamic class creation:
-
-```python
-from dataclasses import dataclass
-import pyfory
-fory = pyfory.Fory(xlang=False, ref=True, strict=False)
-
-def create_local_class():
- class LocalClass:
- def f(self, x):
- return 10 * x
-
- @classmethod
- def g(cls, x):
- return 10 * x
-
- @staticmethod
- def h(x):
- return 10 * x
- return LocalClass
-
-# serialize local class
-data = fory.dumps(create_local_class())
-print(fory.loads(data)().f(10)) # 100
-
-# serialize local class instance method
-data = fory.dumps(create_local_class()().f)
-print(fory.loads(data)(10)) # 100
-
-# serialize local class method
-data = fory.dumps(create_local_class().g)
-print(fory.loads(data)(10)) # 100
-
-# serialize local class static method
-data = fory.dumps(create_local_class().h)
-print(fory.loads(data)(10)) # 100
-```
-
-### Out-of-Band Buffer Serialization
-
-Fory supports pickle5-compatible out-of-band buffer serialization for efficient zero-copy handling of large data structures. This is particularly useful for NumPy arrays, Pandas DataFrames, and other objects with large memory footprints.
-
-Out-of-band serialization separates metadata from the actual data buffers, allowing for:
-
-- **Zero-copy transfers** when sending data over networks or IPC using `memoryview`
-- **Improved performance** for large datasets
-- **Pickle5 compatibility** using `pickle.PickleBuffer`
-- **Flexible stream support** - write to any writable object (files, BytesIO, sockets, etc.)
-
-#### Basic Out-of-Band Serialization
-
-```python
-import pyfory
-import numpy as np
-
-fory = pyfory.Fory(xlang=False, ref=False, strict=False)
-
-# Large numpy array
-array = np.arange(10000, dtype=np.float64)
-
-# Serialize with out-of-band buffers
-buffer_objects = []
-serialized_data = fory.serialize(array, buffer_callback=buffer_objects.append)
-
-# Convert buffer objects to memoryview for zero-copy transmission
-# For contiguous buffers (bytes, numpy arrays), this is zero-copy
-# For non-contiguous data, a copy may be created to ensure contiguity
-buffers = [obj.getbuffer() for obj in buffer_objects]
-
-# Deserialize with out-of-band buffers (accepts memoryview, bytes, or Buffer)
-deserialized_array = fory.deserialize(serialized_data, buffers=buffers)
-
-assert np.array_equal(array, deserialized_array)
-```
-
-#### Out-of-Band with Pandas DataFrames
-
-```python
-import pyfory
-import pandas as pd
-import numpy as np
-
-fory = pyfory.Fory(xlang=False, ref=False, strict=False)
-
-# Create a DataFrame with numeric columns
-df = pd.DataFrame({
- 'a': np.arange(1000, dtype=np.float64),
- 'b': np.arange(1000, dtype=np.int64),
- 'c': ['text'] * 1000
-})
-
-# Serialize with out-of-band buffers
-buffer_objects = []
-serialized_data = fory.serialize(df, buffer_callback=buffer_objects.append)
-buffers = [obj.getbuffer() for obj in buffer_objects]
-
-# Deserialize
-deserialized_df = fory.deserialize(serialized_data, buffers=buffers)
-
-assert df.equals(deserialized_df)
-```
-
-#### Selective Out-of-Band Serialization
-
-You can control which buffers go out-of-band by providing a callback that returns `True` to keep data in-band or `False` (and appending to a list) to send it out-of-band:
-
-```python
-import pyfory
-import numpy as np
-
-fory = pyfory.Fory(xlang=False, ref=True, strict=False)
-
-arr1 = np.arange(1000, dtype=np.float64)
-arr2 = np.arange(2000, dtype=np.float64)
-data = [arr1, arr2]
-
-buffer_objects = []
-counter = 0
-
-def selective_callback(buffer_object):
- global counter
- counter += 1
- # Only send even-numbered buffers out-of-band
- if counter % 2 == 0:
- buffer_objects.append(buffer_object)
- return False # Out-of-band
- return True # In-band
-
-serialized = fory.serialize(data, buffer_callback=selective_callback)
-buffers = [obj.getbuffer() for obj in buffer_objects]
-deserialized = fory.deserialize(serialized, buffers=buffers)
-```
-
-#### Pickle5 Compatibility
-
-Fory's out-of-band serialization is fully compatible with pickle protocol 5. When objects implement `__reduce_ex__(protocol)`, Fory automatically uses protocol 5 to enable `pickle.PickleBuffer` support:
-
-```python
-import pyfory
-import pickle
-
-fory = pyfory.Fory(xlang=False, ref=False, strict=False)
-
-# PickleBuffer objects are automatically supported
-data = b"Large binary data"
-pickle_buffer = pickle.PickleBuffer(data)
-
-# Serialize with buffer callback for out-of-band handling
-buffer_objects = []
-serialized = fory.serialize(pickle_buffer, buffer_callback=buffer_objects.append)
-buffers = [obj.getbuffer() for obj in buffer_objects]
-
-# Deserialize with buffers
-deserialized = fory.deserialize(serialized, buffers=buffers)
-assert bytes(deserialized.raw()) == data
-```
-
-#### Writing Buffers to Different Streams
-
-The `BufferObject.write_to()` method accepts any writable stream object, making it flexible for various use cases:
-
-```python
-import pyfory
-import numpy as np
-import io
-
-fory = pyfory.Fory(xlang=False, ref=False, strict=False)
-
-array = np.arange(1000, dtype=np.float64)
-
-# Collect out-of-band buffers
-buffer_objects = []
-serialized = fory.serialize(array, buffer_callback=buffer_objects.append)
-
-# Write to different stream types
-for buffer_obj in buffer_objects:
- # Write to BytesIO (in-memory stream)
- bytes_stream = io.BytesIO()
- buffer_obj.write_to(bytes_stream)
-
- # Write to file
- with open('/tmp/buffer_data.bin', 'wb') as f:
- buffer_obj.write_to(f)
-
- # Get zero-copy memoryview (for contiguous buffers)
- mv = buffer_obj.getbuffer()
- assert isinstance(mv, memoryview)
-```
-
-**Note**: For contiguous memory buffers (like bytes, numpy arrays), `getbuffer()` returns a zero-copy `memoryview`. For non-contiguous data, a copy may be created to ensure contiguity.
-
-## 🏃♂️ Cross-Language Object Graph Serialization
-
-`pyfory` supports cross-language object graph serialization, allowing you to serialize data in Python and deserialize it in Java, Go, Rust, or other supported languages.
-
-The binary protocol and API are similar to `pyfory`'s python-native mode, but Python-native mode can serialize any Python object—including global functions, local functions, lambdas, local classes, and types with customized serialization using `__getstate__/__reduce__/__reduce_ex__`, which are not allowed in xlang mode.
-
-To use xlang mode, create `Fory` with `xlang=True`. This mode is for xlang serialization applications:
-
-```python
-import pyfory
-fory = pyfory.Fory(xlang=True, ref=False, strict=True)
-```
-
-### Cross-Language Sserialization
-
-Serialize data in Python and deserialize it in Java, Go, Rust, or other supported languages. Both sides must register the same type with matching names:
-
-**Python (Serializer)**
-
-```python
-import pyfory
-
-# Cross-language mode for interoperability
-f = pyfory.Fory(xlang=True, ref=True)
-
-# Register type for cross-language compatibility
-@dataclass
-class Person:
- name: str
- age: pyfory.int32
-
-f.register(Person, typename="example.Person")
-
-person = Person("Charlie", 35)
-binary_data = f.serialize(person)
-# binary_data can now be sent to Java, Go, etc.
-```
-
-**Java (Deserializer)**
-
-```java
-import org.apache.fory.*;
-
-public class Person {
- public String name;
- public int age;
-}
-
-Fory fory = Fory.builder()
- .withLanguage(Language.XLANG)
- .withRefTracking(true)
- .build();
-
-fory.register(Person.class, "example.Person");
-Person person = (Person) fory.deserialize(binaryData);
-```
-
-## 📊 Row Format - Zero-Copy Processing
-
-Apache Fury™ provides a random-access row format that enables reading nested fields from binary data without full deserialization. This drastically reduces overhead when working with large objects where only partial data access is needed. The format also supports memory-mapped files for ultra-low memory footprint.
-
-### Basic Row Format Usage
-
-Encode objects to row format for random access without full deserialization. Ideal for large datasets:
-
-**Python**
-
-```python
-import pyfory
-import pyarrow as pa
-from dataclasses import dataclass
-from typing import List, Dict
-
-@dataclass
-class Bar:
- f1: str
- f2: List[pa.int64]
-
-@dataclass
-class Foo:
- f1: pa.int32
- f2: List[pa.int32]
- f3: Dict[str, pa.int32]
- f4: List[Bar]
-
-# Create encoder for row format
-encoder = pyfory.encoder(Foo)
-
-# Create large dataset
-foo = Foo(
- f1=10,
- f2=list(range(1_000_000)),
- f3={f"k{i}": i for i in range(1_000_000)},
- f4=[Bar(f1=f"s{i}", f2=list(range(10))) for i in range(1_000_000)]
-)
-
-# Encode to row format
-binary: bytes = encoder.to_row(foo).to_bytes()
-
-# Zero-copy access - no full deserialization needed!
-foo_row = pyfory.RowData(encoder.schema, binary)
-print(foo_row.f2[100000]) # Access 100,000th element directly
-print(foo_row.f4[100000].f1) # Access nested field directly
-print(foo_row.f4[200000].f2[5]) # Access deeply nested field directly
-```
-
-### Cross-Language Compatibility
-
-Row format works across languages. Here's the same data structure accessed in Java:
-
-**Java**
-
-```java
-public class Bar {
- String f1;
- List<Long> f2;
-}
-
-public class Foo {
- int f1;
- List<Integer> f2;
- Map<String, Integer> f3;
- List<Bar> f4;
-}
-
-RowEncoder<Foo> encoder = Encoders.bean(Foo.class);
-
-// Create large dataset
-Foo foo = new Foo();
-foo.f1 = 10;
-foo.f2 = IntStream.range(0, 1_000_000).boxed().collect(Collectors.toList());
-foo.f3 = IntStream.range(0, 1_000_000).boxed().collect(Collectors.toMap(i -> "k" + i, i -> i));
-List<Bar> bars = new ArrayList<>(1_000_000);
-for (int i = 0; i < 1_000_000; i++) {
- Bar bar = new Bar();
- bar.f1 = "s" + i;
- bar.f2 = LongStream.range(0, 10).boxed().collect(Collectors.toList());
- bars.add(bar);
-}
-foo.f4 = bars;
-
-// Encode to row format (cross-language compatible with Python)
-BinaryRow binaryRow = encoder.toRow(foo);
-
-// Zero-copy random access without full deserialization
-BinaryArray f2Array = binaryRow.getArray(1); // Access f2 list
-BinaryArray f4Array = binaryRow.getArray(3); // Access f4 list
-BinaryRow bar10 = f4Array.getStruct(10); // Access 11th Bar
-long value = bar10.getArray(1).getInt64(5); // Access 6th element of bar.f2
-
-// Partial deserialization - only deserialize what you need
-RowEncoder<Bar> barEncoder = Encoders.bean(Bar.class);
-Bar bar1 = barEncoder.fromRow(f4Array.getStruct(10)); // Deserialize 11th Bar only
-Bar bar2 = barEncoder.fromRow(f4Array.getStruct(20)); // Deserialize 21st Bar only
-
-// Full deserialization when needed
-Foo newFoo = encoder.fromRow(binaryRow);
-```
-
-**C++**
-
-And in C++ with compile-time type information:
-
-```cpp
-#include "fory/encoder/row_encoder.h"
-#include "fory/row/writer.h"
-
-struct Bar {
- std::string f1;
- std::vector<int64_t> f2;
-};
-
-FORY_FIELD_INFO(Bar, f1, f2);
-
-struct Foo {
- int32_t f1;
- std::vector<int32_t> f2;
- std::map<std::string, int32_t> f3;
- std::vector<Bar> f4;
-};
-
-FORY_FIELD_INFO(Foo, f1, f2, f3, f4);
-
-// Create large dataset
-Foo foo;
-foo.f1 = 10;
-for (int i = 0; i < 1000000; i++) {
- foo.f2.push_back(i);
- foo.f3["k" + std::to_string(i)] = i;
-}
-for (int i = 0; i < 1000000; i++) {
- Bar bar;
- bar.f1 = "s" + std::to_string(i);
- for (int j = 0; j < 10; j++) {
- bar.f2.push_back(j);
- }
- foo.f4.push_back(bar);
-}
-
-// Encode to row format (cross-language compatible with Python/Java)
-fory::encoder::RowEncoder<Foo> encoder;
-encoder.Encode(foo);
-auto row = encoder.GetWriter().ToRow();
-
-// Zero-copy random access without full deserialization
-auto f2_array = row->GetArray(1); // Access f2 list
-auto f4_array = row->GetArray(3); // Access f4 list
-auto bar10 = f4_array->GetStruct(10); // Access 11th Bar
-int64_t value = bar10->GetArray(1)->GetInt64(5); // Access 6th element of bar.f2
-std::string str = bar10->GetString(0); // Access bar.f1
-```
-
-### Key Benefits
-
-- **Zero-Copy Access**: Read nested fields without deserializing the entire object
-- **Memory Efficiency**: Memory-map large datasets directly from disk
-- **Cross-Language**: Binary format is compatible between Python, Java, and other Fury implementations
-- **Partial Deserialization**: Deserialize only the specific elements you need
-- **High Performance**: Skip unnecessary data parsing for analytics and big data workloads
-
-## 🏗️ Core API Reference
-
-### Fory Class
-
-The main serialization interface:
-
-```python
-class Fory:
- def __init__(
- self,
- xlang: bool = False,
- ref: bool = False,
- strict: bool = True,
- compatible: bool = False,
- max_depth: int = 50
- )
-```
-
-### ThreadSafeFory Class
-
-Thread-safe serialization interface using thread-local storage:
-
-```python
-class ThreadSafeFory:
- def __init__(
- self,
- xlang: bool = False,
- ref: bool = False,
- strict: bool = True,
- compatible: bool = False,
- max_depth: int = 50
- )
-```
-
-`ThreadSafeFory` provides thread-safe serialization by maintaining a pool of `Fory` instances protected by a lock. When a thread needs to serialize/deserialize, it gets an instance from the pool, uses it, and returns it. All type registrations must be done before any serialization to ensure consistency across all instances.
-
-**Thread Safety Example:**
-
-```python
-import pyfory
-import threading
-from dataclasses import dataclass
-
-@dataclass
-class Person:
- name: str
- age: int
-
-# Create thread-safe Fory instance
-fory = pyfory.ThreadSafeFory(xlang=False, ref=True)
-fory.register(Person)
-
-# Use in multiple threads safely
-def serialize_in_thread(thread_id):
- person = Person(name=f"User{thread_id}", age=25 + thread_id)
- data = fory.serialize(person)
- result = fory.deserialize(data)
- print(f"Thread {thread_id}: {result}")
-
-threads = [threading.Thread(target=serialize_in_thread, args=(i,)) for i in range(10)]
-for t in threads: t.start()
-for t in threads: t.join()
-```
-
-**Key Features:**
-
-- **Instance Pool**: Maintains a pool of `Fory` instances protected by a lock for thread safety
-- **Shared Configuration**: All registrations must be done upfront and are applied to all instances
-- **Same API**: Drop-in replacement for `Fory` class with identical methods
-- **Registration Safety**: Prevents registration after first use to ensure consistency
-
-**When to Use:**
-
-- **Multi-threaded Applications**: Web servers, concurrent workers, parallel processing
-- **Shared Fory Instances**: When multiple threads need to serialize/deserialize data
-- **Thread Pools**: Applications using thread pools or concurrent.futures
-
-**Parameters:**
-
-- **`xlang`** (`bool`, default=`False`): Enable cross-language serialization. When `False`, enables Python-native mode supporting all Python objects. When `True`, enables cross-language mode compatible with Java, Go, Rust, etc.
-- **`ref`** (`bool`, default=`False`): Enable reference tracking for shared/circular references. Disable for better performance if your data has no shared references.
-- **`strict`** (`bool`, default=`True`): Require type registration for security. **Highly recommended** for production. Only disable in trusted environments.
-- **`compatible`** (`bool`, default=`False`): Enable schema evolution in cross-language mode, allowing fields to be added/removed while maintaining compatibility.
-- **`max_depth`** (`int`, default=`50`): Maximum deserialization depth for security, preventing stack overflow attacks.
-
-**Key Methods:**
-
-```python
-# Serialization (serialize/deserialize are identical to dumps/loads)
-data: bytes = fory.serialize(obj)
-obj = fory.deserialize(data)
-
-# Alternative API (aliases)
-data: bytes = fory.dumps(obj)
-obj = fory.loads(data)
-
-# Type registration by id (for Python mode)
-fory.register(MyClass, type_id=123)
-fory.register(MyClass, type_id=123, serializer=custom_serializer)
-
-# Type registration by name (for cross-language mode)
-fory.register(MyClass, typename="my.package.MyClass")
-fory.register(MyClass, typename="my.package.MyClass", serializer=custom_serializer)
-```
-
-### Language Modes Comparison
-
-| Feature | Python Mode (`xlang=False`) | Cross-Language Mode (`xlang=True`) |
-| --------------------- | ------------------------------------ | ------------------------------------- |
-| **Use Case** | Pure Python applications | Multi-language systems |
-| **Compatibility** | Python only | Java, Go, Rust, C++, JavaScript, etc. |
-| **Supported Types** | All Python types | Cross-language compatible types only |
-| **Functions/Lambdas** | ✓ Supported | ✗ Not allowed |
-| **Local Classes** | ✓ Supported | ✗ Not allowed |
-| **Dynamic Classes** | ✓ Supported | ✗ Not allowed |
-| **Schema Evolution** | ✓ Supported (with `compatible=True`) | ✓ Supported (with `compatible=True`) |
-| **Performance** | Extremely fast | Very fast |
-| **Data Size** | Compact | Compact with type metadata |
-
-#### Python Mode (`xlang=False`)
-
-Python mode supports all Python types including functions, classes, and closures. Perfect for pure Python applications:
-
-```python
-import pyfory
-
-# Full Python compatibility mode
-fory = pyfory.Fory(xlang=False, ref=True, strict=False)
-
-# Supports ALL Python objects:
-data = fory.dumps({
- 'function': lambda x: x * 2, # Functions and lambdas
- 'class': type('Dynamic', (), {}), # Dynamic classes
- 'method': str.upper, # Methods
- 'nested': {'circular_ref': None} # Circular references (when ref=True)
-})
-
-# Drop-in replacement for pickle/cloudpickle
-import pickle
-obj = [1, 2, {"nested": [3, 4]}]
-assert fory.loads(fory.dumps(obj)) == pickle.loads(pickle.dumps(obj))
-
-# Significantly faster and more compact than pickle
-import timeit
-obj = {f"key{i}": f"value{i}" for i in range(10000)}
-print(f"Fory: {timeit.timeit(lambda: fory.dumps(obj), number=1000):.3f}s")
-print(f"Pickle: {timeit.timeit(lambda: pickle.dumps(obj), number=1000):.3f}s")
-```
-
-#### Cross-Language Mode (`xlang=True`)
-
-Cross-language mode restricts types to those compatible across all Fory implementations. Use for multi-language systems:
-
-```python
-import pyfory
-
-# Cross-language compatibility mode
-f = pyfory.Fory(xlang=True, ref=True)
-
-# Only supports cross-language compatible types
-f.register(MyDataClass, typename="com.example.MyDataClass")
-
-# Data can be read by Java, Go, Rust, etc.
-data = f.serialize(MyDataClass(field1="value", field2=42))
-```
-
-## 🔧 Advanced Features
-
-### Reference Tracking & Circular References
-
-Handle shared references and circular dependencies safely. Set `ref=True` to deduplicate objects:
-
-```python
-import pyfory
-
-f = pyfory.Fory(ref=True) # Enable reference tracking
-
-# Handle circular references safely
-class Node:
- def __init__(self, value):
- self.value = value
- self.children = []
- self.parent = None
-
-root = Node("root")
-child = Node("child")
-child.parent = root # Circular reference
-root.children.append(child)
-
-# Serializes without infinite recursion
-data = f.serialize(root)
-result = f.deserialize(data)
-assert result.children[0].parent is result # Reference preserved
-```
-
-### Type Registration & Security
-
-In strict mode, only registered types can be deserialized. This prevents arbitrary code execution:
-
-```python
-import pyfory
-
-# Strict mode (recommended for production)
-f = pyfory.Fory(strict=True)
-
-class SafeClass:
- def __init__(self, data):
- self.data = data
-
-# Must register types in strict mode
-f.register(SafeClass, typename="com.example.SafeClass")
-
-# Now serialization works
-obj = SafeClass("safe data")
-data = f.serialize(obj)
-result = f.deserialize(data)
-
-# Unregistered types will raise an exception
-class UnsafeClass:
- pass
-
-# This will fail in strict mode
-try:
- f.serialize(UnsafeClass())
-except Exception as e:
- print("Security protection activated!")
-```
-
-### Custom Serializers
-
-Implement custom serialization logic for specialized types. Override `write/read` for Python mode, `xwrite/xread` for cross-language:
-
-```python
-import pyfory
-from pyfory.serializer import Serializer
-from dataclasses import dataclass
-
-@dataclass
-class Foo:
- f1: int
- f2: str
-
-class FooSerializer(Serializer):
- def __init__(self, fory, cls):
- super().__init__(fory, cls)
-
- def write(self, buffer, obj: Foo):
- # Custom serialization logic
- buffer.write_varint32(obj.f1)
- buffer.write_string(obj.f2)
-
- def read(self, buffer):
- # Custom deserialization logic
- f1 = buffer.read_varint32()
- f2 = buffer.read_string()
- return Foo(f1, f2)
-
- # For cross-language mode
- def xwrite(self, buffer, obj: Foo):
- buffer.write_int32(obj.f1)
- buffer.write_string(obj.f2)
-
- def xread(self, buffer):
- return Foo(buffer.read_int32(), buffer.read_string())
-
-f = pyfory.Fory()
-f.register(Foo, type_id=100, serializer=FooSerializer(f, Foo))
-
-# Now Foo uses your custom serializer
-data = f.dumps(Foo(42, "hello"))
-result = f.loads(data)
-print(result) # Foo(f1=42, f2='hello')
-```
-
-### Numpy & Scientific Computing
-
-Fory natively supports numpy arrays with optimized serialization. Large arrays use zero-copy when possible:
-
-```python
-import pyfory
-import numpy as np
-
-f = pyfory.Fory()
-
-# Numpy arrays are supported natively
-arrays = {
- 'matrix': np.random.rand(1000, 1000),
- 'vector': np.arange(10000),
- 'bool_mask': np.random.choice([True, False], size=5000)
-}
-
-data = f.serialize(arrays)
-result = f.deserialize(data)
-
-# Zero-copy for compatible array types
-assert np.array_equal(arrays['matrix'], result['matrix'])
-```
-
-## 💡 Best Practices
-
-### Production Configuration
-
-Use these recommended settings to balance security, performance, and functionality in production:
-
-```python
-import pyfory
-
-# Recommended settings for production
-fory = pyfory.Fory(
- xlang=False, # Use True if you need cross-language support
- ref=False, # Enable if you have shared/circular references
- strict=True, # CRITICAL: Always True in production
- compatible=False, # Enable only if you need schema evolution
- max_depth=20 # Adjust based on your data structure depth
-)
-
-# Register all types upfront
-fory.register(UserModel, type_id=100)
-fory.register(OrderModel, type_id=101)
-fory.register(ProductModel, type_id=102)
-```
-
-### Performance Tips
-
-Optimize serialization speed and memory usage with these guidelines:
-
-1. **Disable `ref=True` if not needed**: Reference tracking has overhead
-2. **Use type_id instead of typename**: Integer IDs are faster than string names
-3. **Reuse Fory instances**: Create once, use many times
-4. **Enable Cython**: Make sure `ENABLE_FORY_CYTHON_SERIALIZATION=1`, should be enabled by default
-5. **Use row format for large arrays**: Zero-copy access for analytics
-
-```python
-# Good: Reuse instance
-fory = pyfory.Fory()
-for obj in objects:
- data = fory.dumps(obj)
-
-# Bad: Create new instance each time
-for obj in objects:
- fory = pyfory.Fory() # Wasteful!
- data = fory.dumps(obj)
-```
-
-### Type Registration Patterns
-
-Choose the right registration approach for your use case:
-
-```python
-# Pattern 1: Simple registration
-fory.register(MyClass, type_id=100)
-
-# Pattern 2: Cross-language with typename
-fory.register(MyClass, typename="com.example.MyClass")
-
-# Pattern 3: With custom serializer
-fory.register(MyClass, type_id=100, serializer=MySerializer(fory, MyClass))
-
-# Pattern 4: Batch registration
-type_id = 100
-for model_class in [User, Order, Product, Invoice]:
- fory.register(model_class, type_id=type_id)
- type_id += 1
-```
-
-### Error Handling
-
-Handle common serialization errors gracefully. Catch specific exceptions for better error recovery:
-
-```python
-import pyfory
-from pyfory.error import TypeUnregisteredError, TypeNotCompatibleError
-
-fory = pyfory.Fory(strict=True)
-
-try:
- data = fory.dumps(my_object)
-except TypeUnregisteredError as e:
- print(f"Type not registered: {e}")
- # Register the type and retry
- fory.register(type(my_object), type_id=100)
- data = fory.dumps(my_object)
-except Exception as e:
- print(f"Serialization failed: {e}")
-
-try:
- obj = fory.loads(data)
-except TypeNotCompatibleError as e:
- print(f"Schema mismatch: {e}")
- # Handle version mismatch
-except Exception as e:
- print(f"Deserialization failed: {e}")
-```
-
-## 🛠️ Migration Guide
-
-### From Pickle
-
-Replace pickle with Fory for better performance while keeping the same API:
-
-```python
-# Before (pickle)
-import pickle
-data = pickle.dumps(obj)
-result = pickle.loads(data)
-
-# After (Fory - drop-in replacement with better performance)
-import pyfory
-f = pyfory.Fory(xlang=False, ref=True, strict=False)
-data = f.dumps(obj) # Faster and more compact
-result = f.loads(data) # Faster deserialization
-
-# Benefits:
-# - 2-10x faster serialization
-# - 2-5x faster deserialization
-# - Up to 3x smaller data size
-# - Same API, better performance
-```
-
-### From JSON
-
-Unlike JSON, Fory supports arbitrary Python types including functions:
-
-```python
-# Before (JSON - limited types)
-import json
-data = json.dumps({"name": "Alice", "age": 30})
-result = json.loads(data)
-
-# After (Fory - all Python types)
-import pyfory
-f = pyfory.Fory()
-data = f.dumps({"name": "Alice", "age": 30, "func": lambda x: x})
-result = f.loads(data)
-```
-
-## 🚨 Security Best Practices
-
-### Production Configuration
-
-Never disable `strict=True` in production unless your environment is completely trusted:
-
-```python
-import pyfory
-
-# Recommended production settings
-f = pyfory.Fory(
- xlang=False, # or True for cross-language
- ref=True, # Handle circular references
- strict=True, # IMPORTANT: Prevent malicious data
- max_depth=100 # Prevent deep recursion attacks
-)
-
-# Explicitly register allowed types
-f.register(UserModel, type_id=100)
-f.register(OrderModel, type_id=101)
-# Never set strict=False in production with untrusted data!
-```
-
-### Development vs Production
-
-Use environment variables to switch between development and production configurations:
-
-```python
-import pyfory
-import os
-
-# Development configuration
-if os.getenv('ENV') == 'development':
- fory = pyfory.Fory(
- xlang=False,
- ref=True,
- strict=False, # Allow any type for development
- max_depth=1000 # Higher limit for development
- )
-else:
- # Production configuration (security hardened)
- fory = pyfory.Fory(
- xlang=False,
- ref=True,
- strict=True, # CRITICAL: Require registration
- max_depth=100 # Reasonable limit
- )
- # Register only known safe types
- for idx, model_class in enumerate([UserModel, ProductModel, OrderModel]):
- fory.register(model_class, type_id=100 + idx)
-```
-
-### DeserializationPolicy
-
-When `strict=False` is necessary (e.g., deserializing functions/lambdas), use `DeserializationPolicy` to implement fine-grained security controls during deserialization. This provides protection similar to `pickle.Unpickler.find_class()` but with more comprehensive hooks.
-
-**Why use DeserializationPolicy?**
-
-- Block dangerous classes/modules (e.g., `subprocess.Popen`)
-- Intercept and validate `__reduce__` callables before invocation
-- Sanitize sensitive data during `__setstate__`
-- Replace or reject deserialized objects based on custom rules
-
-**Example: Blocking Dangerous Classes**
-
-```python
-import pyfory
-from pyfory import DeserializationPolicy
-
-dangerous_modules = {'subprocess', 'os', '__builtin__'}
-
-class SafeDeserializationPolicy(DeserializationPolicy):
- """Block potentially dangerous classes during deserialization."""
-
- def validate_class(self, cls, is_local, **kwargs):
- # Block dangerous modules
- if cls.__module__ in dangerous_modules:
- raise ValueError(f"Blocked dangerous class: {cls.__module__}.{cls.__name__}")
- return None
-
- def intercept_reduce_call(self, callable_obj, args, **kwargs):
- # Block specific callable invocations during __reduce__
- if getattr(callable_obj, '__name__', "") == 'Popen':
- raise ValueError("Blocked attempt to invoke subprocess.Popen")
- return None
-
- def intercept_setstate(self, obj, state, **kwargs):
- # Sanitize sensitive data
- if isinstance(state, dict) and 'password' in state:
- state['password'] = '***REDACTED***'
- return None
-
-# Create Fory with custom security policy
-policy = SafeDeserializationPolicy()
-fory = pyfory.Fory(xlang=False, ref=True, strict=False, policy=policy)
-
-# Now deserialization is protected by your custom policy
-data = fory.serialize(my_object)
-result = fory.deserialize(data) # Policy hooks will be invoked
-```
-
-**Available Policy Hooks:**
-
-- `validate_class(cls, is_local)` - Validate/block class types during deserialization
-- `validate_module(module, is_local)` - Validate/block module imports
-- `validate_function(func, is_local)` - Validate/block function references
-- `intercept_reduce_call(callable_obj, args)` - Intercept `__reduce__` invocations
-- `inspect_reduced_object(obj)` - Inspect/replace objects created via `__reduce__`
-- `intercept_setstate(obj, state)` - Sanitize state before `__setstate__`
-- `authorize_instantiation(cls, args, kwargs)` - Control class instantiation
-
-**See also:** `pyfory/policy.py` contains detailed documentation and examples for each hook.
-
-## 🐛 Troubleshooting
-
-### Common Issues
-
-**Q: ImportError with format features**
-
-```python
-# A: Install Row format support
-pip install pyfory[format]
-
-# Or install from source with format support
-pip install -e ".[format]"
-```
-
-**Q: Slow serialization performance**
-
-```python
-# A: Check if Cython acceleration is enabled
-import pyfory
-print(pyfory.ENABLE_FORY_CYTHON_SERIALIZATION) # Should be True
-
-# If False, Cython extension may not be compiled correctly
-# Reinstall with: pip install --force-reinstall --no-cache-dir pyfory
-
-# For debugging, you can disable Cython mode before importing
-import os
-os.environ['ENABLE_FORY_CYTHON_SERIALIZATION'] = '0'
-import pyfory # Now uses pure Python mode
-```
-
-**Q: Cross-language compatibility issues**
-
-```python
-# A: Use explicit type registration with consistent naming
-f = pyfory.Fory(xlang=True)
-f.register(MyClass, typename="com.package.MyClass") # Use same name in all languages
-```
-
-**Q: Circular reference errors or duplicate data**
-
-```python
-# A: Enable reference tracking
-f = pyfory.Fory(ref=True) # Required for circular references
-
-# Example with circular reference
-class Node:
- def __init__(self, value):
- self.value = value
- self.next = None
-
-node1 = Node(1)
-node2 = Node(2)
-node1.next = node2
-node2.next = node1 # Circular reference
-
-data = f.dumps(node1)
-result = f.loads(data)
-assert result.next.next is result # Circular reference preserved
-```
-
-### Debug Mode
-
-```python
-# Set environment variable BEFORE importing pyfory to disable Cython for debugging
-import os
-os.environ['ENABLE_FORY_CYTHON_SERIALIZATION'] = '0'
-import pyfory # Now uses pure Python implementation
-
-# This is useful for:
-# 1. Debugging protocol issues
-# 2. Understanding serialization behavior
-# 3. Development without recompiling Cython
-```
-
-**Q: Schema evolution not working**
-
-```python
-# A: Enable compatible mode for schema evolution
-f = pyfory.Fory(xlang=True, compatible=True)
-
-# Version 1: Original class
-@dataclass
-class User:
- name: str
- age: int
-
-f.register(User, typename="User")
-data = f.dumps(User("Alice", 30))
-
-# Version 2: Add new field (backward compatible)
-@dataclass
-class User:
- name: str
- age: int
- email: str = "unknown@example.com" # New field with default
-
-# Can still deserialize old data
-user = f.loads(data)
-print(user.email) # "unknown@example.com"
-```
-
-**Q: Type registration errors in strict mode**
-
-```python
-# A: Register all custom types before serialization
-f = pyfory.Fory(strict=True)
-
-# Must register before use
-f.register(MyClass, type_id=100)
-f.register(AnotherClass, type_id=101)
-
-# Or disable strict mode (NOT recommended for production)
-f = pyfory.Fory(strict=False) # Use only in trusted environments
-```
-
-## 🤝 Contributing
-
-Apache Fory™ is an open-source project under the Apache Software Foundation. We welcome all forms of contributions:
-
-### How to Contribute
-
-1. **Report Issues**: Found a bug? [Open an issue](https://github.com/apache/fory/issues)
-2. **Suggest Features**: Have an idea? Start a discussion
-3. **Improve Docs**: Documentation improvements are always welcome
-4. **Submit Code**: See our [Contributing Guide](https://github.com/apache/fory/blob/main/CONTRIBUTING.md)
-
-### Development Setup
-
-```bash
-git clone https://github.com/apache/fory.git
-cd fory/python
-
-# Install dependencies
-pip install -e ".[dev,format]"
-
-# Run tests
-pytest -v -s .
-
-# Run specific test
-pytest -v -s pyfory/tests/test_serializer.py
-
-# Format code
-ruff format .
-ruff check --fix .
-```
-
-## 📄 License
-
-Apache License 2.0. See [LICENSE](https://github.com/apache/fory/blob/main/LICENSE) for details.
-
----
-
-**Apache Fory™** - Blazing fast, secure, and versatile serialization for modern applications.
-
-## 🔗 Links
-
-- **Documentation**: https://fory.apache.org/docs/latest/python_guide/
-- **GitHub**: https://github.com/apache/fory
-- **PyPI**: https://pypi.org/project/pyfory/
-- **Issues**: https://github.com/apache/fory/issues
-- **Slack**: https://join.slack.com/t/fory-project/shared_invite/zt-36g0qouzm-kcQSvV_dtfbtBKHRwT5gsw
-- **Benchmarks**: https://fory.apache.org/docs/latest/benchmarks/
-
-## 🌟 Community
-
-We welcome contributions! Whether it's bug reports, feature requests, documentation improvements, or code contributions, we appreciate your help.
-
-- Star the project on [GitHub](https://github.com/apache/fory) ⭐
-- Join our [Slack community](https://join.slack.com/t/fory-project/shared_invite/zt-36g0qouzm-kcQSvV_dtfbtBKHRwT5gsw) 💬
-- Follow us on [X/Twitter](https://x.com/ApacheFory) 🐦
diff --git a/docs/docs/guide/row_format_guide.md b/docs/docs/guide/row_format_guide.md
deleted file mode 100644
index ec6df6d63..000000000
--- a/docs/docs/guide/row_format_guide.md
+++ /dev/null
@@ -1,205 +0,0 @@
----
-title: Row Format Guide
-sidebar_position: 5
-id: row_format
-license: |
- Licensed to the Apache Software Foundation (ASF) under one or more
- contributor license agreements. See the NOTICE file distributed with
- this work for additional information regarding copyright ownership.
- The ASF licenses this file to You under the Apache License, Version 2.0
- (the "License"); you may not use this file except in compliance with
- the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
----
-
-## Java
-
-```java
-public class Bar {
- String f1;
- List<Long> f2;
-}
-
-public class Foo {
- int f1;
- List<Integer> f2;
- Map<String, Integer> f3;
- List<Bar> f4;
-}
-
-RowEncoder<Foo> encoder = Encoders.bean(Foo.class);
-Foo foo = new Foo();
-foo.f1 = 10;
-foo.f2 = IntStream.range(0, 1000000).boxed().collect(Collectors.toList());
-foo.f3 = IntStream.range(0, 1000000).boxed().collect(Collectors.toMap(i -> "k"+i, i->i));
-List<Bar> bars = new ArrayList<>(1000000);
-for (int i = 0; i < 1000000; i++) {
- Bar bar = new Bar();
- bar.f1 = "s"+i;
- bar.f2 = LongStream.range(0, 10).boxed().collect(Collectors.toList());
- bars.add(bar);
-}
-foo.f4 = bars;
-// Can be zero-copy read by python
-BinaryRow binaryRow = encoder.toRow(foo);
-// can be data from python
-Foo newFoo = encoder.fromRow(binaryRow);
-// zero-copy read List<Integer> f2
-BinaryArray binaryArray2 = binaryRow.getArray(1);
-// zero-copy read List<Bar> f4
-BinaryArray binaryArray4 = binaryRow.getArray(3);
-// zero-copy read 11th element of `readList<Bar> f4`
-BinaryRow barStruct = binaryArray4.getStruct(10);
-
-// zero-copy read 6th of f2 of 11th element of `readList<Bar> f4`
-barStruct.getArray(1).getInt64(5);
-RowEncoder<Bar> barEncoder = Encoders.bean(Bar.class);
-// deserialize part of data.
-Bar newBar = barEncoder.fromRow(barStruct);
-Bar newBar2 = barEncoder.fromRow(binaryArray4.getStruct(20));
-```
-
-## Python
-
-```python
-@dataclass
-class Bar:
- f1: str
- f2: List[pa.int64]
-@dataclass
-class Foo:
- f1: pa.int32
- f2: List[pa.int32]
- f3: Dict[str, pa.int32]
- f4: List[Bar]
-
-encoder = pyfory.encoder(Foo)
-foo = Foo(f1=10, f2=list(range(1000_000)),
- f3={f"k{i}": i for i in range(1000_000)},
- f4=[Bar(f1=f"s{i}", f2=list(range(10))) for i in range(1000_000)])
-binary: bytes = encoder.to_row(foo).to_bytes()
-print(f"start: {datetime.datetime.now()}")
-foo_row = pyfory.RowData(encoder.schema, binary)
-print(foo_row.f2[100000], foo_row.f4[100000].f1, foo_row.f4[200000].f2[5])
-print(f"end: {datetime.datetime.now()}")
-
-binary = pickle.dumps(foo)
-print(f"pickle start: {datetime.datetime.now()}")
-new_foo = pickle.loads(binary)
-print(new_foo.f2[100000], new_foo.f4[100000].f1, new_foo.f4[200000].f2[5])
-print(f"pickle end: {datetime.datetime.now()}")
-```
-
-### Apache Arrow Support
-
-Apache Fory™ Row Format also supports automatic conversion from/to Arrow Table/RecordBatch.
-
-Java:
-
-```java
-Schema schema = TypeInference.inferSchema(BeanA.class);
-ArrowWriter arrowWriter = ArrowUtils.createArrowWriter(schema);
-Encoder<BeanA> encoder = Encoders.rowEncoder(BeanA.class);
-for (int i = 0; i < 10; i++) {
- BeanA beanA = BeanA.createBeanA(2);
- arrowWriter.write(encoder.toRow(beanA));
-}
-return arrowWriter.finishAsRecordBatch();
-```
-
-## Support for Interface and Extension Types
-
-Fory now supports row format mapping for Java `interface` types and subclassed (`extends`) types, enabling more dynamic and flexible data schemas.
-
-These enhancements were introduced in [#2243](https://github.com/apache/fory/pull/2243), [#2250](https://github.com/apache/fory/pull/2250), and [#2256](https://github.com/apache/fory/pull/2256).
-
-### Example: Interface Mapping with RowEncoder
-
-```java
-public interface Animal {
- String speak();
-}
-
-public class Dog implements Animal {
- public String name;
-
- @Override
- public String speak() {
- return "Woof";
- }
-}
-
-// Encode and decode using RowEncoder with interface type
-RowEncoder<Animal> encoder = Encoders.bean(Animal.class);
-Dog dog = new Dog();
-dog.name = "Bingo";
-BinaryRow row = encoder.toRow(dog);
-Animal decoded = encoder.fromRow(row);
-System.out.println(decoded.speak()); // Woof
-
-```
-
-### Example: Extension Type with RowEncoder
-
-```java
-public class Parent {
- public String parentField;
-}
-
-public class Child extends Parent {
- public String childField;
-}
-
-// Encode and decode using RowEncoder with parent class type
-RowEncoder<Parent> encoder = Encoders.bean(Parent.class);
-Child child = new Child();
-child.parentField = "Hello";
-child.childField = "World";
-BinaryRow row = encoder.toRow(child);
-Parent decoded = encoder.fromRow(row);
-
-```
-
-Python:
-
-```python
-import pyfory
-encoder = pyfory.encoder(Foo)
-encoder.to_arrow_record_batch([foo] * 10000)
-encoder.to_arrow_table([foo] * 10000)
-```
-
-C++
-
-```c++
-std::shared_ptr<ArrowWriter> arrow_writer;
-EXPECT_TRUE(
- ArrowWriter::Make(schema, ::arrow::default_memory_pool(), &arrow_writer)
- .ok());
-for (auto &row : rows) {
- EXPECT_TRUE(arrow_writer->Write(row).ok());
-}
-std::shared_ptr<::arrow::RecordBatch> record_batch;
-EXPECT_TRUE(arrow_writer->Finish(&record_batch).ok());
-EXPECT_TRUE(record_batch->Validate().ok());
-EXPECT_EQ(record_batch->num_columns(), schema->num_fields());
-EXPECT_EQ(record_batch->num_rows(), row_nums);
-```
-
-```java
-Schema schema = TypeInference.inferSchema(BeanA.class);
-ArrowWriter arrowWriter = ArrowUtils.createArrowWriter(schema);
-Encoder<BeanA> encoder = Encoders.rowEncoder(BeanA.class);
-for (int i = 0; i < 10; i++) {
- BeanA beanA = BeanA.createBeanA(2);
- arrowWriter.write(encoder.toRow(beanA));
-}
-return arrowWriter.finishAsRecordBatch();
-```
diff --git a/docs/docs/guide/rust_guide.md b/docs/docs/guide/rust_guide.md
deleted file mode 100644
index 5eed8c467..000000000
--- a/docs/docs/guide/rust_guide.md
+++ /dev/null
@@ -1,729 +0,0 @@
----
-title: Rust Serialization
-sidebar_position: 2
-id: rust_serialization
-license: |
- Licensed to the Apache Software Foundation (ASF) under one or more
- contributor license agreements. See the NOTICE file distributed with
- this work for additional information regarding copyright ownership.
- The ASF licenses this file to You under the Apache License, Version 2.0
- (the "License"); you may not use this file except in compliance with
- the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
----
-# Apache Fory™ Rust
-
-[](https://crates.io/crates/fory)
-[](https://docs.rs/fory)
-[](https://github.com/apache/fory/blob/main/LICENSE)
-
-**Apache Fory™** is a blazing fast multi-language serialization framework powered by **JIT compilation** and **zero-copy** techniques, providing up to **ultra-fast performance** while maintaining ease of use and safety.
-
-The Rust implementation provides versatile and high-performance serialization with automatic memory management and compile-time type safety.
-
-## 🚀 Why Apache Fory™ Rust?
-
-- **🔥 Blazingly Fast**: Zero-copy deserialization and optimized binary protocols
-- **🌍 Cross-Language**: Seamlessly serialize/deserialize data across Java, Python, C++, Go, JavaScript, and Rust
-- **🎯 Type-Safe**: Compile-time type checking with derive macros
-- **🔄 Circular References**: Automatic tracking of shared and circular references with `Rc`/`Arc` and weak pointers
-- **🧬 Polymorphic**: Serialize trait objects with `Box<dyn Trait>`, `Rc<dyn Trait>`, and `Arc<dyn Trait>`
-- **📦 Schema Evolution**: Compatible mode for independent schema changes
-- **⚡ Two Modes**: Object graph serialization and zero-copy row-based format
-
-## 📦 Crates
-
-| Crate | Description | Version |
-| --------------------------------------------------------------------------- | --------------------------------- | ----------------------------------------------------------------------------------------------------- |
-| [`fory`](https://github.com/apache/fory/blob/main/rust/fory) | High-level API with derive macros | [](https://crates.io/crates/fory) |
-| [`fory-core`](https://github.com/apache/fory/blob/main/rust/fory-core/) | Core serialization engine | [](https://crates.io/crates/fory-core) |
-| [`fory-derive`](https://github.com/apache/fory/blob/main/rust/fory-derive/) | Procedural macros | [](https://crates.io/crates/fory-derive) |
-
-## 🏃 Quick Start
-
-Add Apache Fory™ to your `Cargo.toml`:
-
-```toml
-[dependencies]
-fory = "0.13"
-```
-
-### Basic Example
-
-```rust
-use fory::{Fory, Error, Reader};
-use fory::ForyObject;
-
-#[derive(ForyObject, Debug, PartialEq)]
-struct User {
- name: String,
- age: i32,
- email: String,
-}
-
-fn main() -> Result<(), Error> {
- let mut fory = Fory::default();
- fory.register::<User>(1)?;
-
- let user = User {
- name: "Alice".to_string(),
- age: 30,
- email: "alice@example.com".to_string(),
- };
-
- // Serialize
- let bytes = fory.serialize(&user)?;
- // Deserialize
- let decoded: User = fory.deserialize(&bytes)?;
- assert_eq!(user, decoded);
-
- // Serialize to specified buffer
- let mut buf: Vec<u8> = vec![];
- fory.serialize_to(&user, &mut buf)?;
- // Deserialize from specified buffer
- let mut reader = Reader::new(&buf);
- let decoded: User = fory.deserialize_from(&mut reader)?;
- assert_eq!(user, decoded);
- Ok(())
-}
-```
-
-## 📚 Core Features
-
-### 1. Object Graph Serialization
-
-Apache Fory™ provides automatic serialization of complex object graphs, preserving the structure and relationships between objects. The `#[derive(ForyObject)]` macro generates efficient serialization code at compile time, eliminating runtime overhead.
-
-**Key capabilities:**
-
-- Nested struct serialization with arbitrary depth
-- Collection types (Vec, HashMap, HashSet, BTreeMap)
-- Optional fields with `Option<T>`
-- Automatic handling of primitive types and strings
-- Efficient binary encoding with variable-length integers
-
-```rust
-use fory::{Fory, Error};
-use fory::ForyObject;
-use std::collections::HashMap;
-
-#[derive(ForyObject, Debug, PartialEq)]
-struct Person {
- name: String,
- age: i32,
- address: Address,
- hobbies: Vec<String>,
- metadata: HashMap<String, String>,
-}
-
-#[derive(ForyObject, Debug, PartialEq)]
-struct Address {
- street: String,
- city: String,
- country: String,
-}
-
-let mut fory = Fory::default();
-fory.register::<Address>(100);
-fory.register::<Person>(200);
-
-let person = Person {
- name: "John Doe".to_string(),
- age: 30,
- address: Address {
- street: "123 Main St".to_string(),
- city: "New York".to_string(),
- country: "USA".to_string(),
- },
- hobbies: vec!["reading".to_string(), "coding".to_string()],
- metadata: HashMap::from([
- ("role".to_string(), "developer".to_string()),
- ]),
-};
-
-let bytes = fory.serialize(&person);
-let decoded: Person = fory.deserialize(&bytes)?;
-assert_eq!(person, decoded);
-```
-
-### 2. Shared and Circular References
-
-Apache Fory™ automatically tracks and preserves reference identity for shared objects using `Rc<T>` and `Arc<T>`. When the same object is referenced multiple times, Fory serializes it only once and uses reference IDs for subsequent occurrences. This ensures:
-
-- **Space efficiency**: No data duplication in serialized output
-- **Reference identity preservation**: Deserialized objects maintain the same sharing relationships
-- **Circular reference support**: Use `RcWeak<T>` and `ArcWeak<T>` to break cycles
-
-#### Shared References with Rc/Arc
-
-```rust
-use fory::Fory;
-use std::rc::Rc;
-
-let fory = Fory::default();
-
-// Create a shared value
-let shared = Rc::new(String::from("shared_value"));
-
-// Reference it multiple times
-let data = vec![shared.clone(), shared.clone(), shared.clone()];
-
-// The shared value is serialized only once
-let bytes = fory.serialize(&data);
-let decoded: Vec<Rc<String>> = fory.deserialize(&bytes)?;
-
-// Verify reference identity is preserved
-assert_eq!(decoded.len(), 3);
-assert_eq!(*decoded[0], "shared_value");
-
-// All three Rc pointers point to the same object
-assert!(Rc::ptr_eq(&decoded[0], &decoded[1]));
-assert!(Rc::ptr_eq(&decoded[1], &decoded[2]));
-```
-
-For thread-safe shared references, use `Arc<T>`.
-
-#### Circular References with Weak Pointers
-
-To serialize circular references like parent-child relationships or doubly-linked structures, use `RcWeak<T>` or `ArcWeak<T>` to break the cycle. These weak pointers are serialized as references to their strong counterparts, preserving the graph structure without causing memory leaks or infinite recursion.
-
-**How it works:**
-
-- Weak pointers serialize as references to their target objects
-- If the strong pointer has been dropped, weak serializes as `Null`
-- Forward references (weak appearing before target) are resolved via callbacks
-- All clones of a weak pointer share the same internal cell for automatic updates
-
-```rust
-use fory::{Fory, Error};
-use fory::ForyObject;
-use fory::RcWeak;
-use std::rc::Rc;
-use std::cell::RefCell;
-
-#[derive(ForyObject, Debug)]
-struct Node {
- value: i32,
- parent: RcWeak<RefCell<Node>>,
- children: Vec<Rc<RefCell<Node>>>,
-}
-
-let mut fory = Fory::default();
-fory.register::<Node>(2000);
-
-// Build a parent-child tree
-let parent = Rc::new(RefCell::new(Node {
- value: 1,
- parent: RcWeak::new(),
- children: vec![],
-}));
-
-let child1 = Rc::new(RefCell::new(Node {
- value: 2,
- parent: RcWeak::from(&parent),
- children: vec![],
-}));
-
-let child2 = Rc::new(RefCell::new(Node {
- value: 3,
- parent: RcWeak::from(&parent),
- children: vec![],
-}));
-
-parent.borrow_mut().children.push(child1.clone());
-parent.borrow_mut().children.push(child2.clone());
-
-// Serialize and deserialize the circular structure
-let bytes = fory.serialize(&parent);
-let decoded: Rc<RefCell<Node>> = fory.deserialize(&bytes)?;
-
-// Verify the circular relationship
-assert_eq!(decoded.borrow().children.len(), 2);
-for child in &decoded.borrow().children {
- let upgraded_parent = child.borrow().parent.upgrade().unwrap();
- assert!(Rc::ptr_eq(&decoded, &upgraded_parent));
-}
-```
-
-### 3. Trait Object Serialization
-
-Apache Fory™ supports polymorphic serialization through trait objects, enabling dynamic dispatch and type flexibility. This is essential for plugin systems, heterogeneous collections, and extensible architectures.
-
-**Supported trait object types:**
-
-- `Box<dyn Trait>` - Owned trait objects
-- `Rc<dyn Trait>` - Reference-counted trait objects
-- `Arc<dyn Trait>` - Thread-safe reference-counted trait objects
-- `Box<dyn Any>`/`Rc<dyn Any>`/`Arc<dyn Any>` - Any trait type objects
-- `Vec<Box<dyn Trait>>`, `HashMap<K, Box<dyn Trait>>` - Collections of trait objects
-
-**Basic Trait Object Serialization Example:**
-
-```rust
-use fory::{Fory, register_trait_type};
-use fory::Serializer;
-use fory::ForyObject;
-
-trait Animal: Serializer {
- fn speak(&self) -> String;
- fn name(&self) -> &str;
-}
-
-#[derive(ForyObject)]
-struct Dog { name: String, breed: String }
-
-impl Animal for Dog {
- fn speak(&self) -> String { "Woof!".to_string() }
- fn name(&self) -> &str { &self.name }
-}
-
-#[derive(ForyObject)]
-struct Cat { name: String, color: String }
-
-impl Animal for Cat {
- fn speak(&self) -> String { "Meow!".to_string() }
- fn name(&self) -> &str { &self.name }
-}
-
-// Register trait implementations
-register_trait_type!(Animal, Dog, Cat);
-
-#[derive(ForyObject)]
-struct Zoo {
- star_animal: Box<dyn Animal>,
-}
-
-let mut fory = Fory::default().compatible(true);
-fory.register::<Dog>(100);
-fory.register::<Cat>(101);
-fory.register::<Zoo>(102);
-
-let zoo = Zoo {
- star_animal: Box::new(Dog {
- name: "Buddy".to_string(),
- breed: "Labrador".to_string(),
- }),
-};
-
-let bytes = fory.serialize(&zoo);
-let decoded: Zoo = fory.deserialize(&bytes)?;
-
-assert_eq!(decoded.star_animal.name(), "Buddy");
-assert_eq!(decoded.star_animal.speak(), "Woof!");
-```
-
-### 4. Schema Evolution
-
-Apache Fory™ supports schema evolution in **Compatible mode**, allowing serialization and deserialization peers to have different type definitions. This enables independent evolution of services in distributed systems without breaking compatibility.
-
-**Features:**
-
-- Add new fields with default values
-- Remove obsolete fields (skipped during deserialization)
-- Change field nullability (`T` ↔ `Option<T>`)
-- Reorder fields (matched by name, not position)
-- Type-safe fallback to default values for missing fields
-
-**Compatibility rules:**
-
-- Field names must match (case-sensitive)
-- Type changes are not supported (except nullable/non-nullable)
-- Nested struct types must be registered on both sides
-
-```rust
-use fory::Fory;
-use fory::ForyObject;
-use std::collections::HashMap;
-
-#[derive(ForyObject, Debug)]
-struct PersonV1 {
- name: String,
- age: i32,
- address: String,
-}
-
-#[derive(ForyObject, Debug)]
-struct PersonV2 {
- name: String,
- age: i32,
- // address removed
- // phone added
- phone: Option<String>,
- metadata: HashMap<String, String>,
-}
-
-let mut fory1 = Fory::default().compatible(true);
-fory1.register::<PersonV1>(1);
-
-let mut fory2 = Fory::default().compatible(true);
-fory2.register::<PersonV2>(1);
-
-let person_v1 = PersonV1 {
- name: "Alice".to_string(),
- age: 30,
- address: "123 Main St".to_string(),
-};
-
-// Serialize with V1
-let bytes = fory1.serialize(&person_v1);
-
-// Deserialize with V2 - missing fields get default values
-let person_v2: PersonV2 = fory2.deserialize(&bytes)?;
-assert_eq!(person_v2.name, "Alice");
-assert_eq!(person_v2.age, 30);
-assert_eq!(person_v2.phone, None);
-```
-
-### 5. Enum Support
-
-Apache Fory™ supports three types of enum variants with full schema evolution in Compatible mode:
-
-**Variant Types:**
-
-- **Unit**: C-style enums (`Status::Active`)
-- **Unnamed**: Tuple-like variants (`Message::Pair(String, i32)`)
-- **Named**: Struct-like variants (`Event::Click { x: i32, y: i32 }`)
-
-**Features:**
-
-- Efficient varint encoding for variant ordinals
-- Schema evolution support (add/remove variants, add/remove fields)
-- Default variant support with `#[default]`
-- Automatic type mismatch handling
-
-```rust
-use fory::{Fory, ForyObject};
-
-#[derive(Default, ForyObject, Debug, PartialEq)]
-enum Value {
- #[default]
- Null,
- Bool(bool),
- Number(f64),
- Text(String),
- Object { name: String, value: i32 },
-}
-
-let mut fory = Fory::default();
-fory.register::<Value>(1)?;
-
-let value = Value::Object { name: "score".to_string(), value: 100 };
-let bytes = fory.serialize(&value)?;
-let decoded: Value = fory.deserialize(&bytes)?;
-assert_eq!(value, decoded);
-```
-
-**Evolution capabilities:**
-
-- **Unknown variants** → Falls back to default variant
-- **Named variant fields** → Add/remove fields (missing fields use defaults)
-- **Unnamed variant elements** → Add/remove elements (extras skipped, missing use defaults)
-- **Variant type mismatches** → Automatically uses default value for current variant
-
-**Best practices:**
-
-- Always mark a default variant with `#[default]`
-- Named variants provide better evolution than unnamed
-- Use compatible mode for cross-version communication
-
-### 6. Tuple Support
-
-Apache Fory™ supports tuples up to 22 elements out of the box with efficient serialization in both compatible and non-compatible modes.
-
-**Features:**
-
-- Automatic serialization for tuples from 1 to 22 elements
-- Heterogeneous type support (each element can be a different type)
-- Schema evolution in Compatible mode (handles missing/extra elements)
-
-**Serialization modes:**
-
-1. **Non-compatible mode**: Serializes elements sequentially without collection headers for minimal overhead
-2. **Compatible mode**: Uses collection protocol with type metadata for schema evolution
-
-```rust
-use fory::{Fory, Error};
-
-let mut fory = Fory::default();
-
-// Tuple with heterogeneous types
-let data: (i32, String, bool, Vec<i32>) = (
- 42,
- "hello".to_string(),
- true,
- vec![1, 2, 3],
-);
-
-let bytes = fory.serialize(&data)?;
-let decoded: (i32, String, bool, Vec<i32>) = fory.deserialize(&bytes)?;
-assert_eq!(data, decoded);
-```
-
-### 7. Custom Serializers
-
-For types that don't support `#[derive(ForyObject)]`, implement the `Serializer` trait manually. This is useful for:
-
-- External types from other crates
-- Types with special serialization requirements
-- Legacy data format compatibility
-- Performance-critical custom encoding
-
-```rust
-use fory::{Fory, ReadContext, WriteContext, Serializer, ForyDefault, Error};
-use std::any::Any;
-
-#[derive(Debug, PartialEq)]
-struct CustomType {
- value: i32,
- name: String,
-}
-
-impl Serializer for CustomType {
- fn fory_write_data(&self, context: &mut WriteContext, is_field: bool) {
- context.writer.write_i32(self.value);
- context.writer.write_varuint32(self.name.len() as u32);
- context.writer.write_utf8_string(&self.name);
- }
-
- fn fory_read_data(context: &mut ReadContext, is_field: bool) -> Result<Self, Error> {
- let value = context.reader.read_i32();
- let len = context.reader.read_varuint32() as usize;
- let name = context.reader.read_utf8_string(len);
- Ok(Self { value, name })
- }
-
- fn fory_type_id_dyn(&self, type_resolver: &TypeResolver) -> u32 {
- Self::fory_get_type_id(type_resolver)
- }
-
- fn as_any(&self) -> &dyn Any {
- self
- }
-}
-
-impl ForyDefault for CustomType {
- fn fory_default() -> Self {
- Self::default()
- }
-}
-
-let mut fory = Fory::default();
-fory.register_serializer::<CustomType>(100);
-
-let custom = CustomType {
- value: 42,
- name: "test".to_string(),
-};
-let bytes = fory.serialize(&custom);
-let decoded: CustomType = fory.deserialize(&bytes)?;
-assert_eq!(custom, decoded);
-```
-
-### 7. Row-Based Serialization
-
-Apache Fory™ provides a high-performance **row format** for zero-copy deserialization. Unlike traditional object serialization that reconstructs entire objects in memory, row format enables **random access** to fields directly from binary data without full deserialization.
-
-**Key benefits:**
-
-- **Zero-copy access**: Read fields without allocating or copying data
-- **Partial deserialization**: Access only the fields you need
-- **Memory-mapped files**: Work with data larger than RAM
-- **Cache-friendly**: Sequential memory layout for better CPU cache utilization
-- **Lazy evaluation**: Defer expensive operations until field access
-
-**When to use row format:**
-
-- Analytics workloads with selective field access
-- Large datasets where only a subset of fields is needed
-- Memory-constrained environments
-- High-throughput data pipelines
-- Reading from memory-mapped files or shared memory
-
-**How it works:**
-
-- Fields are encoded in a binary row with fixed offsets for primitives
-- Variable-length data (strings, collections) stored with offset pointers
-- Null bitmap tracks which fields are present
-- Nested structures supported through recursive row encoding
-
-```rust
-use fory::{to_row, from_row};
-use fory::ForyRow;
-use std::collections::BTreeMap;
-
-#[derive(ForyRow)]
-struct UserProfile {
- id: i64,
- username: String,
- email: String,
- scores: Vec<i32>,
- preferences: BTreeMap<String, String>,
- is_active: bool,
-}
-
-let profile = UserProfile {
- id: 12345,
- username: "alice".to_string(),
- email: "alice@example.com".to_string(),
- scores: vec![95, 87, 92, 88],
- preferences: BTreeMap::from([
- ("theme".to_string(), "dark".to_string()),
- ("language".to_string(), "en".to_string()),
- ]),
- is_active: true,
-};
-
-// Serialize to row format
-let row_data = to_row(&profile);
-
-// Zero-copy deserialization - no object allocation!
-let row = from_row::<UserProfile>(&row_data);
-
-// Access fields directly from binary data
-assert_eq!(row.id(), 12345);
-assert_eq!(row.username(), "alice");
-assert_eq!(row.email(), "alice@example.com");
-assert_eq!(row.is_active(), true);
-
-// Access collections efficiently
-let scores = row.scores();
-assert_eq!(scores.size(), 4);
-assert_eq!(scores.get(0), 95);
-assert_eq!(scores.get(1), 87);
-
-let prefs = row.preferences();
-assert_eq!(prefs.keys().size(), 2);
-assert_eq!(prefs.keys().get(0), "language");
-assert_eq!(prefs.values().get(0), "en");
-```
-
-**Performance comparison:**
-
-| Operation | Object Format | Row Format |
-| -------------------- | ----------------------------- | ------------------------------- |
-| Full deserialization | Allocates all objects | Zero allocation |
-| Single field access | Full deserialization required | Direct offset read |
-| Memory usage | Full object graph in memory | Only accessed fields in memory |
-| Suitable for | Small objects, full access | Large objects, selective access |
-
-## 🌍 Cross-Language Serialization
-
-Apache Fory™ supports seamless data exchange across multiple languages:
-
-```rust
-use fory::Fory;
-
-// Enable cross-language mode
-let mut fory = Fory::default()
- .compatible(true)
- .xlang(true);
-
-// Register types with consistent IDs across languages
-fory.register::<MyStruct>(100);
-
-// Or use namespace-based registration
-fory.register_by_namespace::<MyStruct>("com.example", "MyStruct");
-```
-
-See [xlang_type_mapping.md](https://fory.apache.org/docs/specification/xlang_type_mapping) for type mapping across languages.
-
-## ⚡ Performance
-
-Apache Fory™ Rust is designed for maximum performance:
-
-- **Zero-Copy Deserialization**: Row format enables direct memory access without copying
-- **Buffer Pre-allocation**: Minimizes memory allocations during serialization
-- **Compact Encoding**: Variable-length encoding for space efficiency
-- **Little-Endian**: Optimized for modern CPU architectures
-- **Reference Deduplication**: Shared objects serialized only once
-
-Run benchmarks:
-
-```bash
-cd benchmarks/rust_benchmark
-cargo bench
-```
-
-## 📖 Documentation
-
-- **[User Guide](https://fory.apache.org/docs/next/docs/guide/rust)** - Comprehensive User documents
-- **[API Documentation](https://docs.rs/fory)** - Complete API reference
-- **[Protocol Specification](https://fory.apache.org/docs/specification/fory_xlang_serialization_spec)** - Serialization protocol details
-- **[Type Mapping](https://fory.apache.org/docs/specification/xlang_type_mapping)** - Cross-language type mappings
-- **[Source](https://github.com/apache/fory/tree/main/docs/guide/rust)** - Source code for doc
-
-## 🎯 Use Cases
-
-### Object Serialization
-
-- Complex data structures with nested objects and references
-- Cross-language communication in microservices
-- General-purpose serialization with full type safety
-- Schema evolution with compatible mode
-- Graph-like data structures with circular references
-
-### Row-Based Serialization
-
-- High-throughput data processing
-- Analytics workloads requiring fast field access
-- Memory-constrained environments
-- Real-time data streaming applications
-- Zero-copy scenarios
-
-## 🛠️ Development
-
-### Building
-
-```bash
-cd rust
-cargo build
-```
-
-### Testing
-
-```bash
-# Run all tests
-cargo test --features tests
-
-# Run specific test
-cargo test -p tests --test test_complex_struct
-```
-
-### Code Quality
-
-```bash
-# Format code
-cargo fmt
-
-# Check formatting
-cargo fmt --check
-
-# Run linter
-cargo clippy --all-targets --all-features -- -D warnings
-```
-
-## 📄 License
-
-Licensed under the Apache License, Version 2.0. See [LICENSE](https://github.com/apache/fory/blob/main/LICENSE) for details.
-
-## 🤝 Contributing
-
-We welcome contributions! Please see our [Contributing Guide](https://github.com/apache/fory/blob/main/CONTRIBUTING.md) for details.
-
-## 📞 Support
-
-- **Documentation**: [docs.rs/fory](https://docs.rs/fory)
-- **Issues**: [GitHub Issues](https://github.com/apache/fory/issues)
-- **Discussions**: [GitHub Discussions](https://github.com/apache/fory/discussions)
-- **Slack**: [Apache Fory Slack](https://join.slack.com/t/fory-project/shared_invite/zt-1u8soj4qc-ieYEu7ciHOqA2mo47llS8A)
-
----
-
-**Apache Fory™** - Blazingly fast multi-language serialization framework.
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org
For additional commands, e-mail: commits-help@fory.apache.org
(fory-site) 02/02: refine docs
Posted by ch...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
chaokunyang pushed a commit to branch update_guide_structure
in repository https://gitbox.apache.org/repos/asf/fory-site.git
commit ba6426c57238786bf00564a277e638dc37397c96
Author: chaokunyang <sh...@gmail.com>
AuthorDate: Wed Dec 10 22:53:24 2025 +0800
refine docs
---
docs/docs/guide/cpp/_category_.json | 4 +
docs/docs/guide/java/_category_.json | 4 +
docs/docs/guide/python/_category_.json | 4 +
docs/docs/guide/rust/_category_.json | 4 +
docs/docs/guide/scala/_category_.json | 4 +
docs/docs/guide/scala_guide.md | 313 --------------
docs/docs/guide/xlang/_category_.json | 4 +
docs/docs/guide/xlang_serialization_guide.md | 610 ---------------------------
8 files changed, 24 insertions(+), 923 deletions(-)
diff --git a/docs/docs/guide/cpp/_category_.json b/docs/docs/guide/cpp/_category_.json
new file mode 100644
index 000000000..1832d9760
--- /dev/null
+++ b/docs/docs/guide/cpp/_category_.json
@@ -0,0 +1,4 @@
+{
+ "label": "C++",
+ "collapsed": true
+}
\ No newline at end of file
diff --git a/docs/docs/guide/java/_category_.json b/docs/docs/guide/java/_category_.json
new file mode 100644
index 000000000..77437fd8b
--- /dev/null
+++ b/docs/docs/guide/java/_category_.json
@@ -0,0 +1,4 @@
+{
+ "label": "Java",
+ "collapsed": true
+}
diff --git a/docs/docs/guide/python/_category_.json b/docs/docs/guide/python/_category_.json
new file mode 100644
index 000000000..d07a16f69
--- /dev/null
+++ b/docs/docs/guide/python/_category_.json
@@ -0,0 +1,4 @@
+{
+ "label": "Python",
+ "collapsed": true
+}
diff --git a/docs/docs/guide/rust/_category_.json b/docs/docs/guide/rust/_category_.json
new file mode 100644
index 000000000..ad9884ddf
--- /dev/null
+++ b/docs/docs/guide/rust/_category_.json
@@ -0,0 +1,4 @@
+{
+ "label": "Rust",
+ "collapsed": true
+}
diff --git a/docs/docs/guide/scala/_category_.json b/docs/docs/guide/scala/_category_.json
new file mode 100644
index 000000000..7ff63f8a8
--- /dev/null
+++ b/docs/docs/guide/scala/_category_.json
@@ -0,0 +1,4 @@
+{
+ "label": "Scala",
+ "collapsed": true
+}
\ No newline at end of file
diff --git a/docs/docs/guide/scala_guide.md b/docs/docs/guide/scala_guide.md
deleted file mode 100644
index f2d42e0dd..000000000
--- a/docs/docs/guide/scala_guide.md
+++ /dev/null
@@ -1,313 +0,0 @@
----
-title: Scala Serialization Guide
-sidebar_position: 3
-id: scala_serialization
-license: |
- Licensed to the Apache Software Foundation (ASF) under one or more
- contributor license agreements. See the NOTICE file distributed with
- this work for additional information regarding copyright ownership.
- The ASF licenses this file to You under the Apache License, Version 2.0
- (the "License"); you may not use this file except in compliance with
- the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
----
-
-Apache Fory™ supports all scala object serialization:
-
-- `case` class serialization supported
-- `pojo/bean` class serialization supported
-- `object` singleton serialization supported
-- `collection` serialization supported
-- other types such as `tuple/either` and basic types are all supported too.
-
-Scala 2 and 3 are both supported.
-
-## Install
-
-To add a dependency on Apache Fory™ scala for with sbt, use the following:
-
-```sbt
-libraryDependencies += "org.apache.fory" %% "fory-scala" % "0.13.2"
-```
-
-## Quick Start
-
-```scala
-case class Person(name: String, id: Long, github: String)
-case class Point(x : Int, y : Int, z : Int)
-
-object ScalaExample {
- val fory: Fory = Fory.builder().withScalaOptimizationEnabled(true).build()
- // Register optimized fory serializers for scala
- ScalaSerializers.registerSerializers(fory)
- fory.register(classOf[Person])
- fory.register(classOf[Point])
-
- def main(args: Array[String]): Unit = {
- val p = Person("Shawn Yang", 1, "https://github.com/chaokunyang")
- println(fory.deserialize(fory.serialize(p)))
- println(fory.deserialize(fory.serialize(Point(1, 2, 3))))
- }
-}
-```
-
-## Fory creation
-
-When using fory for scala serialization, you should create fory at least with following options:
-
-```scala
-import org.apache.fory.Fory
-import org.apache.fory.serializer.scala.ScalaSerializers
-
-val fory = Fory.builder().withScalaOptimizationEnabled(true).build()
-
-// Register optimized fory serializers for scala
-ScalaSerializers.registerSerializers(fory)
-```
-
-Depending on the object types you serialize, you may need to register some scala internal types:
-
-```scala
-fory.register(Class.forName("scala.Enumeration.Val"))
-```
-
-If you want to avoid such registration, you can disable class registration by `ForyBuilder#requireClassRegistration(false)`.
-Note that this option allow to deserialize objects unknown types, more flexible but may be insecure if the classes contains malicious code.
-
-And circular references are common in scala, `Reference tracking` should be enabled by `ForyBuilder#withRefTracking(true)`. If you don't enable reference tracking, [StackOverflowError](https://github.com/apache/fory/issues/1032) may happen for some scala versions when serializing scala Enumeration.
-
-Note that fory instance should be shared between multiple serialization, the creation of fory instance is not cheap.
-
-If you use shared fory instance across multiple threads, you should create `ThreadSafeFory` instead by `ForyBuilder#buildThreadSafeFory()` instead.
-
-## Serialize case class
-
-```scala
-case class Person(github: String, age: Int, id: Long)
-val p = Person("https://github.com/chaokunyang", 18, 1)
-println(fory.deserialize(fory.serialize(p)))
-println(fory.deserializeJavaObject(fory.serializeJavaObject(p)))
-```
-
-## Serialize pojo
-
-```scala
-class Foo(f1: Int, f2: String) {
- override def toString: String = s"Foo($f1, $f2)"
-}
-println(fory.deserialize(fory.serialize(Foo(1, "chaokunyang"))))
-```
-
-## Serialize object singleton
-
-```scala
-object singleton {
-}
-val o1 = fory.deserialize(fory.serialize(singleton))
-val o2 = fory.deserialize(fory.serialize(singleton))
-println(o1 == o2)
-```
-
-## Serialize collection
-
-```scala
-val seq = Seq(1,2)
-val list = List("a", "b")
-val map = Map("a" -> 1, "b" -> 2)
-println(fory.deserialize(fory.serialize(seq)))
-println(fory.deserialize(fory.serialize(list)))
-println(fory.deserialize(fory.serialize(map)))
-```
-
-## Serialize Tuple
-
-```scala
-val tuple = (100, 10000L) //Tuple2
-println(fory.deserialize(fory.serialize(tuple)))
-val tuple = (100, 10000L, 10000L, "str") //Tuple4
-println(fory.deserialize(fory.serialize(tuple)))
-```
-
-## Serialize Enum
-
-### Scala3 Enum
-
-```scala
-enum Color { case Red, Green, Blue }
-println(fory.deserialize(fory.serialize(Color.Green)))
-```
-
-### Scala2 Enum
-
-```scala
-object ColorEnum extends Enumeration {
- type ColorEnum = Value
- val Red, Green, Blue = Value
-}
-println(fory.deserialize(fory.serialize(ColorEnum.Green)))
-```
-
-## Serialize Option
-
-```scala
-val opt: Option[Long] = Some(100)
-println(fory.deserialize(fory.serialize(opt)))
-val opt1: Option[Long] = None
-println(fory.deserialize(fory.serialize(opt1)))
-```
-
-## Scala Class Default Values Support
-
-Fory supports Scala class default values during deserialization when using compatible mode. This feature enables forward/backward compatibility when case classes or regular Scala classes have default parameters.
-
-### Overview
-
-When a Scala class has default parameters, the Scala compiler generates methods in the companion object (for case classes) or in the class itself (for regular Scala classes) like `apply$default$1`, `apply$default$2`, etc. that return the default values. Fory can detect these methods and use them when deserializing objects where certain fields are missing from the serialized data.
-
-### Supported Class Types
-
-Fory supports default values for:
-
-- **Case classes** with default parameters
-- **Regular Scala classes** with default parameters in their primary constructor
-- **Nested case classes** with default parameters
-- **Deeply nested case classes** with default parameters
-
-### How It Works
-
-1. **Detection**: Fory detects if a class is a Scala class by checking for the presence of default value methods (`apply$default$N` or `$default$N`).
-
-2. **Default Value Discovery**:
- - For case classes: Fory scans the companion object for methods named `apply$default$1`, `apply$default$2`, etc.
- - For regular Scala classes: Fory scans the class itself for methods named `$default$1`, `$default$2`, etc.
-
-3. **Field Mapping**: During deserialization, Fory identifies fields that exist in the target class but are missing from the serialized data.
-
-4. **Value Application**: After reading all available fields from the serialized data, Fory applies default values to any missing fields using direct field access for optimal performance.
-
-### Usage
-
-This feature is automatically enabled when:
-
-- Compatible mode is enabled (`withCompatibleMode(CompatibleMode.COMPATIBLE)`)
-- The target class is detected as a Scala class with default values
-- A field is missing from the serialized data but exists in the target class
-
-No additional configuration is required.
-
-### Examples
-
-#### Case Class with Default Values
-
-```scala
-// Class WITHOUT default values (for serialization)
-case class PersonNoDefaults(name: String)
-
-// Class WITH default values (for deserialization)
-case class PersonWithDefaults(name: String, age: Int = 25, city: String = "Unknown")
-
-val fory = Fory.builder()
- .withCompatibleMode(CompatibleMode.COMPATIBLE)
- .withScalaOptimizationEnabled(true)
- .build()
-
-// Serialize using class without default values
-val original = PersonNoDefaults("John")
-val serialized = fory.serialize(original)
-
-// Deserialize into class with default values - missing fields will use defaults
-val deserialized = fory.deserialize(serialized, classOf[PersonWithDefaults])
-// deserialized.name will be "John"
-// deserialized.age will be 25 (default)
-// deserialized.city will be "Unknown" (default)
-```
-
-#### Regular Scala Class with Default Values
-
-```scala
-// Class WITHOUT default values (for serialization)
-class EmployeeNoDefaults(val name: String)
-
-// Class WITH default values (for deserialization)
-class EmployeeWithDefaults(val name: String, val age: Int = 30, val department: String = "Engineering")
-
-val fory = Fory.builder()
- .withCompatibleMode(CompatibleMode.COMPATIBLE)
- .withScalaOptimizationEnabled(true)
- .build()
-
-// Serialize using class without default values
-val original = new EmployeeNoDefaults("Jane")
-val serialized = fory.serialize(original)
-
-// Deserialize into class with default values - missing fields will use defaults
-val deserialized = fory.deserialize(serialized, classOf[EmployeeWithDefaults])
-// deserialized.name will be "Jane"
-// deserialized.age will be 30 (default)
-// deserialized.department will be "Engineering" (default)
-```
-
-#### Complex Default Values
-
-```scala
-// Class WITHOUT default values (for serialization)
-case class ConfigurationNoDefaults(name: String)
-
-// Class WITH default values (for deserialization)
-case class ConfigurationWithDefaults(
- name: String,
- settings: Map[String, String] = Map("default" -> "value"),
- tags: List[String] = List("default"),
- enabled: Boolean = true
-)
-
-val fory = Fory.builder()
- .withCompatibleMode(CompatibleMode.COMPATIBLE)
- .withScalaOptimizationEnabled(true)
- .build()
-
-// Serialize using class without default values
-val original = ConfigurationNoDefaults("myConfig")
-val serialized = fory.serialize(original)
-
-// Deserialize into class with default values - missing fields will use defaults
-val deserialized = fory.deserialize(serialized, classOf[ConfigurationWithDefaults])
-// deserialized.name will be "myConfig"
-// deserialized.settings will be Map("default" -> "value")
-// deserialized.tags will be List("default")
-// deserialized.enabled will be true
-```
-
-#### Nested Case Classes
-
-```scala
-object NestedClasses {
- // Class WITHOUT default values (for serialization)
- case class SimplePerson(name: String)
-
- // Class WITH default values (for deserialization)
- case class Address(street: String, city: String = "DefaultCity")
- case class PersonWithDefaults(name: String, address: Address = Address("DefaultStreet"))
-}
-
-val fory = Fory.builder()
- .withCompatibleMode(CompatibleMode.COMPATIBLE)
- .withScalaOptimizationEnabled(true)
- .build()
-
-// Serialize using class without default values
-val original = NestedClasses.SimplePerson("Alice")
-val serialized = fory.serialize(original)
-
-// Deserialize into class with default values - missing address field will use default
-val deserialized = fory.deserialize(serialized, classOf[NestedClasses.PersonWithDefaults])
-// deserialized.name will be "Alice"
-// deserialized.address will be Address("DefaultStreet", "DefaultCity")
-```
diff --git a/docs/docs/guide/xlang/_category_.json b/docs/docs/guide/xlang/_category_.json
new file mode 100644
index 000000000..a7b9da285
--- /dev/null
+++ b/docs/docs/guide/xlang/_category_.json
@@ -0,0 +1,4 @@
+{
+ "label": "Cross Language",
+ "collapsed": true
+}
\ No newline at end of file
diff --git a/docs/docs/guide/xlang_serialization_guide.md b/docs/docs/guide/xlang_serialization_guide.md
deleted file mode 100644
index a5715a60a..000000000
--- a/docs/docs/guide/xlang_serialization_guide.md
+++ /dev/null
@@ -1,610 +0,0 @@
----
-title: Xlang Serialization Guide
-sidebar_position: 4
-id: xlang_serialization
-license: |
- Licensed to the Apache Software Foundation (ASF) under one or more
- contributor license agreements. See the NOTICE file distributed with
- this work for additional information regarding copyright ownership.
- The ASF licenses this file to You under the Apache License, Version 2.0
- (the "License"); you may not use this file except in compliance with
- the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
----
-
-## Serialize built-in types
-
-Common types can be serialized automatically: primitive numeric types, string, binary, array, list, map and so on.
-
-**Java**
-
-```java
-import org.apache.fory.*;
-import org.apache.fory.config.*;
-
-import java.util.*;
-
-public class Example1 {
- public static void main(String[] args) {
- Fory fory = Fory.builder().withLanguage(Language.XLANG).build();
- List<Object> list = ofArrayList(true, false, "str", -1.1, 1, new int[100], new double[20]);
- byte[] bytes = fory.serialize(list);
- // bytes can be data serialized by other languages.
- fory.deserialize(bytes);
- Map<Object, Object> map = new HashMap<>();
- map.put("k1", "v1");
- map.put("k2", list);
- map.put("k3", -1);
- bytes = fory.serialize(map);
- // bytes can be data serialized by other languages.
- fory.deserialize(bytes);
- }
-}
-```
-
-**Python**
-
-```python
-import pyfory
-import numpy as np
-
-fory = pyfory.Fory()
-object_list = [True, False, "str", -1.1, 1,
- np.full(100, 0, dtype=np.int32), np.full(20, 0.0, dtype=np.double)]
-data = fory.serialize(object_list)
-# bytes can be data serialized by other languages.
-new_list = fory.deserialize(data)
-object_map = {"k1": "v1", "k2": object_list, "k3": -1}
-data = fory.serialize(object_map)
-# bytes can be data serialized by other languages.
-new_map = fory.deserialize(data)
-print(new_map)
-```
-
-**Golang**
-
-```go
-package main
-
-import forygo "github.com/apache/fory/fory/go/fory"
-import "fmt"
-
-func main() {
- list := []interface{}{true, false, "str", -1.1, 1, make([]int32, 10), make([]float64, 20)}
- fory := forygo.NewFory()
- bytes, err := fory.Marshal(list)
- if err != nil {
- panic(err)
- }
- var newValue interface{}
- // bytes can be data serialized by other languages.
- if err := fory.Unmarshal(bytes, &newValue); err != nil {
- panic(err)
- }
- fmt.Println(newValue)
- dict := map[string]interface{}{
- "k1": "v1",
- "k2": list,
- "k3": -1,
- }
- bytes, err = fory.Marshal(dict)
- if err != nil {
- panic(err)
- }
- // bytes can be data serialized by other languages.
- if err := fory.Unmarshal(bytes, &newValue); err != nil {
- panic(err)
- }
- fmt.Println(newValue)
-}
-```
-
-**JavaScript**
-
-```javascript
-import Fory from "@apache-fory/fory";
-
-/**
- * @apache-fory/hps use v8's fast-calls-api that can be called directly by jit, ensure that the version of Node is 20 or above.
- * Experimental feature, installation success cannot be guaranteed at this moment
- * If you are unable to install the module, replace it with `const hps = null;`
- **/
-import hps from "@apache-fory/hps";
-
-const fory = new Fory({ hps });
-const input = fory.serialize("hello fory");
-const result = fory.deserialize(input);
-console.log(result);
-```
-
-**Rust**
-
-```rust
-use chrono::{NaiveDate, NaiveDateTime};
-use fory::{from_buffer, to_buffer, Fory};
-use std::collections::HashMap;
-
-fn run() {
- let bin: Vec<u8> = to_buffer(&"hello".to_string());
- let obj: String = from_buffer(&bin).expect("should success");
- assert_eq!("hello".to_string(), obj);
-}
-```
-
-## Serialize custom types
-
-Serializing user-defined types needs registering the custom type using the register API to establish the mapping relationship between the type in different languages.
-
-**Java**
-
-```java
-import org.apache.fory.*;
-import org.apache.fory.config.*;
-import java.util.*;
-
-public class Example2 {
- public static class SomeClass1 {
- Object f1;
- Map<Byte, Integer> f2;
- }
-
- public static class SomeClass2 {
- Object f1;
- String f2;
- List<Object> f3;
- Map<Byte, Integer> f4;
- Byte f5;
- Short f6;
- Integer f7;
- Long f8;
- Float f9;
- Double f10;
- short[] f11;
- List<Short> f12;
- }
-
- public static Object createObject() {
- SomeClass1 obj1 = new SomeClass1();
- obj1.f1 = true;
- obj1.f2 = ofHashMap((byte) -1, 2);
- SomeClass2 obj = new SomeClass2();
- obj.f1 = obj1;
- obj.f2 = "abc";
- obj.f3 = ofArrayList("abc", "abc");
- obj.f4 = ofHashMap((byte) 1, 2);
- obj.f5 = Byte.MAX_VALUE;
- obj.f6 = Short.MAX_VALUE;
- obj.f7 = Integer.MAX_VALUE;
- obj.f8 = Long.MAX_VALUE;
- obj.f9 = 1.0f / 2;
- obj.f10 = 1 / 3.0;
- obj.f11 = new short[]{(short) 1, (short) 2};
- obj.f12 = ofArrayList((short) -1, (short) 4);
- return obj;
- }
-
- // mvn exec:java -Dexec.mainClass="org.apache.fory.examples.Example2"
- public static void main(String[] args) {
- Fory fory = Fory.builder().withLanguage(Language.XLANG).build();
- fory.register(SomeClass1.class, "example.SomeClass1");
- fory.register(SomeClass2.class, "example.SomeClass2");
- byte[] bytes = fory.serialize(createObject());
- // bytes can be data serialized by other languages.
- System.out.println(fory.deserialize(bytes));
- }
-}
-```
-
-**Python**
-
-```python
-from dataclasses import dataclass
-from typing import List, Dict, Any
-import pyfory, array
-
-
-@dataclass
-class SomeClass1:
- f1: Any
- f2: Dict[pyfory.Int8Type, pyfory.Int32Type]
-
-
-@dataclass
-class SomeClass2:
- f1: Any = None
- f2: str = None
- f3: List[str] = None
- f4: Dict[pyfory.Int8Type, pyfory.Int32Type] = None
- f5: pyfory.Int8Type = None
- f6: pyfory.Int16Type = None
- f7: pyfory.Int32Type = None
- # int type will be taken as `pyfory.Int64Type`.
- # use `pyfory.Int32Type` for type hint if peer
- # are more narrow type.
- f8: int = None
- f9: pyfory.Float32Type = None
- # float type will be taken as `pyfory.Float64Type`
- f10: float = None
- f11: pyfory.Int16ArrayType = None
- f12: List[pyfory.Int16Type] = None
-
-
-if __name__ == "__main__":
- f = pyfory.Fory()
- f.register_type(SomeClass1, typename="example.SomeClass1")
- f.register_type(SomeClass2, typename="example.SomeClass2")
- obj1 = SomeClass1(f1=True, f2={-1: 2})
- obj = SomeClass2(
- f1=obj1,
- f2="abc",
- f3=["abc", "abc"],
- f4={1: 2},
- f5=2 ** 7 - 1,
- f6=2 ** 15 - 1,
- f7=2 ** 31 - 1,
- f8=2 ** 63 - 1,
- f9=1.0 / 2,
- f10=1 / 3.0,
- f11=array.array("h", [1, 2]),
- f12=[-1, 4],
- )
- data = f.serialize(obj)
- # bytes can be data serialized by other languages.
- print(f.deserialize(data))
-```
-
-**Golang**
-
-```go
-package main
-
-import forygo "github.com/apache/fory/fory/go/fory"
-import "fmt"
-
-func main() {
- type SomeClass1 struct {
- F1 interface{}
- F2 string
- F3 []interface{}
- F4 map[int8]int32
- F5 int8
- F6 int16
- F7 int32
- F8 int64
- F9 float32
- F10 float64
- F11 []int16
- F12 fory.Int16Slice
- }
-
- type SomeClas2 struct {
- F1 interface{}
- F2 map[int8]int32
- }
- fory := forygo.NewFory()
- if err := fory.RegisterNamedType(SomeClass1{}, "example.SomeClass1"); err != nil {
- panic(err)
- }
- if err := fory.RegisterNamedType(SomeClass2{}, "example.SomeClass2"); err != nil {
- panic(err)
- }
- obj1 := &SomeClass1{}
- obj1.F1 = true
- obj1.F2 = map[int8]int32{-1: 2}
- obj := &SomeClass1{}
- obj.F1 = obj1
- obj.F2 = "abc"
- obj.F3 = []interface{}{"abc", "abc"}
- f4 := map[int8]int32{1: 2}
- obj.F4 = f4
- obj.F5 = fory.MaxInt8
- obj.F6 = fory.MaxInt16
- obj.F7 = fory.MaxInt32
- obj.F8 = fory.MaxInt64
- obj.F9 = 1.0 / 2
- obj.F10 = 1 / 3.0
- obj.F11 = []int16{1, 2}
- obj.F12 = []int16{-1, 4}
- bytes, err := fory.Marshal(obj);
- if err != nil {
- panic(err)
- }
- var newValue interface{}
- // bytes can be data serialized by other languages.
- if err := fory.Unmarshal(bytes, &newValue); err != nil {
- panic(err)
- }
- fmt.Println(newValue)
-}
-```
-
-**JavaScript**
-
-```javascript
-import Fory, { Type, InternalSerializerType } from "@apache-fory/fory";
-
-/**
- * @apache-fory/hps use v8's fast-calls-api that can be called directly by jit, ensure that the version of Node is 20 or above.
- * Experimental feature, installation success cannot be guaranteed at this moment
- * If you are unable to install the module, replace it with `const hps = null;`
- **/
-import hps from "@apache-fory/hps";
-
-// Now we describe data structures using JSON, but in the future, we will use more ways.
-const description = Type.object("example.foo", {
- foo: Type.string(),
-});
-const fory = new Fory({ hps });
-const { serialize, deserialize } = fory.registerSerializer(description);
-const input = serialize({ foo: "hello fory" });
-const result = deserialize(input);
-console.log(result);
-```
-
-**Rust**
-
-```rust
-use chrono::{NaiveDate, NaiveDateTime};
-use fory::{from_buffer, to_buffer, Fory};
-use std::collections::HashMap;
-
-#[test]
-fn complex_struct() {
- #[derive(Fory, Debug, PartialEq)]
- #[tag("example.foo2")]
- struct Animal {
- category: String,
- }
-
- #[derive(Fory, Debug, PartialEq)]
- #[tag("example.foo")]
- struct Person {
- c1: Vec<u8>, // binary
- c2: Vec<i16>, // primitive array
- animal: Vec<Animal>,
- c3: Vec<Vec<u8>>,
- name: String,
- c4: HashMap<String, String>,
- age: u16,
- op: Option<String>,
- op2: Option<String>,
- date: NaiveDate,
- time: NaiveDateTime,
- c5: f32,
- c6: f64,
- }
- let person: Person = Person {
- c1: vec![1, 2, 3],
- c2: vec![5, 6, 7],
- c3: vec![vec![1, 2], vec![1, 3]],
- animal: vec![Animal {
- category: "Dog".to_string(),
- }],
- c4: HashMap::from([
- ("hello1".to_string(), "hello2".to_string()),
- ("hello2".to_string(), "hello3".to_string()),
- ]),
- age: 12,
- name: "helo".to_string(),
- op: Some("option".to_string()),
- op2: None,
- date: NaiveDate::from_ymd_opt(2025, 12, 12).unwrap(),
- time: NaiveDateTime::from_timestamp_opt(1689912359, 0).unwrap(),
- c5: 2.0,
- c6: 4.0,
- };
-
- let bin: Vec<u8> = to_buffer(&person);
- let obj: Person = from_buffer(&bin).expect("should success");
- assert_eq!(person, obj);
-}
-```
-
-## Serialize Shared Reference and Circular Reference
-
-Shared reference and circular reference can be serialized automatically, no duplicate data or recursion error.
-
-**Java**
-
-```java
-import org.apache.fory.*;
-import org.apache.fory.config.*;
-import java.util.*;
-
-public class ReferenceExample {
- public static class SomeClass {
- SomeClass f1;
- Map<String, String> f2;
- Map<String, String> f3;
- }
-
- public static Object createObject() {
- SomeClass obj = new SomeClass();
- obj.f1 = obj;
- obj.f2 = ofHashMap("k1", "v1", "k2", "v2");
- obj.f3 = obj.f2;
- return obj;
- }
-
- // mvn exec:java -Dexec.mainClass="org.apache.fory.examples.ReferenceExample"
- public static void main(String[] args) {
- Fory fory = Fory.builder().withLanguage(Language.XLANG)
- .withRefTracking(true).build();
- fory.register(SomeClass.class, "example.SomeClass");
- byte[] bytes = fory.serialize(createObject());
- // bytes can be data serialized by other languages.
- System.out.println(fory.deserialize(bytes));
- }
-}
-```
-
-**Python**
-
-```python
-from typing import Dict
-import pyfory
-
-class SomeClass:
- f1: "SomeClass"
- f2: Dict[str, str]
- f3: Dict[str, str]
-
-fory = pyfory.Fory(ref_tracking=True)
-fory.register_type(SomeClass, typename="example.SomeClass")
-obj = SomeClass()
-obj.f2 = {"k1": "v1", "k2": "v2"}
-obj.f1, obj.f3 = obj, obj.f2
-data = fory.serialize(obj)
-# bytes can be data serialized by other languages.
-print(fory.deserialize(data))
-```
-
-**Golang**
-
-```go
-package main
-
-import forygo "github.com/apache/fory/fory/go/fory"
-import "fmt"
-
-func main() {
- type SomeClass struct {
- F1 *SomeClass
- F2 map[string]string
- F3 map[string]string
- }
- fory := forygo.NewFory(true)
- if err := fory.Register(SomeClass{}, 65); err != nil {
- panic(err)
- }
- value := &SomeClass{F2: map[string]string{"k1": "v1", "k2": "v2"}}
- value.F3 = value.F2
- value.F1 = value
- bytes, err := fory.Marshal(value)
- if err != nil {
- }
- var newValue interface{}
- // bytes can be data serialized by other languages.
- if err := fory.Unmarshal(bytes, &newValue); err != nil {
- panic(err)
- }
- fmt.Println(newValue)
-}
-```
-
-**JavaScript**
-
-```javascript
-import Fory, { Type } from '@apache-fory/fory';
-/**
- * @apache-fory/hps use v8's fast-calls-api that can be called directly by jit, ensure that the version of Node is 20 or above.
- * Experimental feature, installation success cannot be guaranteed at this moment
- * If you are unable to install the module, replace it with `const hps = null;`
- **/
-import hps from '@apache-fory/hps';
-
-const description = Type.object('example.foo', {
- foo: Type.string(),
- bar: Type.object('example.foo'),
-});
-
-const fory = new Fory({ hps });
-const { serialize, deserialize } = fory.registerSerializer(description);
-const data: any = {
- foo: 'hello fory',
-};
-data.bar = data;
-const input = serialize(data);
-const result = deserialize(input);
-console.log(result.bar.foo === result.foo);
-```
-
-**JavaScript**
-Reference cannot be implemented because of rust ownership restrictions
-
-## Zero-Copy Serialization
-
-**Java**
-
-```java
-import org.apache.fory.*;
-import org.apache.fory.config.*;
-import org.apache.fory.serializer.BufferObject;
-import org.apache.fory.memory.MemoryBuffer;
-
-import java.util.*;
-import java.util.stream.Collectors;
-
-public class ZeroCopyExample {
- // mvn exec:java -Dexec.mainClass="io.ray.fory.examples.ZeroCopyExample"
- public static void main(String[] args) {
- Fory fory = Fory.builder().withLanguage(Language.XLANG).build();
- List<Object> list = ofArrayList("str", new byte[1000], new int[100], new double[100]);
- Collection<BufferObject> bufferObjects = new ArrayList<>();
- byte[] bytes = fory.serialize(list, e -> !bufferObjects.add(e));
- // bytes can be data serialized by other languages.
- List<MemoryBuffer> buffers = bufferObjects.stream()
- .map(BufferObject::toBuffer).collect(Collectors.toList());
- System.out.println(fory.deserialize(bytes, buffers));
- }
-}
-```
-
-**Python**
-
-```python
-import array
-import pyfory
-import numpy as np
-
-fory = pyfory.Fory()
-list_ = ["str", bytes(bytearray(1000)),
- array.array("i", range(100)), np.full(100, 0.0, dtype=np.double)]
-serialized_objects = []
-data = fory.serialize(list_, buffer_callback=serialized_objects.append)
-buffers = [o.to_buffer() for o in serialized_objects]
-# bytes can be data serialized by other languages.
-print(fory.deserialize(data, buffers=buffers))
-```
-
-**Golang**
-
-```go
-package main
-
-import forygo "github.com/apache/fory/fory/go/fory"
-import "fmt"
-
-func main() {
- fory := forygo.NewFory()
- list := []interface{}{"str", make([]byte, 1000)}
- buf := fory.NewByteBuffer(nil)
- var bufferObjects []fory.BufferObject
- fory.Serialize(buf, list, func(o fory.BufferObject) bool {
- bufferObjects = append(bufferObjects, o)
- return false
- })
- var newList []interface{}
- var buffers []*fory.ByteBuffer
- for _, o := range bufferObjects {
- buffers = append(buffers, o.ToBuffer())
- }
- if err := fory.Deserialize(buf, &newList, buffers); err != nil {
- panic(err)
- }
- fmt.Println(newList)
-}
-```
-
-**JavaScript**
-
-```javascript
-// Coming soon
-```
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org
For additional commands, e-mail: commits-help@fory.apache.org