You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@fory.apache.org by "chaokunyang (via GitHub)" <gi...@apache.org> on 2025/11/03 15:36:17 UTC

[I] [Rust][XLang] Use varint32/64 as type id for int32/64 [fory]

chaokunyang opened a new issue, #2885:
URL: https://github.com/apache/fory/issues/2885

   ### Feature Request
   
   Use varint32/64 type id  as type id for int32/64 type , so we can use ` int32/64` type id for uncompressed int32/64 read/write.
   
   ### Is your feature request related to a problem? Please describe
   
   _No response_
   
   ### Describe the solution you'd like
   
   _No response_
   
   ### Describe alternatives you've considered
   
   _No response_
   
   ### Additional context
   
   _No response_


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org
For additional commands, e-mail: commits-help@fory.apache.org


Re: [I] [Rust][XLang] Use varint32/64 as type id for int32/64 [fory]

Posted by "chaokunyang (via GitHub)" <gi...@apache.org>.
chaokunyang closed issue #2885: [Rust][XLang] Use varint32/64 as type id for int32/64
URL: https://github.com/apache/fory/issues/2885


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org
For additional commands, e-mail: commits-help@fory.apache.org


Re: [I] [Rust][XLang] Use varint32/64 as type id for int32/64 [fory]

Posted by "urlyy (via GitHub)" <gi...@apache.org>.
urlyy commented on issue #2885:
URL: https://github.com/apache/fory/issues/2885#issuecomment-3595431951

   For the same `int32` field, if locally it is set to `compress=false`, but the type_id received from remote is `VAR_INT32`, how should it be handled? If different processing is required based on the remote type_id, passing the type_id and conditional branches to `fory_read_data()` might affect performance.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org
For additional commands, e-mail: commits-help@fory.apache.org


Re: [I] [Rust][XLang] Use varint32/64 as type id for int32/64 [fory]

Posted by "chaokunyang (via GitHub)" <gi...@apache.org>.
chaokunyang commented on issue #2885:
URL: https://github.com/apache/fory/issues/2885#issuecomment-3649026533

   @urlyy The `ForyObject` macro has already skiped use `i32/i64` serializer tro handle read/write, it generate code to invoke `write_varint32/write_varint64` directly. You can just update marco codegen logic to generate buffer `read/writexxx` code directly based on macro attrs introduced in this issue


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org
For additional commands, e-mail: commits-help@fory.apache.org


Re: [I] [Rust][XLang] Use varint32/64 as type id for int32/64 [fory]

Posted by "urlyy (via GitHub)" <gi...@apache.org>.
urlyy commented on issue #2885:
URL: https://github.com/apache/fory/issues/2885#issuecomment-3636484143

   > For the same `int32` field, if locally it is set to `compress=false`, but the type_id received from remote is `VAR_INT32`, how should it be handled? If different processing is required based on the remote type_id, passing the type_id and add conditional branches to `fory_read_data()` might affect performance.
   
   @chaokunyang 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org
For additional commands, e-mail: commits-help@fory.apache.org


Re: [I] [Rust][XLang] Use varint32/64 as type id for int32/64 [fory]

Posted by "chaokunyang (via GitHub)" <gi...@apache.org>.
chaokunyang commented on issue #2885:
URL: https://github.com/apache/fory/issues/2885#issuecomment-3637583271

   > For the same `int32` field, if locally it is set to `compress=false`, but the type_id received from remote is `VAR_INT32`, how should it be handled? If different processing is required based on the remote type_id, passing the type_id and add conditional branches to `fory_read_data()` might affect performance.
   
   We read field based remote type meta, since remote field already say it's varint32, we just read by varint32. 
   
   This is essentially a type mismatch, we support such cases, but we should put it into a type id mismatch default case, and use a predefined switch case function to handle that.
   
   @urlyy 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org
For additional commands, e-mail: commits-help@fory.apache.org


Re: [I] [Rust][XLang] Use varint32/64 as type id for int32/64 [fory]

Posted by "urlyy (via GitHub)" <gi...@apache.org>.
urlyy commented on issue #2885:
URL: https://github.com/apache/fory/issues/2885#issuecomment-3648959438

   @chaokunyang This is the solution I believe requires the least modifications to the current code structure to resolve this PR, it is using `generic constant parameters` to eliminate branches at compile time and reduce runtime overhead. 
   
   However, this is not very user-friendly for `EXT` users, and if more compression modes need to be added in the future, it will also result in very long code.
   
   Here is the demo code:
   ```rust
   use std::collections::HashMap;
   
   trait Serializer {
       fn fory_read_data<const COMPRESS_INT: bool, const COMPRESS_LONG_MODE: u8>() -> Self;
   }
   
   impl Serializer for i32 {
       fn fory_read_data<const COMPRESS_INT: bool, const COMPRESS_LONG_MODE: u8>() -> Self {
           match COMPRESS_INT {
               true => {
                   println!("read_varint32");
                   return 21
               }
               false => {
                   println!("read_i32");
                   return 42;
               }
           }
       }
   }
   
   // for container, should pass the const generic parameters to the element.
   impl<K: Serializer + Eq + std::hash::Hash, V: Serializer> Serializer for HashMap<K, V> {
       fn fory_read_data<const COMPRESS_INT: bool, const COMPRESS_LONG_MODE: u8>() -> Self {
           K::fory_read_data::<COMPRESS_INT, COMPRESS_LONG_MODE>();
           V::fory_read_data::<COMPRESS_INT, COMPRESS_LONG_MODE>();
           unimplemented!()
       }
   }
   
   // same as i64
   // And for more compression, we should add more const generic parameters.
   
   
   // but for EXT type, user should write all const generic parameters when implement.
   struct CustomType;
   impl Serializer for CustomType {
       fn fory_read_data<const COMPRESS_INT: bool, const COMPRESS_LONG_MODE: u8>() -> Self {
           unimplemented!()
       }
   }
   
   #[repr(u8)]
   enum CompressLongMode {
       None,
       Var,
       Sli,
   }
   
   fn main() {
       // Based on #[fory(compress_int = true)] and #[fory(compress_long = xx)], pass different generic constant parameters at compile time.
       let a = i32::fory_read_data::<true, {CompressLongMode::None as u8}>();
       println!("Result: {}", a);
       let b = i32::fory_read_data::<false, {CompressLongMode::Sli as u8}>();
       println!("Result: {}", b);
   }
   ```
   I've also considered custom types like `I32Uncompressed`, but it cannot return an `i32` during reading. 
   Or if we add an extra `type_id` parameter to `read_data`/`write_data` and introduce branches for various compression modes of `i32` and `i64`, it would increase runtime overhead. 
   
   Do you have any suggestions?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org
For additional commands, e-mail: commits-help@fory.apache.org


Re: [I] [Rust][XLang] Use varint32/64 as type id for int32/64 [fory]

Posted by "urlyy (via GitHub)" <gi...@apache.org>.
urlyy commented on issue #2885:
URL: https://github.com/apache/fory/issues/2885#issuecomment-3645270363

   @chaokunyang Ok, But you said 
   ```rust
   struct Foo {
     [#fory(compress=False)]
     f1: i32
   }
   ```
   But i64 has three compression types. A single bool attribute can't handle that. Can we use a string form like `#[fory(compress = "varint")]?`
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org
For additional commands, e-mail: commits-help@fory.apache.org


Re: [I] [Rust][XLang] Use varint32/64 as type id for int32/64 [fory]

Posted by "chaokunyang (via GitHub)" <gi...@apache.org>.
chaokunyang commented on issue #2885:
URL: https://github.com/apache/fory/issues/2885#issuecomment-3649381962

   We can only suo=pport for direct struct fields, and ignore for `map/list` fields


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org
For additional commands, e-mail: commits-help@fory.apache.org


Re: [I] [Rust][XLang] Use varint32/64 as type id for int32/64 [fory]

Posted by "chaokunyang (via GitHub)" <gi...@apache.org>.
chaokunyang commented on issue #2885:
URL: https://github.com/apache/fory/issues/2885#issuecomment-3645407454

   We can support like:
   ```rust
   struct Foo {
       #[fory(compress = false)]
       f1: i64,
       #[fory(compress = "sli64")]
       f2: i64,
       #[fory(compress = "varint64")]
       f2: i64,
   }
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org
For additional commands, e-mail: commits-help@fory.apache.org


Re: [I] [Rust][XLang] Use varint32/64 as type id for int32/64 [fory]

Posted by "urlyy (via GitHub)" <gi...@apache.org>.
urlyy commented on issue #2885:
URL: https://github.com/apache/fory/issues/2885#issuecomment-3649140189

   > The `ForyObject` macro has already skiped using `i32/i64` serializer tro handle read/write, it generate code to invoke `write_varint32/write_varint64` directly. You can just update marco codegen logic to generate buffer `read/writexxx` code directly based on macro attrs introduced in this issue
   
   Yes, but it's hard to change the behavior of  `i32/i64::fory_write_type_info` at compile-time for each field. For example, `K::fory_write_type_info(context)` in `map.rs#write_map_data`. After obtaining the compression mode setting for this field, how can we make the types `K` and `V` use the corresponding read/write methods for that compression mode?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org
For additional commands, e-mail: commits-help@fory.apache.org