You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@fory.apache.org by ch...@apache.org on 2025/10/07 00:20:45 UTC
(fory) branch main updated: perf(Rust): Use SIMD to se/de string (#2716)
This is an automated email from the ASF dual-hosted git repository.
chaokunyang pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/fory.git
The following commit(s) were added to refs/heads/main by this push:
new f63084038 perf(Rust): Use SIMD to se/de string (#2716)
f63084038 is described below
commit f63084038832319cf5fe1032ddb7d1ba4bc32648
Author: urlyy <li...@gmail.com>
AuthorDate: Tue Oct 7 08:20:41 2025 +0800
perf(Rust): Use SIMD to se/de string (#2716)
## What does this PR do?
Use SIMD to se/de string
## Related issues
close #2713
close #2658
## Benchmark
hardware：
```
Chip: Apple M1 Pro
Total Number of Cores: 10 (8 performance and 2 efficiency)
Memory: 16 GB
```
output:
```
Write Latin-1 SIMD size 100
time: [119.07 ns 119.88 ns 120.74 ns]
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high mild
Write Latin-1 Standard size 100
time: [327.71 ns 329.71 ns 332.30 ns]
Found 9 outliers among 100 measurements (9.00%)
8 (8.00%) high mild
1 (1.00%) high severe
Write Latin-1 SIMD size 1000
time: [826.19 ns 827.90 ns 829.97 ns]
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high mild
Write Latin-1 Standard size 1000
time: [2.3436 µs 2.3459 µs 2.3488 µs]
Found 8 outliers among 100 measurements (8.00%)
1 (1.00%) low mild
4 (4.00%) high mild
3 (3.00%) high severe
Write Latin-1 SIMD size 10000
time: [7.0065 µs 7.0100 µs 7.0138 µs]
Found 2 outliers among 100 measurements (2.00%)
1 (1.00%) high mild
1 (1.00%) high severe
Write Latin-1 Standard size 10000
time: [21.003 µs 21.030 µs 21.064 µs]
Found 11 outliers among 100 measurements (11.00%)
1 (1.00%) low mild
3 (3.00%) high mild
7 (7.00%) high severe
Write Latin-1 SIMD size 100000
time: [71.804 µs 71.995 µs 72.170 µs]
Found 2 outliers among 100 measurements (2.00%)
1 (1.00%) low mild
1 (1.00%) high mild
Write Latin-1 Standard size 100000
time: [211.09 µs 211.51 µs 211.93 µs]
Found 5 outliers among 100 measurements (5.00%)
2 (2.00%) low mild
1 (1.00%) high mild
2 (2.00%) high severe
Write UTF-8 SIMD size 100
time: [136.54 ns 136.63 ns 136.74 ns]
Found 8 outliers among 100 measurements (8.00%)
6 (6.00%) high mild
2 (2.00%) high severe
Write UTF-8 Standard size 100
time: [4.2362 µs 4.2396 µs 4.2431 µs]
Found 6 outliers among 100 measurements (6.00%)
1 (1.00%) low mild
4 (4.00%) high mild
1 (1.00%) high severe
Write UTF-8 SIMD size 1000
time: [1.2469 µs 1.2498 µs 1.2533 µs]
Found 7 outliers among 100 measurements (7.00%)
1 (1.00%) low mild
5 (5.00%) high mild
1 (1.00%) high severe
Write UTF-8 Standard size 1000
time: [39.329 µs 39.369 µs 39.430 µs]
Found 13 outliers among 100 measurements (13.00%)
1 (1.00%) low mild
5 (5.00%) high mild
7 (7.00%) high severe
Write UTF-8 SIMD size 10000
time: [12.243 µs 12.254 µs 12.266 µs]
Found 6 outliers among 100 measurements (6.00%)
3 (3.00%) high mild
3 (3.00%) high severe
Write UTF-8 Standard size 10000
time: [400.42 µs 401.68 µs 403.37 µs]
Found 11 outliers among 100 measurements (11.00%)
1 (1.00%) low mild
3 (3.00%) high mild
7 (7.00%) high severe
Write UTF-8 SIMD size 100000
time: [189.73 µs 190.37 µs 191.04 µs]
Found 12 outliers among 100 measurements (12.00%)
10 (10.00%) high mild
2 (2.00%) high severe
Write UTF-8 Standard size 100000
time: [4.0438 ms 4.0524 ms 4.0617 ms]
Found 2 outliers among 100 measurements (2.00%)
1 (1.00%) high mild
1 (1.00%) high severe
Write UTF-16 SIMD size 100
time: [1.6355 µs 1.6384 µs 1.6420 µs]
Found 8 outliers among 100 measurements (8.00%)
3 (3.00%) high mild
5 (5.00%) high severe
Write UTF-16 Standard size 100
time: [4.7163 µs 4.7338 µs 4.7535 µs]
Found 5 outliers among 100 measurements (5.00%)
2 (2.00%) high mild
3 (3.00%) high severe
Write UTF-16 SIMD size 1000
time: [14.996 µs 15.017 µs 15.039 µs]
Found 4 outliers among 100 measurements (4.00%)
3 (3.00%) high mild
1 (1.00%) high severe
Write UTF-16 Standard size 1000
time: [41.669 µs 41.742 µs 41.827 µs]
Found 5 outliers among 100 measurements (5.00%)
5 (5.00%) high mild
Write UTF-16 SIMD size 10000
time: [145.72 µs 146.24 µs 146.81 µs]
Found 2 outliers among 100 measurements (2.00%)
1 (1.00%) high mild
1 (1.00%) high severe
Write UTF-16 Standard size 10000
time: [437.97 µs 452.79 µs 471.40 µs]
Found 13 outliers among 100 measurements (13.00%)
4 (4.00%) high mild
9 (9.00%) high severe
Benchmarking Write UTF-16 SIMD size 100000: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 9.3s, enable flat sampling, or reduce sample count to 50.
Write UTF-16 SIMD size 100000
time: [1.7727 ms 1.7795 ms 1.7878 ms]
Found 6 outliers among 100 measurements (6.00%)
6 (6.00%) high mild
Write UTF-16 Standard size 100000
time: [4.7168 ms 4.7426 ms 4.7709 ms]
Found 4 outliers among 100 measurements (4.00%)
2 (2.00%) low mild
1 (1.00%) high mild
1 (1.00%) high severe
```
```
Read Latin-1 SIMD size 100
time: [28.438 ns 28.466 ns 28.493 ns]
Found 3 outliers among 100 measurements (3.00%)
1 (1.00%) low mild
1 (1.00%) high mild
1 (1.00%) high severe
Read Latin-1 Standard size 100
time: [239.24 ns 239.80 ns 240.55 ns]
Found 7 outliers among 100 measurements (7.00%)
4 (4.00%) high mild
3 (3.00%) high severe
Read Latin-1 SIMD size 1000
time: [66.150 ns 66.183 ns 66.222 ns]
Found 4 outliers among 100 measurements (4.00%)
2 (2.00%) high mild
2 (2.00%) high severe
Read Latin-1 Standard size 1000
time: [2.2034 µs 2.2050 µs 2.2066 µs]
Found 6 outliers among 100 measurements (6.00%)
1 (1.00%) low mild
4 (4.00%) high mild
1 (1.00%) high severe
Read Latin-1 SIMD size 10000
time: [503.37 ns 506.51 ns 509.43 ns]
Found 19 outliers among 100 measurements (19.00%)
19 (19.00%) high mild
Read Latin-1 Standard size 10000
time: [21.271 µs 21.316 µs 21.370 µs]
Found 2 outliers among 100 measurements (2.00%)
2 (2.00%) high mild
Read Latin-1 SIMD size 100000
time: [5.5075 µs 5.5178 µs 5.5285 µs]
Found 5 outliers among 100 measurements (5.00%)
4 (4.00%) high mild
1 (1.00%) high severe
Read Latin-1 Standard size 100000
time: [212.06 µs 212.32 µs 212.63 µs]
Found 8 outliers among 100 measurements (8.00%)
3 (3.00%) high mild
5 (5.00%) high severe
Read UTF-8 SIMD size 100
time: [21.300 ns 21.314 ns 21.331 ns]
Found 8 outliers among 100 measurements (8.00%)
5 (5.00%) high mild
3 (3.00%) high severe
Read UTF-8 Standard size 100
time: [76.610 ns 76.689 ns 76.779 ns]
Found 5 outliers among 100 measurements (5.00%)
3 (3.00%) high mild
2 (2.00%) high severe
Read UTF-8 SIMD size 1000
time: [90.750 ns 91.046 ns 91.425 ns]
Found 7 outliers among 100 measurements (7.00%)
2 (2.00%) low mild
3 (3.00%) high mild
2 (2.00%) high severe
Read UTF-8 Standard size 1000
time: [530.43 ns 530.67 ns 530.95 ns]
Found 10 outliers among 100 measurements (10.00%)
8 (8.00%) high mild
2 (2.00%) high severe
Read UTF-8 SIMD size 10000
time: [402.37 ns 403.34 ns 404.30 ns]
Found 3 outliers among 100 measurements (3.00%)
1 (1.00%) high mild
2 (2.00%) high severe
Read UTF-8 Standard size 10000
time: [4.8775 µs 4.8913 µs 4.9069 µs]
Found 6 outliers among 100 measurements (6.00%)
6 (6.00%) high mild
Read UTF-8 SIMD size 100000
time: [8.2674 µs 8.2749 µs 8.2830 µs]
Found 9 outliers among 100 measurements (9.00%)
1 (1.00%) low severe
3 (3.00%) low mild
3 (3.00%) high mild
2 (2.00%) high severe
Read UTF-8 Standard size 100000
time: [47.938 µs 47.991 µs 48.054 µs]
Found 9 outliers among 100 measurements (9.00%)
5 (5.00%) high mild
4 (4.00%) high severe
Read UTF-16 SIMD size 100
time: [210.00 ns 210.46 ns 210.90 ns]
Found 2 outliers among 100 measurements (2.00%)
2 (2.00%) high mild
Read UTF-16 Standard size 100
time: [294.07 ns 299.92 ns 308.96 ns]
Found 8 outliers among 100 measurements (8.00%)
2 (2.00%) high mild
6 (6.00%) high severe
Read UTF-16 SIMD size 1000
time: [1.4022 µs 1.4076 µs 1.4141 µs]
Found 4 outliers among 100 measurements (4.00%)
3 (3.00%) high mild
1 (1.00%) high severe
Read UTF-16 Standard size 1000
time: [2.2546 µs 2.2960 µs 2.3543 µs]
Found 9 outliers among 100 measurements (9.00%)
4 (4.00%) high mild
5 (5.00%) high severe
Read UTF-16 SIMD size 10000
time: [13.036 µs 13.291 µs 13.566 µs]
Found 9 outliers among 100 measurements (9.00%)
5 (5.00%) high mild
4 (4.00%) high severe
Read UTF-16 Standard size 10000
time: [20.146 µs 20.273 µs 20.421 µs]
Found 6 outliers among 100 measurements (6.00%)
4 (4.00%) high mild
2 (2.00%) high severe
Read UTF-16 SIMD size 100000
time: [131.25 µs 133.50 µs 136.36 µs]
Found 8 outliers among 100 measurements (8.00%)
2 (2.00%) high mild
6 (6.00%) high severe
Read UTF-16 Standard size 100000
time: [214.45 µs 219.54 µs 225.14 µs]
Found 12 outliers among 100 measurements (12.00%)
5 (5.00%) high mild
7 (7.00%) high severe
```
---
rust/fory-core/benches/simd_bench.rs | 176 +++++++++
rust/fory-core/src/buffer.rs | 51 +--
rust/fory-core/src/meta/mod.rs | 2 +-
rust/fory-core/src/meta/string_util.rs | 618 ++++++++++++++++++++++++++++++++
rust/fory-core/src/serializer/string.rs | 11 +-
5 files changed, 813 insertions(+), 45 deletions(-)
diff --git a/rust/fory-core/benches/simd_bench.rs b/rust/fory-core/benches/simd_bench.rs
index 7649533fc..3f8b7daac 100644
--- a/rust/fory-core/benches/simd_bench.rs
+++ b/rust/fory-core/benches/simd_bench.rs
@@ -19,6 +19,12 @@ use criterion::{black_box, criterion_group, criterion_main, Criterion};
#[cfg(target_feature = "avx2")]
use std::arch::x86_64::*;
+use fory_core::buffer::{Reader, Writer};
+use fory_core::meta::buffer_rw_string::{
+ read_latin1_simd, read_latin1_standard, read_utf16_simd, read_utf16_standard, read_utf8_simd,
+ read_utf8_standard, write_latin1_simd, write_latin1_standard, write_utf16_simd,
+ write_utf16_standard, write_utf8_simd, write_utf8_standard,
+};
#[cfg(target_feature = "sse2")]
use std::arch::x86_64::*;
@@ -78,6 +84,168 @@ fn is_latin_std(s: &str) -> bool {
s.bytes().all(|b| b.is_ascii())
}
+fn benchmark_write_latin1(c: &mut Criterion) {
+ let sizes = [100, 1000, 10000, 100000];
+ let ascii_string = "abcdefghijklmnopqrstuvwxyz0123456789";
+
+ for &size in &sizes {
+ let s_ascii = ascii_string.repeat(size / ascii_string.len() + 1);
+
+ let name_simd = format!("Write Latin-1 SIMD size {}", size);
+ c.bench_function(&name_simd, |b| {
+ b.iter(|| {
+ let mut w = Writer::default();
+ write_latin1_simd(black_box(&mut w), black_box(&s_ascii));
+ })
+ });
+
+ let name_scalar = format!("Write Latin-1 Standard size {}", size);
+ c.bench_function(&name_scalar, |b| {
+ b.iter(|| {
+ let mut w = Writer::default();
+ write_latin1_standard(black_box(&mut w), black_box(&s_ascii));
+ })
+ });
+ }
+}
+
+fn benchmark_write_utf8(c: &mut Criterion) {
+ let sizes = [100, 1000, 10000, 100000];
+ for &size in &sizes {
+ let s = "Hello, 世界! 🌍".repeat(size);
+
+ let name_simd = format!("Write UTF-8 SIMD size {}", size);
+ c.bench_function(&name_simd, |b| {
+ b.iter(|| {
+ let mut w = Writer::default();
+ write_utf8_simd(black_box(&mut w), black_box(&s));
+ })
+ });
+
+ let name_scalar = format!("Write UTF-8 Standard size {}", size);
+ c.bench_function(&name_scalar, |b| {
+ b.iter(|| {
+ let mut w = Writer::default();
+ write_utf8_standard(black_box(&mut w), black_box(&s));
+ })
+ });
+ }
+}
+
+fn benchmark_write_utf16(c: &mut Criterion) {
+ let sizes = [100, 1000, 10000, 100000];
+ for &size in &sizes {
+ let s = "Hello, 世界! 🌍".repeat(size);
+
+ let name_simd = format!("Write UTF-16 SIMD size {}", size);
+ c.bench_function(&name_simd, |b| {
+ b.iter(|| {
+ let mut w = Writer::default();
+ let utf16: Vec<u16> = s.encode_utf16().collect();
+ write_utf16_simd(black_box(&mut w), black_box(&utf16));
+ })
+ });
+
+ let name_scalar = format!("Write UTF-16 Standard size {}", size);
+ c.bench_function(&name_scalar, |b| {
+ b.iter(|| {
+ let mut w = Writer::default();
+ let utf16: Vec<u16> = s.encode_utf16().collect();
+ write_utf16_standard(black_box(&mut w), black_box(&utf16));
+ })
+ });
+ }
+}
+
+fn benchmark_read_latin1(c: &mut Criterion) {
+ let sizes = [100, 1000, 10000, 100000];
+ let ascii_string = "abcdefghijklmnopqrstuvwxyz0123456789";
+
+ for &size in &sizes {
+ let s_ascii = ascii_string.repeat(size / ascii_string.len() + 1);
+ let mut writer = Writer::default();
+ writer.write_latin1_string(&s_ascii);
+ let data = writer.dump();
+
+ let name_simd = format!("Read Latin-1 SIMD size {}", size);
+ c.bench_function(&name_simd, |b| {
+ b.iter(|| {
+ let mut reader = Reader::new(black_box(&data));
+ read_latin1_simd(black_box(&mut reader), black_box(s_ascii.len()));
+ })
+ });
+
+ let name_scalar = format!("Read Latin-1 Standard size {}", size);
+ c.bench_function(&name_scalar, |b| {
+ b.iter(|| {
+ let mut reader = Reader::new(black_box(&data));
+ read_latin1_standard(black_box(&mut reader), black_box(s_ascii.len()));
+ })
+ });
+ }
+}
+
+fn benchmark_read_utf8(c: &mut Criterion) {
+ let sizes = [100, 1000, 10000, 100000];
+ let test_string = "Hello, 世界! 🌍";
+
+ for &size in &sizes {
+ let s = test_string.repeat(size / test_string.len() + 1);
+ let mut writer = Writer::default();
+ writer.write_utf8_string(&s);
+ let data = writer.dump();
+
+ let name_simd = format!("Read UTF-8 SIMD size {}", size);
+ c.bench_function(&name_simd, |b| {
+ b.iter(|| {
+ let mut reader = Reader::new(black_box(&data));
+ read_utf8_simd(black_box(&mut reader), black_box(s.len()));
+ })
+ });
+
+ let name_scalar = format!("Read UTF-8 Standard size {}", size);
+ c.bench_function(&name_scalar, |b| {
+ b.iter(|| {
+ let mut reader = Reader::new(black_box(&data));
+ read_utf8_standard(black_box(&mut reader), black_box(s.len()));
+ })
+ });
+ }
+}
+
+fn benchmark_read_utf16(c: &mut Criterion) {
+ let sizes = [100, 1000, 10000, 100000];
+ let test_string = "Hello, 世界! 🌍";
+
+ for &size in &sizes {
+ let s = test_string.repeat(size / test_string.len() + 1);
+ // 构造 UTF-16 LE 字节数据
+ let mut data: Vec<u8> = Vec::with_capacity(s.len() * 2);
+ for u in s.encode_utf16() {
+ let lo = (u & 0x00FF) as u8;
+ let hi = (u >> 8) as u8;
+ data.push(lo);
+ data.push(hi);
+ }
+
+ let name_simd = format!("Read UTF-16 SIMD size {}", size);
+ c.bench_function(&name_simd, |b| {
+ b.iter(|| {
+ let mut reader = Reader::new(black_box(&data));
+ read_utf16_simd(black_box(&mut reader), black_box(data.len()));
+ })
+ });
+
+ let name_scalar = format!("Read UTF-16 Standard size {}", size);
+ c.bench_function(&name_scalar, |b| {
+ b.iter(|| {
+ let mut reader = Reader::new(black_box(&data));
+ read_utf16_standard(black_box(&mut reader), black_box(data.len()));
+ })
+ });
+ }
+}
+
fn criterion_benchmark(c: &mut Criterion) {
let test_str_short = "Hello, World!";
let test_str_long = "Hello, World! ".repeat(1000);
@@ -106,6 +274,14 @@ fn criterion_benchmark(c: &mut Criterion) {
c.bench_function("Standard long", |b| {
b.iter(|| is_latin_std(black_box(&test_str_long)))
});
+
+ benchmark_write_latin1(c);
+ benchmark_write_utf8(c);
+ benchmark_write_utf16(c);
+
+ benchmark_read_latin1(c);
+ benchmark_read_utf8(c);
+ benchmark_read_utf16(c);
}
criterion_group!(benches, criterion_benchmark);
diff --git a/rust/fory-core/src/buffer.rs b/rust/fory-core/src/buffer.rs
index 6f0de5444..b7fefe179 100644
--- a/rust/fory-core/src/buffer.rs
+++ b/rust/fory-core/src/buffer.rs
@@ -15,11 +15,15 @@
// specific language governing permissions and limitations
// under the License.
+use crate::meta::buffer_rw_string::{
+ read_latin1_simd, read_utf16_simd, read_utf8_simd, write_latin1_simd, write_utf16_simd,
+ write_utf8_simd,
+};
use byteorder::{ByteOrder, LittleEndian, WriteBytesExt};
#[derive(Default)]
pub struct Writer {
- bf: Vec<u8>,
+ pub(crate) bf: Vec<u8>,
reserved: usize,
}
@@ -287,27 +291,22 @@ impl Writer {
}
}
- // TODO: simd
pub fn write_latin1_string(&mut self, s: &str) {
- for c in s.chars() {
- let b = c as u32;
- assert!(b <= 0xFF, "Non-Latin1 character found");
- self.write_u8(b as u8);
- }
+ write_latin1_simd(self, s);
}
- // TODO: simd
pub fn write_utf8_string(&mut self, s: &str) {
- let bytes = s.as_bytes();
- for &b in bytes {
- self.write_u8(b);
- }
+ write_utf8_simd(self, s);
+ }
+
+ pub fn write_utf16_bytes(&mut self, bytes: &[u16]) {
+ write_utf16_simd(self, bytes);
}
}
pub struct Reader<'de> {
- bf: &'de [u8],
- cursor: usize,
+ pub(crate) bf: &'de [u8],
+ pub(crate) cursor: usize,
}
impl<'bf> Reader<'bf> {
@@ -315,7 +314,7 @@ impl<'bf> Reader<'bf> {
Reader { bf, cursor: 0 }
}
- fn move_next(&mut self, additional: usize) {
+ pub(crate) fn move_next(&mut self, additional: usize) {
self.cursor += additional;
}
@@ -498,31 +497,15 @@ impl<'bf> Reader<'bf> {
}
pub fn read_latin1_string(&mut self, len: usize) -> String {
- let slice = &self.bf[self.cursor..self.cursor + len];
- let result: String = slice.iter().map(|&b| b as char).collect();
- self.move_next(len);
- result
+ read_latin1_simd(self, len)
}
pub fn read_utf8_string(&mut self, len: usize) -> String {
- let result = String::from_utf8_lossy(&self.bf[self.cursor..self.cursor + len]).to_string();
- self.move_next(len);
- result
+ read_utf8_simd(self, len)
}
pub fn read_utf16_string(&mut self, len: usize) -> String {
- assert!(len % 2 == 0, "UTF-16 length must be even");
- let slice = &self.bf[self.cursor..self.cursor + len];
- let units: Vec<u16> = slice
- .chunks(2)
- // little endian
- .map(|c| (c[0] as u16) | ((c[1] as u16) << 8))
- .collect();
- let result = String::from_utf16(&units)
- // lossy
- .unwrap_or_else(|_| String::from("�"));
- self.move_next(len);
- result
+ read_utf16_simd(self, len)
}
pub fn read_varuint36small(&mut self) -> u64 {
diff --git a/rust/fory-core/src/meta/mod.rs b/rust/fory-core/src/meta/mod.rs
index 4eea1ae6b..7ec3239aa 100644
--- a/rust/fory-core/src/meta/mod.rs
+++ b/rust/fory-core/src/meta/mod.rs
@@ -24,7 +24,7 @@ pub use meta_string::{
FIELD_NAME_DECODER, FIELD_NAME_ENCODER, NAMESPACE_DECODER, NAMESPACE_ENCODER,
TYPE_NAME_DECODER, TYPE_NAME_ENCODER,
};
-pub use string_util::{get_latin1_length, is_latin, murmurhash3_x64_128};
+pub use string_util::{buffer_rw_string, get_latin1_length, is_latin, murmurhash3_x64_128};
pub use type_meta::{
FieldInfo, FieldType, NullableFieldType, TypeMeta, TypeMetaLayer, NAMESPACE_ENCODINGS,
TYPE_NAME_ENCODINGS,
diff --git a/rust/fory-core/src/meta/string_util.rs b/rust/fory-core/src/meta/string_util.rs
index f9ec88a9a..ba77c63e0 100644
--- a/rust/fory-core/src/meta/string_util.rs
+++ b/rust/fory-core/src/meta/string_util.rs
@@ -509,3 +509,621 @@ mod test_hash {
== (9455322759164802692, 17863277201603478371));
}
}
+
+pub mod buffer_rw_string {
+ #[cfg(all(target_arch = "aarch64", target_feature = "neon"))]
+ use std::arch::aarch64::*;
+ #[cfg(all(target_arch = "x86_64", target_feature = "avx2"))]
+ use std::arch::x86_64::*;
+ #[cfg(all(
+ any(target_arch = "x86", target_arch = "x86_64"),
+ target_feature = "sse2",
+ not(target_feature = "avx2")
+ ))]
+ use std::arch::x86_64::*;
+
+ #[cfg(all(target_arch = "x86_64", target_feature = "avx2"))]
+ const SIMD_CHUNK_SIZE: usize = 64;
+ #[cfg(all(
+ any(target_arch = "x86", target_arch = "x86_64"),
+ target_feature = "sse2",
+ not(target_feature = "avx2")
+ ))]
+ const SIMD_CHUNK_SIZE: usize = 32;
+ #[cfg(all(target_arch = "aarch64", target_feature = "neon"))]
+ const SIMD_CHUNK_SIZE: usize = 32;
+
+ use crate::buffer::{Reader, Writer};
+
+ #[inline]
+ pub fn write_latin1_standard(writer: &mut Writer, s: &str) {
+ for c in s.chars() {
+ let b = c as u32;
+ assert!(b <= 0xFF, "Non-Latin1 character found");
+ writer.write_u8(b as u8);
+ }
+ }
+
+ #[inline]
+ pub fn write_utf8_standard(writer: &mut Writer, s: &str) {
+ let bytes = s.as_bytes();
+ for &b in bytes {
+ writer.write_u8(b);
+ }
+ }
+
+ #[inline]
+ pub fn write_utf16_standard(writer: &mut Writer, utf16: &[u16]) {
+ for unit in utf16 {
+ #[cfg(target_endian = "little")]
+ {
+ writer.write_u16(*unit);
+ }
+ #[cfg(target_endian = "big")]
+ {
+ unimplemented!()
+ }
+ }
+ }
+
+ #[inline]
+ pub fn read_latin1_standard(reader: &mut Reader, len: usize) -> String {
+ let slice = &reader.bf[reader.cursor..reader.cursor + len];
+ let result: String = slice.iter().map(|&b| b as char).collect();
+ reader.move_next(len);
+ result
+ }
+
+ #[inline]
+ pub fn read_utf8_standard(reader: &mut Reader, len: usize) -> String {
+ let result =
+ String::from_utf8_lossy(&reader.bf[reader.cursor..reader.cursor + len]).to_string();
+ reader.move_next(len);
+ result
+ }
+
+ #[inline]
+ pub fn read_utf16_standard(reader: &mut Reader, len: usize) -> String {
+ assert!(len % 2 == 0, "UTF-16 length must be even");
+ let slice = &reader.bf[reader.cursor..reader.cursor + len];
+ let units: Vec<u16> = slice
+ .chunks(2)
+ // little endian
+ .map(|c| (c[0] as u16) | ((c[1] as u16) << 8))
+ .collect();
+ let result = String::from_utf16(&units)
+ // lossy
+ .unwrap_or_else(|_| String::from("�"));
+ reader.move_next(len);
+ result
+ }
+
+ #[inline]
+ fn write_bytes_simd(writer: &mut Writer, bytes: &[u8]) {
+ let len = bytes.len();
+ let mut i = 0usize;
+
+ if len == 0 {
+ return;
+ }
+
+ writer.bf.reserve(len);
+
+ #[cfg(any(
+ all(target_arch = "x86_64", target_feature = "avx2"),
+ all(target_arch = "x86_64", target_feature = "sse2"),
+ all(target_arch = "aarch64", target_feature = "neon")
+ ))]
+ unsafe {
+ #[cfg(all(target_arch = "x86_64", target_feature = "avx2"))]
+ {
+ const CHUNK: usize = 64;
+ while i + CHUNK <= len {
+ let ptr = bytes.as_ptr().add(i);
+ let chunk1 = _mm256_loadu_si256(ptr as *const __m256i);
+ let chunk2 = _mm256_loadu_si256(ptr.add(32) as *const __m256i);
+
+ let current_len = writer.bf.len();
+ writer.bf.set_len(current_len + CHUNK);
+ let dest_ptr = writer.bf.as_mut_ptr().add(current_len);
+
+ _mm256_storeu_si256(dest_ptr as *mut __m256i, chunk1);
+ _mm256_storeu_si256(dest_ptr.add(32) as *mut __m256i, chunk2);
+ i += CHUNK;
+ }
+ }
+
+ #[cfg(all(
+ target_arch = "x86_64",
+ not(target_feature = "avx2"),
+ target_feature = "sse2"
+ ))]
+ {
+ const CHUNK: usize = 32;
+ while i + CHUNK <= len {
+ let ptr = bytes.as_ptr().add(i);
+ let chunk1 = _mm_loadu_si128(ptr as *const __m128i);
+ let chunk2 = _mm_loadu_si128(ptr.add(16) as *const __m128i);
+
+ let current_len = writer.bf.len();
+ writer.bf.set_len(current_len + CHUNK);
+ let dest_ptr = writer.bf.as_mut_ptr().add(current_len);
+
+ _mm_storeu_si128(dest_ptr as *mut __m128i, chunk1);
+ _mm_storeu_si128(dest_ptr.add(16) as *mut __m128i, chunk2);
+ i += CHUNK;
+ }
+ }
+
+ #[cfg(all(target_arch = "aarch64", target_feature = "neon"))]
+ {
+ const CHUNK: usize = 32;
+ while i + CHUNK <= len {
+ let ptr = bytes.as_ptr().add(i);
+ let chunk1 = vld1q_u8(ptr);
+ let chunk2 = vld1q_u8(ptr.add(16));
+
+ let current_len = writer.bf.len();
+ writer.bf.set_len(current_len + CHUNK);
+ let dest_ptr = writer.bf.as_mut_ptr().add(current_len);
+
+ vst1q_u8(dest_ptr, chunk1);
+ vst1q_u8(dest_ptr.add(16), chunk2);
+ i += CHUNK;
+ }
+ }
+ }
+
+ const MEDIUM_CHUNK: usize = 16;
+ while i + MEDIUM_CHUNK <= len {
+ writer.bf.extend_from_slice(&bytes[i..i + MEDIUM_CHUNK]);
+ i += MEDIUM_CHUNK;
+ }
+ if i < len {
+ writer.bf.extend_from_slice(&bytes[i..]);
+ }
+ }
+
+ #[inline]
+ pub fn write_latin1_simd(writer: &mut Writer, s: &str) {
+ if s.is_empty() {
+ return;
+ }
+ let mut buf: Vec<u8> = Vec::with_capacity(s.len());
+ for c in s.chars() {
+ let v = c as u32;
+ assert!(v <= 0xFF, "Non-Latin1 character found");
+ buf.push(v as u8);
+ }
+ write_bytes_simd(writer, &buf);
+ }
+
+ #[inline]
+ pub fn write_utf8_simd(writer: &mut Writer, s: &str) {
+ let bytes = s.as_bytes();
+ write_bytes_simd(writer, bytes);
+ }
+
+ pub fn write_utf16_simd(writer: &mut Writer, utf16: &[u16]) {
+ if utf16.is_empty() {
+ return;
+ }
+
+ #[cfg(target_endian = "big")]
+ {
+ unimplemented!("Big-endian UTF-16 writing is not implemented");
+ }
+
+ #[cfg(target_endian = "little")]
+ unsafe {
+ let total_bytes = utf16.len() * 2;
+ let old_len = writer.bf.len();
+ writer.bf.reserve(total_bytes);
+ let dest = writer.bf.as_mut_ptr().add(old_len);
+ let src = utf16.as_ptr() as *const u8;
+
+ let mut i = 0usize;
+
+ #[cfg(all(target_arch = "x86_64", target_feature = "avx2"))]
+ {
+ const CHUNK: usize = 32;
+ while i + CHUNK <= total_bytes {
+ let chunk = _mm256_loadu_si256(src.add(i) as *const __m256i);
+ _mm256_storeu_si256(dest.add(i) as *mut __m256i, chunk);
+ i += CHUNK;
+ }
+ }
+
+ #[cfg(all(
+ any(target_arch = "x86", target_arch = "x86_64"),
+ target_feature = "sse2",
+ not(target_feature = "avx2")
+ ))]
+ {
+ const CHUNK: usize = 16;
+ while i + CHUNK <= total_bytes {
+ let chunk = _mm_loadu_si128(src.add(i) as *const __m128i);
+ _mm_storeu_si128(dest.add(i) as *mut __m128i, chunk);
+ i += CHUNK;
+ }
+ }
+
+ #[cfg(all(target_arch = "aarch64", target_feature = "neon"))]
+ {
+ const CHUNK: usize = 16;
+ while i + CHUNK <= total_bytes {
+ let chunk = vld1q_u8(src.add(i));
+ vst1q_u8(dest.add(i), chunk);
+ i += CHUNK;
+ }
+ }
+
+ // fallback for remaining bytes
+ if i < total_bytes {
+ std::ptr::copy_nonoverlapping(src.add(i), dest.add(i), total_bytes - i);
+ }
+
+ // set length only after all writes
+ writer.bf.set_len(old_len + total_bytes);
+ }
+ }
+
+ #[inline]
+ pub fn read_latin1_simd(reader: &mut Reader, len: usize) -> String {
+ if len == 0 {
+ return String::new();
+ }
+ let src = &reader.bf[reader.cursor..reader.cursor + len];
+
+ let mut out: Vec<u8> = Vec::with_capacity(len + len / 4);
+
+ unsafe {
+ let mut i = 0usize;
+
+ // ---- AVX2 fast-path (32 bytes) ----
+ #[cfg(target_arch = "x86_64")]
+ {
+ if std::arch::is_x86_feature_detected!("avx2") {
+ use std::arch::x86_64::*;
+ while i + 32 <= len {
+ let ptr = src.as_ptr().add(i) as *const __m256i;
+ let chunk = _mm256_loadu_si256(ptr);
+ let mask = _mm256_movemask_epi8(chunk);
+ if mask == 0 {
+ let mut buf32: [u8; 32] = std::mem::zeroed();
+ _mm256_storeu_si256(buf32.as_mut_ptr() as *mut __m256i, chunk);
+ out.extend_from_slice(&buf32);
+ i += 32;
+ continue;
+ } else {
+ break;
+ }
+ }
+ }
+ }
+
+ // ---- SSE2 fast-path (16 bytes) ----
+ #[cfg(any(target_arch = "x86", target_arch = "x86_64"))]
+ {
+ if std::arch::is_x86_feature_detected!("sse2") {
+ use std::arch::x86_64::*;
+ while i + 16 <= len {
+ let ptr = src.as_ptr().add(i) as *const __m128i;
+ let chunk = _mm_loadu_si128(ptr);
+ let mask = _mm_movemask_epi8(chunk);
+ if mask == 0 {
+ let mut buf16: [u8; 16] = std::mem::zeroed();
+ _mm_storeu_si128(buf16.as_mut_ptr() as *mut __m128i, chunk);
+ out.extend_from_slice(&buf16);
+ i += 16;
+ continue;
+ } else {
+ break;
+ }
+ }
+ }
+ }
+
+ // ---- NEON fast-path (16 bytes) ----
+ #[cfg(target_arch = "aarch64")]
+ {
+ if std::arch::is_aarch64_feature_detected!("neon") {
+ use std::arch::aarch64::*;
+ while i + 16 <= len {
+ let ptr = src.as_ptr().add(i);
+ let v = vld1q_u8(ptr);
+ let cmp = vcgeq_u8(v, vdupq_n_u8(128));
+
+ let mut mask_arr: [u8; 16] = std::mem::zeroed();
+ vst1q_u8(mask_arr.as_mut_ptr(), cmp);
+
+ if mask_arr.iter().all(|&x| x == 0) {
+ let mut buf16: [u8; 16] = std::mem::zeroed();
+ vst1q_u8(buf16.as_mut_ptr(), v);
+ out.extend_from_slice(&buf16);
+ i += 16;
+ continue;
+ } else {
+ break;
+ }
+ }
+ }
+ }
+
+ // ---- scalar fallback for remaining bytes ----
+ while i < len {
+ let b = *src.get_unchecked(i);
+ if b < 0x80 {
+ out.push(b);
+ } else {
+ out.push(0xC0 | (b >> 6));
+ out.push(0x80 | (b & 0x3F));
+ }
+ i += 1;
+ }
+ }
+
+ reader.move_next(len);
+ unsafe { String::from_utf8_unchecked(out) }
+ }
+
+ #[inline]
+ pub fn read_utf8_simd(reader: &mut Reader, len: usize) -> String {
+ if len == 0 {
+ return String::new();
+ }
+
+ let src = &reader.bf[reader.cursor..reader.cursor + len];
+ let mut result = String::with_capacity(len);
+
+ unsafe {
+ let mut i = 0usize;
+
+ #[cfg(all(target_arch = "x86_64", target_feature = "avx2"))]
+ {
+ const CHUNK: usize = 32;
+ while i + CHUNK <= len {
+ let chunk = _mm256_loadu_si256(src.as_ptr().add(i) as *const __m256i);
+ let mut buf = [0u8; CHUNK];
+ _mm256_storeu_si256(buf.as_mut_ptr() as *mut __m256i, chunk);
+ result.push_str(std::str::from_utf8_unchecked(&buf));
+ i += CHUNK;
+ }
+ }
+
+ #[cfg(all(
+ any(target_arch = "x86", target_arch = "x86_64"),
+ target_feature = "sse2",
+ not(target_feature = "avx2")
+ ))]
+ {
+ const CHUNK: usize = 16;
+ while i + CHUNK <= len {
+ let chunk = _mm_loadu_si128(src.as_ptr().add(i) as *const __m128i);
+ let mut buf = [0u8; CHUNK];
+ _mm_storeu_si128(buf.as_mut_ptr() as *mut __m128i, chunk);
+ result.push_str(std::str::from_utf8_unchecked(&buf));
+ i += CHUNK;
+ }
+ }
+
+ #[cfg(all(target_arch = "aarch64", target_feature = "neon"))]
+ {
+ const CHUNK: usize = 16;
+ while i + CHUNK <= len {
+ let chunk = vld1q_u8(src.as_ptr().add(i));
+ let mut buf = [0u8; CHUNK];
+ vst1q_u8(buf.as_mut_ptr(), chunk);
+ result.push_str(std::str::from_utf8_unchecked(&buf));
+ i += CHUNK;
+ }
+ }
+
+ if i < len {
+ result.push_str(std::str::from_utf8_unchecked(&src[i..len]));
+ }
+ }
+
+ reader.move_next(len);
+ result
+ }
+
+ #[inline]
+ pub fn read_utf16_simd(reader: &mut Reader, len: usize) -> String {
+ assert_eq!(len % 2, 0, "UTF-16 length must be even");
+ unsafe fn simd_impl(bytes: &[u8]) -> String {
+ let len = bytes.len();
+ let unit_len = len / 2;
+ let mut units: Vec<u16> = vec![0u16; unit_len];
+
+ let dest_u8 = units.as_mut_ptr() as *mut u8;
+ let src_u8 = bytes.as_ptr();
+ let mut i = 0usize;
+
+ #[cfg(all(target_arch = "x86_64", target_feature = "avx2"))]
+ {
+ while i + SIMD_CHUNK_SIZE <= len {
+ let c1 = _mm256_loadu_si256(src_u8.add(i) as *const __m256i);
+ let c2 = _mm256_loadu_si256(src_u8.add(i + 32) as *const __m256i);
+ _mm256_storeu_si256(dest_u8.add(i) as *mut __m256i, c1);
+ _mm256_storeu_si256(dest_u8.add(i + 32) as *mut __m256i, c2);
+ i += SIMD_CHUNK_SIZE;
+ }
+ while i + 32 <= len {
+ let c = _mm256_loadu_si256(src_u8.add(i) as *const __m256i);
+ _mm256_storeu_si256(dest_u8.add(i) as *mut __m256i, c);
+ i += 32;
+ }
+ }
+
+ #[cfg(all(
+ any(target_arch = "x86", target_arch = "x86_64"),
+ target_feature = "sse2",
+ not(target_feature = "avx2")
+ ))]
+ {
+ while i + SIMD_CHUNK_SIZE <= len {
+ let c1 = _mm_loadu_si128(src_u8.add(i) as *const __m128i);
+ let c2 = _mm_loadu_si128(src_u8.add(i + 16) as *const __m128i);
+ _mm_storeu_si128(dest_u8.add(i) as *mut __m128i, c1);
+ _mm_storeu_si128(dest_u8.add(i + 16) as *mut __m128i, c2);
+ i += SIMD_CHUNK_SIZE;
+ }
+ while i + 16 <= len {
+ let c = _mm_loadu_si128(src_u8.add(i) as *const __m128i);
+ _mm_storeu_si128(dest_u8.add(i) as *mut __m128i, c);
+ i += 16;
+ }
+ }
+
+ #[cfg(all(target_arch = "aarch64", target_feature = "neon"))]
+ {
+ while i + SIMD_CHUNK_SIZE <= len {
+ let c1 = vld1q_u8(src_u8.add(i));
+ let c2 = vld1q_u8(src_u8.add(i + 16));
+ vst1q_u8(dest_u8.add(i), c1);
+ vst1q_u8(dest_u8.add(i + 16), c2);
+ i += SIMD_CHUNK_SIZE;
+ }
+ while i + 16 <= len {
+ let c = vld1q_u8(src_u8.add(i));
+ vst1q_u8(dest_u8.add(i), c);
+ i += 16;
+ }
+ }
+
+ if i < len {
+ std::ptr::copy_nonoverlapping(src_u8.add(i), dest_u8.add(i), len - i);
+ }
+
+ String::from_utf16(&units).unwrap_or_else(|_| String::new())
+ }
+
+ let slice = &reader.bf[reader.cursor..reader.cursor + len];
+ #[cfg(target_arch = "x86_64")]
+ {
+ if std::arch::is_x86_feature_detected!("avx2") {
+ let s = unsafe { simd_impl(slice) };
+ reader.move_next(len);
+ return s;
+ }
+ }
+ #[cfg(any(target_arch = "x86", target_arch = "x86_64"))]
+ {
+ if std::arch::is_x86_feature_detected!("sse2") {
+ let s = unsafe { simd_impl(slice) };
+ reader.move_next(len);
+ return s;
+ }
+ }
+ #[cfg(target_arch = "aarch64")]
+ {
+ if std::arch::is_aarch64_feature_detected!("neon") {
+ let s = unsafe { simd_impl(slice) };
+ reader.move_next(len);
+ return s;
+ }
+ }
+
+ // ---- fallback ----
+ read_utf16_standard(reader, len)
+ }
+
+ #[cfg(test)]
+ mod tests {
+ use super::*;
+ use crate::buffer::{Reader, Writer};
+
+ #[test]
+ fn test_latin1() {
+ let samples = [
+ "Hello World!",
+ "Rusty Café",
+ "1234567890",
+ "ÀÁÂÃÄÅÆÇÈÉÊËÌÍÎÏÐÑÒÓÔÕÖ×ØÙÚÛÜÝ",
+ ];
+
+ for s in samples {
+ let mut writer = Writer::default();
+ write_latin1_simd(&mut writer, s);
+ write_latin1_simd(&mut writer, s);
+ let bytes = &*writer.dump();
+ let bytes_len = bytes.len() / 2;
+ let mut reader = Reader::new(bytes);
+ assert_eq!(read_latin1_standard(&mut reader, bytes_len), s);
+ assert_eq!(read_latin1_standard(&mut reader, bytes_len), s);
+
+ let mut writer = Writer::default();
+ write_latin1_standard(&mut writer, s);
+ write_latin1_standard(&mut writer, s);
+ let bytes = &*writer.dump();
+ let bytes_len = bytes.len() / 2;
+ let mut reader = Reader::new(bytes);
+ assert_eq!(read_latin1_simd(&mut reader, bytes_len), s);
+ assert_eq!(read_latin1_simd(&mut reader, bytes_len), s);
+ }
+ }
+
+ #[test]
+ fn test_utf8() {
+ let samples = [
+ "hello",
+ "rust语言",
+ "你好，世界",
+ "emoji 😀😃😄😁",
+ "mixed ASCII + 中文 + emoji 😁",
+ ];
+
+ for s in samples {
+ let bytes_len = s.len();
+
+ let mut writer = Writer::default();
+ write_utf8_standard(&mut writer, s);
+ write_utf8_standard(&mut writer, s);
+ let bytes = &*writer.dump();
+ let mut reader = Reader::new(bytes);
+ assert_eq!(read_utf8_simd(&mut reader, bytes_len), s);
+ assert_eq!(read_utf8_simd(&mut reader, bytes_len), s);
+
+ let mut writer = Writer::default();
+ write_utf8_simd(&mut writer, s);
+ write_utf8_simd(&mut writer, s);
+ let bytes = &*writer.dump();
+ let mut reader = Reader::new(bytes);
+ assert_eq!(read_utf8_standard(&mut reader, bytes_len), s);
+ assert_eq!(read_utf8_standard(&mut reader, bytes_len), s);
+ }
+ }
+
+ #[test]
+ fn test_utf16() {
+ let samples = [
+ "hello",
+ "rust语言",
+ "你好，世界",
+ "emoji 😀😃😄😁",
+ "混合文字 + emoji 🐍💻🦀",
+ ];
+ for s in samples {
+ let utf16: Vec<u16> = s.encode_utf16().collect();
+ let bytes_len = utf16.len() * 2;
+
+ let mut writer = Writer::default();
+ write_utf16_standard(&mut writer, &utf16);
+ write_utf16_standard(&mut writer, &utf16);
+ let bytes = &*writer.dump();
+ let mut reader = Reader::new(bytes);
+ assert_eq!(read_utf16_simd(&mut reader, bytes_len), s);
+ assert_eq!(read_utf16_simd(&mut reader, bytes_len), s);
+
+ let mut writer = Writer::default();
+ write_utf16_simd(&mut writer, &utf16);
+ write_utf16_simd(&mut writer, &utf16);
+ let bytes = &*writer.dump();
+ let mut reader = Reader::new(bytes);
+ assert_eq!(read_utf16_standard(&mut reader, bytes_len), s);
+ assert_eq!(read_utf16_standard(&mut reader, bytes_len), s);
+ }
+ }
+ }
+}
diff --git a/rust/fory-core/src/serializer/string.rs b/rust/fory-core/src/serializer/string.rs
index b82811079..1ffa47328 100644
--- a/rust/fory-core/src/serializer/string.rs
+++ b/rust/fory-core/src/serializer/string.rs
@@ -47,16 +47,7 @@ impl Serializer for String {
let utf16: Vec<u16> = self.encode_utf16().collect();
let bitor = (utf16.len() as u64 * 2) << 2 | StrEncoding::Utf16 as u64;
context.writer.write_varuint36_small(bitor);
- for unit in utf16 {
- #[cfg(target_endian = "little")]
- {
- context.writer.write_u16(unit);
- }
- #[cfg(target_endian = "big")]
- {
- unimplemented!()
- }
- }
+ context.writer.write_utf16_bytes(&utf16);
}
}
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org
For additional commands, e-mail: commits-help@fory.apache.org