You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@fury.apache.org by "chaokunyang (via GitHub)" <gi...@apache.org> on 2025/01/26 05:04:38 UTC
[PR] feat(java): new implementation for chunk based map serialization [fury]
chaokunyang opened a new pull request, #2025:
URL: https://github.com/apache/fury/pull/2025
## What does this PR do?
This pr provides a new implementation for chunk based map serialization
## Related issues
#1722
#925
## Does this PR introduce any user-facing change?
<!--
If any user-facing interface changes, please [open an issue](https://github.com/apache/fury/issues/new/choose) describing the need to do so and update the document if necessary.
-->
- [ ] Does this PR introduce any public API change?
- [ ] Does this PR introduce any binary protocol compatibility change?
## Benchmark
<!--
When the PR has an impact on performance (if you don't know whether the PR will have an impact on performance, you can submit the PR first, and if it will have impact on performance, the code reviewer will explain it), be sure to attach a benchmark data here.
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@fury.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fury.apache.org
For additional commands, e-mail: commits-help@fury.apache.org
Re: [PR] feat(java): new implementation and protocol refine for chunk based map serialization [fury]
Posted by "chaokunyang (via GitHub)" <gi...@apache.org>.
chaokunyang commented on PR #2025:
URL: https://github.com/apache/fury/pull/2025#issuecomment-2614292874
plot script:
```python
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import io
data = """
"Benchmark","Mode","Threads","Samples","Score","Score Error","Unit","enableChunkEncoding","mapSize"
"org.apache.fury.benchmark.MapSerializationSuite.deserializeIntMap","thrpt",1,9,4466051.463287,873291.405075,"ops/s",false,5
"org.apache.fury.benchmark.MapSerializationSuite.deserializeIntMap","thrpt",1,9,1350641.974999,81715.924297,"ops/s",false,20
"org.apache.fury.benchmark.MapSerializationSuite.deserializeIntMap","thrpt",1,9,497254.149014,68862.874865,"ops/s",false,50
"org.apache.fury.benchmark.MapSerializationSuite.deserializeIntMap","thrpt",1,9,266334.668251,35573.243688,"ops/s",false,100
"org.apache.fury.benchmark.MapSerializationSuite.deserializeIntMap","thrpt",1,9,122097.002001,27652.497466,"ops/s",false,200
"org.apache.fury.benchmark.MapSerializationSuite.deserializeIntMap","thrpt",1,9,5888333.717480,739656.066973,"ops/s",true,5
"org.apache.fury.benchmark.MapSerializationSuite.deserializeIntMap","thrpt",1,9,2036689.629240,254260.659777,"ops/s",true,20
"org.apache.fury.benchmark.MapSerializationSuite.deserializeIntMap","thrpt",1,9,734567.196053,49056.877252,"ops/s",true,50
"org.apache.fury.benchmark.MapSerializationSuite.deserializeIntMap","thrpt",1,9,367623.736912,59614.675972,"ops/s",true,100
"org.apache.fury.benchmark.MapSerializationSuite.deserializeIntMap","thrpt",1,9,170496.570903,32886.215740,"ops/s",true,200
"org.apache.fury.benchmark.MapSerializationSuite.serializeIntMap","thrpt",1,9,7893446.540629,808837.061250,"ops/s",false,5
"org.apache.fury.benchmark.MapSerializationSuite.serializeIntMap","thrpt",1,9,2584359.305934,129956.799737,"ops/s",false,20
"org.apache.fury.benchmark.MapSerializationSuite.serializeIntMap","thrpt",1,9,1036138.030458,84703.530824,"ops/s",false,50
"org.apache.fury.benchmark.MapSerializationSuite.serializeIntMap","thrpt",1,9,394205.916586,25123.147428,"ops/s",false,100
"org.apache.fury.benchmark.MapSerializationSuite.serializeIntMap","thrpt",1,9,206653.480783,30347.851656,"ops/s",false,200
"org.apache.fury.benchmark.MapSerializationSuite.serializeIntMap","thrpt",1,9,7337531.043356,437811.548408,"ops/s",true,5
"org.apache.fury.benchmark.MapSerializationSuite.serializeIntMap","thrpt",1,9,2474583.537284,379139.624785,"ops/s",true,20
"org.apache.fury.benchmark.MapSerializationSuite.serializeIntMap","thrpt",1,9,806053.192966,112383.449082,"ops/s",true,50
"org.apache.fury.benchmark.MapSerializationSuite.serializeIntMap","thrpt",1,9,434183.111785,64625.641813,"ops/s",true,100
"org.apache.fury.benchmark.MapSerializationSuite.serializeIntMap","thrpt",1,9,208393.763140,22779.649134,"ops/s",true,200
"""
# Read the data into a DataFrame
df = pd.read_csv(io.StringIO(data))
# Function to plot the data for each benchmark type
def plot_benchmark(dataframe, benchmark_name):
# Filter the data for the given benchmark name
df_benchmark = dataframe[dataframe['Benchmark'].str.contains(benchmark_name)]
# Set up the matplotlib figure
plt.figure(figsize=(10, 6))
# Plot the scores with and without Chunk Encoding
sns.barplot(data=df_benchmark, x='mapSize', y='Score', hue='enableChunkEncoding')
# Add title and labels
plt.title(f'{benchmark_name} performance')
plt.xlabel('Map Size')
plt.ylabel('Score (ops/s)')
# Show the plot
plt.legend(title='Chunk Encoding')
plt.show()
# Plot for each benchmark type
benchmark_types = [
"deserializeIntMap",
"deserializeStringMap",
"serializeIntMap",
"serializeStringMap"
]
for benchmark in benchmark_types:
plot_benchmark(df, benchmark)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@fury.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fury.apache.org
For additional commands, e-mail: commits-help@fury.apache.org
Re: [PR] feat(java): new implementation and protocol refine for chunk based map serialization [fury]
Posted by "pandalee99 (via GitHub)" <gi...@apache.org>.
pandalee99 merged PR #2025:
URL: https://github.com/apache/fury/pull/2025
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@fury.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fury.apache.org
For additional commands, e-mail: commits-help@fury.apache.org
Re: [PR] feat(java): new implementation and protocol refine for chunk based map serialization [fury]
Posted by "chaokunyang (via GitHub)" <gi...@apache.org>.
chaokunyang commented on PR #2025:
URL: https://github.com/apache/fury/pull/2025#issuecomment-2614296872
@theweipeng This pr udpates teh xlang map serialization protocol, the furyjs may need to udpate the implementation too.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@fury.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fury.apache.org
For additional commands, e-mail: commits-help@fury.apache.org