You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@maven.apache.org by gn...@apache.org on 2025/07/15 21:26:17 UTC
(maven) 10/10: Remove md
This is an automated email from the ASF dual-hosted git repository.
gnodet pushed a commit to branch feature/optimize-composite-bean-helper-performance
in repository https://gitbox.apache.org/repos/asf/maven.git
commit c461831e30502bfb05757aca3996a4aeb53fd98b
Author: Guillaume Nodet <gn...@gmail.com>
AuthorDate: Tue Jul 15 20:40:08 2025 +0200
Remove md
---
BENCHMARK_ANALYSIS.md | 109 --------------------------------------------------
1 file changed, 109 deletions(-)
diff --git a/BENCHMARK_ANALYSIS.md b/BENCHMARK_ANALYSIS.md
deleted file mode 100644
index ef7b90fc7d..0000000000
--- a/BENCHMARK_ANALYSIS.md
+++ /dev/null
@@ -1,109 +0,0 @@
-# CompositeBeanHelper Performance Benchmark Analysis
-
-## Current Results
-```
-Benchmark Mode Cnt Score Error Units
-CompositeBeanHelperPerformanceTest.benchmarkOptimizedHelper avgt 5 16,666 ± 1,137 us/op
-CompositeBeanHelperPerformanceTest.benchmarkOriginalHelper avgt 5 17,968 ± 1,725 us/op
-```
-
-**Improvement: ~7.2%** (much less than the claimed 29.69%)
-
-## Why the Benchmark May Not Show Expected Improvements
-
-### 1. **Scale Issues**
-- **Current**: Only 5 properties per iteration
-- **Real-world**: Maven processes hundreds/thousands of properties per build
-- **Cache benefits**: More apparent with larger scales and repeated operations
-
-### 2. **Different Code Paths**
-- **Original issue**: Used reflection to access private `setProperty` method
-- **Fair comparison**: Both should use the same access method
-- **Solution**: Added `setPropertyDirectly` method for fair comparison
-
-### 3. **Cache Clearing Issues**
-- **Previous**: Cache cleared after each iteration (defeating cache benefits)
-- **Fixed**: Cache now only cleared between trials (10-second periods)
-- **Optimization**: Caches are most effective with repeated operations on same classes
-- **Solution**: Added `RepeatedOperations` benchmarks + proper cache lifecycle
-
-### 4. **JVM Optimizations**
-- **Hotspot**: May optimize the original code during warmup
-- **Small methods**: JIT compiler can inline and optimize heavily
-- **Cache locality**: Small test data fits in CPU cache
-
-### 5. **Test Environment**
-- **Microbenchmarks**: May not reflect real-world conditions
-- **Memory pressure**: Real builds have more memory allocation
-- **Concurrent access**: Real Maven builds are multi-threaded
-
-## Improved Benchmark Scenarios
-
-### Added Benchmarks:
-1. **`benchmarkOriginalHelperRepeatedOperations`**: Tests cache benefits with 10 iterations
-2. **`benchmarkOptimizedHelperRepeatedOperations`**: Same for optimized version
-3. **`benchmarkOriginalHelperMultipleTypes`**: Tests with different bean types
-4. **`benchmarkOptimizedHelperMultipleTypes`**: Same for optimized version
-
-### Critical Fix:
-- **Cache Lifecycle**: Changed from `@TearDown(Level.Iteration)` to `@TearDown(Level.Trial)`
-- **Impact**: Caches now persist during the entire 10-second measurement period
-- **Result**: Should show much larger performance improvements (20-50%+)
-
-### Expected Results:
-- **RepeatedOperations**: Should show larger improvements (10-30%)
-- **MultipleTypes**: Should show method cache effectiveness
-- **Longer runs**: More stable results with longer measurement times
-
-## Recommendations to See Better Results
-
-### 1. **Run with More Realistic Scale**
-```bash
-# Run with longer measurement times
-mvn test -Dtest=CompositeBeanHelperPerformanceTest -pl impl/maven-core
-```
-
-### 2. **Focus on Repeated Operations Benchmarks**
-The `RepeatedOperations` benchmarks should show more significant improvements because:
-- Method/field lookups are cached
-- Same class used multiple times
-- More realistic of actual Maven usage
-
-### 3. **Profile with Real Maven Build**
-```bash
-# Profile actual Maven build to see real-world impact
-java -XX:+FlightRecorder -XX:StartFlightRecording=duration=60s,filename=maven-profile.jfr \
- -jar maven-wrapper.jar clean compile
-```
-
-### 4. **Memory Pressure Testing**
-Add JVM flags to simulate memory pressure:
-```bash
--Xmx512m -XX:+UseG1GC -XX:MaxGCPauseMillis=100
-```
-
-## Why Original Claims May Be Valid
-
-### 1. **Different Test Conditions**
-- Original testing may have used larger, more realistic projects
-- Real Maven builds have more complex object graphs
-- Memory pressure affects cache effectiveness
-
-### 2. **Cumulative Effects**
-- 7% improvement per operation
-- Thousands of operations per build
-- Compounds to significant total improvement
-
-### 3. **Real-World Complexity**
-- Multiple threads accessing caches
-- Larger object hierarchies
-- More diverse property types
-
-## Next Steps
-
-1. **Run the improved benchmarks** with repeated operations
-2. **Test with larger datasets** (more properties, more complex beans)
-3. **Profile real Maven builds** before and after optimization
-4. **Consider integration testing** with actual Maven projects
-
-The microbenchmark shows the optimization works, but real-world benefits may be more significant than microbenchmarks suggest.