You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@felix.apache.org by pd...@apache.org on 2014/07/17 10:51:45 UTC

svn commit: r1611293 - /felix/sandbox/pderop/dependencymanager-prototype/org.apache.felix.dm.benchmark/README

Author: pderop
Date: Thu Jul 17 08:51:45 2014
New Revision: 1611293

URL: http://svn.apache.org/r1611293
Log:
fixed README

Modified:
    felix/sandbox/pderop/dependencymanager-prototype/org.apache.felix.dm.benchmark/README

Modified: felix/sandbox/pderop/dependencymanager-prototype/org.apache.felix.dm.benchmark/README
URL: http://svn.apache.org/viewvc/felix/sandbox/pderop/dependencymanager-prototype/org.apache.felix.dm.benchmark/README?rev=1611293&r1=1611292&r2=1611293&view=diff
==============================================================================
--- felix/sandbox/pderop/dependencymanager-prototype/org.apache.felix.dm.benchmark/README (original)
+++ felix/sandbox/pderop/dependencymanager-prototype/org.apache.felix.dm.benchmark/README Thu Jul 17 08:51:45 2014
@@ -11,7 +11,7 @@ adjust the project specific setting and 
 How to launch the stress test under bndtools:
 ============================================
 
-The stress test performs two kind of tests on different Felix DI framework (currently, we support Felix DM4, Felix SCR, Felix iPOJO):
+The stress test performs two kind of tests on different Felix DI framework (currently, we support Felix DM4, Felix SCR):
 
 1) first kind of test: starts/stops several times each tested bundles. When started, the test bundle is expected to register/unregister several services.
 And no processing is done at all in each component activation/deactivation methods (start/stop methods).
@@ -30,32 +30,43 @@ g! 
 	+++++ Starting benchmarks without processing done in components start/stop methods.
 
 Benchmarking bundle:
-  org.apache.felix.dm.benchmark.dependencymanager:..............................
-  results=165645599,240534693,250335703,276343059,458505648
+  org.apache.felix.dm.benchmark.dependencymanager:....................................................................................................
+  results=138362237,182590925,238169860,365712274,8246024452
 
 Benchmarking bundle:
-  org.apache.felix.dm.benchmark.scr:..............................
-  results=193493527,203784076,257764377,365117336,469242290
+  org.apache.felix.dm.benchmark.dependencymanager.parallel:....................................................................................................
+  results=58007254,113886816,161644014,248809313,964751449
 
 Benchmarking bundle:
-  org.apache.felix.dm.benchmark.ipojo:..............................
-  results=3996621923,5619860774,10963277969,19170373702,27043085149
+  org.apache.felix.dm.benchmark.scr:....................................................................................................
+  results=181468797,199847561,240200634,318892594,723789524
 
 	+++++ Starting benchmarks with processing done in components start/stop methods.
 
 Benchmarking bundle:
   org.apache.felix.dm.benchmark.dependencymanager:.....
-  results=3228612945,3230318727,3244609156,3272021057,3275854755
+  results=3203962830,3204232653,3205555311,3220331851,3319273387
 
 Benchmarking bundle:
-  org.apache.felix.dm.benchmark.scr:.....
-  results=3293150735,3293724043,3297981700,3325328872,3347331154
+  org.apache.felix.dm.benchmark.dependencymanager.parallel:.....
+  results=820258470,825405394,862318292,885332987,890429938
 
 Benchmarking bundle:
-  org.apache.felix.dm.benchmark.ipojo:.....
-  results=29970616791,30866086557,31311315987,32831635310,33242710589
+  org.apache.felix.dm.benchmark.scr:.....
+  results=3257080433,3260117641,3271632555,3308487828,3330361529
 -----------------------------------------------------------------------------------------------------------------
 
+Test bundles
+============
+
+- org.apache.felix.dm.benchmark.dependencymanager: components are declared using dependency manager API to run the scenario.
+
+- org.apache.felix.dm.benchmark.dependencymanager.parallel: uses parallel dependency manager where components 
+dependency management and components activation/deactivation processing is performed concurrently, using a fixed
+thread pool.
+
+- org.apache.felix.dm.benchmark.scr: components are declared using declarative service.
+
 How to interpret results:
 ========================
 
@@ -65,12 +76,14 @@ for example: 
   results=165645599,240534693,250335703,276343059,458505648
 
 Here is how to interpret the results: when testing a bundle, the benchmark controller starts/stops it many times, then the 
-elapsed time consumed to 1- start the bundle; 2- activate all services; 3- deactivate all services; 4- stop the bundle is recorded in a list.
+elapsed time used to start the bundle, activate/deactivate all services, and stop the bundle is recorded in a list.
 Then this list is sorted: the first entry is the fastest execution time, the last entry is the slowest. the middle one is the
 average.
-We display the first entry (fastest), the entry at 1/4 of the list, the middle of the list (average), the entry at 3/4 of the list, 
-and the last entry (slowest time)
+We display the first entry (fastest), the entry at 1/4 of the list, the middle of the list, the entry at 3/4 of the list, 
+and the last entry (slowest time).
 
+We don't do an average, because usually, when running benchmark, some measurements does not reflect reality, especially, when there is a full GC or when the JVM is warming up.
+(we actually do the same as in Java Chronicle: https://github.com/peter-lawrey/Java-Chronicle).
 
 Stress test scenario description
 --------------------------------
@@ -82,33 +95,33 @@ We have the following services: 
 "Album" service: is a single release of musics, comprising several music Tracks. One Album depends on several Track services.
 "Track" service: A piece of music.
 
+By default, a scenario consists in registering 5 artists, each artist having 10 albums, and each album having 5 music tracks (~ 300 services).
+
 The scenario is implemented in the following bundles
 
 - org.apache.felix.dm.benchmark.scenario: defines the interfaces.
 - org.apache.felix.dm.benchmark.scenario.impl: defines the basic implementations for the services.
 
 The Scenario Controller (see org.apache.felix.dm.benchmark.scenario/org.apache.felix.dm.benchmark.scenario.impl) is in charge of starting/stopping many times
-some DI specific bundles (DM4, SCR, iPOJO), which are expected to create all expected Artist/Album/Track services.
+some DI specific bundles (DM4, SCR), which are expected to create all expected Artist/Album/Track services.
 
 By default, when a tested bundle is started, it will create several Artists (see Artists.ARTISTS constant). each Artist depends on several Albums (see Artists.ALBUMS constant),
 and each Album depends on several music Tracks (see Artists.TRACKS constants).
 
-Here is the list of bundles which are testing dependency manager, scr, or iPojo:
+Here is the list of bundles which are testing dependency manager, scr:
 
 - org.apache.felix.dm.benchmark.dependencymanager: 
-It contains a simple activator, which creates the various services.
+It contains a simple activator, which creates the various services using dependency manager API.
 
 - org.apache.felix.dm.benchmark.dependencymanager.parallel: 
-same as above, but using parallel dependency manager (Warning: This is a new feature, not yet committed, still under discussion with Marcel and Xander).
+same as above, but using parallel dependency manager (Warning: we still have to find a way to inject the thread pool in dependency manager without having
+to modify application activators).
 
 - org.apache.felix.dm.benchmark.scr: 
 contains a DS xml descriptor for declaring the Artist/Album/Track services. If you change the constants values of Artists.ARTISTS/Artists.ALBUMS/Artists.TRACKS in order to
 support more or less services, then you have to regenerate the xml descriptor in ./src/org/apache/felix/dm/benchmark/scr/scr.xml.
 To do so, you can modify the constants (in the Artists interface), then you have to run under eclipse the src/org/apache/felix/dm/benchmark/scr/XMLGenerator.java class.
-It will generate in /tmp/scr.xml the new xml descriptor, which you have to copy to ./src/org/apache/felix/dm/benchmark/scr/scr.xml.
-
-- org.apache.felix.dm.benchmark.ipojo:
-contains an iPojo Activator, using the iPojo API for declaring all the services.
+It will generate "/tmp/scr.xml" which you have to copy to ./src/org/apache/felix/dm/benchmark/scr/scr.xml.
 
 The org.apache.felix.dm.benchmark.controller bundle, when starting, first stops all tested bundles.
 Then for each one, it performs the following test (multiple times):
@@ -124,6 +137,3 @@ All the elapsed time (nanoseconds) used 
 When enough iterations are done, the list is sorted (that is : the first entry in the list corresponds to the fastest execution time, and the
 last entry corresponds to the slowest execution time).
 Then, we display some meaningful entries in the list (like the first entry, the entry in the middle of the list (average), and the last entry (slowest).
-
-We don't do an average, because usually, when running benchmark, some measurements does not reflect reality, especially, when there is a full GC or when the JVM is warming up.
-(we actually do the same as in Java Chronicle: https://github.com/peter-lawrey/Java-Chronicle).
\ No newline at end of file