You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@jackrabbit.apache.org by De...@signal-iduna.de on 2013/01/18 15:45:27 UTC
bad performance in a clustered environment under heavy load
Hello all,
we experience a problem with jackrabbit (Version 2.2.5) in a clustered
environment.
The Situation:
We are running a webaplication in a clustered environment (12 nodes,
WebSphere Platform 7.0.0.25) and are using the deployment model "Shared
J2EE Resource".
So in fact we have a jackrabbit cluster with 12 nodes.
We use a DB2 UDB as persistent storage with the following configuration:
<DataSources>
<DataSource name="ds1">
<param name="driver" value="com.ibm.db2.jcc.DB2Driver"/>
<param name="url" value="jdbc:db2://xxxxxxxxxxxxxxxxx"/>
<param name="user" value="xxxxxxxxxxxxx"/>
<param name="password" value="xxxxxx"/>
<param name="databaseType" value="db2"/>
<param name="maxPoolSize" value="80"/>
</DataSource>
</DataSources>
<FileSystem class="org.apache.jackrabbit.core.fs.db.DbFileSystem">
<param name="dataSourceName" value="ds1"/>
<param name="schemaObjectPrefix" value="J_R_FS_"/>
</FileSystem>
<DataStore class="org.apache.jackrabbit.core.data.db.DbDataStore">
<param name="dataSourceName" value="ds1"/>
<param name="schemaObjectPrefix" value="J_R_"/>
<param name="minRecordLength" value="1024"/>
<param name="maxConnections" value="3"/>
<param name="copyWhenReading" value="true"/>
<param name="tablePrefix" value=""/>
</DataStore>
<Workspaces rootPath="${rep.home}/workspaces" defaultWorkspace="blubb"
/>
<Workspace name="${wsp.name}">
<FileSystem class=
"org.apache.jackrabbit.core.fs.local.LocalFileSystem">
<param name="path" value="${wsp.home}"/>
</FileSystem>
<PersistenceManager class=
"org.apache.jackrabbit.core.persistence.pool.BundleDbPersistenceManager">
<param name="dataSourceName" value="ds1"/>
<param name="schemaObjectPrefix" value="J_PM_${wsp.name}_"/>
<param name="databaseType" value="db2"/>
<param name="bundleCacheSize" value="8"/>
<param name="consistencyCheck" value="false"/>
<param name="consistencyFix" value="false"/>
<param name="minBlobSize" value="4096"/>
<param name="errorHandling" value=""/>
<param name="blockOnConnectionLoss" value="false"/>
<param name="schemaCheckEnabled" value="true"/>
</PersistenceManager>
</Workspace>
<Versioning rootPath="${rep.home}/version">
<FileSystem class=
"org.apache.jackrabbit.core.fs.local.LocalFileSystem">
<param name="path" value="${rep.home}/version"/>
</FileSystem>
<PersistenceManager class=
"org.apache.jackrabbit.core.persistence.pool.BundleDbPersistenceManager">
<param name="dataSourceName" value="ds1"/>
<param name="schemaObjectPrefix" value="J_V_PM_"/>
<param name="databaseType" value="db2"/>
<param name="bundleCacheSize" value="8"/>
<param name="consistencyCheck" value="false"/>
<param name="consistencyFix" value="false"/>
<param name="minBlobSize" value="4096"/>
<param name="errorHandling" value=""/>
<param name="blockOnConnectionLoss" value="false"/>
<param name="schemaCheckEnabled" value="true"/>
</PersistenceManager>
</Versioning>
<Cluster syncDelay="5">
<Journal class=
"org.apache.jackrabbit.core.journal.DatabaseJournal">
<param name="dataSourceName" value="ds1"/>
<param name="revision" value="${rep.home}/revision.log"/>
</Journal>
</Cluster>
The clusterId is provided as system.property.
As soon as we provide heavy load on some (max. 4) Jackrabbit-Nodes the
Perfomance drops.
It looks like too many locks on tables GLOBAL_REVISION and/or
LOCAL_REVISIONS.
The first action we tried is to increase the (very low) value 5 (ms) for
the syncDelay up to 800ms (default is 5000ms).
The next action will be to update to the jackrabbit-revision 2.2.13.
Any advice the community can provide will be greatly appreciated.
If you need more detailed Information, please let me know, i will try to
provide it.
Also important:
We would also like to consult professional support for our problem.
Can you please give advice where we can find professional support for
Jackrabbit to get this running?
Best regards,
Detlev Zühlke
SIGNAL IDUNA Gruppe
AES5-97350
Neue Rabenstraße 15-19
20354 Hamburg
Tel.: (040) 4124-2954
Fax: (040) 4124-4902954
email: detlev.zuehlke@signal-iduna.de
SIGNAL Krankenversicherung a. G., Sitz: Dortmund, HR B 2405, AG Dortmund
IDUNA Vereinigte Lebensversicherung aG für Handwerk, Handel und Gewerbe,
Sitz: Hamburg, HR B 2740, AG Hamburg
Deutscher Ring Krankenversicherungsverein a.G., Sitz: Hamburg,
HR B 4673, AG Hamburg,
SIGNAL IDUNA Allgemeine Versicherung AG, Sitz: Dortmund, HR B 19108,
AG Dortmund
Vorstände: Reinhold Schulte (Vorsitzender),
Dr. Karl-Josef Bierth, Marlies Hirschberg-Tafel,
Michael Johnigk, Ulrich Leitermann, Michael Petmecky,
Dr. Klaus Sticker, Prof. Dr. Markus Warg
Vorsitzender der Aufsichtsräte: Günter Kutz
SIGNAL IDUNA Gruppe Hauptverwaltungen, Internet: www.signal-iduna.de
44121 Dortmund, Hausanschrift: Joseph-Scherer-Str. 3, 44139 Dortmund
20351 Hamburg, Hausanschrift: Neue Rabenstraße 15-19, 20354 Hamburg
Re: bad performance in a clustered environment under heavy load
Posted by Morrell Jacobs <mj...@maned.com>.
Hello Detlev,
We recently experienced problems with JackRabbit performance. Our problems were slightly different in that they appeared both clustered and non-clustered environments, but one problem we found was a spike in garbage collection times. I found this article was recommended and I found it to be very helpful:
http://www.oracle.com/technetwork/java/javase/gc-tuning-6-140523.html#par_gc.ergonomics
It sounds like you've isolated the problems to the database, but I wanted to offer the insight from my recent experience.
Good Luck,
Morrell
On Jan 18, 2013, at 9:45 AM, <De...@signal-iduna.de>>
<De...@signal-iduna.de>> wrote:
Hello all,
we experience a problem with jackrabbit (Version 2.2.5) in a clustered
environment.
The Situation:
We are running a webaplication in a clustered environment (12 nodes,
WebSphere Platform 7.0.0.25) and are using the deployment model "Shared
J2EE Resource".
So in fact we have a jackrabbit cluster with 12 nodes.
We use a DB2 UDB as persistent storage with the following configuration:
<DataSources>
<DataSource name="ds1">
<param name="driver" value="com.ibm.db2.jcc.DB2Driver"/>
<param name="url" value="jdbc:db2://xxxxxxxxxxxxxxxxx"/>
<param name="user" value="xxxxxxxxxxxxx"/>
<param name="password" value="xxxxxx"/>
<param name="databaseType" value="db2"/>
<param name="maxPoolSize" value="80"/>
</DataSource>
</DataSources>
<FileSystem class="org.apache.jackrabbit.core.fs.db.DbFileSystem">
<param name="dataSourceName" value="ds1"/>
<param name="schemaObjectPrefix" value="J_R_FS_"/>
</FileSystem>
<DataStore class="org.apache.jackrabbit.core.data.db.DbDataStore">
<param name="dataSourceName" value="ds1"/>
<param name="schemaObjectPrefix" value="J_R_"/>
<param name="minRecordLength" value="1024"/>
<param name="maxConnections" value="3"/>
<param name="copyWhenReading" value="true"/>
<param name="tablePrefix" value=""/>
</DataStore>
<Workspaces rootPath="${rep.home}/workspaces" defaultWorkspace="blubb"
/>
<Workspace name="${wsp.name}">
<FileSystem class=
"org.apache.jackrabbit.core.fs.local.LocalFileSystem">
<param name="path" value="${wsp.home}"/>
</FileSystem>
<PersistenceManager class=
"org.apache.jackrabbit.core.persistence.pool.BundleDbPersistenceManager">
<param name="dataSourceName" value="ds1"/>
<param name="schemaObjectPrefix" value="J_PM_${wsp.name}_"/>
<param name="databaseType" value="db2"/>
<param name="bundleCacheSize" value="8"/>
<param name="consistencyCheck" value="false"/>
<param name="consistencyFix" value="false"/>
<param name="minBlobSize" value="4096"/>
<param name="errorHandling" value=""/>
<param name="blockOnConnectionLoss" value="false"/>
<param name="schemaCheckEnabled" value="true"/>
</PersistenceManager>
</Workspace>
<Versioning rootPath="${rep.home}/version">
<FileSystem class=
"org.apache.jackrabbit.core.fs.local.LocalFileSystem">
<param name="path" value="${rep.home}/version"/>
</FileSystem>
<PersistenceManager class=
"org.apache.jackrabbit.core.persistence.pool.BundleDbPersistenceManager">
<param name="dataSourceName" value="ds1"/>
<param name="schemaObjectPrefix" value="J_V_PM_"/>
<param name="databaseType" value="db2"/>
<param name="bundleCacheSize" value="8"/>
<param name="consistencyCheck" value="false"/>
<param name="consistencyFix" value="false"/>
<param name="minBlobSize" value="4096"/>
<param name="errorHandling" value=""/>
<param name="blockOnConnectionLoss" value="false"/>
<param name="schemaCheckEnabled" value="true"/>
</PersistenceManager>
</Versioning>
<Cluster syncDelay="5">
<Journal class=
"org.apache.jackrabbit.core.journal.DatabaseJournal">
<param name="dataSourceName" value="ds1"/>
<param name="revision" value="${rep.home}/revision.log"/>
</Journal>
</Cluster>
The clusterId is provided as system.property.
As soon as we provide heavy load on some (max. 4) Jackrabbit-Nodes the
Perfomance drops.
It looks like too many locks on tables GLOBAL_REVISION and/or
LOCAL_REVISIONS.
The first action we tried is to increase the (very low) value 5 (ms) for
the syncDelay up to 800ms (default is 5000ms).
The next action will be to update to the jackrabbit-revision 2.2.13.
Any advice the community can provide will be greatly appreciated.
If you need more detailed Information, please let me know, i will try to
provide it.
Also important:
We would also like to consult professional support for our problem.
Can you please give advice where we can find professional support for
Jackrabbit to get this running?
Best regards,
Detlev Zühlke
SIGNAL IDUNA Gruppe
AES5-97350
Neue Rabenstraße 15-19
20354 Hamburg
Tel.: (040) 4124-2954
Fax: (040) 4124-4902954
email: detlev.zuehlke@signal-iduna.de<ma...@signal-iduna.de>
SIGNAL Krankenversicherung a. G., Sitz: Dortmund, HR B 2405, AG Dortmund
IDUNA Vereinigte Lebensversicherung aG für Handwerk, Handel und Gewerbe,
Sitz: Hamburg, HR B 2740, AG Hamburg
Deutscher Ring Krankenversicherungsverein a.G., Sitz: Hamburg,
HR B 4673, AG Hamburg,
SIGNAL IDUNA Allgemeine Versicherung AG, Sitz: Dortmund, HR B 19108,
AG Dortmund
Vorstände: Reinhold Schulte (Vorsitzender),
Dr. Karl-Josef Bierth, Marlies Hirschberg-Tafel,
Michael Johnigk, Ulrich Leitermann, Michael Petmecky,
Dr. Klaus Sticker, Prof. Dr. Markus Warg
Vorsitzender der Aufsichtsräte: Günter Kutz
SIGNAL IDUNA Gruppe Hauptverwaltungen, Internet: www.signal-iduna.de<http://www.signal-iduna.de>
44121 Dortmund, Hausanschrift: Joseph-Scherer-Str. 3, 44139 Dortmund
20351 Hamburg, Hausanschrift: Neue Rabenstraße 15-19, 20354 Hamburg
Antwort: bad performance in a clustered environment under heavy load
Posted by De...@signal-iduna.de.
Hello all,
our database admins fount a bottleneck and tried to remove it. They
discovered that the database serves was forced to create the files for the
transaction log too frequently and increased the maximum size of these
files.
The performance is much better after this.
Mit freundlichen Grüßen
Detlev Zühlke
SIGNAL IDUNA Gruppe
AES5-97350
Neue Rabenstraße 15-19
20354 Hamburg
Tel.: (040) 4124-2954
Fax: (040) 4124-4902954
email: detlev.zuehlke@signal-iduna.de
Detlev.Zuehlke@signal-iduna.de
18.01.2013 15:46
Bitte antworten an
users@jackrabbit.apache.org
An
users@jackrabbit.apache.org
Kopie
Thema
bad performance in a clustered environment under heavy load
Hello all,
we experience a problem with jackrabbit (Version 2.2.5) in a clustered
environment.
The Situation:
We are running a webaplication in a clustered environment (12 nodes,
WebSphere Platform 7.0.0.25) and are using the deployment model "Shared
J2EE Resource".
So in fact we have a jackrabbit cluster with 12 nodes.
We use a DB2 UDB as persistent storage with the following configuration:
<DataSources>
<DataSource name="ds1">
<param name="driver" value="com.ibm.db2.jcc.DB2Driver"/>
<param name="url" value="jdbc:db2://xxxxxxxxxxxxxxxxx"/>
<param name="user" value="xxxxxxxxxxxxx"/>
<param name="password" value="xxxxxx"/>
<param name="databaseType" value="db2"/>
<param name="maxPoolSize" value="80"/>
</DataSource>
</DataSources>
<FileSystem class="org.apache.jackrabbit.core.fs.db.DbFileSystem">
<param name="dataSourceName" value="ds1"/>
<param name="schemaObjectPrefix" value="J_R_FS_"/>
</FileSystem>
<DataStore class="org.apache.jackrabbit.core.data.db.DbDataStore">
<param name="dataSourceName" value="ds1"/>
<param name="schemaObjectPrefix" value="J_R_"/>
<param name="minRecordLength" value="1024"/>
<param name="maxConnections" value="3"/>
<param name="copyWhenReading" value="true"/>
<param name="tablePrefix" value=""/>
</DataStore>
<Workspaces rootPath="${rep.home}/workspaces" defaultWorkspace="blubb"
/>
<Workspace name="${wsp.name}">
<FileSystem class=
"org.apache.jackrabbit.core.fs.local.LocalFileSystem">
<param name="path" value="${wsp.home}"/>
</FileSystem>
<PersistenceManager class=
"org.apache.jackrabbit.core.persistence.pool.BundleDbPersistenceManager">
<param name="dataSourceName" value="ds1"/>
<param name="schemaObjectPrefix" value="J_PM_${wsp.name}_"/>
<param name="databaseType" value="db2"/>
<param name="bundleCacheSize" value="8"/>
<param name="consistencyCheck" value="false"/>
<param name="consistencyFix" value="false"/>
<param name="minBlobSize" value="4096"/>
<param name="errorHandling" value=""/>
<param name="blockOnConnectionLoss" value="false"/>
<param name="schemaCheckEnabled" value="true"/>
</PersistenceManager>
</Workspace>
<Versioning rootPath="${rep.home}/version">
<FileSystem class=
"org.apache.jackrabbit.core.fs.local.LocalFileSystem">
<param name="path" value="${rep.home}/version"/>
</FileSystem>
<PersistenceManager class=
"org.apache.jackrabbit.core.persistence.pool.BundleDbPersistenceManager">
<param name="dataSourceName" value="ds1"/>
<param name="schemaObjectPrefix" value="J_V_PM_"/>
<param name="databaseType" value="db2"/>
<param name="bundleCacheSize" value="8"/>
<param name="consistencyCheck" value="false"/>
<param name="consistencyFix" value="false"/>
<param name="minBlobSize" value="4096"/>
<param name="errorHandling" value=""/>
<param name="blockOnConnectionLoss" value="false"/>
<param name="schemaCheckEnabled" value="true"/>
</PersistenceManager>
</Versioning>
<Cluster syncDelay="5">
<Journal class=
"org.apache.jackrabbit.core.journal.DatabaseJournal">
<param name="dataSourceName" value="ds1"/>
<param name="revision" value="${rep.home}/revision.log"/>
</Journal>
</Cluster>
The clusterId is provided as system.property.
As soon as we provide heavy load on some (max. 4) Jackrabbit-Nodes the
Perfomance drops.
It looks like too many locks on tables GLOBAL_REVISION and/or
LOCAL_REVISIONS.
The first action we tried is to increase the (very low) value 5 (ms) for
the syncDelay up to 800ms (default is 5000ms).
The next action will be to update to the jackrabbit-revision 2.2.13.
Any advice the community can provide will be greatly appreciated.
If you need more detailed Information, please let me know, i will try to
provide it.
Also important:
We would also like to consult professional support for our problem.
Can you please give advice where we can find professional support for
Jackrabbit to get this running?
Best regards,
Detlev Zühlke
SIGNAL IDUNA Gruppe
AES5-97350
Neue Rabenstraße 15-19
20354 Hamburg
Tel.: (040) 4124-2954
Fax: (040) 4124-4902954
email: detlev.zuehlke@signal-iduna.de
SIGNAL Krankenversicherung a. G., Sitz: Dortmund, HR B 2405, AG Dortmund
IDUNA Vereinigte Lebensversicherung aG für Handwerk, Handel und Gewerbe,
Sitz: Hamburg, HR B 2740, AG Hamburg
Deutscher Ring Krankenversicherungsverein a.G., Sitz: Hamburg,
HR B 4673, AG Hamburg,
SIGNAL IDUNA Allgemeine Versicherung AG, Sitz: Dortmund, HR B 19108,
AG Dortmund
Vorstände: Reinhold Schulte (Vorsitzender),
Dr. Karl-Josef Bierth, Marlies Hirschberg-Tafel,
Michael Johnigk, Ulrich Leitermann, Michael Petmecky,
Dr. Klaus Sticker, Prof. Dr. Markus Warg
Vorsitzender der Aufsichtsräte: Günter Kutz
SIGNAL IDUNA Gruppe Hauptverwaltungen, Internet: www.signal-iduna.de
44121 Dortmund, Hausanschrift: Joseph-Scherer-Str. 3, 44139 Dortmund
20351 Hamburg, Hausanschrift: Neue Rabenstraße 15-19, 20354 Hamburg
SIGNAL Krankenversicherung a. G., Sitz: Dortmund, HR B 2405, AG Dortmund
IDUNA Vereinigte Lebensversicherung aG für Handwerk, Handel und Gewerbe,
Sitz: Hamburg, HR B 2740, AG Hamburg
Deutscher Ring Krankenversicherungsverein a.G., Sitz: Hamburg,
HR B 4673, AG Hamburg,
SIGNAL IDUNA Allgemeine Versicherung AG, Sitz: Dortmund, HR B 19108,
AG Dortmund
Vorstände: Reinhold Schulte (Vorsitzender),
Dr. Karl-Josef Bierth, Marlies Hirschberg-Tafel,
Michael Johnigk, Ulrich Leitermann, Michael Petmecky,
Dr. Klaus Sticker, Prof. Dr. Markus Warg
Vorsitzender der Aufsichtsräte: Günter Kutz
SIGNAL IDUNA Gruppe Hauptverwaltungen, Internet: www.signal-iduna.de
44121 Dortmund, Hausanschrift: Joseph-Scherer-Str. 3, 44139 Dortmund
20351 Hamburg, Hausanschrift: Neue Rabenstraße 15-19, 20354 Hamburg