You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@camel.apache.org by "Bryan Keller (JIRA)" <ji...@apache.org> on 2011/09/22 02:13:26 UTC
[jira] [Created] (CAMEL-4476) Camel splitter/aggregator 2x slower
in 2.8.1 vs 2.7.3
Camel splitter/aggregator 2x slower in 2.8.1 vs 2.7.3
-----------------------------------------------------
Key: CAMEL-4476
URL: https://issues.apache.org/jira/browse/CAMEL-4476
Project: Camel
Issue Type: Bug
Components: camel-core
Affects Versions: 2.8.1
Environment: Oracle JDK 1.6.0_26 64-bit, CentOS 6 64-bit
Reporter: Bryan Keller
I noticed a sharp decline in the performance of my application when I upgraded from Camel 2.7.3 to 2.8.1. I discovered that the splitter/aggregator had become the bottleneck in my app. I have created a sample app that demonstrates the issue. The aggregator uses a lock, and I suspect lock contention may be at the root of this, but didn't notice any changes around that in the code diff between the versions.
I uploaded the sample app to http://www.vancameron.net/CamelPerf.zip. It uses gradle. Unzip to a directory then run "gradle" from the command line. To change the version from 2.8.1 to 2.7.3, edit the build.gradle file and run again.
The app loads 10 files with 100k lines each. It prints the current time in msec per batch.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CAMEL-4476) Camel splitter/aggregator 2x slower
in 2.8.1 vs 2.7.3
Posted by "Bryan Keller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CAMEL-4476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bryan Keller resolved CAMEL-4476.
---------------------------------
Resolution: Not A Problem
This looks like it was caused by using <simple>xxx</simple> expressions for the completion timeout and completion size. It seems that the expression is being evaluated for each batch in 2.8 but that was not the behavior in 2.7. That caused the performance degradation. Changing to <constant> expressions fixed the performance problem I was having.
> Camel splitter/aggregator 2x slower in 2.8.1 vs 2.7.3
> -----------------------------------------------------
>
> Key: CAMEL-4476
> URL: https://issues.apache.org/jira/browse/CAMEL-4476
> Project: Camel
> Issue Type: Bug
> Components: camel-core
> Affects Versions: 2.8.1
> Environment: Oracle JDK 1.6.0_26 64-bit, CentOS 6 64-bit
> Reporter: Bryan Keller
>
> I noticed a sharp decline in the performance of my application when I upgraded from Camel 2.7.3 to 2.8.1. I discovered that the splitter/aggregator had become the bottleneck in my app. I have created a sample app that demonstrates the issue. The aggregator uses a lock, and I suspect lock contention may be at the root of this, but didn't notice any changes around that in the code diff between the versions.
> I uploaded the sample app to http://www.vancameron.net/CamelPerf.zip. It uses gradle. Unzip to a directory then run "gradle" from the command line. To change the version from 2.8.1 to 2.7.3, edit the build.gradle file and run again.
> The app loads 10 files with 100k lines each. It prints the current time in msec per batch.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CAMEL-4476) Camel splitter/aggregator 2x slower
in 2.8.1 vs 2.7.3
Posted by "Claus Ibsen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CAMEL-4476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Claus Ibsen resolved CAMEL-4476.
--------------------------------
Resolution: Fixed
Fix Version/s: 2.9.0
2.8.2
Thanks for reporting.
I added a test AggregateSimpleExpressionIssueTest which is similar to yours. It can be run manually. And the test now runs as fast for constant/simple language.
> Camel splitter/aggregator 2x slower in 2.8.1 vs 2.7.3
> -----------------------------------------------------
>
> Key: CAMEL-4476
> URL: https://issues.apache.org/jira/browse/CAMEL-4476
> Project: Camel
> Issue Type: Bug
> Components: camel-core
> Affects Versions: 2.8.1
> Environment: Oracle JDK 1.6.0_26 64-bit, CentOS 6 64-bit
> Reporter: Bryan Keller
> Assignee: Claus Ibsen
> Fix For: 2.8.2, 2.9.0
>
>
> I noticed a sharp decline in the performance of my application when I upgraded from Camel 2.7.3 to 2.8.1. I discovered that the splitter/aggregator had become the bottleneck in my app. I have created a sample app that demonstrates the issue. The aggregator uses a lock, and I suspect lock contention may be at the root of this, but didn't notice any changes around that in the code diff between the versions.
> I uploaded the sample app to http://www.vancameron.net/CamelPerf.zip. It uses gradle. Unzip to a directory then run "gradle" from the command line. To change the version from 2.8.1 to 2.7.3, edit the build.gradle file and run again.
> The app loads 10 files with 100k lines each. It prints the current time in msec per batch.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (CAMEL-4476) Camel splitter/aggregator 2x slower
in 2.8.1 vs 2.7.3
Posted by "Bryan Keller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CAMEL-4476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bryan Keller reopened CAMEL-4476:
---------------------------------
Though I uncovered a problem. When using a constant expression, there is no way to inject a property value. So if I want the batch size configurable, it seems the only way to do this is to use a "simple" expression. But using the "simple" expression causes the aforementioned performance issue.
> Camel splitter/aggregator 2x slower in 2.8.1 vs 2.7.3
> -----------------------------------------------------
>
> Key: CAMEL-4476
> URL: https://issues.apache.org/jira/browse/CAMEL-4476
> Project: Camel
> Issue Type: Bug
> Components: camel-core
> Affects Versions: 2.8.1
> Environment: Oracle JDK 1.6.0_26 64-bit, CentOS 6 64-bit
> Reporter: Bryan Keller
>
> I noticed a sharp decline in the performance of my application when I upgraded from Camel 2.7.3 to 2.8.1. I discovered that the splitter/aggregator had become the bottleneck in my app. I have created a sample app that demonstrates the issue. The aggregator uses a lock, and I suspect lock contention may be at the root of this, but didn't notice any changes around that in the code diff between the versions.
> I uploaded the sample app to http://www.vancameron.net/CamelPerf.zip. It uses gradle. Unzip to a directory then run "gradle" from the command line. To change the version from 2.8.1 to 2.7.3, edit the build.gradle file and run again.
> The app loads 10 files with 100k lines each. It prints the current time in msec per batch.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CAMEL-4476) Camel splitter/aggregator 2x slower
in 2.8.1 vs 2.7.3
Posted by "Claus Ibsen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CAMEL-4476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13112510#comment-13112510 ]
Claus Ibsen commented on CAMEL-4476:
------------------------------------
With constant: Time taken 68040 ms
With simple: Time taken 100913 ms
> Camel splitter/aggregator 2x slower in 2.8.1 vs 2.7.3
> -----------------------------------------------------
>
> Key: CAMEL-4476
> URL: https://issues.apache.org/jira/browse/CAMEL-4476
> Project: Camel
> Issue Type: Bug
> Components: camel-core
> Affects Versions: 2.8.1
> Environment: Oracle JDK 1.6.0_26 64-bit, CentOS 6 64-bit
> Reporter: Bryan Keller
> Assignee: Claus Ibsen
>
> I noticed a sharp decline in the performance of my application when I upgraded from Camel 2.7.3 to 2.8.1. I discovered that the splitter/aggregator had become the bottleneck in my app. I have created a sample app that demonstrates the issue. The aggregator uses a lock, and I suspect lock contention may be at the root of this, but didn't notice any changes around that in the code diff between the versions.
> I uploaded the sample app to http://www.vancameron.net/CamelPerf.zip. It uses gradle. Unzip to a directory then run "gradle" from the command line. To change the version from 2.8.1 to 2.7.3, edit the build.gradle file and run again.
> The app loads 10 files with 100k lines each. It prints the current time in msec per batch.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (CAMEL-4476) Camel splitter/aggregator 2x slower
in 2.8.1 vs 2.7.3
Posted by "Claus Ibsen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CAMEL-4476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Claus Ibsen reassigned CAMEL-4476:
----------------------------------
Assignee: Claus Ibsen
> Camel splitter/aggregator 2x slower in 2.8.1 vs 2.7.3
> -----------------------------------------------------
>
> Key: CAMEL-4476
> URL: https://issues.apache.org/jira/browse/CAMEL-4476
> Project: Camel
> Issue Type: Bug
> Components: camel-core
> Affects Versions: 2.8.1
> Environment: Oracle JDK 1.6.0_26 64-bit, CentOS 6 64-bit
> Reporter: Bryan Keller
> Assignee: Claus Ibsen
>
> I noticed a sharp decline in the performance of my application when I upgraded from Camel 2.7.3 to 2.8.1. I discovered that the splitter/aggregator had become the bottleneck in my app. I have created a sample app that demonstrates the issue. The aggregator uses a lock, and I suspect lock contention may be at the root of this, but didn't notice any changes around that in the code diff between the versions.
> I uploaded the sample app to http://www.vancameron.net/CamelPerf.zip. It uses gradle. Unzip to a directory then run "gradle" from the command line. To change the version from 2.8.1 to 2.7.3, edit the build.gradle file and run again.
> The app loads 10 files with 100k lines each. It prints the current time in msec per batch.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CAMEL-4476) Camel splitter/aggregator 2x slower
in 2.8.1 vs 2.7.3
Posted by "Claus Ibsen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CAMEL-4476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13112514#comment-13112514 ]
Claus Ibsen commented on CAMEL-4476:
------------------------------------
Okay got it optimized so its as fast again:
With simple: Time taken 68876 ms
> Camel splitter/aggregator 2x slower in 2.8.1 vs 2.7.3
> -----------------------------------------------------
>
> Key: CAMEL-4476
> URL: https://issues.apache.org/jira/browse/CAMEL-4476
> Project: Camel
> Issue Type: Bug
> Components: camel-core
> Affects Versions: 2.8.1
> Environment: Oracle JDK 1.6.0_26 64-bit, CentOS 6 64-bit
> Reporter: Bryan Keller
> Assignee: Claus Ibsen
>
> I noticed a sharp decline in the performance of my application when I upgraded from Camel 2.7.3 to 2.8.1. I discovered that the splitter/aggregator had become the bottleneck in my app. I have created a sample app that demonstrates the issue. The aggregator uses a lock, and I suspect lock contention may be at the root of this, but didn't notice any changes around that in the code diff between the versions.
> I uploaded the sample app to http://www.vancameron.net/CamelPerf.zip. It uses gradle. Unzip to a directory then run "gradle" from the command line. To change the version from 2.8.1 to 2.7.3, edit the build.gradle file and run again.
> The app loads 10 files with 100k lines each. It prints the current time in msec per batch.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira