You are viewing a plain text version of this content. The canonical link for it is here.
Posted to derby-dev@db.apache.org by "Myrna van Lunteren (JIRA)" <ji...@apache.org> on 2009/05/13 02:32:45 UTC
[jira] Created: (DERBY-4227) performance degradation of 10.5.1.1
with simple queries & no data retrieval
performance degradation of 10.5.1.1 with simple queries & no data retrieval
---------------------------------------------------------------------------
Key: DERBY-4227
URL: https://issues.apache.org/jira/browse/DERBY-4227
Project: Derby
Issue Type: Bug
Affects Versions: 10.5.1.1, 10.6.0.0
Reporter: Myrna van Lunteren
I have been executing some non-contributed (and not easily contributable) performance tests, and found that across the board, with the exception of some blob streaming and complicated queries, 10.5.1.1 performs significantly worse from 10.4.2.0.
I've turned one of these tests into a little repro script.
It's decidedly hokey, but if you execute this repeatedly (removing the wombatC db and derby.log each time) you'll likely find the same. I executed this on an MS-Windows 2000 machine, with ibm 1.5. jvm.
As a quick comparison, I got the following data out of executing the repro 4 times with a fairly recent 10.3.3.1 build (706492), 10.4.2.0, and 10.5.1.1. Time is the elapsed time printed out at the end by the repro.
version 10.3.3.1 10.4.2.0 10.5.1.1
1st run 187 219 328
2nd run 172 172 328
3rd run 203 234 313
4th run 187 281 344
We should identify why a simple select without actually fetching the data would do so much worse with 10.5.1.1.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-4227) performance degradation of 10.5.1.1
with simple queries & no data retrieval
Posted by "Knut Anders Hatlen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-4227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12708808#action_12708808 ]
Knut Anders Hatlen commented on DERBY-4227:
-------------------------------------------
I'm wondering if this could be the same that we see in the nightly performance regression tests, mentioned here: http://mail-archives.apache.org/mod_mbox/db-derby-dev/200904.mbox/%3cwjo8vdon4vco.fsf_-_@sun.com%3e
Those tests also perform table scans, similar to the repro class you uploaded, but the performance drop seen there is much smaller than what you report.
> performance degradation of 10.5.1.1 with simple queries & no data retrieval
> ---------------------------------------------------------------------------
>
> Key: DERBY-4227
> URL: https://issues.apache.org/jira/browse/DERBY-4227
> Project: Derby
> Issue Type: Bug
> Affects Versions: 10.5.1.1, 10.6.0.0
> Reporter: Myrna van Lunteren
> Attachments: scanperf.java
>
>
> I have been executing some non-contributed (and not easily contributable) performance tests, and found that across the board, with the exception of some blob streaming and complicated queries, 10.5.1.1 performs significantly worse from 10.4.2.0.
> I've turned one of these tests into a little repro script.
> It's decidedly hokey, but if you execute this repeatedly (removing the wombatC db and derby.log each time) you'll likely find the same. I executed this on an MS-Windows 2000 machine, with ibm 1.5. jvm.
> As a quick comparison, I got the following data out of executing the repro 4 times with a fairly recent 10.3.3.1 build (706492), 10.4.2.0, and 10.5.1.1. Time is the elapsed time printed out at the end by the repro.
> version 10.3.3.1 10.4.2.0 10.5.1.1
> 1st run 187 219 328
> 2nd run 172 172 328
> 3rd run 203 234 313
> 4th run 187 281 344
> We should identify why a simple select without actually fetching the data would do so much worse with 10.5.1.1.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (DERBY-4227) performance degradation of 10.5.1.1
with simple queries & no data retrieval
Posted by "Myrna van Lunteren (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-4227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Myrna van Lunteren updated DERBY-4227:
--------------------------------------
Attachment: scanperf.java
Small repro class that will get the system time in a long before and after selecting 10000 rows and only counting the number of rows in the resultset, then prints out the diff.
To run: compile,add class to classpath, run (for instance redirecting output to a file), remove derby.log and wombatC, run again.
> performance degradation of 10.5.1.1 with simple queries & no data retrieval
> ---------------------------------------------------------------------------
>
> Key: DERBY-4227
> URL: https://issues.apache.org/jira/browse/DERBY-4227
> Project: Derby
> Issue Type: Bug
> Affects Versions: 10.5.1.1, 10.6.0.0
> Reporter: Myrna van Lunteren
> Attachments: scanperf.java
>
>
> I have been executing some non-contributed (and not easily contributable) performance tests, and found that across the board, with the exception of some blob streaming and complicated queries, 10.5.1.1 performs significantly worse from 10.4.2.0.
> I've turned one of these tests into a little repro script.
> It's decidedly hokey, but if you execute this repeatedly (removing the wombatC db and derby.log each time) you'll likely find the same. I executed this on an MS-Windows 2000 machine, with ibm 1.5. jvm.
> As a quick comparison, I got the following data out of executing the repro 4 times with a fairly recent 10.3.3.1 build (706492), 10.4.2.0, and 10.5.1.1. Time is the elapsed time printed out at the end by the repro.
> version 10.3.3.1 10.4.2.0 10.5.1.1
> 1st run 187 219 328
> 2nd run 172 172 328
> 3rd run 203 234 313
> 4th run 187 281 344
> We should identify why a simple select without actually fetching the data would do so much worse with 10.5.1.1.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-4227) performance degradation of 10.5.1.1
with simple queries & no data retrieval
Posted by "Myrna van Lunteren (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-4227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12711391#action_12711391 ]
Myrna van Lunteren commented on DERBY-4227:
-------------------------------------------
I have finished running all my tests, and they show no noteworthy differences between 10.4 and 10.5.
> performance degradation of 10.5.1.1 with simple queries & no data retrieval
> ---------------------------------------------------------------------------
>
> Key: DERBY-4227
> URL: https://issues.apache.org/jira/browse/DERBY-4227
> Project: Derby
> Issue Type: Bug
> Affects Versions: 10.5.1.1, 10.6.0.0
> Reporter: Myrna van Lunteren
> Attachments: scanperf.java
>
>
> I have been executing some non-contributed (and not easily contributable) performance tests, and found that across the board, with the exception of some blob streaming and complicated queries, 10.5.1.1 performs significantly worse from 10.4.2.0.
> I've turned one of these tests into a little repro script.
> It's decidedly hokey, but if you execute this repeatedly (removing the wombatC db and derby.log each time) you'll likely find the same. I executed this on an MS-Windows 2000 machine, with ibm 1.5. jvm.
> As a quick comparison, I got the following data out of executing the repro 4 times with a fairly recent 10.3.3.1 build (706492), 10.4.2.0, and 10.5.1.1. Time is the elapsed time printed out at the end by the repro.
> version 10.3.3.1 10.4.2.0 10.5.1.1
> 1st run 187 219 328
> 2nd run 172 172 328
> 3rd run 203 234 313
> 4th run 187 281 344
> We should identify why a simple select without actually fetching the data would do so much worse with 10.5.1.1.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-4227) performance degradation of 10.5.1.1
with simple queries & no data retrieval
Posted by "Knut Anders Hatlen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-4227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12708844#action_12708844 ]
Knut Anders Hatlen commented on DERBY-4227:
-------------------------------------------
I printed some more timestamps and found that of the ~80 ms reported by the test, ~30 ms was spent compiling the statement. To get a better picture of the performance of executing the statement, it's probably a good idea to put PreparedStatement.executeQuery()+ResultSet.next() in a loop and execute it, say, 10000 times.
> performance degradation of 10.5.1.1 with simple queries & no data retrieval
> ---------------------------------------------------------------------------
>
> Key: DERBY-4227
> URL: https://issues.apache.org/jira/browse/DERBY-4227
> Project: Derby
> Issue Type: Bug
> Affects Versions: 10.5.1.1, 10.6.0.0
> Reporter: Myrna van Lunteren
> Attachments: scanperf.java
>
>
> I have been executing some non-contributed (and not easily contributable) performance tests, and found that across the board, with the exception of some blob streaming and complicated queries, 10.5.1.1 performs significantly worse from 10.4.2.0.
> I've turned one of these tests into a little repro script.
> It's decidedly hokey, but if you execute this repeatedly (removing the wombatC db and derby.log each time) you'll likely find the same. I executed this on an MS-Windows 2000 machine, with ibm 1.5. jvm.
> As a quick comparison, I got the following data out of executing the repro 4 times with a fairly recent 10.3.3.1 build (706492), 10.4.2.0, and 10.5.1.1. Time is the elapsed time printed out at the end by the repro.
> version 10.3.3.1 10.4.2.0 10.5.1.1
> 1st run 187 219 328
> 2nd run 172 172 328
> 3rd run 203 234 313
> 4th run 187 281 344
> We should identify why a simple select without actually fetching the data would do so much worse with 10.5.1.1.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-4227) performance degradation of 10.5.1.1
with simple queries & no data retrieval
Posted by "Kristian Waagan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-4227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12709067#action_12709067 ]
Kristian Waagan commented on DERBY-4227:
----------------------------------------
Myrna,
I'm curious about what results you see when running with the 10.5 insane jars!
Do you still have the rig up, so you can take Derby for another spin?
> performance degradation of 10.5.1.1 with simple queries & no data retrieval
> ---------------------------------------------------------------------------
>
> Key: DERBY-4227
> URL: https://issues.apache.org/jira/browse/DERBY-4227
> Project: Derby
> Issue Type: Bug
> Affects Versions: 10.5.1.1, 10.6.0.0
> Reporter: Myrna van Lunteren
> Attachments: scanperf.java
>
>
> I have been executing some non-contributed (and not easily contributable) performance tests, and found that across the board, with the exception of some blob streaming and complicated queries, 10.5.1.1 performs significantly worse from 10.4.2.0.
> I've turned one of these tests into a little repro script.
> It's decidedly hokey, but if you execute this repeatedly (removing the wombatC db and derby.log each time) you'll likely find the same. I executed this on an MS-Windows 2000 machine, with ibm 1.5. jvm.
> As a quick comparison, I got the following data out of executing the repro 4 times with a fairly recent 10.3.3.1 build (706492), 10.4.2.0, and 10.5.1.1. Time is the elapsed time printed out at the end by the repro.
> version 10.3.3.1 10.4.2.0 10.5.1.1
> 1st run 187 219 328
> 2nd run 172 172 328
> 3rd run 203 234 313
> 4th run 187 281 344
> We should identify why a simple select without actually fetching the data would do so much worse with 10.5.1.1.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-4227) performance degradation of 10.5.1.1
with simple queries & no data retrieval
Posted by "Myrna van Lunteren (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-4227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12709093#action_12709093 ]
Myrna van Lunteren commented on DERBY-4227:
-------------------------------------------
Yes, it's running with the *in*sane jars now. I'll post any conclusions here after I've looked at the results...
> performance degradation of 10.5.1.1 with simple queries & no data retrieval
> ---------------------------------------------------------------------------
>
> Key: DERBY-4227
> URL: https://issues.apache.org/jira/browse/DERBY-4227
> Project: Derby
> Issue Type: Bug
> Affects Versions: 10.5.1.1, 10.6.0.0
> Reporter: Myrna van Lunteren
> Attachments: scanperf.java
>
>
> I have been executing some non-contributed (and not easily contributable) performance tests, and found that across the board, with the exception of some blob streaming and complicated queries, 10.5.1.1 performs significantly worse from 10.4.2.0.
> I've turned one of these tests into a little repro script.
> It's decidedly hokey, but if you execute this repeatedly (removing the wombatC db and derby.log each time) you'll likely find the same. I executed this on an MS-Windows 2000 machine, with ibm 1.5. jvm.
> As a quick comparison, I got the following data out of executing the repro 4 times with a fairly recent 10.3.3.1 build (706492), 10.4.2.0, and 10.5.1.1. Time is the elapsed time printed out at the end by the repro.
> version 10.3.3.1 10.4.2.0 10.5.1.1
> 1st run 187 219 328
> 2nd run 172 172 328
> 3rd run 203 234 313
> 4th run 187 281 344
> We should identify why a simple select without actually fetching the data would do so much worse with 10.5.1.1.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-4227) performance degradation of 10.5.1.1
with simple queries & no data retrieval
Posted by "Knut Anders Hatlen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-4227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12709060#action_12709060 ]
Knut Anders Hatlen commented on DERBY-4227:
-------------------------------------------
And just for the record, I have not been able to verify the degradation seen in the nightly regression tests. I've tried to rerun the tests with jar files from each day the last two and a half months, but I don't see any degradation, so I assume it's a problem with the environment. The test in question runs the following command:
java -server -Dderby.storage.pageCacheSize=25000 org.apache.derbyTesting.perf.clients.Runner -load sr_select -load_opts nonIndexed
> performance degradation of 10.5.1.1 with simple queries & no data retrieval
> ---------------------------------------------------------------------------
>
> Key: DERBY-4227
> URL: https://issues.apache.org/jira/browse/DERBY-4227
> Project: Derby
> Issue Type: Bug
> Affects Versions: 10.5.1.1, 10.6.0.0
> Reporter: Myrna van Lunteren
> Attachments: scanperf.java
>
>
> I have been executing some non-contributed (and not easily contributable) performance tests, and found that across the board, with the exception of some blob streaming and complicated queries, 10.5.1.1 performs significantly worse from 10.4.2.0.
> I've turned one of these tests into a little repro script.
> It's decidedly hokey, but if you execute this repeatedly (removing the wombatC db and derby.log each time) you'll likely find the same. I executed this on an MS-Windows 2000 machine, with ibm 1.5. jvm.
> As a quick comparison, I got the following data out of executing the repro 4 times with a fairly recent 10.3.3.1 build (706492), 10.4.2.0, and 10.5.1.1. Time is the elapsed time printed out at the end by the repro.
> version 10.3.3.1 10.4.2.0 10.5.1.1
> 1st run 187 219 328
> 2nd run 172 172 328
> 3rd run 203 234 313
> 4th run 187 281 344
> We should identify why a simple select without actually fetching the data would do so much worse with 10.5.1.1.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Closed: (DERBY-4227) performance degradation of 10.5.1.1
with simple queries & no data retrieval
Posted by "Myrna van Lunteren (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-4227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Myrna van Lunteren closed DERBY-4227.
-------------------------------------
Resolution: Invalid
Thanks Knut for looking at this.
I went back and looked at my jars and am ashamed to say - I used sane jars for 10.5.1.1.
I apologize for the trouble.
> performance degradation of 10.5.1.1 with simple queries & no data retrieval
> ---------------------------------------------------------------------------
>
> Key: DERBY-4227
> URL: https://issues.apache.org/jira/browse/DERBY-4227
> Project: Derby
> Issue Type: Bug
> Affects Versions: 10.5.1.1, 10.6.0.0
> Reporter: Myrna van Lunteren
> Attachments: scanperf.java
>
>
> I have been executing some non-contributed (and not easily contributable) performance tests, and found that across the board, with the exception of some blob streaming and complicated queries, 10.5.1.1 performs significantly worse from 10.4.2.0.
> I've turned one of these tests into a little repro script.
> It's decidedly hokey, but if you execute this repeatedly (removing the wombatC db and derby.log each time) you'll likely find the same. I executed this on an MS-Windows 2000 machine, with ibm 1.5. jvm.
> As a quick comparison, I got the following data out of executing the repro 4 times with a fairly recent 10.3.3.1 build (706492), 10.4.2.0, and 10.5.1.1. Time is the elapsed time printed out at the end by the repro.
> version 10.3.3.1 10.4.2.0 10.5.1.1
> 1st run 187 219 328
> 2nd run 172 172 328
> 3rd run 203 234 313
> 4th run 187 281 344
> We should identify why a simple select without actually fetching the data would do so much worse with 10.5.1.1.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-4227) performance degradation of 10.5.1.1
with simple queries & no data retrieval
Posted by "Knut Anders Hatlen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-4227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12708814#action_12708814 ]
Knut Anders Hatlen commented on DERBY-4227:
-------------------------------------------
I downloaded the test class and ran it 100 times with 10.4.2.0 and 100 times with 10.5.1.1. On average, it took 82 ms with 10.4 and 84 ms with 10.5, so I don't see any significant changes in performance between the releases.
I ran the test on OpenSolaris and
java version "1.6.0_13"
Java(TM) SE Runtime Environment (build 1.6.0_13-b03)
Java HotSpot(TM) Client VM (build 11.3-b02, mixed mode)
> performance degradation of 10.5.1.1 with simple queries & no data retrieval
> ---------------------------------------------------------------------------
>
> Key: DERBY-4227
> URL: https://issues.apache.org/jira/browse/DERBY-4227
> Project: Derby
> Issue Type: Bug
> Affects Versions: 10.5.1.1, 10.6.0.0
> Reporter: Myrna van Lunteren
> Attachments: scanperf.java
>
>
> I have been executing some non-contributed (and not easily contributable) performance tests, and found that across the board, with the exception of some blob streaming and complicated queries, 10.5.1.1 performs significantly worse from 10.4.2.0.
> I've turned one of these tests into a little repro script.
> It's decidedly hokey, but if you execute this repeatedly (removing the wombatC db and derby.log each time) you'll likely find the same. I executed this on an MS-Windows 2000 machine, with ibm 1.5. jvm.
> As a quick comparison, I got the following data out of executing the repro 4 times with a fairly recent 10.3.3.1 build (706492), 10.4.2.0, and 10.5.1.1. Time is the elapsed time printed out at the end by the repro.
> version 10.3.3.1 10.4.2.0 10.5.1.1
> 1st run 187 219 328
> 2nd run 172 172 328
> 3rd run 203 234 313
> 4th run 187 281 344
> We should identify why a simple select without actually fetching the data would do so much worse with 10.5.1.1.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.