[jira] Commented: (DERBY-3791) Excessive memory usage when fetching small Clobs

["Kristian Waagan (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/DERBY-3791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12615582#action_12615582 ] 

Kristian Waagan commented on DERBY-3791:
----------------------------------------

I ran some tests using SingleRecordSelectClient (found in the svn repository).
I used default settings and ran five times with one client and five times with 16 clients. The results are "normalized", where the throughput obtained with 10.2.2.1 is defined as 100%.

100% 100% 10.2.2.1
075% 216% 10.3.3.0
079% 231% 10.4.1.3
078% 231% trunk
146% 394% trunk-buffer-fix
167% 456% trunk-StringClob-fix

As can be seen, we do have a regression when using one client. The results obtained with SingleRecordSelectClient are a bit different than those I saw with the LobPerf repro posted under DERBY-3312. This might be because the repro doesn't commit after each select, but SingleRecordSelectClient does. This can cause the list of open Clobs to get large and might further reduce the performance.

I've played with two fixes. One is really simple, where one buffer is removed and another one is adjusted according to the Clob size. Without the fix, we allocated at least 32KB character buffers for each Clob. Needless to say, this is quite a lot when the Clob itself has between one and five characters.

The second fix adds another InternalClob implementation, which I have called StringClob. This is used for Clobs that are too small to be represented as streams. Currently these Clobs are represented as a byte array in memory. It turns out we start out with a byte array (from store), convert it to a string, convert it back to a byte array and then finally we convert whatever data the user requests to strings again.
Adding yet another internal Clob representation is not exactly good regarding testing and code volume, but because it is so simple I consider doing it for the extra performance.

Before I continue working on the StringClob fix, I want to clean up the InternalClob interface slightly.

> Excessive memory usage when fetching small Clobs
> ------------------------------------------------
>
>                 Key: DERBY-3791
>                 URL: https://issues.apache.org/jira/browse/DERBY-3791
>             Project: Derby
>          Issue Type: Bug
>          Components: JDBC
>    Affects Versions: 10.2.2.0, 10.3.1.4, 10.4.1.3, 10.5.0.0
>            Reporter: Kristian Waagan
>            Assignee: Kristian Waagan
>            Priority: Minor
>
> When investigating DERBY-3312 I found out that performance with the embedded driver has decreased a lot as well.
> Analysis on trunk indicate excessive memory usage, causing high allocation and garbage collection costs.
> I believe there was another major performance problem in 10.3, but it appears fixed in trunk. I have not spent time identifying this problem.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.