[jira] Commented: (DERBY-3571) LOB locators are not released if the LOB columns are not accessed by the client

["Kristian Waagan (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/DERBY-3571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12584049#action_12584049 ] 

Kristian Waagan commented on DERBY-3571:
----------------------------------------

Thanks for all the reviewing Knut and Kathey.

This is still rather "unstable" code, and I have newer versions in the pipeline. Let me comment on the simple things first.

a) No release of LOBs in the first row.
  This is a bug introduced when I made it a requirement that checkCurrentRow should only be called on a vaild row. I'll fix this.

b) Setting trackColumn to true in checkCurrentRow.
  Won't it always be true with the current code? To enter the if-block it must be true, and we don't need to set it again. Otherwise we enter the else-block and set it to true. I do see the point of setting it to true unconditionally for readability though. In a newer patch (not published yet), I have renamed "trackColumn" to "accessed".

c) noRelease
  I'll rewrite it using Arrays and remove the check variable. Another question is whether an exception should be thrown in any case, as providing an invalid index seems like a programming error.
  Further, I was thinking about using noRelease to thrown an exception if the LOB column had already been accessed, but there might be other approaches one can use to make Derby fail if a LOB column is accessed twice.

d) The comment with the typo will be removed, and the code in createLOBColumTracker partly rewritten.
   We now need a functioning tracker also when locators are unsupported.

e) !connection_.autoCommit_ condition for calling checkCurrentRow
   A good further optimization. Thanks. I think we need to think about how to handle this for statements with multiple result sets though, as that might cause the autocommit to be skipped even though the connection has autocommit set.


Regarding the control question above, yes there will be multiple release mechanisms. I have thought of these names:
 * NoReleaseLOBTracker
 * SingleReleaseLOBTracker
 * BatchReleaseLOBTracker
 * A tracker using piggybacking.

Originally I introduced an interface and created different implementations for the tracker to avoid maintaining state I didn't need. Now that we plan to use the tracker to also track accesses to LOB columns, we need some common code in all of the implementations.
Would it perhaps be better to have only one tracker implementation and instead implement multiple "releasers"?
I think the tracker should be able to determine the proper release mechanism itself in the constructor.

Any other ideas?
If not, I will implement a first version of the approach I have described and post a patch for review later today.

> LOB locators are not released if the LOB columns are not accessed by the client
> -------------------------------------------------------------------------------
>
>                 Key: DERBY-3571
>                 URL: https://issues.apache.org/jira/browse/DERBY-3571
>             Project: Derby
>          Issue Type: Bug
>          Components: JDBC, Network Client
>    Affects Versions: 10.3.2.1, 10.4.0.0, 10.5.0.0
>            Reporter: Kristian Waagan
>            Assignee: Kristian Waagan
>         Attachments: derby-3571-1a-client_track_lob_fix.diff, derby-3571-1a-client_track_lob_fix.stat, derby-3571-1b-client_track_lob_fix.diff, derby-3571-1c-client_track_lob_fix.diff, derby-3571-1d-client_track_lob_fix.diff, derby-3571-1e-client_track_lob_fix.diff
>
>
> If the client creates a result set containing LOB locator columns and iterates through it without actually accessing the LOB columns, the locators are not released.
> The amount of locators and their associated LOB objects causes the server to consume large amounts of memory and it eventually gets an OOME.
> There are a few workarounds for this bug:
>  a) Access and/or properly close the LOBs (i.e. Blob.free).
>     This is partly dependent on DERBY-2892.
>  b) Invoke Connection.commit (or rollback) periodically, which causes all locators on the connection to be released.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.