You are viewing a plain text version of this content. The canonical link for it is here.
Posted to derby-dev@db.apache.org by "Kristian Waagan (JIRA)" <ji...@apache.org> on 2007/05/15 09:08:16 UTC
[jira] Created: (DERBY-2646) Cleanup of LOB control/support
structures
Cleanup of LOB control/support structures
-----------------------------------------
Key: DERBY-2646
URL: https://issues.apache.org/jira/browse/DERBY-2646
Project: Derby
Issue Type: Sub-task
Reporter: Kristian Waagan
Assigned To: Kristian Waagan
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (DERBY-2646) Cleanup of Clob control/support
structures
Posted by "Myrna van Lunteren (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Myrna van Lunteren updated DERBY-2646:
--------------------------------------
Fix Version/s: 10.3.0.0
> Cleanup of Clob control/support structures
> ------------------------------------------
>
> Key: DERBY-2646
> URL: https://issues.apache.org/jira/browse/DERBY-2646
> Project: Derby
> Issue Type: Sub-task
> Components: JDBC
> Affects Versions: 10.3.0.0
> Reporter: Kristian Waagan
> Assignee: Kristian Waagan
> Fix For: 10.3.0.0
>
> Attachments: derby-2646-01a_trailing_space.diff, derby-2646-01a_trailing_space.stat, derby-2646-02a-clob-reorg.diff, derby-2646-02a-clob-reorg.stat, derby-2646-03a-internalclob.diff
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (DERBY-2646) Cleanup of Clob control/support
structures
Posted by "Kristian Waagan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kristian Waagan updated DERBY-2646:
-----------------------------------
Attachment: derby-2646-02a-clob-reorg.diff
derby-2646-02a-clob-reorg.diff
'derby-2646-02a-clob-reorg.diff' is a stab at reorganizing the upper level Clob code in the embedded driver.
I have run suites.All without failures.
Please note that the patch, although operational, is preliminary. I'm sorry it became so big, but I think it will be hard to split it up into smaller parts. I'm also very short on time, but I still feel this is worth getting done for 10.3.
For now I'm primarily interested in design feedback, but other kinds of feedback (bugs, suggestions etc) are welcome too.
Although I plan to address more cleanup and bugs in later patches, a few fixes were incorporated into the 2a patch. For instance, ClobTest has 3 failures/errors when run against trunk, but none when run with the patch.
Comments on the patch:
a) EmbedClob now works against the new interface InternalClob
b) There are two different kinds of internal Clob representations; StoreStreamClob (which is read-only) and ClobStreamControl (which I might rename, name suggestions? TemporaryClob?). If you fetch an existing Clob from the database, you start out with a StoreStreamClob. If you modify it, the Clob content is copied into ClobStreamControl. The switch is done in EmbedClob.
c) I need to look into exception handling. For various reasons it seems many methods must throw SQLExceptions. StoreStreamControl tends to get StandardException from store, ClobStreamControl tends to get StandardException/SQLException from LOBStreamControl. And of course IOException is well represented.
d) I need to look into/verify synchronization (on the connection child). Can anyone please enlighten me on the importance of calling setupContextStack/restoreContextStack?
e) All character positions in InternalClob are 1-based. Byte positions are 0-based.
f) Positioning streams at character positions are done by reading the byte stream from position 0 and counting characters. This implies UTF-8 decoding. Changing this will not happen in this patch/issue. Different improvements might be suitable for the store streams and the temporary Clobs (i.e. caching positions or changing to UTF-16).
My apologies for bringing a rather big change to the community this close to the freeze date. However, unless I get serious pushback, I will drive this forward to completion for 10.3.
> Cleanup of Clob control/support structures
> ------------------------------------------
>
> Key: DERBY-2646
> URL: https://issues.apache.org/jira/browse/DERBY-2646
> Project: Derby
> Issue Type: Sub-task
> Components: JDBC
> Affects Versions: 10.3.0.0
> Reporter: Kristian Waagan
> Assigned To: Kristian Waagan
> Attachments: derby-2646-01a_trailing_space.diff, derby-2646-01a_trailing_space.stat, derby-2646-02a-clob-reorg.diff, derby-2646-02a-clob-reorg.diff
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-2646) Cleanup of Clob control/support
structures
Posted by "Knut Anders Hatlen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12501522 ]
Knut Anders Hatlen commented on DERBY-2646:
-------------------------------------------
In UTF8Util, there is a private method called isDerbyEOFMarker() which isn't used. Is it a left-over that should be removed?
I was also wondering, would it be cleaner if internalSkip() returned an instance of a private inner class with two long members instead of an array of longs?
> Cleanup of Clob control/support structures
> ------------------------------------------
>
> Key: DERBY-2646
> URL: https://issues.apache.org/jira/browse/DERBY-2646
> Project: Derby
> Issue Type: Sub-task
> Components: JDBC
> Affects Versions: 10.3.0.0
> Reporter: Kristian Waagan
> Assignee: Kristian Waagan
> Fix For: 10.3.0.0
>
> Attachments: derby-2646-01a_trailing_space.diff, derby-2646-01a_trailing_space.stat, derby-2646-02a-clob-reorg.diff, derby-2646-02a-clob-reorg.stat, derby-2646-03a-internalclob.diff, derby-2646-04a-storestreamclob_utf8util.diff, derby-2646-04a-storestreamclob_utf8util.stat, derby-2646-04b-storestreamclob_utf8util.diff, derby-2646-05a-tests.diff, derby-2646-05a-tests.stat, derby-2646-05b-tests.diff
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Closed: (DERBY-2646) Cleanup of Clob control/support
structures
Posted by "Kristian Waagan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kristian Waagan closed DERBY-2646.
----------------------------------
> Cleanup of Clob control/support structures
> ------------------------------------------
>
> Key: DERBY-2646
> URL: https://issues.apache.org/jira/browse/DERBY-2646
> Project: Derby
> Issue Type: Sub-task
> Components: JDBC
> Affects Versions: 10.3.1.4
> Reporter: Kristian Waagan
> Assignee: Kristian Waagan
> Fix For: 10.3.1.4
>
> Attachments: derby-2646-01a_trailing_space.diff, derby-2646-01a_trailing_space.stat, derby-2646-02a-clob-reorg.diff, derby-2646-02a-clob-reorg.stat, derby-2646-03a-internalclob.diff, derby-2646-04a-storestreamclob_utf8util.diff, derby-2646-04a-storestreamclob_utf8util.stat, derby-2646-04b-storestreamclob_utf8util.diff, derby-2646-05a-tests.diff, derby-2646-05a-tests.stat, derby-2646-05b-tests.diff, derby-2646-06a-embedclob_clobstreamcontrol.diff, derby-2646-06a-embedclob_clobstreamcontrol.stat, derby-2646-06b-embedclob_clobstreamcontrol.diff, derby-2646-06b-embedclob_clobstreamcontrol.stat
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (DERBY-2646) Cleanup of Clob control/support
structures
Posted by "Kristian Waagan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kristian Waagan updated DERBY-2646:
-----------------------------------
Attachment: derby-2646-06b-embedclob_clobstreamcontrol.stat
derby-2646-06b-embedclob_clobstreamcontrol.diff
'derby-2646-06b-embedclob_clobstreamcontrol.diff' is based on 06a, but includes the following changes:
* Enables the new jdbcapi/ClobTest.
* Fixed two bugs in ClobStreamControl related to the position caching.
* A few doc changes.
* Cleaned up some code in the jdbc4/ClobTest and enabled a few more methods to be tested for throwing an exception after free is called. There is a bug in this area (code not tested earlier), but it requires addition of SQLStates and to define the correct behavior of setString so I will report/handle it separately.
This is the last major patch for this issue. Please test the Clob functionality.
If you find bugs, please ask if it is known already, or file a Jira for it.
Committed version 06b to trunk with revision 544814.
Will resolve/close in a few days if appropriate.
> Cleanup of Clob control/support structures
> ------------------------------------------
>
> Key: DERBY-2646
> URL: https://issues.apache.org/jira/browse/DERBY-2646
> Project: Derby
> Issue Type: Sub-task
> Components: JDBC
> Affects Versions: 10.3.0.0
> Reporter: Kristian Waagan
> Assignee: Kristian Waagan
> Fix For: 10.3.0.0
>
> Attachments: derby-2646-01a_trailing_space.diff, derby-2646-01a_trailing_space.stat, derby-2646-02a-clob-reorg.diff, derby-2646-02a-clob-reorg.stat, derby-2646-03a-internalclob.diff, derby-2646-04a-storestreamclob_utf8util.diff, derby-2646-04a-storestreamclob_utf8util.stat, derby-2646-04b-storestreamclob_utf8util.diff, derby-2646-05a-tests.diff, derby-2646-05a-tests.stat, derby-2646-05b-tests.diff, derby-2646-06a-embedclob_clobstreamcontrol.diff, derby-2646-06a-embedclob_clobstreamcontrol.stat, derby-2646-06b-embedclob_clobstreamcontrol.diff, derby-2646-06b-embedclob_clobstreamcontrol.stat
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (DERBY-2646) Cleanup of Clob control/support
structures
Posted by "Kristian Waagan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kristian Waagan updated DERBY-2646:
-----------------------------------
Attachment: derby-2646-04b-storestreamclob_utf8util.diff
'derby-2646-04b-storestreamclob_utf8util.diff' is the same as version 04a, except that the methods StoreStreamClob.getByteLength() and StoreStreamClob.resetStoreStream() (private) were changed to honor and take care of the encoded length in the store stream.
Committed to trunk with revision 544111.
> Cleanup of Clob control/support structures
> ------------------------------------------
>
> Key: DERBY-2646
> URL: https://issues.apache.org/jira/browse/DERBY-2646
> Project: Derby
> Issue Type: Sub-task
> Components: JDBC
> Affects Versions: 10.3.0.0
> Reporter: Kristian Waagan
> Assignee: Kristian Waagan
> Fix For: 10.3.0.0
>
> Attachments: derby-2646-01a_trailing_space.diff, derby-2646-01a_trailing_space.stat, derby-2646-02a-clob-reorg.diff, derby-2646-02a-clob-reorg.stat, derby-2646-03a-internalclob.diff, derby-2646-04a-storestreamclob_utf8util.diff, derby-2646-04a-storestreamclob_utf8util.stat, derby-2646-04b-storestreamclob_utf8util.diff
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-2646) Cleanup of Clob control/support
structures
Posted by "Kristian Waagan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12499929 ]
Kristian Waagan commented on DERBY-2646:
----------------------------------------
Committed patch version 03a (InternalClob) to trunk with revision 542641.
This piece of code is not yet in use.
> Cleanup of Clob control/support structures
> ------------------------------------------
>
> Key: DERBY-2646
> URL: https://issues.apache.org/jira/browse/DERBY-2646
> Project: Derby
> Issue Type: Sub-task
> Components: JDBC
> Affects Versions: 10.3.0.0
> Reporter: Kristian Waagan
> Assignee: Kristian Waagan
> Attachments: derby-2646-01a_trailing_space.diff, derby-2646-01a_trailing_space.stat, derby-2646-02a-clob-reorg.diff, derby-2646-02a-clob-reorg.stat, derby-2646-03a-internalclob.diff
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-2646) Cleanup of Clob control/support
structures
Posted by "Kristian Waagan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12501576 ]
Kristian Waagan commented on DERBY-2646:
----------------------------------------
Yes you are right, they are now redundant.
I say now because earlier the variables were not accessed in the constructor but simply tucked away, and you would get the NPE/CCE when you executed some other method on the Clob object. For the reasons you mention, they should now be removed. Committed check removals and JavaDoc additions to trunk with revision 544503.
Thanks a lot!
> Cleanup of Clob control/support structures
> ------------------------------------------
>
> Key: DERBY-2646
> URL: https://issues.apache.org/jira/browse/DERBY-2646
> Project: Derby
> Issue Type: Sub-task
> Components: JDBC
> Affects Versions: 10.3.0.0
> Reporter: Kristian Waagan
> Assignee: Kristian Waagan
> Fix For: 10.3.0.0
>
> Attachments: derby-2646-01a_trailing_space.diff, derby-2646-01a_trailing_space.stat, derby-2646-02a-clob-reorg.diff, derby-2646-02a-clob-reorg.stat, derby-2646-03a-internalclob.diff, derby-2646-04a-storestreamclob_utf8util.diff, derby-2646-04a-storestreamclob_utf8util.stat, derby-2646-04b-storestreamclob_utf8util.diff, derby-2646-05a-tests.diff, derby-2646-05a-tests.stat, derby-2646-05b-tests.diff
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-2646) Cleanup of Clob control/support
structures
Posted by "Knut Anders Hatlen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12501566 ]
Knut Anders Hatlen commented on DERBY-2646:
-------------------------------------------
Thanks for explaining the synchronization and opening a new issue!
One more minor comment. Aren't these checks in StoreStreamClob's constructor redundant?
if (stream == null) {
throw new IllegalArgumentException("Stream cannot be <null>");
}
if (conChild == null) {
throw new IllegalArgumentException("conChild cannot be <null>");
}
if (!(stream instanceof Resetable)) {
throw new IllegalArgumentException("Stream must be resetable: " +
stream.toString());
}
Without these checks, the constructor would have thrown NullPointerException (for the first two) and ClassCastException (for the last one). I'm not sure throwing IllegalArgumentException instead of NPE/CCE adds any value. Perhaps we should remove the checks and add @throws tags to the javadoc stating explicitly that we get NPE when stream or conChild is null and CCE if stream is not a Resetable?
> Cleanup of Clob control/support structures
> ------------------------------------------
>
> Key: DERBY-2646
> URL: https://issues.apache.org/jira/browse/DERBY-2646
> Project: Derby
> Issue Type: Sub-task
> Components: JDBC
> Affects Versions: 10.3.0.0
> Reporter: Kristian Waagan
> Assignee: Kristian Waagan
> Fix For: 10.3.0.0
>
> Attachments: derby-2646-01a_trailing_space.diff, derby-2646-01a_trailing_space.stat, derby-2646-02a-clob-reorg.diff, derby-2646-02a-clob-reorg.stat, derby-2646-03a-internalclob.diff, derby-2646-04a-storestreamclob_utf8util.diff, derby-2646-04a-storestreamclob_utf8util.stat, derby-2646-04b-storestreamclob_utf8util.diff, derby-2646-05a-tests.diff, derby-2646-05a-tests.stat, derby-2646-05b-tests.diff
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-2646) Cleanup of Clob control/support
structures
Posted by "Knut Anders Hatlen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12501575 ]
Knut Anders Hatlen commented on DERBY-2646:
-------------------------------------------
> d) I need to look into/verify synchronization (on the connection
> child). Can anyone please enlighten me on the importance of calling
> setupContextStack/restoreContextStack?
I wondered about this myself when I looked at the patch. It seems like the existing clob/blob implementation sets up the context stack most of the times it accesses the stream to/from store, but not always. Perhaps someone else knows the details about when it is required?
> Cleanup of Clob control/support structures
> ------------------------------------------
>
> Key: DERBY-2646
> URL: https://issues.apache.org/jira/browse/DERBY-2646
> Project: Derby
> Issue Type: Sub-task
> Components: JDBC
> Affects Versions: 10.3.0.0
> Reporter: Kristian Waagan
> Assignee: Kristian Waagan
> Fix For: 10.3.0.0
>
> Attachments: derby-2646-01a_trailing_space.diff, derby-2646-01a_trailing_space.stat, derby-2646-02a-clob-reorg.diff, derby-2646-02a-clob-reorg.stat, derby-2646-03a-internalclob.diff, derby-2646-04a-storestreamclob_utf8util.diff, derby-2646-04a-storestreamclob_utf8util.stat, derby-2646-04b-storestreamclob_utf8util.diff, derby-2646-05a-tests.diff, derby-2646-05a-tests.stat, derby-2646-05b-tests.diff
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-2646) Cleanup of Clob control/support
structures
Posted by "Kristian Waagan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12497486 ]
Kristian Waagan commented on DERBY-2646:
----------------------------------------
I'm planning to do a review and cleanup of the underlying structures of Clob in the embedded driver. I foresee that ClobStreamControl and maybe EmbedClob will be affected by this, but some other classes might have to be changed as well.
My current plans:
1) Write a new test for Clob.
Only methods defined by JDBC 3 or earlier will be tested. Currently these tests reside in jdbcapi/ClobTest.java (in my sandbox), but there is already a ClobTest in the jdbc4 package. I would prefer to rename the jdbc4 one to something else. Any opinions on this?
2) Review and rewrite ClobStreamControl as appropriate.
I will focus especially on getting the byte and clob positions right, as it has been suggested these have been mixed up on occasions. Changes might propagate to EmbedClob, but I hope the changes there will be minimal.
Preliminary findings:
When executing the new tests, I quickly got tangled up in the current difference of behavior between the embedded and the client driver for setBytes etc. I do not plan to work on these, as I understand other work going on at the moment will fix these issues when the switch to locators is made effective.
I also see differences in handling of the "ASCII" encoding, probably related to the client using US-ASCII and embedded ISO-8859-1. I believe the latter is the correct one, and that enabling locators might resolve the issues.
Last, I see an UTFDataFormat exception when inserting some Norwegian characters with setAsciiStream.
If people have comments, questions or additions to my little plan, please let me know.
I do plan to get this issue done for 10.3.
> Cleanup of Clob control/support structures
> ------------------------------------------
>
> Key: DERBY-2646
> URL: https://issues.apache.org/jira/browse/DERBY-2646
> Project: Derby
> Issue Type: Sub-task
> Components: JDBC
> Affects Versions: 10.3.0.0
> Reporter: Kristian Waagan
> Assigned To: Kristian Waagan
> Attachments: derby-2646-01a_trailing_space.diff, derby-2646-01a_trailing_space.stat
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (DERBY-2646) Cleanup of Clob control/support
structures
Posted by "Kristian Waagan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kristian Waagan updated DERBY-2646:
-----------------------------------
Attachment: derby-2646-03a-internalclob.diff
'derby-2646-03a-internalclob.diff' deprecates version 02a.
The patch defines a new interface for internal Clob objects. The point of this is to make better use of OO programming techniques, and also to split the code into smaller pieces. This will make it easer to both review and debug.
I plan to have two imlementations:
a) StoreStreamClob - a read-only Clob working on streams from store
b) ClobStreamControl - a writable Clob working on a memory or disk image of the content
I plan to rename the latter to TemporaryClob, as Derby operates on copies of the Clob value when modifications are done.
The switch between the implementations/representations is one-way; you operate on store streams until you modify the Clob. Note that using Connection.createClob() will create a temporary Clob.
As a side note, the matter of representing the temporary Clob in memory or on disk is left to LOBStreamControl.
This patch will be committed shortly.
More patches will follow.
> Cleanup of Clob control/support structures
> ------------------------------------------
>
> Key: DERBY-2646
> URL: https://issues.apache.org/jira/browse/DERBY-2646
> Project: Derby
> Issue Type: Sub-task
> Components: JDBC
> Affects Versions: 10.3.0.0
> Reporter: Kristian Waagan
> Assignee: Kristian Waagan
> Attachments: derby-2646-01a_trailing_space.diff, derby-2646-01a_trailing_space.stat, derby-2646-02a-clob-reorg.diff, derby-2646-02a-clob-reorg.stat, derby-2646-03a-internalclob.diff
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (DERBY-2646) Cleanup of Clob control/support
structures
Posted by "Kristian Waagan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kristian Waagan updated DERBY-2646:
-----------------------------------
Derby Info: [Patch Available]
> Cleanup of Clob control/support structures
> ------------------------------------------
>
> Key: DERBY-2646
> URL: https://issues.apache.org/jira/browse/DERBY-2646
> Project: Derby
> Issue Type: Sub-task
> Components: JDBC
> Affects Versions: 10.3.0.0
> Reporter: Kristian Waagan
> Assigned To: Kristian Waagan
> Attachments: derby-2646-01a_trailing_space.diff, derby-2646-01a_trailing_space.stat, derby-2646-02a-clob-reorg.diff, derby-2646-02a-clob-reorg.diff
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-2646) Cleanup of Clob control/support
structures
Posted by "Kristian Waagan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12501527 ]
Kristian Waagan commented on DERBY-2646:
----------------------------------------
Thanks for the comments Knut Anders.
I'm wondering if you could create a separate Jira for these issues?
My plan was to create a utility class that works nicely, which obviously requires some more work. When we have confidence that it is fast enough and bug-free, we can start using it other places in Derby.
If you create the new Jira I can address you comments in a separate patch. The method you commented should be removed, or maybe rewritten and fitted into skipInternal?
Using a private internal class also sounds like a good idea.
> Cleanup of Clob control/support structures
> ------------------------------------------
>
> Key: DERBY-2646
> URL: https://issues.apache.org/jira/browse/DERBY-2646
> Project: Derby
> Issue Type: Sub-task
> Components: JDBC
> Affects Versions: 10.3.0.0
> Reporter: Kristian Waagan
> Assignee: Kristian Waagan
> Fix For: 10.3.0.0
>
> Attachments: derby-2646-01a_trailing_space.diff, derby-2646-01a_trailing_space.stat, derby-2646-02a-clob-reorg.diff, derby-2646-02a-clob-reorg.stat, derby-2646-03a-internalclob.diff, derby-2646-04a-storestreamclob_utf8util.diff, derby-2646-04a-storestreamclob_utf8util.stat, derby-2646-04b-storestreamclob_utf8util.diff, derby-2646-05a-tests.diff, derby-2646-05a-tests.stat, derby-2646-05b-tests.diff
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (DERBY-2646) Cleanup of Clob control/support
structures
Posted by "Kristian Waagan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kristian Waagan updated DERBY-2646:
-----------------------------------
Attachment: derby-2646-04a-storestreamclob_utf8util.stat
derby-2646-04a-storestreamclob_utf8util.diff
'derby-2646-04a-storestreamclob_utf8util.diff' includes the following changes:
A java/engine/org/apache/derby/impl/jdbc/StoreStreamClob.java
Added StoreStreamClob, which implements InternalClob. It represents
a Clob on top of a stream from store, and is read-only.
M java/engine/org/apache/derby/impl/jdbc/InternalClob.java
Modified JavaDoc. An important aspect of some of the methods is only
described by the JavaDoc, it is not possible to see from the method
signature alone. This is that EOFException is thrown if the position
is larger than the length of the Clob. Should the signature include
both EOFException and IOException in the throws-clause?
A java/engine/org/apache/derby/iapi/util/UTF8Util.java
Created a utility class for operations on byte streams encoded as
UTF-8. Currently only a few skip methods are in there. The goal of
this class is to centralize the error-prone activity of decoding
UTF-8 encoded streams. Please review!
The main internal method does look a bit verbose to me, but I don't
think it is worth trimming down and "obfuscating" it until it is
shown to have performance problems.
Again, none of this code is enabled. This will happen in the next patch,
where EmbedClob and StoreStreamControl will be modified. There will also
be a separate patch for additional tests.
I expect to commit this code really fast.
> Cleanup of Clob control/support structures
> ------------------------------------------
>
> Key: DERBY-2646
> URL: https://issues.apache.org/jira/browse/DERBY-2646
> Project: Derby
> Issue Type: Sub-task
> Components: JDBC
> Affects Versions: 10.3.0.0
> Reporter: Kristian Waagan
> Assignee: Kristian Waagan
> Fix For: 10.3.0.0
>
> Attachments: derby-2646-01a_trailing_space.diff, derby-2646-01a_trailing_space.stat, derby-2646-02a-clob-reorg.diff, derby-2646-02a-clob-reorg.stat, derby-2646-03a-internalclob.diff, derby-2646-04a-storestreamclob_utf8util.diff, derby-2646-04a-storestreamclob_utf8util.stat
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-2646) Cleanup of Clob control/support
structures
Posted by "Knut Anders Hatlen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12501532 ]
Knut Anders Hatlen commented on DERBY-2646:
-------------------------------------------
>I'm wondering if you could create a separate Jira for these issues?
Done. Logged as DERBY-2760.
> Cleanup of Clob control/support structures
> ------------------------------------------
>
> Key: DERBY-2646
> URL: https://issues.apache.org/jira/browse/DERBY-2646
> Project: Derby
> Issue Type: Sub-task
> Components: JDBC
> Affects Versions: 10.3.0.0
> Reporter: Kristian Waagan
> Assignee: Kristian Waagan
> Fix For: 10.3.0.0
>
> Attachments: derby-2646-01a_trailing_space.diff, derby-2646-01a_trailing_space.stat, derby-2646-02a-clob-reorg.diff, derby-2646-02a-clob-reorg.stat, derby-2646-03a-internalclob.diff, derby-2646-04a-storestreamclob_utf8util.diff, derby-2646-04a-storestreamclob_utf8util.stat, derby-2646-04b-storestreamclob_utf8util.diff, derby-2646-05a-tests.diff, derby-2646-05a-tests.stat, derby-2646-05b-tests.diff
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (DERBY-2646) Cleanup of Clob control/support
structures
Posted by "Kristian Waagan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kristian Waagan updated DERBY-2646:
-----------------------------------
Attachment: (was: derby-2646-02a-clob-reorg.diff)
> Cleanup of Clob control/support structures
> ------------------------------------------
>
> Key: DERBY-2646
> URL: https://issues.apache.org/jira/browse/DERBY-2646
> Project: Derby
> Issue Type: Sub-task
> Components: JDBC
> Affects Versions: 10.3.0.0
> Reporter: Kristian Waagan
> Assigned To: Kristian Waagan
> Attachments: derby-2646-01a_trailing_space.diff, derby-2646-01a_trailing_space.stat, derby-2646-02a-clob-reorg.diff
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (DERBY-2646) Cleanup of Clob control/support
structures
Posted by "Kristian Waagan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kristian Waagan updated DERBY-2646:
-----------------------------------
Urgency: (was: Urgent)
Summary: Cleanup of Clob control/support structures (was: Cleanup of LOB control/support structures)
Changed the name of the issue to reflect the fact that only work related to Clob will be tracked here.
> Cleanup of Clob control/support structures
> ------------------------------------------
>
> Key: DERBY-2646
> URL: https://issues.apache.org/jira/browse/DERBY-2646
> Project: Derby
> Issue Type: Sub-task
> Components: JDBC
> Affects Versions: 10.3.0.0
> Reporter: Kristian Waagan
> Assigned To: Kristian Waagan
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-2646) Cleanup of Clob control/support
structures
Posted by "Knut Anders Hatlen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12501548 ]
Knut Anders Hatlen commented on DERBY-2646:
-------------------------------------------
StoreStreamClob has a volatile boolean field released. Could you elaborate a bit on why it needs to be volatile?
The field storeStream has a comment which says @GuardedBy("synchronizationObject"). I'm not sure I understand, since there are many methods that access storeStream without synchronizing on synchronizationObject.
> Cleanup of Clob control/support structures
> ------------------------------------------
>
> Key: DERBY-2646
> URL: https://issues.apache.org/jira/browse/DERBY-2646
> Project: Derby
> Issue Type: Sub-task
> Components: JDBC
> Affects Versions: 10.3.0.0
> Reporter: Kristian Waagan
> Assignee: Kristian Waagan
> Fix For: 10.3.0.0
>
> Attachments: derby-2646-01a_trailing_space.diff, derby-2646-01a_trailing_space.stat, derby-2646-02a-clob-reorg.diff, derby-2646-02a-clob-reorg.stat, derby-2646-03a-internalclob.diff, derby-2646-04a-storestreamclob_utf8util.diff, derby-2646-04a-storestreamclob_utf8util.stat, derby-2646-04b-storestreamclob_utf8util.diff, derby-2646-05a-tests.diff, derby-2646-05a-tests.stat, derby-2646-05b-tests.diff
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (DERBY-2646) Cleanup of Clob control/support
structures
Posted by "Kristian Waagan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kristian Waagan updated DERBY-2646:
-----------------------------------
Attachment: derby-2646-01a_trailing_space.stat
derby-2646-01a_trailing_space.diff
'derby-2646-01a_trailing_space.diff' is a patch for removing trailing spaces in ClobStreamControl and EmbedClob.
I'm committing this without review, it only deletes trailing spaces. It is a preparation for other changes, to avoid potentially confusing diffs.
Committed to trunk with revision 540083.
> Cleanup of Clob control/support structures
> ------------------------------------------
>
> Key: DERBY-2646
> URL: https://issues.apache.org/jira/browse/DERBY-2646
> Project: Derby
> Issue Type: Sub-task
> Components: JDBC
> Affects Versions: 10.3.0.0
> Reporter: Kristian Waagan
> Assigned To: Kristian Waagan
> Attachments: derby-2646-01a_trailing_space.diff, derby-2646-01a_trailing_space.stat
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (DERBY-2646) Cleanup of LOB control/support
structures
Posted by "Kristian Waagan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kristian Waagan updated DERBY-2646:
-----------------------------------
Component/s: JDBC
Urgency: Urgent
Affects Version/s: 10.3.0.0
> Cleanup of LOB control/support structures
> -----------------------------------------
>
> Key: DERBY-2646
> URL: https://issues.apache.org/jira/browse/DERBY-2646
> Project: Derby
> Issue Type: Sub-task
> Components: JDBC
> Affects Versions: 10.3.0.0
> Reporter: Kristian Waagan
> Assigned To: Kristian Waagan
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (DERBY-2646) Cleanup of Clob control/support
structures
Posted by "Kristian Waagan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kristian Waagan updated DERBY-2646:
-----------------------------------
Attachment: derby-2646-02a-clob-reorg.stat
> Cleanup of Clob control/support structures
> ------------------------------------------
>
> Key: DERBY-2646
> URL: https://issues.apache.org/jira/browse/DERBY-2646
> Project: Derby
> Issue Type: Sub-task
> Components: JDBC
> Affects Versions: 10.3.0.0
> Reporter: Kristian Waagan
> Assigned To: Kristian Waagan
> Attachments: derby-2646-01a_trailing_space.diff, derby-2646-01a_trailing_space.stat, derby-2646-02a-clob-reorg.diff, derby-2646-02a-clob-reorg.stat
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (DERBY-2646) Cleanup of Clob control/support
structures
Posted by "Kristian Waagan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kristian Waagan resolved DERBY-2646.
------------------------------------
Resolution: Fixed
Derby Info: (was: [Patch Available])
> Cleanup of Clob control/support structures
> ------------------------------------------
>
> Key: DERBY-2646
> URL: https://issues.apache.org/jira/browse/DERBY-2646
> Project: Derby
> Issue Type: Sub-task
> Components: JDBC
> Affects Versions: 10.3.0.0
> Reporter: Kristian Waagan
> Assignee: Kristian Waagan
> Fix For: 10.3.0.0
>
> Attachments: derby-2646-01a_trailing_space.diff, derby-2646-01a_trailing_space.stat, derby-2646-02a-clob-reorg.diff, derby-2646-02a-clob-reorg.stat, derby-2646-03a-internalclob.diff, derby-2646-04a-storestreamclob_utf8util.diff, derby-2646-04a-storestreamclob_utf8util.stat, derby-2646-04b-storestreamclob_utf8util.diff, derby-2646-05a-tests.diff, derby-2646-05a-tests.stat, derby-2646-05b-tests.diff, derby-2646-06a-embedclob_clobstreamcontrol.diff, derby-2646-06a-embedclob_clobstreamcontrol.stat, derby-2646-06b-embedclob_clobstreamcontrol.diff, derby-2646-06b-embedclob_clobstreamcontrol.stat
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (DERBY-2646) Cleanup of Clob control/support
structures
Posted by "Kristian Waagan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kristian Waagan updated DERBY-2646:
-----------------------------------
Attachment: derby-2646-05b-tests.diff
'derby-2646-05b-tests.diff' fixes a few things I forgot in 05a; license headers, removed some temporary testing stuff, also added a one more test to ClobTest.
Note that these tests are still not enabled.
Committed 'derby-2646-05b-tests.diff' to trunk with revision 544445.
> Cleanup of Clob control/support structures
> ------------------------------------------
>
> Key: DERBY-2646
> URL: https://issues.apache.org/jira/browse/DERBY-2646
> Project: Derby
> Issue Type: Sub-task
> Components: JDBC
> Affects Versions: 10.3.0.0
> Reporter: Kristian Waagan
> Assignee: Kristian Waagan
> Fix For: 10.3.0.0
>
> Attachments: derby-2646-01a_trailing_space.diff, derby-2646-01a_trailing_space.stat, derby-2646-02a-clob-reorg.diff, derby-2646-02a-clob-reorg.stat, derby-2646-03a-internalclob.diff, derby-2646-04a-storestreamclob_utf8util.diff, derby-2646-04a-storestreamclob_utf8util.stat, derby-2646-04b-storestreamclob_utf8util.diff, derby-2646-05a-tests.diff, derby-2646-05a-tests.stat, derby-2646-05b-tests.diff
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (DERBY-2646) Cleanup of Clob control/support
structures
Posted by "Kristian Waagan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kristian Waagan updated DERBY-2646:
-----------------------------------
Attachment: derby-2646-05a-tests.stat
derby-2646-05a-tests.diff
'derby-2646-05a-tests.diff' contains a series of tests for Clob.
More tests can/should be added, they are far from complete.
There is one "normal" test, 'jdbcapi/ClobTest'. It is not enabled in any suite yet, as two of the tests fails with the current implementation due to UTFDataFormatException (which again is due to positioning/boundary errors). They will be enabled as part of the next patch for this issue.
There is also four package-private test classes, all based off the abstract superclass 'InternalClobTest'.
The test classes test the following cases:
SmallStoreStreamTest: Small read-only Clobs from a store stream
BiggerStoreStreamTest: Bigger read-only Clobs from a store stream, size should cause crossing of various buffer sizes and boundaries.
SmallTemporaryClobTest: Temorary read-write Clobs held in memory.
BiggerTemporaryClobTest: Temporary read-write Clobs stored on disk due to their size.
Transition from read-only to read-write is not tested in the package-private tests, as transition is the responsibility of EmbedClob. I also considered adding encryption to BiggerTemporaryClobTest, but decided to put this on hold and address it in a later patch. I think we have some performance issues with encrypted Clobs, but it can be due to the test implementation and should be determined before any conclusions are taken.
The required framework for package-private tests are not in place. I will possibly address this later, but for now the package private tests must be compiled and run manually.
The simplest is to issue for instance this javac command from trunk after having compiled Derby the normal way:
javac -d classes/ -classpath classes/:tools/java/junit.jar java/testing/org/apache/derby/impl/jdbc/*java
Then run them with 'classes/' in the classpath:
java junit.textui.TestRunner org.apache.derby.impl.jdbc._Suite
> Cleanup of Clob control/support structures
> ------------------------------------------
>
> Key: DERBY-2646
> URL: https://issues.apache.org/jira/browse/DERBY-2646
> Project: Derby
> Issue Type: Sub-task
> Components: JDBC
> Affects Versions: 10.3.0.0
> Reporter: Kristian Waagan
> Assignee: Kristian Waagan
> Fix For: 10.3.0.0
>
> Attachments: derby-2646-01a_trailing_space.diff, derby-2646-01a_trailing_space.stat, derby-2646-02a-clob-reorg.diff, derby-2646-02a-clob-reorg.stat, derby-2646-03a-internalclob.diff, derby-2646-04a-storestreamclob_utf8util.diff, derby-2646-04a-storestreamclob_utf8util.stat, derby-2646-04b-storestreamclob_utf8util.diff, derby-2646-05a-tests.diff, derby-2646-05a-tests.stat
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (DERBY-2646) Cleanup of Clob control/support
structures
Posted by "Kristian Waagan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kristian Waagan updated DERBY-2646:
-----------------------------------
Attachment: derby-2646-06a-embedclob_clobstreamcontrol.stat
derby-2646-06a-embedclob_clobstreamcontrol.diff
'derby-2646-06a-embedclob_clobstreamcontrol.diff' is the last planned patch for this issue.
It changes EmbedClob to use the new InternalClob interface, and updates ClobStreamControl accordingly (basically removes functionality moved to StoreStreamClob).
A few comments on changes in ClobStreamControl:
* I added a simple caching mechanism for the last character position accessed. The cache simply stores the corresponding byte position. Its primary objective is to reduce the penalty taken when writing/reading one byte/char at a time.
* It no longer extends LOBStreamControl, but keeps a reference to an instance of it (composition). This caused the addition of some methods that only forwards the calls to LOBStreamControl.
There will be a version 06b of this patch, which I plan to commit tomorrow morning, but I wanted to give people a change to have a look. For the curious, I suggest applying the patch and looking at the Java source files instead of the diff. The changes are partly very extensive.
What remains for 06b is any last minute changes I might see fit and the enabling of the tests. Note that also this patch will have some shortcomings that have to be addressed through other Jiras (for instance DERBY-2762 and the streaming issues).
All feedback is appreciated!
> Cleanup of Clob control/support structures
> ------------------------------------------
>
> Key: DERBY-2646
> URL: https://issues.apache.org/jira/browse/DERBY-2646
> Project: Derby
> Issue Type: Sub-task
> Components: JDBC
> Affects Versions: 10.3.0.0
> Reporter: Kristian Waagan
> Assignee: Kristian Waagan
> Fix For: 10.3.0.0
>
> Attachments: derby-2646-01a_trailing_space.diff, derby-2646-01a_trailing_space.stat, derby-2646-02a-clob-reorg.diff, derby-2646-02a-clob-reorg.stat, derby-2646-03a-internalclob.diff, derby-2646-04a-storestreamclob_utf8util.diff, derby-2646-04a-storestreamclob_utf8util.stat, derby-2646-04b-storestreamclob_utf8util.diff, derby-2646-05a-tests.diff, derby-2646-05a-tests.stat, derby-2646-05b-tests.diff, derby-2646-06a-embedclob_clobstreamcontrol.diff, derby-2646-06a-embedclob_clobstreamcontrol.stat
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-2646) Cleanup of Clob control/support
structures
Posted by "Kristian Waagan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12501559 ]
Kristian Waagan commented on DERBY-2646:
----------------------------------------
I was planning to use it to ensure visibility of the variable without having to obtain a lock. This it achieves, but since release() is closing the store stream as well, I need to obtain a lock to ensure the stream is not closed while an operation is being performed on it. Thus the volatile modifier must go away (I will fix this at once).
And the reason why you don't understand much of the synchronization policy, is because it is not yet in place :)
The task grew a bit bigger than expected, and I need to push it out to "get things done". It is also bad to track all these things under this issue.
I have created DERBY-2762 to track the synchronization issue. There are also some related issues already filed, and I hope we can link them up soon.
Even though there are important things not yet addressed regarded to Clob, all our tests passes...
As a little warning, we might see some instabilities related to Clobs. There are two reasons; this rewrite on the embedded side, and the enabling of locator support on the client side. I think we should get these improvements into trunk ASAP and then address the remaining issues.
> Cleanup of Clob control/support structures
> ------------------------------------------
>
> Key: DERBY-2646
> URL: https://issues.apache.org/jira/browse/DERBY-2646
> Project: Derby
> Issue Type: Sub-task
> Components: JDBC
> Affects Versions: 10.3.0.0
> Reporter: Kristian Waagan
> Assignee: Kristian Waagan
> Fix For: 10.3.0.0
>
> Attachments: derby-2646-01a_trailing_space.diff, derby-2646-01a_trailing_space.stat, derby-2646-02a-clob-reorg.diff, derby-2646-02a-clob-reorg.stat, derby-2646-03a-internalclob.diff, derby-2646-04a-storestreamclob_utf8util.diff, derby-2646-04a-storestreamclob_utf8util.stat, derby-2646-04b-storestreamclob_utf8util.diff, derby-2646-05a-tests.diff, derby-2646-05a-tests.stat, derby-2646-05b-tests.diff
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.