You are viewing a plain text version of this content. The canonical link for it is here.
Posted to derby-dev@db.apache.org by "Tomohito Nakayama (JIRA)" <de...@db.apache.org> on 2005/12/23 06:24:31 UTC
[jira] Created: (DERBY-782) Improvement on org.apache.derby.impl.jdbc.EmbedBlob#length()
Improvement on org.apache.derby.impl.jdbc.EmbedBlob#length()
------------------------------------------------------------
Key: DERBY-782
URL: http://issues.apache.org/jira/browse/DERBY-782
Project: Derby
Type: Bug
Components: JDBC
Reporter: Tomohito Nakayama
Now, org.apache.derby.impl.jdbc.EmbedBlob#length() method read out whole BinaryToRawStream to know exact length.
On the other hand, BinaryToRawStream have some commented-out inplementation of having information for length.
I think the information of lengh in BinaryToRawStream should be restored to be used in .org.apache.derby.impl.jdbc.EmbedBlob#length(), because read out whole stream can be expensive processing when streamed information was large.
There exists a subject that reliability of lengh information in BinaryToRawStream is unknown.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Commented: (DERBY-782) Improvement on
org.apache.derby.impl.jdbc.EmbedBlob#length()
Posted by "Tomohito Nakayama (JIRA)" <de...@db.apache.org>.
[ http://issues.apache.org/jira/browse/DERBY-782?page=comments#action_12377492 ]
Tomohito Nakayama commented on DERBY-782:
-----------------------------------------
I read modification of DERBY-438
that length of blob is memoried in org.apache.derby.impl.jdbc.EmbedBlob,
if once length() method was called and all of blob was read out.
My original intention in this task was to use information of lengh in BinaryToRawStream,
which was found as invalid.
Then my original intention was not achieved.
I'm not exactly sure why information of length in BinaryToRawStream is invalid,
though it is likely to be invalid.
Here, "likely" means that I think keeping information for length of blob in BinaryToRawStream valid may be difficult and fragile.
Well ....
The information for length of blob in BinaryToRawStream is a little mystery for me,
because I do'nt know the history why it is invalid.
> Improvement on org.apache.derby.impl.jdbc.EmbedBlob#length()
> ------------------------------------------------------------
>
> Key: DERBY-782
> URL: http://issues.apache.org/jira/browse/DERBY-782
> Project: Derby
> Type: Bug
> Components: JDBC
> Reporter: Tomohito Nakayama
> Fix For: 10.2.0.0
> Attachments: testLob4.java
>
> Now, org.apache.derby.impl.jdbc.EmbedBlob#length() method read out whole BinaryToRawStream to know exact length.
> On the other hand, BinaryToRawStream have some commented-out inplementation of having information for length.
> I think the information of lengh in BinaryToRawStream should be restored to be used in .org.apache.derby.impl.jdbc.EmbedBlob#length(), because read out whole stream can be expensive processing when streamed information was large.
> There exists a subject that reliability of lengh information in BinaryToRawStream is unknown.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Updated: (DERBY-782) Improvement on org.apache.derby.impl.jdbc.EmbedBlob#length()
Posted by "Tomohito Nakayama (JIRA)" <de...@db.apache.org>.
[ http://issues.apache.org/jira/browse/DERBY-782?page=all ]
Tomohito Nakayama updated DERBY-782:
------------------------------------
Comment: was deleted
> Improvement on org.apache.derby.impl.jdbc.EmbedBlob#length()
> ------------------------------------------------------------
>
> Key: DERBY-782
> URL: http://issues.apache.org/jira/browse/DERBY-782
> Project: Derby
> Type: Bug
> Components: JDBC
> Reporter: Tomohito Nakayama
> Attachments: testLob4.java
>
> Now, org.apache.derby.impl.jdbc.EmbedBlob#length() method read out whole BinaryToRawStream to know exact length.
> On the other hand, BinaryToRawStream have some commented-out inplementation of having information for length.
> I think the information of lengh in BinaryToRawStream should be restored to be used in .org.apache.derby.impl.jdbc.EmbedBlob#length(), because read out whole stream can be expensive processing when streamed information was large.
> There exists a subject that reliability of lengh information in BinaryToRawStream is unknown.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Updated: (DERBY-782) Improvement on org.apache.derby.impl.jdbc.EmbedBlob#length()
Posted by "Tomohito Nakayama (JIRA)" <de...@db.apache.org>.
[ http://issues.apache.org/jira/browse/DERBY-782?page=all ]
Tomohito Nakayama updated DERBY-782:
------------------------------------
Attachment: testLob4.java
Current implementation of org.apache.derby.impl.jdbc.EmbedBlob#length() have some problematic behavior.
If length() method was called when InputStream was being read, the InputStream was finished.
Attached testLob4.java shows this behavior.
naka@rufelza:~/derby/test/20051223$ java testLob4
Here goes 1st stream
0
1
2
length:1048576
Here continue 1st stream
-1
-1
-1
I found next description in javadoc comment in org.apache.derby.impl.jdbc.EmbedBlob.
This means that we have limited control over the state of the stream, since the user can read bytes from it at any time.
Thus all methods here reset the stream to the beginning before doing any work
This problematic behavior may be resolved spontaneously in this issue.
> Improvement on org.apache.derby.impl.jdbc.EmbedBlob#length()
> ------------------------------------------------------------
>
> Key: DERBY-782
> URL: http://issues.apache.org/jira/browse/DERBY-782
> Project: Derby
> Type: Bug
> Components: JDBC
> Reporter: Tomohito Nakayama
> Attachments: testLob4.java
>
> Now, org.apache.derby.impl.jdbc.EmbedBlob#length() method read out whole BinaryToRawStream to know exact length.
> On the other hand, BinaryToRawStream have some commented-out inplementation of having information for length.
> I think the information of lengh in BinaryToRawStream should be restored to be used in .org.apache.derby.impl.jdbc.EmbedBlob#length(), because read out whole stream can be expensive processing when streamed information was large.
> There exists a subject that reliability of lengh information in BinaryToRawStream is unknown.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
Question around length of long column in disk page format (Re: [jira]
Commented: (DERBY-782) Improvement on org.apache.derby.impl.jdbc.EmbedBlob#length())
Posted by TomohitoNakayama <to...@basil.ocn.ne.jp>.
Hello.
I have surveyed code where length information is/may be written to disk.
Reading code around StoredPage and tracing execution with jdb,
I could not find code to write whole length information of long column
as value in field header in disk page format.
// I read that length of information divided in pages seems to be recorded.
// But not sure why this divided length was not be seen in jdb execution
of test programs...
I read length information of long column value is skipped to be recorded
and
think this is why testLob4.java was resulted as in
http://issues.apache.org/jira/browse/DERBY-782#action_12361194.
I have next questions.
Q1 :
In the view of architecture of disk page format, is it impossible to
record length information of long column?
I know very little about disk page format,
however, there seems to be no explicit description in the document
(http://db.apache.org/derby/papers/pageformats.html)
to conclude that it is impossible.
Q2 :
I read that length information of long column value is skipped to be
recorded.
However, does any other one have other information ?
I'm not so confident ...
Best regards.
Tomohito Nakayama (JIRA) wrote:
> [ http://issues.apache.org/jira/browse/DERBY-782?page=comments#action_12361194 ]
>
>Tomohito Nakayama commented on DERBY-782:
>-----------------------------------------
>
>I have tried and found length information, which is ignored now, does not have correct value.
>
>
>naka@rufelza:~/derby/test/20051223$ runProgram.sh testLob4
>Initializing jdb ...
>
>
>>stop at org.apache.derby.impl.jdbc.BinaryToRawStream:94
>>
>>
>Deferring breakpoint org.apache.derby.impl.jdbc.BinaryToRawStream:94.
>It will be set after the class is loaded.
>
>
>>run
>>
>>
>run testLob4
>Set uncaught java.lang.Throwable
>Set deferred uncaught java.lang.Throwable
>
>
>VM Started: Set deferred breakpoint org.apache.derby.impl.jdbc.BinaryToRawStream:94
>
>Breakpoint hit: "thread=main", org.apache.derby.impl.jdbc.BinaryToRawStream.<init>(), line=94 bci=252
>94 int lenInBits = (((bl & 0xff) << 24) | ((v2 & 0xff) << 16) | ((v3 & 0xff) << 8) | (v4 & 0xff));
>
>main[1] eval b1
>com.sun.tools.example.debug.expr.ParseException: Name unknown: b1
> b1 = null
>main[1] eval bl
> bl = 0
>main[1] eval v2
> v2 = 0
>main[1] eval v3
> v3 = 0
>main[1] eval v4
> v4 = 0
>main[1] cont
>Here goes 1st stream
>0
>1
>2
>length:0
>Here continue 1st stream
>3
>4
>5
>
>
>The application exited
>
>
>
>>Improvement on org.apache.derby.impl.jdbc.EmbedBlob#length()
>>------------------------------------------------------------
>>
>> Key: DERBY-782
>> URL: http://issues.apache.org/jira/browse/DERBY-782
>> Project: Derby
>> Type: Bug
>> Components: JDBC
>> Reporter: Tomohito Nakayama
>> Attachments: testLob4.java
>>
>>Now, org.apache.derby.impl.jdbc.EmbedBlob#length() method read out whole BinaryToRawStream to know exact length.
>>On the other hand, BinaryToRawStream have some commented-out inplementation of having information for length.
>>I think the information of lengh in BinaryToRawStream should be restored to be used in .org.apache.derby.impl.jdbc.EmbedBlob#length(), because read out whole stream can be expensive processing when streamed information was large.
>>There exists a subject that reliability of lengh information in BinaryToRawStream is unknown.
>>
>>
>
>
>
--
/*
Tomohito Nakayama
tomonaka@basil.ocn.ne.jp
tomohito@rose.zero.ad.jp
tmnk@apache.org
Naka
http://www5.ocn.ne.jp/~tomohito/TopPage.html
*/
[jira] Commented: (DERBY-782) Improvement on org.apache.derby.impl.jdbc.EmbedBlob#length()
Posted by "Tomohito Nakayama (JIRA)" <de...@db.apache.org>.
[ http://issues.apache.org/jira/browse/DERBY-782?page=comments#action_12361194 ]
Tomohito Nakayama commented on DERBY-782:
-----------------------------------------
I have tried and found length information, which is ignored now, does not have correct value.
naka@rufelza:~/derby/test/20051223$ runProgram.sh testLob4
Initializing jdb ...
> stop at org.apache.derby.impl.jdbc.BinaryToRawStream:94
Deferring breakpoint org.apache.derby.impl.jdbc.BinaryToRawStream:94.
It will be set after the class is loaded.
> run
run testLob4
Set uncaught java.lang.Throwable
Set deferred uncaught java.lang.Throwable
>
VM Started: Set deferred breakpoint org.apache.derby.impl.jdbc.BinaryToRawStream:94
Breakpoint hit: "thread=main", org.apache.derby.impl.jdbc.BinaryToRawStream.<init>(), line=94 bci=252
94 int lenInBits = (((bl & 0xff) << 24) | ((v2 & 0xff) << 16) | ((v3 & 0xff) << 8) | (v4 & 0xff));
main[1] eval b1
com.sun.tools.example.debug.expr.ParseException: Name unknown: b1
b1 = null
main[1] eval bl
bl = 0
main[1] eval v2
v2 = 0
main[1] eval v3
v3 = 0
main[1] eval v4
v4 = 0
main[1] cont
Here goes 1st stream
0
1
2
length:0
Here continue 1st stream
3
4
5
>
The application exited
> Improvement on org.apache.derby.impl.jdbc.EmbedBlob#length()
> ------------------------------------------------------------
>
> Key: DERBY-782
> URL: http://issues.apache.org/jira/browse/DERBY-782
> Project: Derby
> Type: Bug
> Components: JDBC
> Reporter: Tomohito Nakayama
> Attachments: testLob4.java
>
> Now, org.apache.derby.impl.jdbc.EmbedBlob#length() method read out whole BinaryToRawStream to know exact length.
> On the other hand, BinaryToRawStream have some commented-out inplementation of having information for length.
> I think the information of lengh in BinaryToRawStream should be restored to be used in .org.apache.derby.impl.jdbc.EmbedBlob#length(), because read out whole stream can be expensive processing when streamed information was large.
> There exists a subject that reliability of lengh information in BinaryToRawStream is unknown.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Closed: (DERBY-782) Improvement on
org.apache.derby.impl.jdbc.EmbedBlob#length()
Posted by "Daniel John Debrunner (JIRA)" <de...@db.apache.org>.
[ http://issues.apache.org/jira/browse/DERBY-782?page=all ]
Daniel John Debrunner closed DERBY-782:
---------------------------------------
Fix Version: 10.2.0.0
Resolution: Fixed
Fixed by svn revision 397313 for DERBY-438
http://svn.apache.org/viewcvs?view=rev&rev=397313
> Improvement on org.apache.derby.impl.jdbc.EmbedBlob#length()
> ------------------------------------------------------------
>
> Key: DERBY-782
> URL: http://issues.apache.org/jira/browse/DERBY-782
> Project: Derby
> Type: Bug
> Components: JDBC
> Reporter: Tomohito Nakayama
> Fix For: 10.2.0.0
> Attachments: testLob4.java
>
> Now, org.apache.derby.impl.jdbc.EmbedBlob#length() method read out whole BinaryToRawStream to know exact length.
> On the other hand, BinaryToRawStream have some commented-out inplementation of having information for length.
> I think the information of lengh in BinaryToRawStream should be restored to be used in .org.apache.derby.impl.jdbc.EmbedBlob#length(), because read out whole stream can be expensive processing when streamed information was large.
> There exists a subject that reliability of lengh information in BinaryToRawStream is unknown.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Commented: (DERBY-782) Improvement on org.apache.derby.impl.jdbc.EmbedBlob#length()
Posted by "Tomohito Nakayama (JIRA)" <de...@db.apache.org>.
[ http://issues.apache.org/jira/browse/DERBY-782?page=comments#action_12361165 ]
Tomohito Nakayama commented on DERBY-782:
-----------------------------------------
Current implementation of org.apache.derby.impl.jdbc.EmbedBlob#length() have some problematic behavior.
If length() method was called whenBinaryToRawStream was being read, the BinaryToRawStream was finished.
Attached testLob4.java shows this behavior.
naka@rufelza:~/derby/test/20051223$ java testLob4
Here goes 1st stream
0
1
2
length:1048576
Here continue 1st stream
-1
-1
-1
This problematic behavior may be resolved spontaneously in this issue.
> Improvement on org.apache.derby.impl.jdbc.EmbedBlob#length()
> ------------------------------------------------------------
>
> Key: DERBY-782
> URL: http://issues.apache.org/jira/browse/DERBY-782
> Project: Derby
> Type: Bug
> Components: JDBC
> Reporter: Tomohito Nakayama
> Attachments: testLob4.java
>
> Now, org.apache.derby.impl.jdbc.EmbedBlob#length() method read out whole BinaryToRawStream to know exact length.
> On the other hand, BinaryToRawStream have some commented-out inplementation of having information for length.
> I think the information of lengh in BinaryToRawStream should be restored to be used in .org.apache.derby.impl.jdbc.EmbedBlob#length(), because read out whole stream can be expensive processing when streamed information was large.
> There exists a subject that reliability of lengh information in BinaryToRawStream is unknown.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Commented: (DERBY-782) Improvement on
org.apache.derby.impl.jdbc.EmbedBlob#length()
Posted by "Daniel John Debrunner (JIRA)" <de...@db.apache.org>.
[ http://issues.apache.org/jira/browse/DERBY-782?page=comments#action_12377499 ]
Daniel John Debrunner commented on DERBY-782:
---------------------------------------------
The change 397313 includes as you say saving the length of the stream if it has to read the entire length,
but in addition it has this change.
@@ -221,6 +229,14 @@
setupContextStack();
setPosition(0);
+ // If possible get the length from the encoded
+ // length at the front of the raw stream.
+ if ((myLength = biStream.getLength()) != -1) {
+ biStream.close();
+ return myLength;
+ }
+
Then, I'm sorry I missed referencing additional changes from DERBY-438.
SVN change 394109
This change makes the binary length encoded in the on-disk format be written correctly when
the length is known.
http://svn.apache.org/viewcvs?view=rev&rev=394109
SVN Change 393880
This change saves the encoded length when reading the stream from the disk,
this is then used by EmbedBlob in the 397313 change
http://svn.apache.org/viewcvs?view=rev&rev=393880
Thanks for keeping me honest.
> Improvement on org.apache.derby.impl.jdbc.EmbedBlob#length()
> ------------------------------------------------------------
>
> Key: DERBY-782
> URL: http://issues.apache.org/jira/browse/DERBY-782
> Project: Derby
> Type: Bug
> Components: JDBC
> Reporter: Tomohito Nakayama
> Fix For: 10.2.0.0
> Attachments: testLob4.java
>
> Now, org.apache.derby.impl.jdbc.EmbedBlob#length() method read out whole BinaryToRawStream to know exact length.
> On the other hand, BinaryToRawStream have some commented-out inplementation of having information for length.
> I think the information of lengh in BinaryToRawStream should be restored to be used in .org.apache.derby.impl.jdbc.EmbedBlob#length(), because read out whole stream can be expensive processing when streamed information was large.
> There exists a subject that reliability of lengh information in BinaryToRawStream is unknown.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira