You are viewing a plain text version of this content. The canonical link for it is here.
Posted to derby-dev@db.apache.org by "Kristian Waagan (JIRA)" <ji...@apache.org> on 2007/05/23 23:44:16 UTC
[jira] Created: (DERBY-2686) The skip method for some InputStreams
and Readers return invalid values
The skip method for some InputStreams and Readers return invalid values
-----------------------------------------------------------------------
Key: DERBY-2686
URL: https://issues.apache.org/jira/browse/DERBY-2686
Project: Derby
Issue Type: Bug
Components: JDBC, Store
Affects Versions: 10.3.0.0
Reporter: Kristian Waagan
Assigned To: Kristian Waagan
The Java API docs for InputStream.skip and Reader.skip seem to indicate that returning a negative value is breaking the contract.
The contract for Reader.skip is the more clear one, while I have taken the assumption that all Derby InputStreams will return 0 only when EOF has been reached or 0 is passed in as the amount of bytes to skip.
Bad checking in a skip method also caused Derby to enter an infinite loop in a skip method.
It should also be noted that skipping bytes/characters should be done in a loop, as skip is free to skip a smaller amount of bytes than requested. This is true even if EOF is not reached.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (DERBY-2686) The skip method for some InputStreams
and Readers return invalid values
Posted by "Kristian Waagan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-2686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kristian Waagan updated DERBY-2686:
-----------------------------------
Attachment: derby-2686-2a-reader.stat
derby-2686-2a-reader.diff
'derby-2686-2a-reader.diff' makes UTF8Reader.skip compliant with the contract described in the Java API docs.
suites.All ran without failures. Running derbyall now.
The class is only used in one other class, so the change should be quite safe.
Will wait a little while with the commit in case there are comments.
> The skip method for some InputStreams and Readers return invalid values
> -----------------------------------------------------------------------
>
> Key: DERBY-2686
> URL: https://issues.apache.org/jira/browse/DERBY-2686
> Project: Derby
> Issue Type: Bug
> Components: JDBC, Store
> Affects Versions: 10.3.0.0
> Reporter: Kristian Waagan
> Assigned To: Kristian Waagan
> Attachments: derby-2686-1a.diff, derby-2686-1a.stat, derby-2686-2a-reader.diff, derby-2686-2a-reader.stat
>
>
> The Java API docs for InputStream.skip and Reader.skip seem to indicate that returning a negative value is breaking the contract.
> The contract for Reader.skip is the more clear one, while I have taken the assumption that all Derby InputStreams will return 0 only when EOF has been reached or 0 is passed in as the amount of bytes to skip.
> Bad checking in a skip method also caused Derby to enter an infinite loop in a skip method.
> It should also be noted that skipping bytes/characters should be done in a loop, as skip is free to skip a smaller amount of bytes than requested. This is true even if EOF is not reached.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-2686) The skip method for some
InputStreams and Readers return invalid values
Posted by "Kristian Waagan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-2686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12498624 ]
Kristian Waagan commented on DERBY-2686:
----------------------------------------
FYI, from this years JavaOne, session TS-2707 - "Java Puzzlers Episode VI: The Phantom-Reference Menace/Attack of the Clone/Revenge of the Shift" by Joshua Bloch, Google, Inc.; William Pugh, Univ. of Maryland, pdf page 35 and 36:
"static void skipFully(InputStream in, long nBytes)
throws IOException {
long remaining = nBytes;
while (remaining != 0) {
long skipped = in.skip(remaining);
if (skipped == 0)
throw new EOFException();
remaining -= skipped;
}
}"
And the following statements:
"Moral
• The skip method is hard to use and error prone
• Use your skipFully instead of skip
• There is an RFE to add it to InputStream
• More generally, if an API is broken, wrap it
• For API designers
• Don't violate the principle of least astonishment
• Make it easy to do simple things"
Again, this does not necessarily justify a specific solution/implementation, it is just another data point.
> The skip method for some InputStreams and Readers return invalid values
> -----------------------------------------------------------------------
>
> Key: DERBY-2686
> URL: https://issues.apache.org/jira/browse/DERBY-2686
> Project: Derby
> Issue Type: Bug
> Components: JDBC, Store
> Affects Versions: 10.3.0.0
> Reporter: Kristian Waagan
> Assigned To: Kristian Waagan
> Attachments: derby-2686-1a.diff, derby-2686-1a.stat
>
>
> The Java API docs for InputStream.skip and Reader.skip seem to indicate that returning a negative value is breaking the contract.
> The contract for Reader.skip is the more clear one, while I have taken the assumption that all Derby InputStreams will return 0 only when EOF has been reached or 0 is passed in as the amount of bytes to skip.
> Bad checking in a skip method also caused Derby to enter an infinite loop in a skip method.
> It should also be noted that skipping bytes/characters should be done in a loop, as skip is free to skip a smaller amount of bytes than requested. This is true even if EOF is not reached.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (DERBY-2686) The skip method for some InputStreams
and Readers return invalid values
Posted by "Kristian Waagan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-2686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kristian Waagan updated DERBY-2686:
-----------------------------------
Attachment: derby-2686-3a.stat
derby-2686-3a.diff
'derby-2686-3a.diff' makes a few changes to some streams. The most important change is that BufferedByteHolderStream jumps out of the loop when the parent reader returns 0. It is up to higher level streams/code to make conclusions about the state of the stream (if EOF is reached etc).
suites.All ran without failures. Ready for review.
> The skip method for some InputStreams and Readers return invalid values
> -----------------------------------------------------------------------
>
> Key: DERBY-2686
> URL: https://issues.apache.org/jira/browse/DERBY-2686
> Project: Derby
> Issue Type: Bug
> Components: JDBC, Store
> Affects Versions: 10.3.0.0
> Reporter: Kristian Waagan
> Assigned To: Kristian Waagan
> Attachments: derby-2686-1a.diff, derby-2686-1a.stat, derby-2686-2a-reader.diff, derby-2686-2a-reader.stat, derby-2686-3a.diff, derby-2686-3a.stat
>
>
> The Java API docs for InputStream.skip and Reader.skip seem to indicate that returning a negative value is breaking the contract.
> The contract for Reader.skip is the more clear one, while I have taken the assumption that all Derby InputStreams will return 0 only when EOF has been reached or 0 is passed in as the amount of bytes to skip.
> Bad checking in a skip method also caused Derby to enter an infinite loop in a skip method.
> It should also be noted that skipping bytes/characters should be done in a loop, as skip is free to skip a smaller amount of bytes than requested. This is true even if EOF is not reached.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Closed: (DERBY-2686) The skip method for some InputStreams
and Readers return invalid values
Posted by "Kristian Waagan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-2686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kristian Waagan closed DERBY-2686.
----------------------------------
> The skip method for some InputStreams and Readers return invalid values
> -----------------------------------------------------------------------
>
> Key: DERBY-2686
> URL: https://issues.apache.org/jira/browse/DERBY-2686
> Project: Derby
> Issue Type: Bug
> Components: JDBC, Store
> Affects Versions: 10.3.0.0
> Reporter: Kristian Waagan
> Assignee: Kristian Waagan
> Fix For: 10.3.0.0
>
> Attachments: derby-2686-1a.diff, derby-2686-1a.stat, derby-2686-2a-reader.diff, derby-2686-2a-reader.stat, derby-2686-3a.diff, derby-2686-3a.stat
>
>
> The Java API docs for InputStream.skip and Reader.skip seem to indicate that returning a negative value is breaking the contract.
> The contract for Reader.skip is the more clear one, while I have taken the assumption that all Derby InputStreams will return 0 only when EOF has been reached or 0 is passed in as the amount of bytes to skip.
> Bad checking in a skip method also caused Derby to enter an infinite loop in a skip method.
> It should also be noted that skipping bytes/characters should be done in a loop, as skip is free to skip a smaller amount of bytes than requested. This is true even if EOF is not reached.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (DERBY-2686) The skip method for some InputStreams
and Readers return invalid values
Posted by "Kristian Waagan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-2686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kristian Waagan updated DERBY-2686:
-----------------------------------
Attachment: derby-2686-1a.stat
derby-2686-1a.diff
Attaching a proposal for a patch for this issue. I'm running tests now and will report back tomorrow.
If the tests run fine, I plan to commit the patch rather quickly as it causes Derby to hang infinitely in another patch I am working on. I will of course take feedback into account :)
The Java API docs leave room for interpretation, but since the InputStreams in question are internal I think the assumption that 0 means EOF is safe. We do not depend on this behavior for user-land streams.
> The skip method for some InputStreams and Readers return invalid values
> -----------------------------------------------------------------------
>
> Key: DERBY-2686
> URL: https://issues.apache.org/jira/browse/DERBY-2686
> Project: Derby
> Issue Type: Bug
> Components: JDBC, Store
> Affects Versions: 10.3.0.0
> Reporter: Kristian Waagan
> Assigned To: Kristian Waagan
> Attachments: derby-2686-1a.diff, derby-2686-1a.stat
>
>
> The Java API docs for InputStream.skip and Reader.skip seem to indicate that returning a negative value is breaking the contract.
> The contract for Reader.skip is the more clear one, while I have taken the assumption that all Derby InputStreams will return 0 only when EOF has been reached or 0 is passed in as the amount of bytes to skip.
> Bad checking in a skip method also caused Derby to enter an infinite loop in a skip method.
> It should also be noted that skipping bytes/characters should be done in a loop, as skip is free to skip a smaller amount of bytes than requested. This is true even if EOF is not reached.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (DERBY-2686) The skip method for some InputStreams
and Readers return invalid values
Posted by "Kristian Waagan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-2686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kristian Waagan updated DERBY-2686:
-----------------------------------
Derby Info: [Patch Available]
> The skip method for some InputStreams and Readers return invalid values
> -----------------------------------------------------------------------
>
> Key: DERBY-2686
> URL: https://issues.apache.org/jira/browse/DERBY-2686
> Project: Derby
> Issue Type: Bug
> Components: JDBC, Store
> Affects Versions: 10.3.0.0
> Reporter: Kristian Waagan
> Assigned To: Kristian Waagan
> Attachments: derby-2686-1a.diff, derby-2686-1a.stat, derby-2686-2a-reader.diff, derby-2686-2a-reader.stat, derby-2686-3a.diff, derby-2686-3a.stat
>
>
> The Java API docs for InputStream.skip and Reader.skip seem to indicate that returning a negative value is breaking the contract.
> The contract for Reader.skip is the more clear one, while I have taken the assumption that all Derby InputStreams will return 0 only when EOF has been reached or 0 is passed in as the amount of bytes to skip.
> Bad checking in a skip method also caused Derby to enter an infinite loop in a skip method.
> It should also be noted that skipping bytes/characters should be done in a loop, as skip is free to skip a smaller amount of bytes than requested. This is true even if EOF is not reached.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (DERBY-2686) The skip method for some InputStreams
and Readers return invalid values
Posted by "Kristian Waagan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-2686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kristian Waagan resolved DERBY-2686.
------------------------------------
Resolution: Fixed
Fix Version/s: 10.3.0.0
Derby Info: (was: [Patch Available])
Committed 'derby-2686-2a-reader.diff' to trunk with revision 542269.
No more work is expected on this issue.
> The skip method for some InputStreams and Readers return invalid values
> -----------------------------------------------------------------------
>
> Key: DERBY-2686
> URL: https://issues.apache.org/jira/browse/DERBY-2686
> Project: Derby
> Issue Type: Bug
> Components: JDBC, Store
> Affects Versions: 10.3.0.0
> Reporter: Kristian Waagan
> Assignee: Kristian Waagan
> Fix For: 10.3.0.0
>
> Attachments: derby-2686-1a.diff, derby-2686-1a.stat, derby-2686-2a-reader.diff, derby-2686-2a-reader.stat, derby-2686-3a.diff, derby-2686-3a.stat
>
>
> The Java API docs for InputStream.skip and Reader.skip seem to indicate that returning a negative value is breaking the contract.
> The contract for Reader.skip is the more clear one, while I have taken the assumption that all Derby InputStreams will return 0 only when EOF has been reached or 0 is passed in as the amount of bytes to skip.
> Bad checking in a skip method also caused Derby to enter an infinite loop in a skip method.
> It should also be noted that skipping bytes/characters should be done in a loop, as skip is free to skip a smaller amount of bytes than requested. This is true even if EOF is not reached.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-2686) The skip method for some
InputStreams and Readers return invalid values
Posted by "Kristian Waagan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-2686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12499418 ]
Kristian Waagan commented on DERBY-2686:
----------------------------------------
Committed 'derby-2686-3a.diff' to trunk with revision 542005.
> The skip method for some InputStreams and Readers return invalid values
> -----------------------------------------------------------------------
>
> Key: DERBY-2686
> URL: https://issues.apache.org/jira/browse/DERBY-2686
> Project: Derby
> Issue Type: Bug
> Components: JDBC, Store
> Affects Versions: 10.3.0.0
> Reporter: Kristian Waagan
> Assigned To: Kristian Waagan
> Attachments: derby-2686-1a.diff, derby-2686-1a.stat, derby-2686-2a-reader.diff, derby-2686-2a-reader.stat, derby-2686-3a.diff, derby-2686-3a.stat
>
>
> The Java API docs for InputStream.skip and Reader.skip seem to indicate that returning a negative value is breaking the contract.
> The contract for Reader.skip is the more clear one, while I have taken the assumption that all Derby InputStreams will return 0 only when EOF has been reached or 0 is passed in as the amount of bytes to skip.
> Bad checking in a skip method also caused Derby to enter an infinite loop in a skip method.
> It should also be noted that skipping bytes/characters should be done in a loop, as skip is free to skip a smaller amount of bytes than requested. This is true even if EOF is not reached.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-2686) The skip method for some
InputStreams and Readers return invalid values
Posted by "Daniel John Debrunner (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-2686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12498409 ]
Daniel John Debrunner commented on DERBY-2686:
----------------------------------------------
It might be dangerous to make the assumption that 0 means EOF for internal streams. Future changes may wrap/push non-internal streams (e.g. buffering) around internal streams. Thus some code that thought it was reading an internal stream suddenly starts reading a non-internal stream where as defined by InputStream.skip() 0 is a valid return for any number of reasons:
"This may result from any of a number of conditions; reaching end of file before n bytes have been skipped is only one possibility. "
I think it's wise to use InputStreams according to their contract.
What problem is being solved here?
> The skip method for some InputStreams and Readers return invalid values
> -----------------------------------------------------------------------
>
> Key: DERBY-2686
> URL: https://issues.apache.org/jira/browse/DERBY-2686
> Project: Derby
> Issue Type: Bug
> Components: JDBC, Store
> Affects Versions: 10.3.0.0
> Reporter: Kristian Waagan
> Assigned To: Kristian Waagan
> Attachments: derby-2686-1a.diff, derby-2686-1a.stat
>
>
> The Java API docs for InputStream.skip and Reader.skip seem to indicate that returning a negative value is breaking the contract.
> The contract for Reader.skip is the more clear one, while I have taken the assumption that all Derby InputStreams will return 0 only when EOF has been reached or 0 is passed in as the amount of bytes to skip.
> Bad checking in a skip method also caused Derby to enter an infinite loop in a skip method.
> It should also be noted that skipping bytes/characters should be done in a loop, as skip is free to skip a smaller amount of bytes than requested. This is true even if EOF is not reached.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (DERBY-2686) The skip method for some InputStreams
and Readers return invalid values
Posted by "Kristian Waagan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-2686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kristian Waagan updated DERBY-2686:
-----------------------------------
Derby Info: [Patch Available]
Patch ready for review.
> The skip method for some InputStreams and Readers return invalid values
> -----------------------------------------------------------------------
>
> Key: DERBY-2686
> URL: https://issues.apache.org/jira/browse/DERBY-2686
> Project: Derby
> Issue Type: Bug
> Components: JDBC, Store
> Affects Versions: 10.3.0.0
> Reporter: Kristian Waagan
> Assigned To: Kristian Waagan
> Attachments: derby-2686-1a.diff, derby-2686-1a.stat
>
>
> The Java API docs for InputStream.skip and Reader.skip seem to indicate that returning a negative value is breaking the contract.
> The contract for Reader.skip is the more clear one, while I have taken the assumption that all Derby InputStreams will return 0 only when EOF has been reached or 0 is passed in as the amount of bytes to skip.
> Bad checking in a skip method also caused Derby to enter an infinite loop in a skip method.
> It should also be noted that skipping bytes/characters should be done in a loop, as skip is free to skip a smaller amount of bytes than requested. This is true even if EOF is not reached.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (DERBY-2686) The skip method for some InputStreams
and Readers return invalid values
Posted by "Kristian Waagan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-2686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kristian Waagan updated DERBY-2686:
-----------------------------------
Derby Info: (was: [Patch Available])
As I said, this area is a bit "fuzzzy"...
What exactly is the contract of InputStream.skip?
The problem being solved, is an infinite loop when skip returns 0. Since you really can't tell if the stream just chooses to skip zero bytes, or if EOF has been reached, it is hard to know when to stop calling skip.
To be more on the safe side, I plan to investigate the possibility of checking for 0 inside the higher level skip-method, then do a read() to check if EOF has been reached or not.
Have a look at BufferedByteHolderInputStream.skip for an example of code that hangs.
Secondly, we have readers in Derby that returns -1 in skip, which definitely breaks the contract:
"This method will block until some characters are available, an I/O error occurs, or the end of the stream is reached.
Returns:
The number of characters actually skipped"
I will split the patch I submitted into at least two smaller ones, and the patch 1a is also buggy. I forgot to update a check in EmbedClob from -1 to 0, which causes length() to hang.
> The skip method for some InputStreams and Readers return invalid values
> -----------------------------------------------------------------------
>
> Key: DERBY-2686
> URL: https://issues.apache.org/jira/browse/DERBY-2686
> Project: Derby
> Issue Type: Bug
> Components: JDBC, Store
> Affects Versions: 10.3.0.0
> Reporter: Kristian Waagan
> Assigned To: Kristian Waagan
> Attachments: derby-2686-1a.diff, derby-2686-1a.stat
>
>
> The Java API docs for InputStream.skip and Reader.skip seem to indicate that returning a negative value is breaking the contract.
> The contract for Reader.skip is the more clear one, while I have taken the assumption that all Derby InputStreams will return 0 only when EOF has been reached or 0 is passed in as the amount of bytes to skip.
> Bad checking in a skip method also caused Derby to enter an infinite loop in a skip method.
> It should also be noted that skipping bytes/characters should be done in a loop, as skip is free to skip a smaller amount of bytes than requested. This is true even if EOF is not reached.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.