You are viewing a plain text version of this content. The canonical link for it is here.
Posted to derby-dev@db.apache.org by "Kristian Waagan (JIRA)" <ji...@apache.org> on 2010/01/19 16:13:05 UTC
[jira] Created: (DERBY-4519) Infinite loop in
StreamFileContainer.writeColumn
Infinite loop in StreamFileContainer.writeColumn
------------------------------------------------
Key: DERBY-4519
URL: https://issues.apache.org/jira/browse/DERBY-4519
Project: Derby
Issue Type: Bug
Components: Store
Affects Versions: 10.5.3.0, 10.4.2.0, 10.3.3.0, 10.2.2.0, 10.1.3.1, 10.0.2.1
Reporter: Kristian Waagan
Assignee: Kristian Waagan
Fix For: 10.6.0.0
The offset and length argument have been swapped in the calls to InputStream.read(byte,offset, length) and write(byte,offset, length). This code is inside a do-while (true) loop, and the only normal way out is when InputStream.read returns -1. This will never happen since the stream is asked to read zero bytes. Derby ends up eating up all available CPU, limited to a single core / CPU.
The bug hasn't been observed because Derby is materializing all values when calling this code. Enabling streaming capabilities in the sorter revealed it.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-4519) Infinite loop in
StreamFileContainer.writeColumn
Posted by "Knut Anders Hatlen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-4519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12802757#action_12802757 ]
Knut Anders Hatlen commented on DERBY-4519:
-------------------------------------------
Hi Kristian,
The suggested bounding of the buffer size sounds fine. We should probably also add a short comment about why it's needed.
> Infinite loop in StreamFileContainer.writeColumn
> ------------------------------------------------
>
> Key: DERBY-4519
> URL: https://issues.apache.org/jira/browse/DERBY-4519
> Project: Derby
> Issue Type: Bug
> Components: Store
> Affects Versions: 10.0.2.1, 10.1.3.1, 10.2.2.0, 10.3.3.0, 10.4.2.0, 10.5.3.0, 10.6.0.0
> Reporter: Kristian Waagan
> Assignee: Kristian Waagan
> Fix For: 10.6.0.0
>
> Attachments: derby-4519-1a-argument_swap.diff
>
>
> The offset and length argument have been swapped in the calls to InputStream.read(byte,offset, length) and write(byte,offset, length). This code is inside a do-while (true) loop, and the only normal way out is when InputStream.read returns -1. This will never happen since the stream is asked to read zero bytes. Derby ends up eating up all available CPU, limited to a single core / CPU.
> The bug hasn't been observed because Derby is materializing all values when calling this code. Enabling streaming capabilities in the sorter revealed it.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-4519) Infinite loop in
StreamFileContainer.writeColumn
Posted by "Kristian Waagan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-4519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12802732#action_12802732 ]
Kristian Waagan commented on DERBY-4519:
----------------------------------------
Hi Knut,
I think we can address the additional problem you spotted here as well. The question is which thresholds to use.
Assuming available() will give reasonable answers in most cases (may very well be the case if we are reading from internal store streams), which limits should we impose on the buffer size?
[64, 8192], which could look like something like this?
(BufferedInputStream uses 8 KB as default buffer size)
int bufferLen = Math.min(Math.max(in.available(), 64), 8192).
Based on the experiences from DERBY-3791, I would be careful about defaulting to a largish size if we do get reasonable answers from available().
> Infinite loop in StreamFileContainer.writeColumn
> ------------------------------------------------
>
> Key: DERBY-4519
> URL: https://issues.apache.org/jira/browse/DERBY-4519
> Project: Derby
> Issue Type: Bug
> Components: Store
> Affects Versions: 10.0.2.1, 10.1.3.1, 10.2.2.0, 10.3.3.0, 10.4.2.0, 10.5.3.0, 10.6.0.0
> Reporter: Kristian Waagan
> Assignee: Kristian Waagan
> Fix For: 10.6.0.0
>
> Attachments: derby-4519-1a-argument_swap.diff
>
>
> The offset and length argument have been swapped in the calls to InputStream.read(byte,offset, length) and write(byte,offset, length). This code is inside a do-while (true) loop, and the only normal way out is when InputStream.read returns -1. This will never happen since the stream is asked to read zero bytes. Derby ends up eating up all available CPU, limited to a single core / CPU.
> The bug hasn't been observed because Derby is materializing all values when calling this code. Enabling streaming capabilities in the sorter revealed it.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (DERBY-4519) Infinite loop in
StreamFileContainer.writeColumn
Posted by "Kristian Waagan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-4519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kristian Waagan updated DERBY-4519:
-----------------------------------
Issue & fix info: [Patch Available]
Affects Version/s: 10.6.0.0
> Infinite loop in StreamFileContainer.writeColumn
> ------------------------------------------------
>
> Key: DERBY-4519
> URL: https://issues.apache.org/jira/browse/DERBY-4519
> Project: Derby
> Issue Type: Bug
> Components: Store
> Affects Versions: 10.0.2.1, 10.1.3.1, 10.2.2.0, 10.3.3.0, 10.4.2.0, 10.5.3.0, 10.6.0.0
> Reporter: Kristian Waagan
> Assignee: Kristian Waagan
> Fix For: 10.6.0.0
>
> Attachments: derby-4519-1a-argument_swap.diff
>
>
> The offset and length argument have been swapped in the calls to InputStream.read(byte,offset, length) and write(byte,offset, length). This code is inside a do-while (true) loop, and the only normal way out is when InputStream.read returns -1. This will never happen since the stream is asked to read zero bytes. Derby ends up eating up all available CPU, limited to a single core / CPU.
> The bug hasn't been observed because Derby is materializing all values when calling this code. Enabling streaming capabilities in the sorter revealed it.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (DERBY-4519) Infinite loop in
StreamFileContainer.writeColumn
Posted by "Kristian Waagan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-4519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kristian Waagan resolved DERBY-4519.
------------------------------------
Resolution: Fixed
Fix Version/s: 10.5.3.1
Issue & fix info: (was: [Patch Available])
Thanks for the review, Knut Anders.
I committed patch 2a to trunk with revision 901165.
Back-ported to the 10.5 branch with revision 901167 for good measure, but I don't expect that we will enable the affected code on that branch. For the same reason I'm not planning to back-port to older branches.
> Infinite loop in StreamFileContainer.writeColumn
> ------------------------------------------------
>
> Key: DERBY-4519
> URL: https://issues.apache.org/jira/browse/DERBY-4519
> Project: Derby
> Issue Type: Bug
> Components: Store
> Affects Versions: 10.0.2.1, 10.1.3.1, 10.2.2.0, 10.3.3.0, 10.4.2.0, 10.5.3.0, 10.6.0.0
> Reporter: Kristian Waagan
> Assignee: Kristian Waagan
> Fix For: 10.5.3.1, 10.6.0.0
>
> Attachments: derby-4519-1a-argument_swap.diff, derby-4519-2a-infinite_loop_fixes.diff
>
>
> The offset and length argument have been swapped in the calls to InputStream.read(byte,offset, length) and write(byte,offset, length). This code is inside a do-while (true) loop, and the only normal way out is when InputStream.read returns -1. This will never happen since the stream is asked to read zero bytes. Derby ends up eating up all available CPU, limited to a single core / CPU.
> The bug hasn't been observed because Derby is materializing all values when calling this code. Enabling streaming capabilities in the sorter revealed it.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-4519) Infinite loop in
StreamFileContainer.writeColumn
Posted by "Knut Anders Hatlen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-4519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12802307#action_12802307 ]
Knut Anders Hatlen commented on DERBY-4519:
-------------------------------------------
Another problem with this code is that it uses InputStream.available() to decide the size of the read buffer. If available() returns 0 (which it is allowed to) an empty read buffer is used, and this will also lead to an infinite loop, I think.
The patch looks like an improvement, though. Minor nit: Calling the read(byte[]) method that takes no offset/length arguments would be equivalent to read(bufData,0,bufferLen) and slightly simpler.
> Infinite loop in StreamFileContainer.writeColumn
> ------------------------------------------------
>
> Key: DERBY-4519
> URL: https://issues.apache.org/jira/browse/DERBY-4519
> Project: Derby
> Issue Type: Bug
> Components: Store
> Affects Versions: 10.0.2.1, 10.1.3.1, 10.2.2.0, 10.3.3.0, 10.4.2.0, 10.5.3.0, 10.6.0.0
> Reporter: Kristian Waagan
> Assignee: Kristian Waagan
> Fix For: 10.6.0.0
>
> Attachments: derby-4519-1a-argument_swap.diff
>
>
> The offset and length argument have been swapped in the calls to InputStream.read(byte,offset, length) and write(byte,offset, length). This code is inside a do-while (true) loop, and the only normal way out is when InputStream.read returns -1. This will never happen since the stream is asked to read zero bytes. Derby ends up eating up all available CPU, limited to a single core / CPU.
> The bug hasn't been observed because Derby is materializing all values when calling this code. Enabling streaming capabilities in the sorter revealed it.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Closed: (DERBY-4519) Infinite loop in
StreamFileContainer.writeColumn
Posted by "Kristian Waagan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-4519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kristian Waagan closed DERBY-4519.
----------------------------------
Closing the issue.
> Infinite loop in StreamFileContainer.writeColumn
> ------------------------------------------------
>
> Key: DERBY-4519
> URL: https://issues.apache.org/jira/browse/DERBY-4519
> Project: Derby
> Issue Type: Bug
> Components: Store
> Affects Versions: 10.0.2.1, 10.1.3.1, 10.2.2.0, 10.3.3.0, 10.4.2.0, 10.5.3.0, 10.6.0.0
> Reporter: Kristian Waagan
> Assignee: Kristian Waagan
> Fix For: 10.5.3.1, 10.6.0.0
>
> Attachments: derby-4519-1a-argument_swap.diff, derby-4519-2a-infinite_loop_fixes.diff
>
>
> The offset and length argument have been swapped in the calls to InputStream.read(byte,offset, length) and write(byte,offset, length). This code is inside a do-while (true) loop, and the only normal way out is when InputStream.read returns -1. This will never happen since the stream is asked to read zero bytes. Derby ends up eating up all available CPU, limited to a single core / CPU.
> The bug hasn't been observed because Derby is materializing all values when calling this code. Enabling streaming capabilities in the sorter revealed it.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (DERBY-4519) Infinite loop in
StreamFileContainer.writeColumn
Posted by "Kristian Waagan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-4519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kristian Waagan updated DERBY-4519:
-----------------------------------
Attachment: derby-4519-2a-infinite_loop_fixes.diff
Patch 2a deprecates patch 1a, and it also addresses the issue (and the nit) noted by Knut Anders.
The buffer size will now be determined by InputStream.available() given that it returns a value between 64 and 8192 (8 KB).
Since this code path doesn't appear to be reached currently, I don't expect any changes in behavior at this point.
The fix is an incremental improvement regardless of the exact values of the limits , as the existing code would cause Derby to enter an infinite loop if executed.
Patch ready for review.
> Infinite loop in StreamFileContainer.writeColumn
> ------------------------------------------------
>
> Key: DERBY-4519
> URL: https://issues.apache.org/jira/browse/DERBY-4519
> Project: Derby
> Issue Type: Bug
> Components: Store
> Affects Versions: 10.0.2.1, 10.1.3.1, 10.2.2.0, 10.3.3.0, 10.4.2.0, 10.5.3.0, 10.6.0.0
> Reporter: Kristian Waagan
> Assignee: Kristian Waagan
> Fix For: 10.6.0.0
>
> Attachments: derby-4519-1a-argument_swap.diff, derby-4519-2a-infinite_loop_fixes.diff
>
>
> The offset and length argument have been swapped in the calls to InputStream.read(byte,offset, length) and write(byte,offset, length). This code is inside a do-while (true) loop, and the only normal way out is when InputStream.read returns -1. This will never happen since the stream is asked to read zero bytes. Derby ends up eating up all available CPU, limited to a single core / CPU.
> The bug hasn't been observed because Derby is materializing all values when calling this code. Enabling streaming capabilities in the sorter revealed it.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (DERBY-4519) Infinite loop in
StreamFileContainer.writeColumn
Posted by "Kristian Waagan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-4519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kristian Waagan updated DERBY-4519:
-----------------------------------
Attachment: derby-4519-1a-argument_swap.diff
Attached patch 1a, which swaps the length and offset arguments.
> Infinite loop in StreamFileContainer.writeColumn
> ------------------------------------------------
>
> Key: DERBY-4519
> URL: https://issues.apache.org/jira/browse/DERBY-4519
> Project: Derby
> Issue Type: Bug
> Components: Store
> Affects Versions: 10.0.2.1, 10.1.3.1, 10.2.2.0, 10.3.3.0, 10.4.2.0, 10.5.3.0
> Reporter: Kristian Waagan
> Assignee: Kristian Waagan
> Fix For: 10.6.0.0
>
> Attachments: derby-4519-1a-argument_swap.diff
>
>
> The offset and length argument have been swapped in the calls to InputStream.read(byte,offset, length) and write(byte,offset, length). This code is inside a do-while (true) loop, and the only normal way out is when InputStream.read returns -1. This will never happen since the stream is asked to read zero bytes. Derby ends up eating up all available CPU, limited to a single core / CPU.
> The bug hasn't been observed because Derby is materializing all values when calling this code. Enabling streaming capabilities in the sorter revealed it.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-4519) Infinite loop in
StreamFileContainer.writeColumn
Posted by "Knut Anders Hatlen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-4519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12802807#action_12802807 ]
Knut Anders Hatlen commented on DERBY-4519:
-------------------------------------------
Thanks, Kristian. The patch looks good. +1
> Infinite loop in StreamFileContainer.writeColumn
> ------------------------------------------------
>
> Key: DERBY-4519
> URL: https://issues.apache.org/jira/browse/DERBY-4519
> Project: Derby
> Issue Type: Bug
> Components: Store
> Affects Versions: 10.0.2.1, 10.1.3.1, 10.2.2.0, 10.3.3.0, 10.4.2.0, 10.5.3.0, 10.6.0.0
> Reporter: Kristian Waagan
> Assignee: Kristian Waagan
> Fix For: 10.6.0.0
>
> Attachments: derby-4519-1a-argument_swap.diff, derby-4519-2a-infinite_loop_fixes.diff
>
>
> The offset and length argument have been swapped in the calls to InputStream.read(byte,offset, length) and write(byte,offset, length). This code is inside a do-while (true) loop, and the only normal way out is when InputStream.read returns -1. This will never happen since the stream is asked to read zero bytes. Derby ends up eating up all available CPU, limited to a single core / CPU.
> The bug hasn't been observed because Derby is materializing all values when calling this code. Enabling streaming capabilities in the sorter revealed it.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.