You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Doron Cohen (JIRA)" <ji...@apache.org> on 2007/07/13 00:53:04 UTC
[jira] Created: (LUCENE-957) Lucene RAM Directory doesn't work for
Index Size > 8 GB
Lucene RAM Directory doesn't work for Index Size > 8 GB
-------------------------------------------------------
Key: LUCENE-957
URL: https://issues.apache.org/jira/browse/LUCENE-957
Project: Lucene - Java
Issue Type: Bug
Components: Store
Reporter: Doron Cohen
Assignee: Doron Cohen
from user list - http://www.gossamer-threads.com/lists/lucene/java-user/50982
Problem seems to be casting issues in RAMInputStream.
Line 90:
bufferStart = BUFFER_SIZE * currentBufferIndex;
both rhs are ints while lhs is long.
so a very large product would first overflow MAX_INT, become negative, and only then (auto) casted to long, but this is too late.
Line 91:
bufferLength = (int) (length - bufferStart);
both rhs are longs while lhs is int.
so the (int) cast result may turn negative and the logic that follows would be wrong.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-957) Lucene RAM Directory doesn't work for
Index Size > 8 GB
Posted by "Doron Cohen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doron Cohen updated LUCENE-957:
-------------------------------
Attachment: lucene-957.patch
Patch fixing int / long cast issues in RAMInputStream.
All tests pass but non of those actually demonstrates the bug being fixed. I don't currently have a large enough index to test this.
I will wait for a manual test by Murali (who reported this issue), and for at least one patch review.
> Lucene RAM Directory doesn't work for Index Size > 8 GB
> -------------------------------------------------------
>
> Key: LUCENE-957
> URL: https://issues.apache.org/jira/browse/LUCENE-957
> Project: Lucene - Java
> Issue Type: Bug
> Components: Store
> Reporter: Doron Cohen
> Assignee: Doron Cohen
> Attachments: lucene-957.patch
>
>
> from user list - http://www.gossamer-threads.com/lists/lucene/java-user/50982
> Problem seems to be casting issues in RAMInputStream.
> Line 90:
> bufferStart = BUFFER_SIZE * currentBufferIndex;
> both rhs are ints while lhs is long.
> so a very large product would first overflow MAX_INT, become negative, and only then (auto) casted to long, but this is too late.
> Line 91:
> bufferLength = (int) (length - bufferStart);
> both rhs are longs while lhs is int.
> so the (int) cast result may turn negative and the logic that follows would be wrong.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-957) Lucene RAM Directory doesn't work for
Index Size > 8 GB
Posted by "Doron Cohen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doron Cohen updated LUCENE-957:
-------------------------------
Attachment: lucene-957.patch
Previous patch apparently did not fix the bug - a casting problem in RAMOutputStream had to be fixed.
Updated patch adds a test imitating ramFile larger than maxint.
For this had to make the allocation of a new byte array in RAMFile overridable.
The new test fails before fixing RAMOutputStream (affecting RAMDirectory constructor from FS). However the issues in RAMInputStream in fact do not cause failures, yet they should be fixed.
With a test in place I now feel confident in this fix - will commit it in a day or two if there are no reservations.
> Lucene RAM Directory doesn't work for Index Size > 8 GB
> -------------------------------------------------------
>
> Key: LUCENE-957
> URL: https://issues.apache.org/jira/browse/LUCENE-957
> Project: Lucene - Java
> Issue Type: Bug
> Components: Store
> Reporter: Doron Cohen
> Assignee: Doron Cohen
> Attachments: lucene-957.patch, lucene-957.patch
>
>
> from user list - http://www.gossamer-threads.com/lists/lucene/java-user/50982
> Problem seems to be casting issues in RAMInputStream.
> Line 90:
> bufferStart = BUFFER_SIZE * currentBufferIndex;
> both rhs are ints while lhs is long.
> so a very large product would first overflow MAX_INT, become negative, and only then (auto) casted to long, but this is too late.
> Line 91:
> bufferLength = (int) (length - bufferStart);
> both rhs are longs while lhs is int.
> so the (int) cast result may turn negative and the logic that follows would be wrong.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Resolved: (LUCENE-957) Lucene RAM Directory doesn't work for
Index Size > 8 GB
Posted by "Doron Cohen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doron Cohen resolved LUCENE-957.
--------------------------------
Resolution: Fixed
Lucene Fields: (was: [New])
committed.
> Lucene RAM Directory doesn't work for Index Size > 8 GB
> -------------------------------------------------------
>
> Key: LUCENE-957
> URL: https://issues.apache.org/jira/browse/LUCENE-957
> Project: Lucene - Java
> Issue Type: Bug
> Components: Store
> Reporter: Doron Cohen
> Assignee: Doron Cohen
> Attachments: lucene-957.patch, lucene-957.patch
>
>
> from user list - http://www.gossamer-threads.com/lists/lucene/java-user/50982
> Problem seems to be casting issues in RAMInputStream.
> Line 90:
> bufferStart = BUFFER_SIZE * currentBufferIndex;
> both rhs are ints while lhs is long.
> so a very large product would first overflow MAX_INT, become negative, and only then (auto) casted to long, but this is too late.
> Line 91:
> bufferLength = (int) (length - bufferStart);
> both rhs are longs while lhs is int.
> so the (int) cast result may turn negative and the logic that follows would be wrong.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: [jira] Commented: (LUCENE-957) Lucene RAM Directory doesn't work for
Index Size > 8 GB
Posted by Doron Cohen <DO...@il.ibm.com>.
Murali Varadarajan commented on LUCENE-957:
> It worked !!!
Hi Murali, That's great - thanks for reporting back on the status of this!
Doron
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-957) Lucene RAM Directory doesn't work
for Index Size > 8 GB
Posted by "Murali Varadarajan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12516123 ]
Murali Varadarajan commented on LUCENE-957:
-------------------------------------------
Doron Cohon,
Thanks a tonne. I tried your first patch and it didn't work. Today i saw your message and checked out the source from head and built it.
It worked !!!
:-)
> Lucene RAM Directory doesn't work for Index Size > 8 GB
> -------------------------------------------------------
>
> Key: LUCENE-957
> URL: https://issues.apache.org/jira/browse/LUCENE-957
> Project: Lucene - Java
> Issue Type: Bug
> Components: Store
> Reporter: Doron Cohen
> Assignee: Doron Cohen
> Attachments: lucene-957.patch, lucene-957.patch
>
>
> from user list - http://www.gossamer-threads.com/lists/lucene/java-user/50982
> Problem seems to be casting issues in RAMInputStream.
> Line 90:
> bufferStart = BUFFER_SIZE * currentBufferIndex;
> both rhs are ints while lhs is long.
> so a very large product would first overflow MAX_INT, become negative, and only then (auto) casted to long, but this is too late.
> Line 91:
> bufferLength = (int) (length - bufferStart);
> both rhs are longs while lhs is int.
> so the (int) cast result may turn negative and the logic that follows would be wrong.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org