You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Robert Muir (Created) (JIRA)" <ji...@apache.org> on 2011/10/28 15:58:33 UTC
[jira] [Created] (LUCENE-3541) remove IndexInput.copyBuf
remove IndexInput.copyBuf
-------------------------
Key: LUCENE-3541
URL: https://issues.apache.org/jira/browse/LUCENE-3541
Project: Lucene - Java
Issue Type: Bug
Reporter: Robert Muir
Fix For: 3.5, 4.0
this looks really broken/dangerous as an instance variable.
what happens on clone() ?! copyBytes can instead make its own array inside the method.
its protected, so ill list in the 3.x backwards breaks section since its technically a backwards break.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (LUCENE-3541) remove IndexInput.copyBuf
Posted by "Robert Muir (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Muir updated LUCENE-3541:
--------------------------------
Attachment: LUCENE-3541.patch
here's the obvious fix, but no test yet.
> remove IndexInput.copyBuf
> -------------------------
>
> Key: LUCENE-3541
> URL: https://issues.apache.org/jira/browse/LUCENE-3541
> Project: Lucene - Java
> Issue Type: Bug
> Reporter: Robert Muir
> Priority: Critical
> Fix For: 3.5, 4.0
>
> Attachments: LUCENE-3541.patch
>
>
> this looks really broken/dangerous as an instance variable.
> what happens on clone() ?! copyBytes can instead make its own array inside the method.
> its protected, so ill list in the 3.x backwards breaks section since its technically a backwards break.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (LUCENE-3541) remove IndexInput.copyBuf
Posted by "Uwe Schindler (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13138393#comment-13138393 ]
Uwe Schindler commented on LUCENE-3541:
---------------------------------------
It's horrible: When you clone this IndexInput and then in another thread start another copy operation, both will override the others buffer contents. What's the reason for having this as instance variable. If you create the byte[] in the method and it's unused afterwards it happily resides in eden space....
> remove IndexInput.copyBuf
> -------------------------
>
> Key: LUCENE-3541
> URL: https://issues.apache.org/jira/browse/LUCENE-3541
> Project: Lucene - Java
> Issue Type: Bug
> Reporter: Robert Muir
> Fix For: 3.5, 4.0
>
>
> this looks really broken/dangerous as an instance variable.
> what happens on clone() ?! copyBytes can instead make its own array inside the method.
> its protected, so ill list in the 3.x backwards breaks section since its technically a backwards break.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Assigned] (LUCENE-3541) remove IndexInput.copyBuf
Posted by "Robert Muir (Assigned) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Muir reassigned LUCENE-3541:
-----------------------------------
Assignee: Robert Muir
> remove IndexInput.copyBuf
> -------------------------
>
> Key: LUCENE-3541
> URL: https://issues.apache.org/jira/browse/LUCENE-3541
> Project: Lucene - Java
> Issue Type: Bug
> Reporter: Robert Muir
> Assignee: Robert Muir
> Priority: Critical
> Fix For: 3.5, 4.0
>
> Attachments: LUCENE-3541.patch, LUCENE-3541.patch, LUCENE-3541.patch
>
>
> this looks really broken/dangerous as an instance variable.
> what happens on clone() ?! copyBytes can instead make its own array inside the method.
> its protected, so ill list in the 3.x backwards breaks section since its technically a backwards break.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Resolved] (LUCENE-3541) remove IndexInput.copyBuf
Posted by "Robert Muir (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Muir resolved LUCENE-3541.
---------------------------------
Resolution: Fixed
> remove IndexInput.copyBuf
> -------------------------
>
> Key: LUCENE-3541
> URL: https://issues.apache.org/jira/browse/LUCENE-3541
> Project: Lucene - Java
> Issue Type: Bug
> Reporter: Robert Muir
> Assignee: Robert Muir
> Priority: Critical
> Fix For: 3.5, 4.0
>
> Attachments: LUCENE-3541.patch, LUCENE-3541.patch, LUCENE-3541.patch
>
>
> this looks really broken/dangerous as an instance variable.
> what happens on clone() ?! copyBytes can instead make its own array inside the method.
> its protected, so ill list in the 3.x backwards breaks section since its technically a backwards break.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (LUCENE-3541) remove IndexInput.copyBuf
Posted by "Uwe Schindler (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Uwe Schindler updated LUCENE-3541:
----------------------------------
Priority: Critical (was: Major)
> remove IndexInput.copyBuf
> -------------------------
>
> Key: LUCENE-3541
> URL: https://issues.apache.org/jira/browse/LUCENE-3541
> Project: Lucene - Java
> Issue Type: Bug
> Reporter: Robert Muir
> Priority: Critical
> Fix For: 3.5, 4.0
>
>
> this looks really broken/dangerous as an instance variable.
> what happens on clone() ?! copyBytes can instead make its own array inside the method.
> its protected, so ill list in the 3.x backwards breaks section since its technically a backwards break.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (LUCENE-3541) remove IndexInput.copyBuf
Posted by "Robert Muir (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Muir updated LUCENE-3541:
--------------------------------
Attachment: LUCENE-3541.patch
here's a prototype test (it needs to be toned down, but it does the trick), that fails without the patch, passes with it.
it will only fail with mmapdirectory, all the other core directories override this method and have some custom implementation already.
> remove IndexInput.copyBuf
> -------------------------
>
> Key: LUCENE-3541
> URL: https://issues.apache.org/jira/browse/LUCENE-3541
> Project: Lucene - Java
> Issue Type: Bug
> Reporter: Robert Muir
> Priority: Critical
> Fix For: 3.5, 4.0
>
> Attachments: LUCENE-3541.patch, LUCENE-3541.patch
>
>
> this looks really broken/dangerous as an instance variable.
> what happens on clone() ?! copyBytes can instead make its own array inside the method.
> its protected, so ill list in the 3.x backwards breaks section since its technically a backwards break.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (LUCENE-3541) remove IndexInput.copyBuf
Posted by "Robert Muir (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Muir updated LUCENE-3541:
--------------------------------
Attachment: LUCENE-3541.patch
toned down the test, it still fails with mmapdir without the patch.
i think this is ready to commit.
> remove IndexInput.copyBuf
> -------------------------
>
> Key: LUCENE-3541
> URL: https://issues.apache.org/jira/browse/LUCENE-3541
> Project: Lucene - Java
> Issue Type: Bug
> Reporter: Robert Muir
> Priority: Critical
> Fix For: 3.5, 4.0
>
> Attachments: LUCENE-3541.patch, LUCENE-3541.patch, LUCENE-3541.patch
>
>
> this looks really broken/dangerous as an instance variable.
> what happens on clone() ?! copyBytes can instead make its own array inside the method.
> its protected, so ill list in the 3.x backwards breaks section since its technically a backwards break.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (LUCENE-3541) remove IndexInput.copyBuf
Posted by "Uwe Schindler (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13138458#comment-13138458 ]
Uwe Schindler commented on LUCENE-3541:
---------------------------------------
+1, commit this as fast as possible, otherwise I will make this a Blocker *g*
> remove IndexInput.copyBuf
> -------------------------
>
> Key: LUCENE-3541
> URL: https://issues.apache.org/jira/browse/LUCENE-3541
> Project: Lucene - Java
> Issue Type: Bug
> Reporter: Robert Muir
> Assignee: Robert Muir
> Priority: Critical
> Fix For: 3.5, 4.0
>
> Attachments: LUCENE-3541.patch, LUCENE-3541.patch, LUCENE-3541.patch
>
>
> this looks really broken/dangerous as an instance variable.
> what happens on clone() ?! copyBytes can instead make its own array inside the method.
> its protected, so ill list in the 3.x backwards breaks section since its technically a backwards break.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (LUCENE-3541) remove IndexInput.copyBuf
Posted by "Robert Muir (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13138396#comment-13138396 ]
Robert Muir commented on LUCENE-3541:
-------------------------------------
I agree its horrible, if someone properly clone()'s indexinputs and uses threads they are sharing this same buffer and its just asking for corrumption.
> remove IndexInput.copyBuf
> -------------------------
>
> Key: LUCENE-3541
> URL: https://issues.apache.org/jira/browse/LUCENE-3541
> Project: Lucene - Java
> Issue Type: Bug
> Reporter: Robert Muir
> Priority: Critical
> Fix For: 3.5, 4.0
>
>
> this looks really broken/dangerous as an instance variable.
> what happens on clone() ?! copyBytes can instead make its own array inside the method.
> its protected, so ill list in the 3.x backwards breaks section since its technically a backwards break.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org