You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@accumulo.apache.org by "Jesse Yates (Created) (JIRA)" <ji...@apache.org> on 2011/10/20 03:10:10 UTC
[jira] [Created] (ACCUMULO-53) Multiple deletes cause a
RuntimeException
Multiple deletes cause a RuntimeException
-----------------------------------------
Key: ACCUMULO-53
URL: https://issues.apache.org/jira/browse/ACCUMULO-53
Project: Accumulo
Issue Type: Bug
Components: tserver
Affects Versions: 1.3.5, 1.4.0
Reporter: Jesse Yates
Assignee: Keith Turner
Pretty sure this affects 1.3.5, though it may also on 1.4.0 (haven't tested on the latter, but the code looks the same).
When a BatchDeleter is used multiple times on the same time, it doesn't reset the scan iterators, leading to runtime exceptions when there are multiple calls to delete().
Below, we can see that it adds a scan iterator to get the list of values in the table. Since it just sets the name of the deleter to "org.apache.accumulo.core.client.BatchDeleter.NOVALUE" every time. So when you go to run the scan a second time, it attempts set the same scan iterator again.
{code}
public void delete() throws MutationsRejectedException, TableNotFoundException {
super.addScanIterator(new IteratorSetting(Integer.MAX_VALUE, BatchDeleter.class.getName() + ".NOVALUE", SortedKeyIterator.class));
{code}
This results in a RuntimeException telling you that the 'Iterator name is already in use'.
After doing the delete, the BatchDeleter should remove that iterator (possibly just clear all iterators) when done with the delete.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ACCUMULO-53) Multiple deletes cause a
RuntimeException
Posted by "Jesse Yates (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ACCUMULO-53?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13137303#comment-13137303 ]
Jesse Yates commented on ACCUMULO-53:
-------------------------------------
@John: +1
I'll create a new ticket for doing the refactor of the options and target it at 1.5
@Billie:
I think if the community is ok with the mocking I'm using here, then v1 of the patch can be accepted. It just does the change for the batch deleter and adds unit tests.
> Multiple deletes cause a RuntimeException
> -----------------------------------------
>
> Key: ACCUMULO-53
> URL: https://issues.apache.org/jira/browse/ACCUMULO-53
> Project: Accumulo
> Issue Type: Bug
> Components: client
> Affects Versions: 1.3.5, 1.4.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: java_ACCUMULO-53.patch, java_ACCUMULO-53_v2.patch
>
>
> Pretty sure this affects 1.3.5, though it may also on 1.4.0 (haven't tested on the latter, but the code looks the same).
> When a BatchDeleter is used multiple times on the same time, it doesn't reset the scan iterators, leading to runtime exceptions when there are multiple calls to delete().
> Below, we can see that it adds a scan iterator to get the list of values in the table. Since it just sets the name of the deleter to "org.apache.accumulo.core.client.BatchDeleter.NOVALUE" every time. So when you go to run the scan a second time, it attempts set the same scan iterator again.
> {code}
> public void delete() throws MutationsRejectedException, TableNotFoundException {
> super.addScanIterator(new IteratorSetting(Integer.MAX_VALUE, BatchDeleter.class.getName() + ".NOVALUE", SortedKeyIterator.class));
> {code}
> This results in a RuntimeException telling you that the 'Iterator name is already in use'.
> After doing the delete, the BatchDeleter should remove that iterator (possibly just clear all iterators) when done with the delete.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (ACCUMULO-53) Multiple deletes cause
a RuntimeException
Posted by "Jesse Yates (Issue Comment Edited) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ACCUMULO-53?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13131244#comment-13131244 ]
Jesse Yates edited comment on ACCUMULO-53 at 10/20/11 1:09 AM:
---------------------------------------------------------------
Glad to work put a patch for this, but just wanted to get the community input on the right way to do it (clearing all or just the set one)
was (Author: jesse_yates):
Glad to work put a patch for this, but just wanted to get the community input on the right way to do it.
> Multiple deletes cause a RuntimeException
> -----------------------------------------
>
> Key: ACCUMULO-53
> URL: https://issues.apache.org/jira/browse/ACCUMULO-53
> Project: Accumulo
> Issue Type: Bug
> Components: tserver
> Affects Versions: 1.3.5, 1.4.0
> Reporter: Jesse Yates
> Assignee: Keith Turner
>
> Pretty sure this affects 1.3.5, though it may also on 1.4.0 (haven't tested on the latter, but the code looks the same).
> When a BatchDeleter is used multiple times on the same time, it doesn't reset the scan iterators, leading to runtime exceptions when there are multiple calls to delete().
> Below, we can see that it adds a scan iterator to get the list of values in the table. Since it just sets the name of the deleter to "org.apache.accumulo.core.client.BatchDeleter.NOVALUE" every time. So when you go to run the scan a second time, it attempts set the same scan iterator again.
> {code}
> public void delete() throws MutationsRejectedException, TableNotFoundException {
> super.addScanIterator(new IteratorSetting(Integer.MAX_VALUE, BatchDeleter.class.getName() + ".NOVALUE", SortedKeyIterator.class));
> {code}
> This results in a RuntimeException telling you that the 'Iterator name is already in use'.
> After doing the delete, the BatchDeleter should remove that iterator (possibly just clear all iterators) when done with the delete.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (ACCUMULO-53) Multiple deletes cause a
RuntimeException
Posted by "Jesse Yates (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ACCUMULO-53?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jesse Yates updated ACCUMULO-53:
--------------------------------
Attachment: java_ACCUMULO-53.patch
Fixing BatchDeleterImpl and adding ScannerOptions.removeIterator()
- Adding removeIterator() method
- Moved adding the whole-row-iterator option to BatchDeleterImpl constructor
- Added tests for the above
- Added EasyMock and Powermock for testing.
> Multiple deletes cause a RuntimeException
> -----------------------------------------
>
> Key: ACCUMULO-53
> URL: https://issues.apache.org/jira/browse/ACCUMULO-53
> Project: Accumulo
> Issue Type: Bug
> Components: client
> Affects Versions: 1.3.5, 1.4.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: java_ACCUMULO-53.patch
>
>
> Pretty sure this affects 1.3.5, though it may also on 1.4.0 (haven't tested on the latter, but the code looks the same).
> When a BatchDeleter is used multiple times on the same time, it doesn't reset the scan iterators, leading to runtime exceptions when there are multiple calls to delete().
> Below, we can see that it adds a scan iterator to get the list of values in the table. Since it just sets the name of the deleter to "org.apache.accumulo.core.client.BatchDeleter.NOVALUE" every time. So when you go to run the scan a second time, it attempts set the same scan iterator again.
> {code}
> public void delete() throws MutationsRejectedException, TableNotFoundException {
> super.addScanIterator(new IteratorSetting(Integer.MAX_VALUE, BatchDeleter.class.getName() + ".NOVALUE", SortedKeyIterator.class));
> {code}
> This results in a RuntimeException telling you that the 'Iterator name is already in use'.
> After doing the delete, the BatchDeleter should remove that iterator (possibly just clear all iterators) when done with the delete.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ACCUMULO-53) Multiple deletes cause a
RuntimeException
Posted by "Jesse Yates (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ACCUMULO-53?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13137255#comment-13137255 ]
Jesse Yates commented on ACCUMULO-53:
-------------------------------------
@Billie:
bq. The patch produces errors... Other classes are expecting ScannerOptions to have serverSideIteratorList and serverSideIteratorOptions.
Hmm, my eclipse wasn't throwing any errors on this. However, I think this is the _right_ way that code should be implemented. Looking into the erorrs now.
bq. branch 1.4 so that we can start larger changes in the trunk.
Any reason we need to branch 1.4? The numbering implies it is a major(ish) release, so making this update (since it is under the hood) should be fine. If it was a change on user interfaces, then yeah, branching makes sense.
@Eric
bq. I have some concerns about introducing a new (test-time) dependency for a single unit test
This was an attempt to start using mocking for better test coverage in the unit test suite, which is actually pretty sparse.
I'll put a comment up on dev. around the issue.
> Multiple deletes cause a RuntimeException
> -----------------------------------------
>
> Key: ACCUMULO-53
> URL: https://issues.apache.org/jira/browse/ACCUMULO-53
> Project: Accumulo
> Issue Type: Bug
> Components: client
> Affects Versions: 1.3.5, 1.4.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: java_ACCUMULO-53.patch, java_ACCUMULO-53_v2.patch
>
>
> Pretty sure this affects 1.3.5, though it may also on 1.4.0 (haven't tested on the latter, but the code looks the same).
> When a BatchDeleter is used multiple times on the same time, it doesn't reset the scan iterators, leading to runtime exceptions when there are multiple calls to delete().
> Below, we can see that it adds a scan iterator to get the list of values in the table. Since it just sets the name of the deleter to "org.apache.accumulo.core.client.BatchDeleter.NOVALUE" every time. So when you go to run the scan a second time, it attempts set the same scan iterator again.
> {code}
> public void delete() throws MutationsRejectedException, TableNotFoundException {
> super.addScanIterator(new IteratorSetting(Integer.MAX_VALUE, BatchDeleter.class.getName() + ".NOVALUE", SortedKeyIterator.class));
> {code}
> This results in a RuntimeException telling you that the 'Iterator name is already in use'.
> After doing the delete, the BatchDeleter should remove that iterator (possibly just clear all iterators) when done with the delete.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (ACCUMULO-53) Multiple deletes cause a
RuntimeException
Posted by "Jesse Yates (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ACCUMULO-53?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jesse Yates updated ACCUMULO-53:
--------------------------------
Attachment: java_ACCUMULO-53_v3.patch
Just doing the fix described in the initial ticket as well as doing some cleanup in associated code.
> Multiple deletes cause a RuntimeException
> -----------------------------------------
>
> Key: ACCUMULO-53
> URL: https://issues.apache.org/jira/browse/ACCUMULO-53
> Project: Accumulo
> Issue Type: Bug
> Components: client
> Affects Versions: 1.3.5, 1.4.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: java_ACCUMULO-53.patch, java_ACCUMULO-53_v2.patch, java_ACCUMULO-53_v3.patch
>
>
> Pretty sure this affects 1.3.5, though it may also on 1.4.0 (haven't tested on the latter, but the code looks the same).
> When a BatchDeleter is used multiple times on the same time, it doesn't reset the scan iterators, leading to runtime exceptions when there are multiple calls to delete().
> Below, we can see that it adds a scan iterator to get the list of values in the table. Since it just sets the name of the deleter to "org.apache.accumulo.core.client.BatchDeleter.NOVALUE" every time. So when you go to run the scan a second time, it attempts set the same scan iterator again.
> {code}
> public void delete() throws MutationsRejectedException, TableNotFoundException {
> super.addScanIterator(new IteratorSetting(Integer.MAX_VALUE, BatchDeleter.class.getName() + ".NOVALUE", SortedKeyIterator.class));
> {code}
> This results in a RuntimeException telling you that the 'Iterator name is already in use'.
> After doing the delete, the BatchDeleter should remove that iterator (possibly just clear all iterators) when done with the delete.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (ACCUMULO-53) Multiple deletes cause a
RuntimeException
Posted by "Billie Rinaldi (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ACCUMULO-53?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Billie Rinaldi updated ACCUMULO-53:
-----------------------------------
Component/s: (was: tserver)
client
Assignee: Jesse Yates (was: Keith Turner)
> Multiple deletes cause a RuntimeException
> -----------------------------------------
>
> Key: ACCUMULO-53
> URL: https://issues.apache.org/jira/browse/ACCUMULO-53
> Project: Accumulo
> Issue Type: Bug
> Components: client
> Affects Versions: 1.3.5, 1.4.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
>
> Pretty sure this affects 1.3.5, though it may also on 1.4.0 (haven't tested on the latter, but the code looks the same).
> When a BatchDeleter is used multiple times on the same time, it doesn't reset the scan iterators, leading to runtime exceptions when there are multiple calls to delete().
> Below, we can see that it adds a scan iterator to get the list of values in the table. Since it just sets the name of the deleter to "org.apache.accumulo.core.client.BatchDeleter.NOVALUE" every time. So when you go to run the scan a second time, it attempts set the same scan iterator again.
> {code}
> public void delete() throws MutationsRejectedException, TableNotFoundException {
> super.addScanIterator(new IteratorSetting(Integer.MAX_VALUE, BatchDeleter.class.getName() + ".NOVALUE", SortedKeyIterator.class));
> {code}
> This results in a RuntimeException telling you that the 'Iterator name is already in use'.
> After doing the delete, the BatchDeleter should remove that iterator (possibly just clear all iterators) when done with the delete.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (ACCUMULO-53) Multiple deletes cause a
RuntimeException
Posted by "Billie Rinaldi (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ACCUMULO-53?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Billie Rinaldi updated ACCUMULO-53:
-----------------------------------
Affects Version/s: (was: 1.4.0)
Fix Version/s: 1.4.0
> Multiple deletes cause a RuntimeException
> -----------------------------------------
>
> Key: ACCUMULO-53
> URL: https://issues.apache.org/jira/browse/ACCUMULO-53
> Project: Accumulo
> Issue Type: Bug
> Components: client
> Affects Versions: 1.3.5
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Fix For: 1.4.0
>
> Attachments: java_ACCUMULO-53.patch, java_ACCUMULO-53_v2.patch, java_ACCUMULO-53_v3.patch
>
>
> Pretty sure this affects 1.3.5, though it may also on 1.4.0 (haven't tested on the latter, but the code looks the same).
> When a BatchDeleter is used multiple times on the same time, it doesn't reset the scan iterators, leading to runtime exceptions when there are multiple calls to delete().
> Below, we can see that it adds a scan iterator to get the list of values in the table. Since it just sets the name of the deleter to "org.apache.accumulo.core.client.BatchDeleter.NOVALUE" every time. So when you go to run the scan a second time, it attempts set the same scan iterator again.
> {code}
> public void delete() throws MutationsRejectedException, TableNotFoundException {
> super.addScanIterator(new IteratorSetting(Integer.MAX_VALUE, BatchDeleter.class.getName() + ".NOVALUE", SortedKeyIterator.class));
> {code}
> This results in a RuntimeException telling you that the 'Iterator name is already in use'.
> After doing the delete, the BatchDeleter should remove that iterator (possibly just clear all iterators) when done with the delete.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (ACCUMULO-53) Multiple deletes cause a
RuntimeException
Posted by "Jesse Yates (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ACCUMULO-53?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jesse Yates updated ACCUMULO-53:
--------------------------------
Attachment: java_ACCUMULO-53_v2.patch
Switching ScannerOptions to just keep a map of iterator settings.
Kept BatchDeleter's setting of iterator in the constructor since it is going to be much more efficient to just set it once, rather than do a lookup on the keys and then add it, every time. Only issue is the case where people are messing with the iterators settings on the deleter, and if you are doing that, you should know what you are doing (since you are messing with the internals).
> Multiple deletes cause a RuntimeException
> -----------------------------------------
>
> Key: ACCUMULO-53
> URL: https://issues.apache.org/jira/browse/ACCUMULO-53
> Project: Accumulo
> Issue Type: Bug
> Components: client
> Affects Versions: 1.3.5, 1.4.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: java_ACCUMULO-53.patch, java_ACCUMULO-53_v2.patch
>
>
> Pretty sure this affects 1.3.5, though it may also on 1.4.0 (haven't tested on the latter, but the code looks the same).
> When a BatchDeleter is used multiple times on the same time, it doesn't reset the scan iterators, leading to runtime exceptions when there are multiple calls to delete().
> Below, we can see that it adds a scan iterator to get the list of values in the table. Since it just sets the name of the deleter to "org.apache.accumulo.core.client.BatchDeleter.NOVALUE" every time. So when you go to run the scan a second time, it attempts set the same scan iterator again.
> {code}
> public void delete() throws MutationsRejectedException, TableNotFoundException {
> super.addScanIterator(new IteratorSetting(Integer.MAX_VALUE, BatchDeleter.class.getName() + ".NOVALUE", SortedKeyIterator.class));
> {code}
> This results in a RuntimeException telling you that the 'Iterator name is already in use'.
> After doing the delete, the BatchDeleter should remove that iterator (possibly just clear all iterators) when done with the delete.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ACCUMULO-53) Multiple deletes cause a
RuntimeException
Posted by "Billie Rinaldi (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ACCUMULO-53?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13135062#comment-13135062 ]
Billie Rinaldi commented on ACCUMULO-53:
----------------------------------------
I wonder if we should make the serverSideIteratorList a Map<String,IterInfo> that maps names to iterators. Then we could remove an iterator more easily, and we could have a hasIterator(String iteratorName) method. That would let us keep the addScanIterator statement in the BatchDeleter in the delete() method, with a check to see if it already exists before adding.
We'd have to convert the serverSideIteratorList.values() into a List when it is passed to various other classes (e.g. it's a List that gets passed via thrift from the client to the tablet server -- I'm not sure that's the best data structure for the purpose, but we probably shouldn't change it for this version).
Any thoughts?
> Multiple deletes cause a RuntimeException
> -----------------------------------------
>
> Key: ACCUMULO-53
> URL: https://issues.apache.org/jira/browse/ACCUMULO-53
> Project: Accumulo
> Issue Type: Bug
> Components: client
> Affects Versions: 1.3.5, 1.4.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: java_ACCUMULO-53.patch
>
>
> Pretty sure this affects 1.3.5, though it may also on 1.4.0 (haven't tested on the latter, but the code looks the same).
> When a BatchDeleter is used multiple times on the same time, it doesn't reset the scan iterators, leading to runtime exceptions when there are multiple calls to delete().
> Below, we can see that it adds a scan iterator to get the list of values in the table. Since it just sets the name of the deleter to "org.apache.accumulo.core.client.BatchDeleter.NOVALUE" every time. So when you go to run the scan a second time, it attempts set the same scan iterator again.
> {code}
> public void delete() throws MutationsRejectedException, TableNotFoundException {
> super.addScanIterator(new IteratorSetting(Integer.MAX_VALUE, BatchDeleter.class.getName() + ".NOVALUE", SortedKeyIterator.class));
> {code}
> This results in a RuntimeException telling you that the 'Iterator name is already in use'.
> After doing the delete, the BatchDeleter should remove that iterator (possibly just clear all iterators) when done with the delete.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (ACCUMULO-53) Multiple deletes cause a
RuntimeException
Posted by "Billie Rinaldi (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ACCUMULO-53?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Billie Rinaldi resolved ACCUMULO-53.
------------------------------------
Resolution: Fixed
> Multiple deletes cause a RuntimeException
> -----------------------------------------
>
> Key: ACCUMULO-53
> URL: https://issues.apache.org/jira/browse/ACCUMULO-53
> Project: Accumulo
> Issue Type: Bug
> Components: client
> Affects Versions: 1.3.5
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Fix For: 1.4.0
>
> Attachments: java_ACCUMULO-53.patch, java_ACCUMULO-53_v2.patch, java_ACCUMULO-53_v3.patch
>
>
> Pretty sure this affects 1.3.5, though it may also on 1.4.0 (haven't tested on the latter, but the code looks the same).
> When a BatchDeleter is used multiple times on the same time, it doesn't reset the scan iterators, leading to runtime exceptions when there are multiple calls to delete().
> Below, we can see that it adds a scan iterator to get the list of values in the table. Since it just sets the name of the deleter to "org.apache.accumulo.core.client.BatchDeleter.NOVALUE" every time. So when you go to run the scan a second time, it attempts set the same scan iterator again.
> {code}
> public void delete() throws MutationsRejectedException, TableNotFoundException {
> super.addScanIterator(new IteratorSetting(Integer.MAX_VALUE, BatchDeleter.class.getName() + ".NOVALUE", SortedKeyIterator.class));
> {code}
> This results in a RuntimeException telling you that the 'Iterator name is already in use'.
> After doing the delete, the BatchDeleter should remove that iterator (possibly just clear all iterators) when done with the delete.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ACCUMULO-53) Multiple deletes cause a
RuntimeException
Posted by "Eric Newton (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ACCUMULO-53?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13136971#comment-13136971 ]
Eric Newton commented on ACCUMULO-53:
-------------------------------------
I have some concerns about introducing a new (test-time) dependency for a single unit test. Anyone else?
> Multiple deletes cause a RuntimeException
> -----------------------------------------
>
> Key: ACCUMULO-53
> URL: https://issues.apache.org/jira/browse/ACCUMULO-53
> Project: Accumulo
> Issue Type: Bug
> Components: client
> Affects Versions: 1.3.5, 1.4.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: java_ACCUMULO-53.patch, java_ACCUMULO-53_v2.patch
>
>
> Pretty sure this affects 1.3.5, though it may also on 1.4.0 (haven't tested on the latter, but the code looks the same).
> When a BatchDeleter is used multiple times on the same time, it doesn't reset the scan iterators, leading to runtime exceptions when there are multiple calls to delete().
> Below, we can see that it adds a scan iterator to get the list of values in the table. Since it just sets the name of the deleter to "org.apache.accumulo.core.client.BatchDeleter.NOVALUE" every time. So when you go to run the scan a second time, it attempts set the same scan iterator again.
> {code}
> public void delete() throws MutationsRejectedException, TableNotFoundException {
> super.addScanIterator(new IteratorSetting(Integer.MAX_VALUE, BatchDeleter.class.getName() + ".NOVALUE", SortedKeyIterator.class));
> {code}
> This results in a RuntimeException telling you that the 'Iterator name is already in use'.
> After doing the delete, the BatchDeleter should remove that iterator (possibly just clear all iterators) when done with the delete.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ACCUMULO-53) Multiple deletes cause a
RuntimeException
Posted by "Billie Rinaldi (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ACCUMULO-53?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13131807#comment-13131807 ]
Billie Rinaldi commented on ACCUMULO-53:
----------------------------------------
I would suggest moving the super.addScanIterator statement to the constructor. That way it will just happen once, and the iterator will remain in place for all subsequent calls to delete(). I suppose the user might manually clear the iterators, in which case it wouldn't be put back in place. However, the iterator is for performance rather than necessity, so delete() would still work correctly.
You don't want to clear all iterators, because that will also wipe out any that the user has added.
We could discuss adding some new methods to the ScannerBase interface that would let you see if an iterator name is already in use and clear a specific named iterator.
> Multiple deletes cause a RuntimeException
> -----------------------------------------
>
> Key: ACCUMULO-53
> URL: https://issues.apache.org/jira/browse/ACCUMULO-53
> Project: Accumulo
> Issue Type: Bug
> Components: tserver
> Affects Versions: 1.3.5, 1.4.0
> Reporter: Jesse Yates
> Assignee: Keith Turner
>
> Pretty sure this affects 1.3.5, though it may also on 1.4.0 (haven't tested on the latter, but the code looks the same).
> When a BatchDeleter is used multiple times on the same time, it doesn't reset the scan iterators, leading to runtime exceptions when there are multiple calls to delete().
> Below, we can see that it adds a scan iterator to get the list of values in the table. Since it just sets the name of the deleter to "org.apache.accumulo.core.client.BatchDeleter.NOVALUE" every time. So when you go to run the scan a second time, it attempts set the same scan iterator again.
> {code}
> public void delete() throws MutationsRejectedException, TableNotFoundException {
> super.addScanIterator(new IteratorSetting(Integer.MAX_VALUE, BatchDeleter.class.getName() + ".NOVALUE", SortedKeyIterator.class));
> {code}
> This results in a RuntimeException telling you that the 'Iterator name is already in use'.
> After doing the delete, the BatchDeleter should remove that iterator (possibly just clear all iterators) when done with the delete.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ACCUMULO-53) Multiple deletes cause a
RuntimeException
Posted by "Billie Rinaldi (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ACCUMULO-53?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13137170#comment-13137170 ]
Billie Rinaldi commented on ACCUMULO-53:
----------------------------------------
The patch produces errors. (Also, its formatting does not follow our spec -- see http://incubator.apache.org/accumulo/source.html.) Other classes are expecting ScannerOptions to have serverSideIteratorList and serverSideIteratorOptions. Merging these into a single object throughout the code might be a good idea, but is a much bigger change. If we want to go that route, I think we should do it in 1.5 instead of 1.4. Perhaps we should just do the BatchDeleter fix in this ticket, create another ticket for improving ScannerOptions and IteratorSetting handling, and branch 1.4 so that we can start larger changes in the trunk.
Deciding upon which mock testing frameworks we should use sounds like something we should do as a community. Perhaps start a discussion on the mailing list?
> Multiple deletes cause a RuntimeException
> -----------------------------------------
>
> Key: ACCUMULO-53
> URL: https://issues.apache.org/jira/browse/ACCUMULO-53
> Project: Accumulo
> Issue Type: Bug
> Components: client
> Affects Versions: 1.3.5, 1.4.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: java_ACCUMULO-53.patch, java_ACCUMULO-53_v2.patch
>
>
> Pretty sure this affects 1.3.5, though it may also on 1.4.0 (haven't tested on the latter, but the code looks the same).
> When a BatchDeleter is used multiple times on the same time, it doesn't reset the scan iterators, leading to runtime exceptions when there are multiple calls to delete().
> Below, we can see that it adds a scan iterator to get the list of values in the table. Since it just sets the name of the deleter to "org.apache.accumulo.core.client.BatchDeleter.NOVALUE" every time. So when you go to run the scan a second time, it attempts set the same scan iterator again.
> {code}
> public void delete() throws MutationsRejectedException, TableNotFoundException {
> super.addScanIterator(new IteratorSetting(Integer.MAX_VALUE, BatchDeleter.class.getName() + ".NOVALUE", SortedKeyIterator.class));
> {code}
> This results in a RuntimeException telling you that the 'Iterator name is already in use'.
> After doing the delete, the BatchDeleter should remove that iterator (possibly just clear all iterators) when done with the delete.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ACCUMULO-53) Multiple deletes cause a
RuntimeException
Posted by "Keith Turner (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ACCUMULO-53?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13135108#comment-13135108 ]
Keith Turner commented on ACCUMULO-53:
--------------------------------------
Converting serverSideIteratorList to map seems fine. Doing linear scans of the list is probably ok, as it will normally be a small list. However, a map will make the code more concise.
> Multiple deletes cause a RuntimeException
> -----------------------------------------
>
> Key: ACCUMULO-53
> URL: https://issues.apache.org/jira/browse/ACCUMULO-53
> Project: Accumulo
> Issue Type: Bug
> Components: client
> Affects Versions: 1.3.5, 1.4.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: java_ACCUMULO-53.patch
>
>
> Pretty sure this affects 1.3.5, though it may also on 1.4.0 (haven't tested on the latter, but the code looks the same).
> When a BatchDeleter is used multiple times on the same time, it doesn't reset the scan iterators, leading to runtime exceptions when there are multiple calls to delete().
> Below, we can see that it adds a scan iterator to get the list of values in the table. Since it just sets the name of the deleter to "org.apache.accumulo.core.client.BatchDeleter.NOVALUE" every time. So when you go to run the scan a second time, it attempts set the same scan iterator again.
> {code}
> public void delete() throws MutationsRejectedException, TableNotFoundException {
> super.addScanIterator(new IteratorSetting(Integer.MAX_VALUE, BatchDeleter.class.getName() + ".NOVALUE", SortedKeyIterator.class));
> {code}
> This results in a RuntimeException telling you that the 'Iterator name is already in use'.
> After doing the delete, the BatchDeleter should remove that iterator (possibly just clear all iterators) when done with the delete.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ACCUMULO-53) Multiple deletes cause a
RuntimeException
Posted by "John Vines (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ACCUMULO-53?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13137295#comment-13137295 ]
John Vines commented on ACCUMULO-53:
------------------------------------
To do a full implementation of your changes, which is the way to go IMO, requires a change in the thrift API. And a change that disruptive is not good for something which we have been trying to freeze for a little while now. By branching we can officially do a feature freeze so we can finalize our testing on it without forcing you and other contributors to sit on your patches until we deem otherwise.
> Multiple deletes cause a RuntimeException
> -----------------------------------------
>
> Key: ACCUMULO-53
> URL: https://issues.apache.org/jira/browse/ACCUMULO-53
> Project: Accumulo
> Issue Type: Bug
> Components: client
> Affects Versions: 1.3.5, 1.4.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: java_ACCUMULO-53.patch, java_ACCUMULO-53_v2.patch
>
>
> Pretty sure this affects 1.3.5, though it may also on 1.4.0 (haven't tested on the latter, but the code looks the same).
> When a BatchDeleter is used multiple times on the same time, it doesn't reset the scan iterators, leading to runtime exceptions when there are multiple calls to delete().
> Below, we can see that it adds a scan iterator to get the list of values in the table. Since it just sets the name of the deleter to "org.apache.accumulo.core.client.BatchDeleter.NOVALUE" every time. So when you go to run the scan a second time, it attempts set the same scan iterator again.
> {code}
> public void delete() throws MutationsRejectedException, TableNotFoundException {
> super.addScanIterator(new IteratorSetting(Integer.MAX_VALUE, BatchDeleter.class.getName() + ".NOVALUE", SortedKeyIterator.class));
> {code}
> This results in a RuntimeException telling you that the 'Iterator name is already in use'.
> After doing the delete, the BatchDeleter should remove that iterator (possibly just clear all iterators) when done with the delete.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ACCUMULO-53) Multiple deletes cause a
RuntimeException
Posted by "Jesse Yates (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ACCUMULO-53?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13131827#comment-13131827 ]
Jesse Yates commented on ACCUMULO-53:
-------------------------------------
{quote}
You don't want to clear all iterators, because that will also wipe out any that the user has added.
{quote}
We could go in by hand an delete the iterator after each use of the delete, but that just seems excessive.
I like the idea of just setting the iterator at object creation - its clean and solves the problem.
bq. We could discuss adding some new methods to the ScannerBase interface that would let you see if an iterator name is already in use and clear a specific named iterator.
This could be interesting, but don't know how often this would be used. There are some cases where I've done some nested, automated scanning and you get around that by just having random names for the iterators and rely on the improbability of overlap - mathematically it should work :)
So yeah, adding something like that would be cool.
I'll work on patches for both.
> Multiple deletes cause a RuntimeException
> -----------------------------------------
>
> Key: ACCUMULO-53
> URL: https://issues.apache.org/jira/browse/ACCUMULO-53
> Project: Accumulo
> Issue Type: Bug
> Components: client
> Affects Versions: 1.3.5, 1.4.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
>
> Pretty sure this affects 1.3.5, though it may also on 1.4.0 (haven't tested on the latter, but the code looks the same).
> When a BatchDeleter is used multiple times on the same time, it doesn't reset the scan iterators, leading to runtime exceptions when there are multiple calls to delete().
> Below, we can see that it adds a scan iterator to get the list of values in the table. Since it just sets the name of the deleter to "org.apache.accumulo.core.client.BatchDeleter.NOVALUE" every time. So when you go to run the scan a second time, it attempts set the same scan iterator again.
> {code}
> public void delete() throws MutationsRejectedException, TableNotFoundException {
> super.addScanIterator(new IteratorSetting(Integer.MAX_VALUE, BatchDeleter.class.getName() + ".NOVALUE", SortedKeyIterator.class));
> {code}
> This results in a RuntimeException telling you that the 'Iterator name is already in use'.
> After doing the delete, the BatchDeleter should remove that iterator (possibly just clear all iterators) when done with the delete.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ACCUMULO-53) Multiple deletes cause a
RuntimeException
Posted by "Jesse Yates (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ACCUMULO-53?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13131244#comment-13131244 ]
Jesse Yates commented on ACCUMULO-53:
-------------------------------------
Glad to work put a patch for this, but just wanted to get the community input on the right way to do it.
> Multiple deletes cause a RuntimeException
> -----------------------------------------
>
> Key: ACCUMULO-53
> URL: https://issues.apache.org/jira/browse/ACCUMULO-53
> Project: Accumulo
> Issue Type: Bug
> Components: tserver
> Affects Versions: 1.3.5, 1.4.0
> Reporter: Jesse Yates
> Assignee: Keith Turner
>
> Pretty sure this affects 1.3.5, though it may also on 1.4.0 (haven't tested on the latter, but the code looks the same).
> When a BatchDeleter is used multiple times on the same time, it doesn't reset the scan iterators, leading to runtime exceptions when there are multiple calls to delete().
> Below, we can see that it adds a scan iterator to get the list of values in the table. Since it just sets the name of the deleter to "org.apache.accumulo.core.client.BatchDeleter.NOVALUE" every time. So when you go to run the scan a second time, it attempts set the same scan iterator again.
> {code}
> public void delete() throws MutationsRejectedException, TableNotFoundException {
> super.addScanIterator(new IteratorSetting(Integer.MAX_VALUE, BatchDeleter.class.getName() + ".NOVALUE", SortedKeyIterator.class));
> {code}
> This results in a RuntimeException telling you that the 'Iterator name is already in use'.
> After doing the delete, the BatchDeleter should remove that iterator (possibly just clear all iterators) when done with the delete.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ACCUMULO-53) Multiple deletes cause a
RuntimeException
Posted by "Jesse Yates (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ACCUMULO-53?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13135149#comment-13135149 ]
Jesse Yates commented on ACCUMULO-53:
-------------------------------------
Agreed on the fact that number of iterators is small. Since its small, the memory footprint of the map shouldn't be too bad either - assuming we go with hashmap. I like the idea of switching it over to a map and then deserializing that on the server (this plays into another idea I'm stil percolating).
I'll start working on a revised patch.
> Multiple deletes cause a RuntimeException
> -----------------------------------------
>
> Key: ACCUMULO-53
> URL: https://issues.apache.org/jira/browse/ACCUMULO-53
> Project: Accumulo
> Issue Type: Bug
> Components: client
> Affects Versions: 1.3.5, 1.4.0
> Reporter: Jesse Yates
> Assignee: Jesse Yates
> Attachments: java_ACCUMULO-53.patch
>
>
> Pretty sure this affects 1.3.5, though it may also on 1.4.0 (haven't tested on the latter, but the code looks the same).
> When a BatchDeleter is used multiple times on the same time, it doesn't reset the scan iterators, leading to runtime exceptions when there are multiple calls to delete().
> Below, we can see that it adds a scan iterator to get the list of values in the table. Since it just sets the name of the deleter to "org.apache.accumulo.core.client.BatchDeleter.NOVALUE" every time. So when you go to run the scan a second time, it attempts set the same scan iterator again.
> {code}
> public void delete() throws MutationsRejectedException, TableNotFoundException {
> super.addScanIterator(new IteratorSetting(Integer.MAX_VALUE, BatchDeleter.class.getName() + ".NOVALUE", SortedKeyIterator.class));
> {code}
> This results in a RuntimeException telling you that the 'Iterator name is already in use'.
> After doing the delete, the BatchDeleter should remove that iterator (possibly just clear all iterators) when done with the delete.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira