You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@couchdb.apache.org by "Paul Joseph Davis (JIRA)" <ji...@apache.org> on 2010/12/23 20:57:45 UTC
[jira] Created: (COUCHDB-997) Prevent multiple keys from entering a
btree.
Prevent multiple keys from entering a btree.
--------------------------------------------
Key: COUCHDB-997
URL: https://issues.apache.org/jira/browse/COUCHDB-997
Project: CouchDB
Issue Type: Bug
Reporter: Paul Joseph Davis
s/sort/usort/ at https://github.com/apache/couchdb/blob/trunk/src/couchdb/couch_btree.erl#L181
This should be completely transparent and incur minimal overhead. This hasn't bitten us quite yet, but it would've prevented some of the crazier behavior from COUCHDB-968
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-997) Prevent multiple keys from entering
a btree.
Posted by "Adam Kocoloski (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12974782#action_12974782 ]
Adam Kocoloski commented on COUCHDB-997:
----------------------------------------
Could even restrict the check to the insert operation if you like.
> Prevent multiple keys from entering a btree.
> --------------------------------------------
>
> Key: COUCHDB-997
> URL: https://issues.apache.org/jira/browse/COUCHDB-997
> Project: CouchDB
> Issue Type: Bug
> Reporter: Paul Joseph Davis
>
> s/sort/usort/ at https://github.com/apache/couchdb/blob/trunk/src/couchdb/couch_btree.erl#L181
> This should be completely transparent and incur minimal overhead. This hasn't bitten us quite yet, but it would've prevented some of the crazier behavior from COUCHDB-968
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-997) Prevent multiple keys from entering
a btree.
Posted by "Adam Kocoloski (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12974778#action_12974778 ]
Adam Kocoloski commented on COUCHDB-997:
----------------------------------------
To be clear, I'm talking about adding a clause to the sort like
fun({X, Y, _}, {X, Y, _}) ->
erlang:error(duplicate_action);
....
> Prevent multiple keys from entering a btree.
> --------------------------------------------
>
> Key: COUCHDB-997
> URL: https://issues.apache.org/jira/browse/COUCHDB-997
> Project: CouchDB
> Issue Type: Bug
> Reporter: Paul Joseph Davis
>
> s/sort/usort/ at https://github.com/apache/couchdb/blob/trunk/src/couchdb/couch_btree.erl#L181
> This should be completely transparent and incur minimal overhead. This hasn't bitten us quite yet, but it would've prevented some of the crazier behavior from COUCHDB-968
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-997) Prevent multiple keys from entering
a btree.
Posted by "Robert Newson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12974764#action_12974764 ]
Robert Newson commented on COUCHDB-997:
---------------------------------------
I understand we don't want the cost of the check when it's a very rare event, and I understand that usort will prevent dupes leaking in in, but I'm concerned it will make finding the introduction point of dupes harder to find (actually impossible, since they'll be generated and then deduped).
> Prevent multiple keys from entering a btree.
> --------------------------------------------
>
> Key: COUCHDB-997
> URL: https://issues.apache.org/jira/browse/COUCHDB-997
> Project: CouchDB
> Issue Type: Bug
> Reporter: Paul Joseph Davis
>
> s/sort/usort/ at https://github.com/apache/couchdb/blob/trunk/src/couchdb/couch_btree.erl#L181
> This should be completely transparent and incur minimal overhead. This hasn't bitten us quite yet, but it would've prevented some of the crazier behavior from COUCHDB-968
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-997) Prevent multiple keys from entering
a btree.
Posted by "Robert Newson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12974750#action_12974750 ]
Robert Newson commented on COUCHDB-997:
---------------------------------------
can we consider adding a log statement if duplicates are present (as that indicates a serious bug elsewhere) or simply fail if they are found? Hiding this feels wrong to me.
> Prevent multiple keys from entering a btree.
> --------------------------------------------
>
> Key: COUCHDB-997
> URL: https://issues.apache.org/jira/browse/COUCHDB-997
> Project: CouchDB
> Issue Type: Bug
> Reporter: Paul Joseph Davis
>
> s/sort/usort/ at https://github.com/apache/couchdb/blob/trunk/src/couchdb/couch_btree.erl#L181
> This should be completely transparent and incur minimal overhead. This hasn't bitten us quite yet, but it would've prevented some of the crazier behavior from COUCHDB-968
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-997) Prevent multiple keys from entering
a btree.
Posted by "Robert Newson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12974780#action_12974780 ]
Robert Newson commented on COUCHDB-997:
---------------------------------------
Paul proposed the same on IRC. +1 on the idea.
> Prevent multiple keys from entering a btree.
> --------------------------------------------
>
> Key: COUCHDB-997
> URL: https://issues.apache.org/jira/browse/COUCHDB-997
> Project: CouchDB
> Issue Type: Bug
> Reporter: Paul Joseph Davis
>
> s/sort/usort/ at https://github.com/apache/couchdb/blob/trunk/src/couchdb/couch_btree.erl#L181
> This should be completely transparent and incur minimal overhead. This hasn't bitten us quite yet, but it would've prevented some of the crazier behavior from COUCHDB-968
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-997) Prevent multiple keys from entering
a btree.
Posted by "Adam Kocoloski (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12974777#action_12974777 ]
Adam Kocoloski commented on COUCHDB-997:
----------------------------------------
I'd rather have the btree refuse duplicate actions. We're already using a custom sort function, what about just having the sort bail with erlang:error(duplicate_action) if OpA == OpB ?
> Prevent multiple keys from entering a btree.
> --------------------------------------------
>
> Key: COUCHDB-997
> URL: https://issues.apache.org/jira/browse/COUCHDB-997
> Project: CouchDB
> Issue Type: Bug
> Reporter: Paul Joseph Davis
>
> s/sort/usort/ at https://github.com/apache/couchdb/blob/trunk/src/couchdb/couch_btree.erl#L181
> This should be completely transparent and incur minimal overhead. This hasn't bitten us quite yet, but it would've prevented some of the crazier behavior from COUCHDB-968
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-997) Prevent multiple keys from entering
a btree.
Posted by "Paul Joseph Davis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12974762#action_12974762 ]
Paul Joseph Davis commented on COUCHDB-997:
-------------------------------------------
We could compare the length of the two lists. But that'd add an extra O(N) per query_modify where N is sum of the lengths of the action lists.
> Prevent multiple keys from entering a btree.
> --------------------------------------------
>
> Key: COUCHDB-997
> URL: https://issues.apache.org/jira/browse/COUCHDB-997
> Project: CouchDB
> Issue Type: Bug
> Reporter: Paul Joseph Davis
>
> s/sort/usort/ at https://github.com/apache/couchdb/blob/trunk/src/couchdb/couch_btree.erl#L181
> This should be completely transparent and incur minimal overhead. This hasn't bitten us quite yet, but it would've prevented some of the crazier behavior from COUCHDB-968
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] [Commented] (COUCHDB-997) Prevent multiple keys from
entering a btree.
Posted by "Paul Joseph Davis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13020700#comment-13020700 ]
Paul Joseph Davis commented on COUCHDB-997:
-------------------------------------------
Kinda. We ended up just fixing the other bug. And then forgot that this was open. So it was kind of a cliff hanger conclusion due to May sweeps.
> Prevent multiple keys from entering a btree.
> --------------------------------------------
>
> Key: COUCHDB-997
> URL: https://issues.apache.org/jira/browse/COUCHDB-997
> Project: CouchDB
> Issue Type: Bug
> Reporter: Paul Joseph Davis
>
> s/sort/usort/ at https://github.com/apache/couchdb/blob/trunk/src/couchdb/couch_btree.erl#L181
> This should be completely transparent and incur minimal overhead. This hasn't bitten us quite yet, but it would've prevented some of the crazier behavior from COUCHDB-968
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (COUCHDB-997) Prevent multiple keys from entering
a btree.
Posted by "Damien Katz (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Damien Katz resolved COUCHDB-997.
---------------------------------
Resolution: Won't Fix
Assignee: Damien Katz
We should rely on callers to the btree to make sure they don't introduce duplicates. If they don't, the bug should be fixed in the caller. Adding extra logic to elimnate duplicates silently may cause more problems, and isn't as correct as the callers doing the right things.
> Prevent multiple keys from entering a btree.
> --------------------------------------------
>
> Key: COUCHDB-997
> URL: https://issues.apache.org/jira/browse/COUCHDB-997
> Project: CouchDB
> Issue Type: Bug
> Reporter: Paul Joseph Davis
> Assignee: Damien Katz
>
> s/sort/usort/ at https://github.com/apache/couchdb/blob/trunk/src/couchdb/couch_btree.erl#L181
> This should be completely transparent and incur minimal overhead. This hasn't bitten us quite yet, but it would've prevented some of the crazier behavior from COUCHDB-968
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (COUCHDB-997) Prevent multiple keys from entering
a btree.
Posted by "Paul Joseph Davis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12974783#action_12974783 ]
Paul Joseph Davis commented on COUCHDB-997:
-------------------------------------------
Yeah, I think a clause like below would probably be the best in terms of performance penalty as well as ease of implementation.
fun({insert, Key, _}, {insert, Key, _}) -> erlang:error(duplicate_insert);
> Prevent multiple keys from entering a btree.
> --------------------------------------------
>
> Key: COUCHDB-997
> URL: https://issues.apache.org/jira/browse/COUCHDB-997
> Project: CouchDB
> Issue Type: Bug
> Reporter: Paul Joseph Davis
>
> s/sort/usort/ at https://github.com/apache/couchdb/blob/trunk/src/couchdb/couch_btree.erl#L181
> This should be completely transparent and incur minimal overhead. This hasn't bitten us quite yet, but it would've prevented some of the crazier behavior from COUCHDB-968
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-997) Prevent multiple keys from entering
a btree.
Posted by "Robert Newson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12974772#action_12974772 ]
Robert Newson commented on COUCHDB-997:
---------------------------------------
Unless someone else chimes in, I think the usort change should go in. Yes, it will mask the next bug that introduces duplicates but it'll get fixed before it hits the disk.
Now, if we implemented usort ourselves we could accumulate a count of the duplicates we removed, which shouldn't be slower than usort, right?
> Prevent multiple keys from entering a btree.
> --------------------------------------------
>
> Key: COUCHDB-997
> URL: https://issues.apache.org/jira/browse/COUCHDB-997
> Project: CouchDB
> Issue Type: Bug
> Reporter: Paul Joseph Davis
>
> s/sort/usort/ at https://github.com/apache/couchdb/blob/trunk/src/couchdb/couch_btree.erl#L181
> This should be completely transparent and incur minimal overhead. This hasn't bitten us quite yet, but it would've prevented some of the crazier behavior from COUCHDB-968
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-997) Prevent multiple keys from entering
a btree.
Posted by "Paul Joseph Davis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12974769#action_12974769 ]
Paul Joseph Davis commented on COUCHDB-997:
-------------------------------------------
Fair point.
An alternative I thought of during 968 is to make the check down in modify_kv_node and then barf if a dupe is detected. Still ends up being O(length(num_inserts)) extra though.
> Prevent multiple keys from entering a btree.
> --------------------------------------------
>
> Key: COUCHDB-997
> URL: https://issues.apache.org/jira/browse/COUCHDB-997
> Project: CouchDB
> Issue Type: Bug
> Reporter: Paul Joseph Davis
>
> s/sort/usort/ at https://github.com/apache/couchdb/blob/trunk/src/couchdb/couch_btree.erl#L181
> This should be completely transparent and incur minimal overhead. This hasn't bitten us quite yet, but it would've prevented some of the crazier behavior from COUCHDB-968
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-997) Prevent multiple keys from entering
a btree.
Posted by "Paul Joseph Davis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12975080#action_12975080 ]
Paul Joseph Davis commented on COUCHDB-997:
-------------------------------------------
It suddenly occurs to me, are we guaranteed to actually call that clause in a sort if a duplicate key exists? I could see it going either way but I can't decide.
> Prevent multiple keys from entering a btree.
> --------------------------------------------
>
> Key: COUCHDB-997
> URL: https://issues.apache.org/jira/browse/COUCHDB-997
> Project: CouchDB
> Issue Type: Bug
> Reporter: Paul Joseph Davis
>
> s/sort/usort/ at https://github.com/apache/couchdb/blob/trunk/src/couchdb/couch_btree.erl#L181
> This should be completely transparent and incur minimal overhead. This hasn't bitten us quite yet, but it would've prevented some of the crazier behavior from COUCHDB-968
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-997) Prevent multiple keys from entering
a btree.
Posted by "Paul Joseph Davis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12974719#action_12974719 ]
Paul Joseph Davis commented on COUCHDB-997:
-------------------------------------------
I should note, I would've just committed this, but I want to add a test to the etap suite so we don't accidentally unfix it in the future.
> Prevent multiple keys from entering a btree.
> --------------------------------------------
>
> Key: COUCHDB-997
> URL: https://issues.apache.org/jira/browse/COUCHDB-997
> Project: CouchDB
> Issue Type: Bug
> Reporter: Paul Joseph Davis
>
> s/sort/usort/ at https://github.com/apache/couchdb/blob/trunk/src/couchdb/couch_btree.erl#L181
> This should be completely transparent and incur minimal overhead. This hasn't bitten us quite yet, but it would've prevented some of the crazier behavior from COUCHDB-968
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] [Commented] (COUCHDB-997) Prevent multiple keys from
entering a btree.
Posted by "Jan Lehnardt (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13020684#comment-13020684 ]
Jan Lehnardt commented on COUCHDB-997:
--------------------------------------
Do we have a conclusion here?
> Prevent multiple keys from entering a btree.
> --------------------------------------------
>
> Key: COUCHDB-997
> URL: https://issues.apache.org/jira/browse/COUCHDB-997
> Project: CouchDB
> Issue Type: Bug
> Reporter: Paul Joseph Davis
>
> s/sort/usort/ at https://github.com/apache/couchdb/blob/trunk/src/couchdb/couch_btree.erl#L181
> This should be completely transparent and incur minimal overhead. This hasn't bitten us quite yet, but it would've prevented some of the crazier behavior from COUCHDB-968
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (COUCHDB-997) Prevent multiple keys from entering
a btree.
Posted by "Randall Leeds (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12974718#action_12974718 ]
Randall Leeds commented on COUCHDB-997:
---------------------------------------
+1
> Prevent multiple keys from entering a btree.
> --------------------------------------------
>
> Key: COUCHDB-997
> URL: https://issues.apache.org/jira/browse/COUCHDB-997
> Project: CouchDB
> Issue Type: Bug
> Reporter: Paul Joseph Davis
>
> s/sort/usort/ at https://github.com/apache/couchdb/blob/trunk/src/couchdb/couch_btree.erl#L181
> This should be completely transparent and incur minimal overhead. This hasn't bitten us quite yet, but it would've prevented some of the crazier behavior from COUCHDB-968
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.