You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@couchdb.apache.org by "Chris Anderson (JIRA)" <ji...@apache.org> on 2009/11/29 00:04:20 UTC
[jira] Created: (COUCHDB-582) _changes filter missing updates
_changes filter missing updates
-------------------------------
Key: COUCHDB-582
URL: https://issues.apache.org/jira/browse/COUCHDB-582
Project: CouchDB
Issue Type: Bug
Components: JavaScript View Server
Reporter: Chris Anderson
Assignee: Chris Anderson
Fix For: 0.11
reported on the user mailing list:
http://mail-archives.apache.org/mod_mbox/couchdb-user/200911.mbox/%3c4B10BD2C.5070200@rogerbinns.com%3e
I know there is work to be done on the filter mechanism (specifically around process reuse and concurrency)
The current implementation requires a process per connected user. This should be not that hard to patch.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (COUCHDB-582) _changes filter missing updates
Posted by "Paul Joseph Davis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Paul Joseph Davis updated COUCHDB-582:
--------------------------------------
Skill Level: Regular Contributors Level (Easy to Medium)
> _changes filter missing updates
> -------------------------------
>
> Key: COUCHDB-582
> URL: https://issues.apache.org/jira/browse/COUCHDB-582
> Project: CouchDB
> Issue Type: Bug
> Components: JavaScript View Server
> Reporter: Chris Anderson
> Assignee: Chris Anderson
> Fix For: 0.12
>
> Attachments: bug582.py
>
>
> reported on the user mailing list:
> http://mail-archives.apache.org/mod_mbox/couchdb-user/200911.mbox/%3c4B10BD2C.5070200@rogerbinns.com%3e
> I know there is work to be done on the filter mechanism (specifically around process reuse and concurrency)
> The current implementation requires a process per connected user. This should be not that hard to patch.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-582) _changes filter missing updates
Posted by "Roger Binns (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12784043#action_12784043 ]
Roger Binns commented on COUCHDB-582:
-------------------------------------
The code I attached reproduces the problem. The important thing is that not only should be there changes happening but those changes must also be part of the view and the view must be accessed. The reproduction code includes comments at the top. Briefly a background thread monitors the changes printing them out and posting to a message queue. A foreground thread waits a few seconds for a message queue item - if there is one then it reads the view, else it makes a change.
Output is shown below. If you comment out the two lines accessing the view at the bottom of the file then the output is always correct. Each changes line shows what the changes call returned. Since the filter always returns true and we made an appropriate change there should always be an item listed. The bug is that some of the time no changed item is given (as though the filter returned false). Here is one run where every alternate change did not give an item:
$ python bug582.py
+one
changes {'last_seq': 2, 'results': [{'changes': [{'rev': '1-cc2a517fa3e1a5b6ae9e3fbea460b9b8'}], 'id': 'one', 'seq': 2}]}
+two
changes {'last_seq': 3, 'results': []}
two
changes {'last_seq': 4, 'results': [{'changes': [{'rev': '2-a197bfde810e37b0a88f74a4ae038565'}], 'id': 'two', 'seq': 4}]}
two
changes {'last_seq': 5, 'results': []}
one
changes {'last_seq': 6, 'results': [{'changes': [{'rev': '2-39ea5c560504b84520606003e3d991f6'}], 'id': 'one', 'seq': 6}]}
-one
changes {'last_seq': 7, 'results': []}
-two
changes {'last_seq': 8, 'results': [{'deleted': True, 'changes': [{'rev': '4-be0fbd62ef4241c2b45693002a808fbb'}], 'id': 'two', 'seq': 8}]}
I then did another run a few seconds later (exact same parameters) where there is only one "failure":
$ python bug582.py
+one
changes {'last_seq': 2, 'results': [{'changes': [{'rev': '1-cc2a517fa3e1a5b6ae9e3fbea460b9b8'}], 'id': 'one', 'seq': 2}]}
+two
changes {'last_seq': 3, 'results': [{'changes': [{'rev': '1-3ca2f01d11f16acffd9bfc740e4c963b'}], 'id': 'two', 'seq': 3}]}
two
changes {'last_seq': 4, 'results': [{'changes': [{'rev': '2-7faeecd5faa47385c730933f13fb7872'}], 'id': 'two', 'seq': 4}]}
two
changes {'last_seq': 5, 'results': []}
one
changes {'last_seq': 6, 'results': [{'changes': [{'rev': '2-625ac20a7b3813db296c2e6cec6337bf'}], 'id': 'one', 'seq': 6}]}
-one
changes {'last_seq': 7, 'results': [{'deleted': True, 'changes': [{'rev': '3-4d6a3081511d2f8e7daec512afca1c11'}], 'id': 'one', 'seq': 7}]}
-two
changes {'last_seq': 8, 'results': [{'deleted': True, 'changes': [{'rev': '4-d47e864a9fbb905fdd86c84a04bacebb'}], 'id': 'two', 'seq': 8}]}
In general each run gives different results.
> _changes filter missing updates
> -------------------------------
>
> Key: COUCHDB-582
> URL: https://issues.apache.org/jira/browse/COUCHDB-582
> Project: CouchDB
> Issue Type: Bug
> Components: JavaScript View Server
> Reporter: Chris Anderson
> Assignee: Chris Anderson
> Fix For: 0.11
>
> Attachments: bug582.py
>
>
> reported on the user mailing list:
> http://mail-archives.apache.org/mod_mbox/couchdb-user/200911.mbox/%3c4B10BD2C.5070200@rogerbinns.com%3e
> I know there is work to be done on the filter mechanism (specifically around process reuse and concurrency)
> The current implementation requires a process per connected user. This should be not that hard to patch.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-582) _changes filter missing updates
Posted by "Roger Binns (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12783355#action_12783355 ]
Roger Binns commented on COUCHDB-582:
-------------------------------------
Not quite as good as tests, but here is an exact description of what they should do.
Create a filter function:
function(doc, req) {
// log("filter "+toJSON(doc)); -- useful for debugging and confirming filter was even called
return doc.type=="A" || doc._deleted;
}
Now watch _changes with that filter function. You should verify that there is always at least one row returned. Create several docs in a row with doc.type="A". Delete them. There should always be at least one row returned. (Bonus points for matching up returned rows with creation/deletion activity.)
A related issue is that _changes requests return based on if there was data before filtering. Consequently running the above will return no rows if you create doc.type="B". It would be more efficient to only return if there are rows *after* filtering in which the above test can continue to run and should not return if you create doc.type="B".
> _changes filter missing updates
> -------------------------------
>
> Key: COUCHDB-582
> URL: https://issues.apache.org/jira/browse/COUCHDB-582
> Project: CouchDB
> Issue Type: Bug
> Components: JavaScript View Server
> Reporter: Chris Anderson
> Assignee: Chris Anderson
> Fix For: 0.11
>
>
> reported on the user mailing list:
> http://mail-archives.apache.org/mod_mbox/couchdb-user/200911.mbox/%3c4B10BD2C.5070200@rogerbinns.com%3e
> I know there is work to be done on the filter mechanism (specifically around process reuse and concurrency)
> The current implementation requires a process per connected user. This should be not that hard to patch.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (COUCHDB-582) _changes filter missing updates
Posted by "Noah Slater (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Noah Slater updated COUCHDB-582:
--------------------------------
Fix Version/s: (was: 0.11)
0.12
> _changes filter missing updates
> -------------------------------
>
> Key: COUCHDB-582
> URL: https://issues.apache.org/jira/browse/COUCHDB-582
> Project: CouchDB
> Issue Type: Bug
> Components: JavaScript View Server
> Reporter: Chris Anderson
> Assignee: Chris Anderson
> Fix For: 0.12
>
> Attachments: bug582.py
>
>
> reported on the user mailing list:
> http://mail-archives.apache.org/mod_mbox/couchdb-user/200911.mbox/%3c4B10BD2C.5070200@rogerbinns.com%3e
> I know there is work to be done on the filter mechanism (specifically around process reuse and concurrency)
> The current implementation requires a process per connected user. This should be not that hard to patch.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-582) _changes filter missing updates
Posted by "Paul Joseph Davis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12783316#action_12783316 ]
Paul Joseph Davis commented on COUCHDB-582:
-------------------------------------------
Is there something wrong with a process per connected user? Each connected socket already uses a process per user. Or is this a second process per user?
> _changes filter missing updates
> -------------------------------
>
> Key: COUCHDB-582
> URL: https://issues.apache.org/jira/browse/COUCHDB-582
> Project: CouchDB
> Issue Type: Bug
> Components: JavaScript View Server
> Reporter: Chris Anderson
> Assignee: Chris Anderson
> Fix For: 0.11
>
>
> reported on the user mailing list:
> http://mail-archives.apache.org/mod_mbox/couchdb-user/200911.mbox/%3c4B10BD2C.5070200@rogerbinns.com%3e
> I know there is work to be done on the filter mechanism (specifically around process reuse and concurrency)
> The current implementation requires a process per connected user. This should be not that hard to patch.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (COUCHDB-582) _changes filter missing updates
Posted by "Roger Binns (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Roger Binns updated COUCHDB-582:
--------------------------------
Attachment: bug582.py
Python code to reproduce the problem.
> _changes filter missing updates
> -------------------------------
>
> Key: COUCHDB-582
> URL: https://issues.apache.org/jira/browse/COUCHDB-582
> Project: CouchDB
> Issue Type: Bug
> Components: JavaScript View Server
> Reporter: Chris Anderson
> Assignee: Chris Anderson
> Fix For: 0.11
>
> Attachments: bug582.py
>
>
> reported on the user mailing list:
> http://mail-archives.apache.org/mod_mbox/couchdb-user/200911.mbox/%3c4B10BD2C.5070200@rogerbinns.com%3e
> I know there is work to be done on the filter mechanism (specifically around process reuse and concurrency)
> The current implementation requires a process per connected user. This should be not that hard to patch.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-582) _changes filter missing updates
Posted by "Chris Anderson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12783318#action_12783318 ]
Chris Anderson commented on COUCHDB-582:
----------------------------------------
It's one couchjs process per connected user, so that's not so hot. There's no reason they can't be shared. Or maybe I haven't looked at the code in a while and this optimization is already done.
In any case, what we really need is some failing tests.
> _changes filter missing updates
> -------------------------------
>
> Key: COUCHDB-582
> URL: https://issues.apache.org/jira/browse/COUCHDB-582
> Project: CouchDB
> Issue Type: Bug
> Components: JavaScript View Server
> Reporter: Chris Anderson
> Assignee: Chris Anderson
> Fix For: 0.11
>
>
> reported on the user mailing list:
> http://mail-archives.apache.org/mod_mbox/couchdb-user/200911.mbox/%3c4B10BD2C.5070200@rogerbinns.com%3e
> I know there is work to be done on the filter mechanism (specifically around process reuse and concurrency)
> The current implementation requires a process per connected user. This should be not that hard to patch.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.