You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@couchdb.apache.org by Luca Matteis <lm...@gmail.com> on 2012/05/16 11:40:53 UTC
Hierarchical comments Hacker News style
I'm trying to implement a basic way of displaying comments in the way
that Hacker News provides, using CouchDB. Not only ordered
hierarchically, but also, each level of the tree should be ordered by
a "points" variable. If you're familiar with sites such as Hacker News
or Reddit you know that each level of the comment tree is ordered by
the amount of points and also the date they were posted - however, for
sake of simplicity I'll just talk about a "points" variable.
The idea is that I want a view to return it in the order I except, and
not make many Ajax calls for example, to retrieve them and make them
look like they're ordered correctly.
This is what I got so far: Each document is a "comment". Each comment
has a property `path` which is an ordered list containing all its
parents. So for example, imagine I have 4 comments (with _id `1`, `2`,
`3` and `4`). Comment `2` is children of `1`, comment `3` is children
of `2`, and comment `4` is also children of `1`. This is what the data
would look like:
{ _id: 1, path: ["1"] },
{ _id: 2, path: ["1", "2"] },
{ _id: 3, path: ["1", "2", "3"] }
{ _id: 4, path: ["1", "4"] }
This works quite well for the hierarchy. A simple `view` will already
return things ordered with each comment underneath the correct one.
The issue comes when I want to order each "level" of the tree
independently. So for example documents `2` and `4` belong to the same
branch, but are ordered, on that level, by their ID. Instead I want
them ordered based on a "points" variable that I want to add to the
path - but can't seem to understand where I could be adding this
variable for it to work the way I want it.
Is there a way to do this? Consider that the "points" variable will
change in time.
Thank you,
Luca
Re: Hierarchical comments Hacker News style
Posted by Randall Leeds <ra...@gmail.com>.
On Wed, May 16, 2012 at 2:40 AM, Luca Matteis <lm...@gmail.com> wrote:
> I'm trying to implement a basic way of displaying comments in the way
> that Hacker News provides, using CouchDB. Not only ordered
> hierarchically, but also, each level of the tree should be ordered by
> a "points" variable. If you're familiar with sites such as Hacker News
> or Reddit you know that each level of the comment tree is ordered by
> the amount of points and also the date they were posted - however, for
> sake of simplicity I'll just talk about a "points" variable.
>
> The idea is that I want a view to return it in the order I except, and
> not make many Ajax calls for example, to retrieve them and make them
> look like they're ordered correctly.
>
> This is what I got so far: Each document is a "comment". Each comment
> has a property `path` which is an ordered list containing all its
> parents. So for example, imagine I have 4 comments (with _id `1`, `2`,
> `3` and `4`). Comment `2` is children of `1`, comment `3` is children
> of `2`, and comment `4` is also children of `1`. This is what the data
> would look like:
>
> { _id: 1, path: ["1"] },
> { _id: 2, path: ["1", "2"] },
> { _id: 3, path: ["1", "2", "3"] }
> { _id: 4, path: ["1", "4"] }
>
> This works quite well for the hierarchy. A simple `view` will already
> return things ordered with each comment underneath the correct one.
>
> The issue comes when I want to order each "level" of the tree
> independently. So for example documents `2` and `4` belong to the same
> branch, but are ordered, on that level, by their ID. Instead I want
> them ordered based on a "points" variable that I want to add to the
> path - but can't seem to understand where I could be adding this
> variable for it to work the way I want it.
>
> Is there a way to do this? Consider that the "points" variable will
> change in time.
The standard threading algorithm is jwz's:
http://www.jwz.org/doc/threading.html
Max Ogden ported a ruby implementation to JS:
https://github.com/maxogden/conversationThreading-js/
I've just fixed a couple bugs in there this week as I needed something similar.
It makes sense to me to use the format you have so that a range query
can get a full thread, but then do the sorting in a list function or
client side using the jwz algorithm. It's simply to write a recursive
traversal through the children so you can sort the children in each
iteration of the recursion by whatever property you like.
It's ugly, poorly commented (read: not at all), written in
CoffeeScript, and contains a bunch of d3.js stuff, but you can see a
similar loop I did here:
https://github.com/hypothesis/hypothes.is/blob/master/hypothesis/js/src/hypothesis.coffee#L201
... with the actual use of the message threading library here (which
uses undefined for the subject, as I'm not grouping by "subject" as in
email):
https://github.com/hypothesis/hypothes.is/blob/master/hypothesis/js/src/hypothesis.coffee#L155
In my case the "thread" is a '/'-delimited string rather than an
array, so you'll see some .split() going on.
The advantage of using this algorithm is also that it properly handles
threading even when some nodes are not available (deleted, not
replicated in order, etc), inserting blank containers with no
`message` property of their own.
I didn't read the whole thread, but just scanned to see if anyone
mentioned this. Hope it helps!
-Randall
Re: Hierarchical comments Hacker News style
Posted by Sean Copenhaver <se...@gmail.com>.
Yeah now that you mention it, I swear I read an article about modeling
stackoverflow with a document database and every post, comment, and vote
was it's own document. Then there was a view with a reduce to get
everything. I want to say it took a couple of views and there was some
grouping involved on the query.
Maybe it was it one query to get posts with their score reduced, then
another for when you load a post to get it's comments and their scores.
Then you group 1 or 2 levels in so the score is on a per post or comment
level.
On Wed, May 16, 2012 at 1:11 PM, Luca Matteis <lm...@gmail.com> wrote:
> Would storing each vote as a separate document be completely inappropriate?
>
> On Wed, May 16, 2012 at 6:59 PM, Sean Copenhaver
> <se...@gmail.com> wrote:
> > Well my idea is that there is a separate document that only keeps track
> of
> > the comments by hierarchy, point, and potentially order (once you have
> the
> > points any code should easily be able to get the ordering) for a single
> > post. That document wouldn't containt the document contents, just the
> ids.
> > So when adding a comment you'll need to associate and score it with the
> > post by adding it to this comment tracking document. If _changes is fast
> > enough that would be easy to do but also just making two requests to
> > CouchDB would be fairly easy. Especially if the update handler took care
> of
> > adding an entry for a new comment.
> >
> > So there would be a post document, and a document for each comment, then
> a
> > comment tracker document for each post. I think the comment tracker
> > document would remain fairly small even with a lot of comments since it's
> > only tracking very little information.
> >
> > The disadvantage is of course having to keep up with that comment tracker
> > document. Again an update handler could handle a lot of that but each
> +/- 1
> > or new comment would require a call to it.
> >
> > I hope that's clear.
> >
> > On Wed, May 16, 2012 at 12:25 PM, Luca Matteis <lm...@gmail.com>
> wrote:
> >
> >> Sean,
> >> interesting. What would be the drawbacks of this other than the
> >> conflict issue? How would I order the comments by score - would my
> >> view need to do the logic? Also, what if there's *many* comments to a
> >> post. How much can a single document handle?
> >>
> >> Luca
> >>
> >> On Wed, May 16, 2012 at 5:32 PM, Sean Copenhaver
> >> <se...@gmail.com> wrote:
> >> > Is it not possible to maintain the scores and ordering in a document
> by
> >> > itself? Something like
> >> >
> >> > {
> >> > _id: <whatever>
> >> > post: <post id>
> >> > comments: {
> >> > <top level comment id>: { score: ##, children: <comments of
> >> > similar structure> },
> >> > <another top level comment id> : <etc>
> >> > }
> >> > }
> >> >
> >> > A view could easily allow one call to CouchDB retrieve the original
> post
> >> > and this comment scoring doc. Then you can load whatever level of
> >> comments
> >> > you need to and know the ordering and hierarchy.
> >> >
> >> > Score changes on a comment though you would need to keep updating this
> >> > document which can lead to conflicts. An update handler to increment
> the
> >> > score and reorder the comments will help with that and if a conflict
> >> causes
> >> > the update handler to error back to the client you just submit the
> same
> >> one
> >> > again.
> >> >
> >> > I have not implemented this solution myself, but throwing out ideas.
> I'm
> >> > certainly curious what others think is the best way.
> >> >
> >> > On Wed, May 16, 2012 at 9:33 AM, Dirkjan Ochtman <di...@ochtman.nl>
> >> wrote:
> >> >
> >> >> On Wed, May 16, 2012 at 3:19 PM, Luca Matteis <lm...@gmail.com>
> >> wrote:
> >> >> > Ok. Where would it be appropriate to build such a tree? Can I do
> it in
> >> >> > a list function?
> >> >> > I'm using Couch directly and not interacting with it using other
> >> >> > languages, so I need Couch to do all the work.
> >> >>
> >> >> Should be possible to do it in the list function, but I haven't
> worked
> >> >> with list functions before.
> >> >>
> >> >> Also note that this topic is solidly off-topic for dev@, and should
> >> >> actually be discussed on user@.
> >> >>
> >> >> Cheers,
> >> >>
> >> >> Dirkjan
> >> >>
> >> >
> >> >
> >> >
> >> > --
> >> > “The limits of language are the limits of one's world. “ - Ludwig von
> >> > Wittgenstein
> >> >
> >> > "Water is fluid, soft and yielding. But water will wear away rock,
> which
> >> is
> >> > rigid and cannot yield. As a rule, whatever is fluid, soft and
> yielding
> >> > will overcome whatever is rigid and hard. This is another paradox:
> what
> >> is
> >> > soft is strong." - Lao-Tzu
> >>
> >
> >
> >
> > --
> > “The limits of language are the limits of one's world. “ - Ludwig von
> > Wittgenstein
> >
> > "Water is fluid, soft and yielding. But water will wear away rock, which
> is
> > rigid and cannot yield. As a rule, whatever is fluid, soft and yielding
> > will overcome whatever is rigid and hard. This is another paradox: what
> is
> > soft is strong." - Lao-Tzu
>
--
“The limits of language are the limits of one's world. “ - Ludwig von
Wittgenstein
"Water is fluid, soft and yielding. But water will wear away rock, which is
rigid and cannot yield. As a rule, whatever is fluid, soft and yielding
will overcome whatever is rigid and hard. This is another paradox: what is
soft is strong." - Lao-Tzu
Re: Hierarchical comments Hacker News style
Posted by Luca Matteis <lm...@gmail.com>.
Would storing each vote as a separate document be completely inappropriate?
On Wed, May 16, 2012 at 6:59 PM, Sean Copenhaver
<se...@gmail.com> wrote:
> Well my idea is that there is a separate document that only keeps track of
> the comments by hierarchy, point, and potentially order (once you have the
> points any code should easily be able to get the ordering) for a single
> post. That document wouldn't containt the document contents, just the ids.
> So when adding a comment you'll need to associate and score it with the
> post by adding it to this comment tracking document. If _changes is fast
> enough that would be easy to do but also just making two requests to
> CouchDB would be fairly easy. Especially if the update handler took care of
> adding an entry for a new comment.
>
> So there would be a post document, and a document for each comment, then a
> comment tracker document for each post. I think the comment tracker
> document would remain fairly small even with a lot of comments since it's
> only tracking very little information.
>
> The disadvantage is of course having to keep up with that comment tracker
> document. Again an update handler could handle a lot of that but each +/- 1
> or new comment would require a call to it.
>
> I hope that's clear.
>
> On Wed, May 16, 2012 at 12:25 PM, Luca Matteis <lm...@gmail.com> wrote:
>
>> Sean,
>> interesting. What would be the drawbacks of this other than the
>> conflict issue? How would I order the comments by score - would my
>> view need to do the logic? Also, what if there's *many* comments to a
>> post. How much can a single document handle?
>>
>> Luca
>>
>> On Wed, May 16, 2012 at 5:32 PM, Sean Copenhaver
>> <se...@gmail.com> wrote:
>> > Is it not possible to maintain the scores and ordering in a document by
>> > itself? Something like
>> >
>> > {
>> > _id: <whatever>
>> > post: <post id>
>> > comments: {
>> > <top level comment id>: { score: ##, children: <comments of
>> > similar structure> },
>> > <another top level comment id> : <etc>
>> > }
>> > }
>> >
>> > A view could easily allow one call to CouchDB retrieve the original post
>> > and this comment scoring doc. Then you can load whatever level of
>> comments
>> > you need to and know the ordering and hierarchy.
>> >
>> > Score changes on a comment though you would need to keep updating this
>> > document which can lead to conflicts. An update handler to increment the
>> > score and reorder the comments will help with that and if a conflict
>> causes
>> > the update handler to error back to the client you just submit the same
>> one
>> > again.
>> >
>> > I have not implemented this solution myself, but throwing out ideas. I'm
>> > certainly curious what others think is the best way.
>> >
>> > On Wed, May 16, 2012 at 9:33 AM, Dirkjan Ochtman <di...@ochtman.nl>
>> wrote:
>> >
>> >> On Wed, May 16, 2012 at 3:19 PM, Luca Matteis <lm...@gmail.com>
>> wrote:
>> >> > Ok. Where would it be appropriate to build such a tree? Can I do it in
>> >> > a list function?
>> >> > I'm using Couch directly and not interacting with it using other
>> >> > languages, so I need Couch to do all the work.
>> >>
>> >> Should be possible to do it in the list function, but I haven't worked
>> >> with list functions before.
>> >>
>> >> Also note that this topic is solidly off-topic for dev@, and should
>> >> actually be discussed on user@.
>> >>
>> >> Cheers,
>> >>
>> >> Dirkjan
>> >>
>> >
>> >
>> >
>> > --
>> > “The limits of language are the limits of one's world. “ - Ludwig von
>> > Wittgenstein
>> >
>> > "Water is fluid, soft and yielding. But water will wear away rock, which
>> is
>> > rigid and cannot yield. As a rule, whatever is fluid, soft and yielding
>> > will overcome whatever is rigid and hard. This is another paradox: what
>> is
>> > soft is strong." - Lao-Tzu
>>
>
>
>
> --
> “The limits of language are the limits of one's world. “ - Ludwig von
> Wittgenstein
>
> "Water is fluid, soft and yielding. But water will wear away rock, which is
> rigid and cannot yield. As a rule, whatever is fluid, soft and yielding
> will overcome whatever is rigid and hard. This is another paradox: what is
> soft is strong." - Lao-Tzu
Re: Hierarchical comments Hacker News style
Posted by Sean Copenhaver <se...@gmail.com>.
Well my idea is that there is a separate document that only keeps track of
the comments by hierarchy, point, and potentially order (once you have the
points any code should easily be able to get the ordering) for a single
post. That document wouldn't containt the document contents, just the ids.
So when adding a comment you'll need to associate and score it with the
post by adding it to this comment tracking document. If _changes is fast
enough that would be easy to do but also just making two requests to
CouchDB would be fairly easy. Especially if the update handler took care of
adding an entry for a new comment.
So there would be a post document, and a document for each comment, then a
comment tracker document for each post. I think the comment tracker
document would remain fairly small even with a lot of comments since it's
only tracking very little information.
The disadvantage is of course having to keep up with that comment tracker
document. Again an update handler could handle a lot of that but each +/- 1
or new comment would require a call to it.
I hope that's clear.
On Wed, May 16, 2012 at 12:25 PM, Luca Matteis <lm...@gmail.com> wrote:
> Sean,
> interesting. What would be the drawbacks of this other than the
> conflict issue? How would I order the comments by score - would my
> view need to do the logic? Also, what if there's *many* comments to a
> post. How much can a single document handle?
>
> Luca
>
> On Wed, May 16, 2012 at 5:32 PM, Sean Copenhaver
> <se...@gmail.com> wrote:
> > Is it not possible to maintain the scores and ordering in a document by
> > itself? Something like
> >
> > {
> > _id: <whatever>
> > post: <post id>
> > comments: {
> > <top level comment id>: { score: ##, children: <comments of
> > similar structure> },
> > <another top level comment id> : <etc>
> > }
> > }
> >
> > A view could easily allow one call to CouchDB retrieve the original post
> > and this comment scoring doc. Then you can load whatever level of
> comments
> > you need to and know the ordering and hierarchy.
> >
> > Score changes on a comment though you would need to keep updating this
> > document which can lead to conflicts. An update handler to increment the
> > score and reorder the comments will help with that and if a conflict
> causes
> > the update handler to error back to the client you just submit the same
> one
> > again.
> >
> > I have not implemented this solution myself, but throwing out ideas. I'm
> > certainly curious what others think is the best way.
> >
> > On Wed, May 16, 2012 at 9:33 AM, Dirkjan Ochtman <di...@ochtman.nl>
> wrote:
> >
> >> On Wed, May 16, 2012 at 3:19 PM, Luca Matteis <lm...@gmail.com>
> wrote:
> >> > Ok. Where would it be appropriate to build such a tree? Can I do it in
> >> > a list function?
> >> > I'm using Couch directly and not interacting with it using other
> >> > languages, so I need Couch to do all the work.
> >>
> >> Should be possible to do it in the list function, but I haven't worked
> >> with list functions before.
> >>
> >> Also note that this topic is solidly off-topic for dev@, and should
> >> actually be discussed on user@.
> >>
> >> Cheers,
> >>
> >> Dirkjan
> >>
> >
> >
> >
> > --
> > “The limits of language are the limits of one's world. “ - Ludwig von
> > Wittgenstein
> >
> > "Water is fluid, soft and yielding. But water will wear away rock, which
> is
> > rigid and cannot yield. As a rule, whatever is fluid, soft and yielding
> > will overcome whatever is rigid and hard. This is another paradox: what
> is
> > soft is strong." - Lao-Tzu
>
--
“The limits of language are the limits of one's world. “ - Ludwig von
Wittgenstein
"Water is fluid, soft and yielding. But water will wear away rock, which is
rigid and cannot yield. As a rule, whatever is fluid, soft and yielding
will overcome whatever is rigid and hard. This is another paradox: what is
soft is strong." - Lao-Tzu
Re: Hierarchical comments Hacker News style
Posted by Luca Matteis <lm...@gmail.com>.
Sean,
interesting. What would be the drawbacks of this other than the
conflict issue? How would I order the comments by score - would my
view need to do the logic? Also, what if there's *many* comments to a
post. How much can a single document handle?
Luca
On Wed, May 16, 2012 at 5:32 PM, Sean Copenhaver
<se...@gmail.com> wrote:
> Is it not possible to maintain the scores and ordering in a document by
> itself? Something like
>
> {
> _id: <whatever>
> post: <post id>
> comments: {
> <top level comment id>: { score: ##, children: <comments of
> similar structure> },
> <another top level comment id> : <etc>
> }
> }
>
> A view could easily allow one call to CouchDB retrieve the original post
> and this comment scoring doc. Then you can load whatever level of comments
> you need to and know the ordering and hierarchy.
>
> Score changes on a comment though you would need to keep updating this
> document which can lead to conflicts. An update handler to increment the
> score and reorder the comments will help with that and if a conflict causes
> the update handler to error back to the client you just submit the same one
> again.
>
> I have not implemented this solution myself, but throwing out ideas. I'm
> certainly curious what others think is the best way.
>
> On Wed, May 16, 2012 at 9:33 AM, Dirkjan Ochtman <di...@ochtman.nl> wrote:
>
>> On Wed, May 16, 2012 at 3:19 PM, Luca Matteis <lm...@gmail.com> wrote:
>> > Ok. Where would it be appropriate to build such a tree? Can I do it in
>> > a list function?
>> > I'm using Couch directly and not interacting with it using other
>> > languages, so I need Couch to do all the work.
>>
>> Should be possible to do it in the list function, but I haven't worked
>> with list functions before.
>>
>> Also note that this topic is solidly off-topic for dev@, and should
>> actually be discussed on user@.
>>
>> Cheers,
>>
>> Dirkjan
>>
>
>
>
> --
> “The limits of language are the limits of one's world. “ - Ludwig von
> Wittgenstein
>
> "Water is fluid, soft and yielding. But water will wear away rock, which is
> rigid and cannot yield. As a rule, whatever is fluid, soft and yielding
> will overcome whatever is rigid and hard. This is another paradox: what is
> soft is strong." - Lao-Tzu
Re: Hierarchical comments Hacker News style
Posted by Sean Copenhaver <se...@gmail.com>.
Is it not possible to maintain the scores and ordering in a document by
itself? Something like
{
_id: <whatever>
post: <post id>
comments: {
<top level comment id>: { score: ##, children: <comments of
similar structure> },
<another top level comment id> : <etc>
}
}
A view could easily allow one call to CouchDB retrieve the original post
and this comment scoring doc. Then you can load whatever level of comments
you need to and know the ordering and hierarchy.
Score changes on a comment though you would need to keep updating this
document which can lead to conflicts. An update handler to increment the
score and reorder the comments will help with that and if a conflict causes
the update handler to error back to the client you just submit the same one
again.
I have not implemented this solution myself, but throwing out ideas. I'm
certainly curious what others think is the best way.
On Wed, May 16, 2012 at 9:33 AM, Dirkjan Ochtman <di...@ochtman.nl> wrote:
> On Wed, May 16, 2012 at 3:19 PM, Luca Matteis <lm...@gmail.com> wrote:
> > Ok. Where would it be appropriate to build such a tree? Can I do it in
> > a list function?
> > I'm using Couch directly and not interacting with it using other
> > languages, so I need Couch to do all the work.
>
> Should be possible to do it in the list function, but I haven't worked
> with list functions before.
>
> Also note that this topic is solidly off-topic for dev@, and should
> actually be discussed on user@.
>
> Cheers,
>
> Dirkjan
>
--
“The limits of language are the limits of one's world. “ - Ludwig von
Wittgenstein
"Water is fluid, soft and yielding. But water will wear away rock, which is
rigid and cannot yield. As a rule, whatever is fluid, soft and yielding
will overcome whatever is rigid and hard. This is another paradox: what is
soft is strong." - Lao-Tzu
Re: Hierarchical comments Hacker News style
Posted by Dirkjan Ochtman <di...@ochtman.nl>.
On Wed, May 16, 2012 at 3:19 PM, Luca Matteis <lm...@gmail.com> wrote:
> Ok. Where would it be appropriate to build such a tree? Can I do it in
> a list function?
> I'm using Couch directly and not interacting with it using other
> languages, so I need Couch to do all the work.
Should be possible to do it in the list function, but I haven't worked
with list functions before.
Also note that this topic is solidly off-topic for dev@, and should
actually be discussed on user@.
Cheers,
Dirkjan
Re: Hierarchical comments Hacker News style
Posted by Benoit Chesneau <bc...@gmail.com>.
On Wed, May 16, 2012 at 3:19 PM, Luca Matteis <lm...@gmail.com> wrote:
> Ok. Where would it be appropriate to build such a tree? Can I do it in
> a list function?
> I'm using Couch directly and not interacting with it using other
> languages, so I need Couch to do all the work.
>
> On Wed, May 16, 2012 at 3:14 PM, Dirkjan Ochtman <di...@ochtman.nl> wrote:
>> On Wed, May 16, 2012 at 2:53 PM, Luca Matteis <lm...@gmail.com> wrote:
>>> Isn't there a better way?
>>
>> Since the ordering of descendants depends on the scores of ancestors
>> (if you're talking about getting the whole thread in order), and you
>> don't want to update descendants when ancestors get modified (scores),
>> there by definition is no way to cleanly do this. I think the best way
>> you can do it is to emit the [ancestor_id, score] for each comment,
>> which I think would allow you to build the tree structure in a single
>> loop over all documents:
>>
>> docs, roots = {}, []
>> for key, doc in documents:
>> if key[0] == null:
>> roots.append(doc.id)
>> docs[doc.id] = doc
>> if key[0]:
>> docs[key[0]].children.append(doc.id)
>>
>> (In horrible pseudo-Python-JS.)
>>
>> Cheers,
>>
>> Dirkjan
Have a look here :
https://github.com/benoitc/nymphormation
It exactly implement that as a couchapp.
- benoit
Re: Hierarchical comments Hacker News style
Posted by Luca Matteis <lm...@gmail.com>.
Ok. Where would it be appropriate to build such a tree? Can I do it in
a list function?
I'm using Couch directly and not interacting with it using other
languages, so I need Couch to do all the work.
On Wed, May 16, 2012 at 3:14 PM, Dirkjan Ochtman <di...@ochtman.nl> wrote:
> On Wed, May 16, 2012 at 2:53 PM, Luca Matteis <lm...@gmail.com> wrote:
>> Isn't there a better way?
>
> Since the ordering of descendants depends on the scores of ancestors
> (if you're talking about getting the whole thread in order), and you
> don't want to update descendants when ancestors get modified (scores),
> there by definition is no way to cleanly do this. I think the best way
> you can do it is to emit the [ancestor_id, score] for each comment,
> which I think would allow you to build the tree structure in a single
> loop over all documents:
>
> docs, roots = {}, []
> for key, doc in documents:
> if key[0] == null:
> roots.append(doc.id)
> docs[doc.id] = doc
> if key[0]:
> docs[key[0]].children.append(doc.id)
>
> (In horrible pseudo-Python-JS.)
>
> Cheers,
>
> Dirkjan
Re: Hierarchical comments Hacker News style
Posted by Dirkjan Ochtman <di...@ochtman.nl>.
On Wed, May 16, 2012 at 2:53 PM, Luca Matteis <lm...@gmail.com> wrote:
> Isn't there a better way?
Since the ordering of descendants depends on the scores of ancestors
(if you're talking about getting the whole thread in order), and you
don't want to update descendants when ancestors get modified (scores),
there by definition is no way to cleanly do this. I think the best way
you can do it is to emit the [ancestor_id, score] for each comment,
which I think would allow you to build the tree structure in a single
loop over all documents:
docs, roots = {}, []
for key, doc in documents:
if key[0] == null:
roots.append(doc.id)
docs[doc.id] = doc
if key[0]:
docs[key[0]].children.append(doc.id)
(In horrible pseudo-Python-JS.)
Cheers,
Dirkjan
Re: Hierarchical comments Hacker News style
Posted by Nick North <no...@gmail.com>.
It needs someone much better with CouchDb than I am to find a better way.
It feels hard to me, as changing the score of a document near the top of
the tree can potentially move its whole subtree around in the ordering. If
you want your view to contain all the documents in tree order, then
presumably all the documents in that subtree have to be reindexed.
Fortunately most people on this list are much better with CouchDb than I
am, so they may well provide a good solution.
On 16 May 2012 13:53, Luca Matteis <lm...@gmail.com> wrote:
> Hi Nick,
> This in fact was also something I came up with. I ditched it because,
> like you said, it requires to update all the descendants. Which means
> a lot of extra overhead. Not only the direct descendants, but all the
> descendants. So a top comment could easily be replied more than 20
> times, and each of those other 20, etc. etc. This would mean that I
> would have to update more than 50 documents each time a user clicks on
> the "upvote" button - not to mention the conflicts.
>
> Isn't there a better way?
>
> On Wed, May 16, 2012 at 2:43 PM, Nick North <no...@gmail.com> wrote:
> > I may be misunderstanding the problem, but would it work if you represent
> > each comment by a list of its score and its id? Say comments 1, 2, 3, 4
> > have scores 40, 30, 20, 10 respectively, then the documents would become:
> >
> > { _id: 1, path: [[40,"1"]] },
> > { _id: 2, path: [[40,"1"], [30,"2"]] },
> > { _id: 3, path: [[40,"1"], [30,"2"], [20,"3"]] }
> > { _id: 4, path: [[40,"1"], [10,"4"]] }
> > Then indexing on the path will now put comment 4 before comment 2. If two
> > comments with the same parent have the same score, they will be ordered
> by
> > id. If you want the date in the ordering, that's just another element in
> > the list.
> > The problem with this is that any change to the score of a parent comment
> > requires the paths of all its descendants to be updated. But it feels as
> if
> > that might be inevitable, as changing the parent score may move an entire
> > subtree around in the ordering so the index of all elements of the
> subtree
> > must change.
> >
> > Nick
> >
> > On 16 May 2012 12:27, Luca Matteis <lm...@gmail.com> wrote:
> >
> >> I know how the ranking works actually. I described how I'd like to
> >> concentrate on simply ranking each level by its points.
> >> The issue is actually in the adapting this structure to Couch.
> >>
> >> On Wed, May 16, 2012 at 12:26 PM, Tim McNamara
> >> <pa...@timmcnamara.co.nz> wrote:
> >> > It may be worth looking at the reddit source code (
> >> github.com/reddit/reddit)
> >> > to get some tips. Then you could adapt this into Couch.
> >> > On May 16, 2012 9:41 PM, "Luca Matteis" <lm...@gmail.com> wrote:
> >> >
> >> >> I'm trying to implement a basic way of displaying comments in the way
> >> >> that Hacker News provides, using CouchDB. Not only ordered
> >> >> hierarchically, but also, each level of the tree should be ordered by
> >> >> a "points" variable. If you're familiar with sites such as Hacker
> News
> >> >> or Reddit you know that each level of the comment tree is ordered by
> >> >> the amount of points and also the date they were posted - however,
> for
> >> >> sake of simplicity I'll just talk about a "points" variable.
> >> >>
> >> >> The idea is that I want a view to return it in the order I except,
> and
> >> >> not make many Ajax calls for example, to retrieve them and make them
> >> >> look like they're ordered correctly.
> >> >>
> >> >> This is what I got so far: Each document is a "comment". Each comment
> >> >> has a property `path` which is an ordered list containing all its
> >> >> parents. So for example, imagine I have 4 comments (with _id `1`,
> `2`,
> >> >> `3` and `4`). Comment `2` is children of `1`, comment `3` is children
> >> >> of `2`, and comment `4` is also children of `1`. This is what the
> data
> >> >> would look like:
> >> >>
> >> >> { _id: 1, path: ["1"] },
> >> >> { _id: 2, path: ["1", "2"] },
> >> >> { _id: 3, path: ["1", "2", "3"] }
> >> >> { _id: 4, path: ["1", "4"] }
> >> >>
> >> >> This works quite well for the hierarchy. A simple `view` will already
> >> >> return things ordered with each comment underneath the correct one.
> >> >>
> >> >> The issue comes when I want to order each "level" of the tree
> >> >> independently. So for example documents `2` and `4` belong to the
> same
> >> >> branch, but are ordered, on that level, by their ID. Instead I want
> >> >> them ordered based on a "points" variable that I want to add to the
> >> >> path - but can't seem to understand where I could be adding this
> >> >> variable for it to work the way I want it.
> >> >>
> >> >> Is there a way to do this? Consider that the "points" variable will
> >> >> change in time.
> >> >>
> >> >> Thank you,
> >> >> Luca
> >> >>
> >>
>
Re: Hierarchical comments Hacker News style
Posted by Luca Matteis <lm...@gmail.com>.
Hi Nick,
This in fact was also something I came up with. I ditched it because,
like you said, it requires to update all the descendants. Which means
a lot of extra overhead. Not only the direct descendants, but all the
descendants. So a top comment could easily be replied more than 20
times, and each of those other 20, etc. etc. This would mean that I
would have to update more than 50 documents each time a user clicks on
the "upvote" button - not to mention the conflicts.
Isn't there a better way?
On Wed, May 16, 2012 at 2:43 PM, Nick North <no...@gmail.com> wrote:
> I may be misunderstanding the problem, but would it work if you represent
> each comment by a list of its score and its id? Say comments 1, 2, 3, 4
> have scores 40, 30, 20, 10 respectively, then the documents would become:
>
> { _id: 1, path: [[40,"1"]] },
> { _id: 2, path: [[40,"1"], [30,"2"]] },
> { _id: 3, path: [[40,"1"], [30,"2"], [20,"3"]] }
> { _id: 4, path: [[40,"1"], [10,"4"]] }
> Then indexing on the path will now put comment 4 before comment 2. If two
> comments with the same parent have the same score, they will be ordered by
> id. If you want the date in the ordering, that's just another element in
> the list.
> The problem with this is that any change to the score of a parent comment
> requires the paths of all its descendants to be updated. But it feels as if
> that might be inevitable, as changing the parent score may move an entire
> subtree around in the ordering so the index of all elements of the subtree
> must change.
>
> Nick
>
> On 16 May 2012 12:27, Luca Matteis <lm...@gmail.com> wrote:
>
>> I know how the ranking works actually. I described how I'd like to
>> concentrate on simply ranking each level by its points.
>> The issue is actually in the adapting this structure to Couch.
>>
>> On Wed, May 16, 2012 at 12:26 PM, Tim McNamara
>> <pa...@timmcnamara.co.nz> wrote:
>> > It may be worth looking at the reddit source code (
>> github.com/reddit/reddit)
>> > to get some tips. Then you could adapt this into Couch.
>> > On May 16, 2012 9:41 PM, "Luca Matteis" <lm...@gmail.com> wrote:
>> >
>> >> I'm trying to implement a basic way of displaying comments in the way
>> >> that Hacker News provides, using CouchDB. Not only ordered
>> >> hierarchically, but also, each level of the tree should be ordered by
>> >> a "points" variable. If you're familiar with sites such as Hacker News
>> >> or Reddit you know that each level of the comment tree is ordered by
>> >> the amount of points and also the date they were posted - however, for
>> >> sake of simplicity I'll just talk about a "points" variable.
>> >>
>> >> The idea is that I want a view to return it in the order I except, and
>> >> not make many Ajax calls for example, to retrieve them and make them
>> >> look like they're ordered correctly.
>> >>
>> >> This is what I got so far: Each document is a "comment". Each comment
>> >> has a property `path` which is an ordered list containing all its
>> >> parents. So for example, imagine I have 4 comments (with _id `1`, `2`,
>> >> `3` and `4`). Comment `2` is children of `1`, comment `3` is children
>> >> of `2`, and comment `4` is also children of `1`. This is what the data
>> >> would look like:
>> >>
>> >> { _id: 1, path: ["1"] },
>> >> { _id: 2, path: ["1", "2"] },
>> >> { _id: 3, path: ["1", "2", "3"] }
>> >> { _id: 4, path: ["1", "4"] }
>> >>
>> >> This works quite well for the hierarchy. A simple `view` will already
>> >> return things ordered with each comment underneath the correct one.
>> >>
>> >> The issue comes when I want to order each "level" of the tree
>> >> independently. So for example documents `2` and `4` belong to the same
>> >> branch, but are ordered, on that level, by their ID. Instead I want
>> >> them ordered based on a "points" variable that I want to add to the
>> >> path - but can't seem to understand where I could be adding this
>> >> variable for it to work the way I want it.
>> >>
>> >> Is there a way to do this? Consider that the "points" variable will
>> >> change in time.
>> >>
>> >> Thank you,
>> >> Luca
>> >>
>>
Re: Hierarchical comments Hacker News style
Posted by Nick North <no...@gmail.com>.
I may be misunderstanding the problem, but would it work if you represent
each comment by a list of its score and its id? Say comments 1, 2, 3, 4
have scores 40, 30, 20, 10 respectively, then the documents would become:
{ _id: 1, path: [[40,"1"]] },
{ _id: 2, path: [[40,"1"], [30,"2"]] },
{ _id: 3, path: [[40,"1"], [30,"2"], [20,"3"]] }
{ _id: 4, path: [[40,"1"], [10,"4"]] }
Then indexing on the path will now put comment 4 before comment 2. If two
comments with the same parent have the same score, they will be ordered by
id. If you want the date in the ordering, that's just another element in
the list.
The problem with this is that any change to the score of a parent comment
requires the paths of all its descendants to be updated. But it feels as if
that might be inevitable, as changing the parent score may move an entire
subtree around in the ordering so the index of all elements of the subtree
must change.
Nick
On 16 May 2012 12:27, Luca Matteis <lm...@gmail.com> wrote:
> I know how the ranking works actually. I described how I'd like to
> concentrate on simply ranking each level by its points.
> The issue is actually in the adapting this structure to Couch.
>
> On Wed, May 16, 2012 at 12:26 PM, Tim McNamara
> <pa...@timmcnamara.co.nz> wrote:
> > It may be worth looking at the reddit source code (
> github.com/reddit/reddit)
> > to get some tips. Then you could adapt this into Couch.
> > On May 16, 2012 9:41 PM, "Luca Matteis" <lm...@gmail.com> wrote:
> >
> >> I'm trying to implement a basic way of displaying comments in the way
> >> that Hacker News provides, using CouchDB. Not only ordered
> >> hierarchically, but also, each level of the tree should be ordered by
> >> a "points" variable. If you're familiar with sites such as Hacker News
> >> or Reddit you know that each level of the comment tree is ordered by
> >> the amount of points and also the date they were posted - however, for
> >> sake of simplicity I'll just talk about a "points" variable.
> >>
> >> The idea is that I want a view to return it in the order I except, and
> >> not make many Ajax calls for example, to retrieve them and make them
> >> look like they're ordered correctly.
> >>
> >> This is what I got so far: Each document is a "comment". Each comment
> >> has a property `path` which is an ordered list containing all its
> >> parents. So for example, imagine I have 4 comments (with _id `1`, `2`,
> >> `3` and `4`). Comment `2` is children of `1`, comment `3` is children
> >> of `2`, and comment `4` is also children of `1`. This is what the data
> >> would look like:
> >>
> >> { _id: 1, path: ["1"] },
> >> { _id: 2, path: ["1", "2"] },
> >> { _id: 3, path: ["1", "2", "3"] }
> >> { _id: 4, path: ["1", "4"] }
> >>
> >> This works quite well for the hierarchy. A simple `view` will already
> >> return things ordered with each comment underneath the correct one.
> >>
> >> The issue comes when I want to order each "level" of the tree
> >> independently. So for example documents `2` and `4` belong to the same
> >> branch, but are ordered, on that level, by their ID. Instead I want
> >> them ordered based on a "points" variable that I want to add to the
> >> path - but can't seem to understand where I could be adding this
> >> variable for it to work the way I want it.
> >>
> >> Is there a way to do this? Consider that the "points" variable will
> >> change in time.
> >>
> >> Thank you,
> >> Luca
> >>
>
Re: Hierarchical comments Hacker News style
Posted by Luca Matteis <lm...@gmail.com>.
I know how the ranking works actually. I described how I'd like to
concentrate on simply ranking each level by its points.
The issue is actually in the adapting this structure to Couch.
On Wed, May 16, 2012 at 12:26 PM, Tim McNamara
<pa...@timmcnamara.co.nz> wrote:
> It may be worth looking at the reddit source code (github.com/reddit/reddit)
> to get some tips. Then you could adapt this into Couch.
> On May 16, 2012 9:41 PM, "Luca Matteis" <lm...@gmail.com> wrote:
>
>> I'm trying to implement a basic way of displaying comments in the way
>> that Hacker News provides, using CouchDB. Not only ordered
>> hierarchically, but also, each level of the tree should be ordered by
>> a "points" variable. If you're familiar with sites such as Hacker News
>> or Reddit you know that each level of the comment tree is ordered by
>> the amount of points and also the date they were posted - however, for
>> sake of simplicity I'll just talk about a "points" variable.
>>
>> The idea is that I want a view to return it in the order I except, and
>> not make many Ajax calls for example, to retrieve them and make them
>> look like they're ordered correctly.
>>
>> This is what I got so far: Each document is a "comment". Each comment
>> has a property `path` which is an ordered list containing all its
>> parents. So for example, imagine I have 4 comments (with _id `1`, `2`,
>> `3` and `4`). Comment `2` is children of `1`, comment `3` is children
>> of `2`, and comment `4` is also children of `1`. This is what the data
>> would look like:
>>
>> { _id: 1, path: ["1"] },
>> { _id: 2, path: ["1", "2"] },
>> { _id: 3, path: ["1", "2", "3"] }
>> { _id: 4, path: ["1", "4"] }
>>
>> This works quite well for the hierarchy. A simple `view` will already
>> return things ordered with each comment underneath the correct one.
>>
>> The issue comes when I want to order each "level" of the tree
>> independently. So for example documents `2` and `4` belong to the same
>> branch, but are ordered, on that level, by their ID. Instead I want
>> them ordered based on a "points" variable that I want to add to the
>> path - but can't seem to understand where I could be adding this
>> variable for it to work the way I want it.
>>
>> Is there a way to do this? Consider that the "points" variable will
>> change in time.
>>
>> Thank you,
>> Luca
>>
Re: Hierarchical comments Hacker News style
Posted by Tim McNamara <pa...@timmcnamara.co.nz>.
It may be worth looking at the reddit source code (github.com/reddit/reddit)
to get some tips. Then you could adapt this into Couch.
On May 16, 2012 9:41 PM, "Luca Matteis" <lm...@gmail.com> wrote:
> I'm trying to implement a basic way of displaying comments in the way
> that Hacker News provides, using CouchDB. Not only ordered
> hierarchically, but also, each level of the tree should be ordered by
> a "points" variable. If you're familiar with sites such as Hacker News
> or Reddit you know that each level of the comment tree is ordered by
> the amount of points and also the date they were posted - however, for
> sake of simplicity I'll just talk about a "points" variable.
>
> The idea is that I want a view to return it in the order I except, and
> not make many Ajax calls for example, to retrieve them and make them
> look like they're ordered correctly.
>
> This is what I got so far: Each document is a "comment". Each comment
> has a property `path` which is an ordered list containing all its
> parents. So for example, imagine I have 4 comments (with _id `1`, `2`,
> `3` and `4`). Comment `2` is children of `1`, comment `3` is children
> of `2`, and comment `4` is also children of `1`. This is what the data
> would look like:
>
> { _id: 1, path: ["1"] },
> { _id: 2, path: ["1", "2"] },
> { _id: 3, path: ["1", "2", "3"] }
> { _id: 4, path: ["1", "4"] }
>
> This works quite well for the hierarchy. A simple `view` will already
> return things ordered with each comment underneath the correct one.
>
> The issue comes when I want to order each "level" of the tree
> independently. So for example documents `2` and `4` belong to the same
> branch, but are ordered, on that level, by their ID. Instead I want
> them ordered based on a "points" variable that I want to add to the
> path - but can't seem to understand where I could be adding this
> variable for it to work the way I want it.
>
> Is there a way to do this? Consider that the "points" variable will
> change in time.
>
> Thank you,
> Luca
>