IRC discussion

[Paul Davis <pa...@gmail.com>]

Hey,

Jan asked me to post a discussion from IRC here or on the wiki, and I
didn't see a wiki section that looked like it was a good place for a
RandomIRCDiscussion page.

This was basically a discussion that went from bulk_docs fetch vs.
http pipelining into a discussion about storing graphs in couchdb. Not
sure which part jan wanted so here's all of it.

Paul

jan____	hmmm, bulk fetch....
16:36	jan____	sounds sweet
16:37	davisp	yah, though damienkatz said that it might be replaced by
the pipelining which sounds like just sending a set of http requests
and getting responses to them all at once
16:37	davisp	I'm thinking that might require a custom client library
in most languages though
16:38	jan____	need to read up on http pipelining...
16:39	dreid	Not all at once.
16:39	dreid	You send a series of requests, and the server responds to
them in order.
16:39	davisp	dreid: Sounds about right
16:39	davisp	Totally didn't realize till just now it was part of the
http/1.1 spec
16:40	dreid	There is no promise that the responses will be received
"all at once".
16:40	dreid	I'm not entirely sure I like the idea, because it requires
more logic on the client's part.
16:40	davisp	dreid: How do you mean? Like, the server may decide to
pause, or it may say "Hey fucker, to many requests, try a new
connection"?
16:41	dreid	davisp: A good example is two requests GET
/ubuntu-8.04.iso HTTP/1.1 & GET /favicon.ico HTTP/1.1
16:41	davisp	With you so far
16:41	dreid	If those requests are pipelined, the server will not send
/favicon.ico until it has sent all of /ubuntu-8.04.iso
16:42	dreid	So the pipeline must be served in the order of the requests.
16:42	davisp	well, there is only one pipe :D
16:43	dreid	Another good example is a pipeline of a request of a
couple of documents, and then a view (say tag counts) and then some
more documents.
16:44	davisp	Hmm. reading
http://www.mozilla.org/projects/netlib/http/pipelining-faq.html makes
me feel uncomfortable
16:44	davisp	ie, the client needing more logic seems fairly accurate
16:48	davisp	I think my tree view is working. Yay.
16:53	jan____	ncie
16:53	jan____	*speling
16:54	davisp	I need to add children for each node. And for some of
these, the multipel doc requests will probably start sucking.
Although... I should look and see how deep the tree actually gets
17:02	plum	Created: (COUCHDB-47) Building CouchDB against Spidermonkey
source: https://issues.apache.org/jira/browse/COUCHDB-47
17:07	jan____	meh, RTFW :)
17:12	davisp	jan____: Did you see the mailing list thread about the
issues running behind a firewall?
17:12	jan____	davisp: glanced over that
17:12	davisp	Where in the world did HttpConfig come from
17:13	jan____	ooooooold code
17:13	jan____	that's gone with mochiweb
17:13	davisp	phew. I thought I was missing something
17:15	dreid	davisp: Not to mention that there are very popular
environments that give the actual client no control over wether or not
pipelining is used. (i.e. XHR)
17:16	davisp	dreid: Fair point. all I was looking for was something
like a _bulk_docs get functionality too :(
17:18	jan____	davisp: that might be actually easy to add as a plug-in
once we refactor the HTTP API code
17:18	dreid	Well ... views are like that.
17:18	davisp	dreid: This particular case I can't get the docs in the view
17:19		*** olegandreev joined #couchdb
17:19	dreid	Granted, views are probably inefficient if all you want
is, give me the entire body of the documents specified by this bunch
of document IDs.
17:20	davisp	dreid: Which is exactly what i'm looking for. Although,
the list of ids comes from a view :D
17:20	davisp	Everytime I use an emoticon, a fair dies. :(
17:20	davisp	*fairy
17:21	dreid	davisp: Are you trying to get the documents for a list of children?
17:21	davisp	dreid: Trying to get the documents for a given node in a
DAG so that I can build the entire tree up to the root as well as that
node's immediate children
17:22	dreid	I don't know what a DAG is.
17:22	davisp	directed acyclicl graph
17:22	dreid	I do know that you could do this with a view if every
document had a reference to it's parent.
17:22	davisp	Before inserting documents I make sure that each document
has a reference to every one of it's descendants
17:23	davisp	And a view like for i in descendants: map(
doc.descendants[i], doc._id )
17:23	davisp	dreid: but if you can think of something better, I'm all ears
17:29		*** schapht left.
17:29	dreid	Ok, so how about before inserting documents you make sure
that each document has a reference to it's parent?
17:29	dreid	Then you should be able to have a view that says "get me
all documents whose parent is x"
17:30		*** binary42 quit (Read error: 110 (Connection timed out))
17:31	davisp	Well, that's the basic idea I'm using... kind of. But I'm
more interested in "Give me all documents that have x as a descendant"
which would be equivalent to "Give me X's parent, X's parent's parent,
X's parent's parent's parent" etc. And it also allows for multiple
paths from the root to X with finesse. Drawing the view with different
edges for different relations will require data from the documents,
but once you have the list of ids you just have to get the docs
17:33	davisp	oh, actually, I don't need multiple docs. I can just map(
descendant, doc ) can't I?
17:33	dreid	I have to admit, CouchDB would not have been my first
choice to store a digraph.
17:33	davisp	and in an hour, that should be done reindexing
17:34	davisp	How would you rather store it?
17:38		*** slightlyoff_afk is now known as slightlyoff
17:44		*** naderman quit (Nick collision from services.)
17:44		*** naderman_ joined #couchdb
17:45		*** bobesponja quit (Remote closed the connection)
17:45		*** bobesponja joined #couchdb
17:49		*** jan_____ joined #couchdb
17:49	jan_____	any replies to "How would you rather store it?" that I missed?
17:50	jan_____	(Apple, can you please make Airport work reliably with
airport base stations?)
17:50		*** jan____ quit (Read error: 104 (Connection reset by peer))
17:56	davisp	jan_____: Nope
17:57	jan_____	ok, can you send results to the ML or put it into the
wiki for future reference? gotta sleep now
17:57	davisp	After I'm done reading this book I'm gonna have to spend
some serious time with the indexing.
17:57	jan_____	:)
17:57	davisp	jan_____: Results for what?
17:57	jan_____	you discussion with dreid
17:57	jan_____	*your
17:58	davisp	will do
17:58	jan_____	thanks!
17:58	jan_____	night-o
17:58	davisp	Awesome!
17:58	davisp	Night man
17:58	jan_____	:)
17:58		*** jan_____ left.