You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@couchdb.apache.org by Jason Smith <jh...@proven-corporation.com> on 2009/03/05 09:31:02 UTC
Chance of including CouchDB in Linux distros or desktops?
Hi, all. I had a thought the other day and wanted to share:
What are the chances that the free software movers and shakers could
successfully lobby CouchDB to be included in the Freedesktop.org system?
Consider DBus, which is a mandatory component of Linux (actually,
Freedesktop) desktops, is now understood by all developers, leading to
more and more apps talking to each other over DBus. I suggest that
having a document DB built in to all Linux desktops would be true
innovation for Linux development (especially since the GNOME pundits
want to move to "web-aware" desktops).
If there is any chance in Hell that it could gain traction (I'm
enthusiastic but skeptical--IMHO "Linux desktop innovation" is a myth,
but I digress), I'd definitely volunteer to write code, as I have
relevant experience. I'm thinking of two components:
1. Similar to DBus, you have one CouchDB process per user that runs when
he logs on and exits when he logs out. (Maybe have a system-wide
CouchDB too but I'm not sure if there is a need.)
2. (I'm surprised this doesn't exist already) A DBus CouchDB client API,
so that nobody has to learn or use HTTP in their code, just the
well-known DBus.
Given 1 and 2, any desktop app could just assume a private (for that
user) DB in the same way they assume a per-user-session DBus bus today.
So, in summary, I'm asking if it's desirable that modern distros
bundle a document DB (Couch) for all apps to build from.
--
Jason Smith
Proven Corporation
Bangkok, Thailand
http://www.proven-corporation.com
Re: Chance of including CouchDB in Linux distros or desktops?
Posted by Brian Candler <B....@pobox.com>.
On Sat, Mar 07, 2009 at 08:38:01AM +0700, Jason Smith wrote:
> Still, DBus is tangential to the main argument. My concern with a DBus
> API is you need a persistent process translating between DBus RPC calls
> and HTTP calls. So all DBus gives you is a language-independent API but
> since Couch has bindings for all major languages (and direct HTTP too), I
> could be persuaded that DBus is not worth the effort.
So another point worth pondering is: why did DBus choose its own protocol,
rather than HTTP?
Probably because (a) HTTP is actually an enormous pain to implement
correctly (anybody read RFC 2616 end-to-end recently?); and (b) it's only a
transport, so unless all you're doing is fetching and putting files, you
still have to layer your own application semantics on top.
I'm no expert on DBus, but I believe it provides things like
publish-subscribe, which is not a standard HTTP application. Well, perhaps
Stomp is an ad-hoc standard. Things like asynchronous notification are also
awkward with HTTP - Comet? Supposedly small desktop applets would end up
carrying a lot of baggage if they had to use HTTP+Stomp+Comet.
B.
Re: Chance of including CouchDB in Linux distros or desktops?
Posted by Noah Slater <ns...@apache.org>.
On Sat, Mar 07, 2009 at 08:38:01AM +0700, Jason Smith wrote:
> But while HTTP and REST is easy enough, ...
You'd be surprised.
--
Noah Slater, http://tumbolia.org/nslater
Re: Chance of including CouchDB in Linux distros or desktops?
Posted by David Reid <dr...@dreid.org>.
On Fri, Mar 6, 2009 at 5:38 PM, Jason Smith <jh...@proven-corporation.com>wrote:
> David Reid wrote:
>
>> Consider DBus, which is a mandatory component of Linux (actually,
>>> Freedesktop) desktops, is now understood by all developers, leading to
>>> more
>>> and more apps talking to each other over DBus. I suggest that having a
>>> document DB built in to all Linux desktops would be true innovation for
>>> Linux development (especially since the GNOME pundits want to move to
>>> "web-aware" desktops).
>>>
>>>
>> I reject the notion that any (much less all) developers understand DBus.
>>
>
> I'll concede the point. Maybe DBus isn't appropriate. Mostly I wanted to
> discuss whether desktop applications could or should use a document database
> instead of custom config files and binary formats.
>
> (In the future! When it's ready!)
>
> I agree with Noah's warning about "technobunkum." I disagree that a
> desktop document DB is technobunkum; however since posting the original
> question, I've realized that distros don't embed MySQL like this, and even
> SQLite doesn't have universal adoption among desktop apps. So that forces
> me to acknowledge that people have voted with their feet. (Maybe "the
> people" are fools, but whaddayagonnado?)
>
> 2. (I'm surprised this doesn't exist already) A DBus CouchDB client API,
>>> so
>>> that nobody has to learn or use HTTP in their code, just the well-known
>>> DBus.
>>>
>>>
>> ... having a DBus couchdb API is a TERRIBLE idea. It's right up there
>> with
>> building an XML-RPC or SOAP bridge to CouchDB. I promise you that a great
>> deal more people understand HTTP than DBus.
>>
>
> Oh, right. That is why there aren't any language bindings for Couch. I
> forgot that everybody just uses HTTP :p
There are so many client libraries with various feature sets because HTTP is
so easy to use and implement.
Still, DBus is tangential to the main argument. My concern with a DBus API
> is you need a persistent process translating between DBus RPC calls and
> HTTP calls. So all DBus gives you is a language-independent API but since
> Couch has bindings for all major languages (and direct HTTP too), I could be
> persuaded that DBus is not worth the effort.
>
> Having said that, any programmer in any language can learn DBus in a day.
You overestimate the the competency of programmers.
-David
Re: Chance of including CouchDB in Linux distros or desktops?
Posted by Jason Smith <jh...@proven-corporation.com>.
David Reid wrote:
>> Consider DBus, which is a mandatory component of Linux (actually,
>> Freedesktop) desktops, is now understood by all developers, leading to more
>> and more apps talking to each other over DBus. I suggest that having a
>> document DB built in to all Linux desktops would be true innovation for
>> Linux development (especially since the GNOME pundits want to move to
>> "web-aware" desktops).
>>
>
> I reject the notion that any (much less all) developers understand DBus.
I'll concede the point. Maybe DBus isn't appropriate. Mostly I wanted
to discuss whether desktop applications could or should use a document
database instead of custom config files and binary formats.
(In the future! When it's ready!)
I agree with Noah's warning about "technobunkum." I disagree that a
desktop document DB is technobunkum; however since posting the original
question, I've realized that distros don't embed MySQL like this, and
even SQLite doesn't have universal adoption among desktop apps. So that
forces me to acknowledge that people have voted with their feet. (Maybe
"the people" are fools, but whaddayagonnado?)
>> 2. (I'm surprised this doesn't exist already) A DBus CouchDB client API, so
>> that nobody has to learn or use HTTP in their code, just the well-known
>> DBus.
>>
>
> ... having a DBus couchdb API is a TERRIBLE idea. It's right up there with
> building an XML-RPC or SOAP bridge to CouchDB. I promise you that a great
> deal more people understand HTTP than DBus.
Oh, right. That is why there aren't any language bindings for Couch. I
forgot that everybody just uses HTTP :p
Still, DBus is tangential to the main argument. My concern with a DBus
API is you need a persistent process translating between DBus RPC calls
and HTTP calls. So all DBus gives you is a language-independent API
but since Couch has bindings for all major languages (and direct HTTP
too), I could be persuaded that DBus is not worth the effort.
Having said that, any programmer in any language can learn DBus in a
day. But while HTTP and REST is easy enough, hardly any programmer can
use CouchDB properly without considerable practice. With CouchDB, the
API isn't the hard part. But if programmers haven't already adopted
SQLite by now, well, that's ominous.
I started out just speculating about the role of CouchDB with desktop
apps, perhaps even as an arrow in the free desktop's quiver. "Why not?"
I said. Well, I am starting to answer that question I'm afraid!
--
Jason Smith
Proven Corporation
Bangkok, Thailand
http://www.proven-corporation.com
Re: Chance of including CouchDB in Linux distros or desktops?
Posted by Noah Slater <ns...@apache.org>.
On Fri, Mar 06, 2009 at 03:51:30PM -0800, David Reid wrote:
> ... having a DBus couchdb API is a TERRIBLE idea. It's right up there with
> building an XML-RPC or SOAP bridge to CouchDB. I promise you that a great
> deal more people understand HTTP than DBus.
LOL
--
Noah Slater, http://tumbolia.org/nslater
Re: Chance of including CouchDB in Linux distros or desktops?
Posted by David Reid <dr...@dreid.org>.
Since this thread has veered wildly off course I'm going to just respond to
Jason here.
On Thu, Mar 5, 2009 at 12:31 AM, Jason Smith <jh...@proven-corporation.com>wrote:
> Hi, all. I had a thought the other day and wanted to share:
>
> What are the chances that the free software movers and shakers could
> successfully lobby CouchDB to be included in the Freedesktop.org system?
>
> Consider DBus, which is a mandatory component of Linux (actually,
> Freedesktop) desktops, is now understood by all developers, leading to more
> and more apps talking to each other over DBus. I suggest that having a
> document DB built in to all Linux desktops would be true innovation for
> Linux development (especially since the GNOME pundits want to move to
> "web-aware" desktops).
>
I reject the notion that any (much less all) developers understand DBus.
>
> If there is any chance in Hell that it could gain traction (I'm
> enthusiastic but skeptical--IMHO "Linux desktop innovation" is a myth, but I
> digress), I'd definitely volunteer to write code, as I have relevant
> experience. I'm thinking of two components:
>
> 1. Similar to DBus, you have one CouchDB process per user that runs when he
> logs on and exits when he logs out. (Maybe have a system-wide CouchDB too
> but I'm not sure if there is a need.)
Now that I've said that, I like the idea. Having a single common API for
getting at all the data I put into gnome applications would be great.
Having that be something that I can also use on non-linux platforms is also
a great idea. I don't think the whole thing is very likely to happen but I
would love it if it did.
>
> 2. (I'm surprised this doesn't exist already) A DBus CouchDB client API, so
> that nobody has to learn or use HTTP in their code, just the well-known
> DBus.
>
... having a DBus couchdb API is a TERRIBLE idea. It's right up there with
building an XML-RPC or SOAP bridge to CouchDB. I promise you that a great
deal more people understand HTTP than DBus.
-David
Re: Chance of including CouchDB in Linux distros or desktops?
Posted by Jason Smith <jh...@proven-corporation.com>.
Jan Lehnardt wrote:
> Boy would I like to see browser history syncing without going through
> "The Cloud".
Speak of the Devil. Today on Linux Journal:
http://www.linuxjournal.com/content/sync-firefox-command-line
I mean, all you have to do is set lpath="$(find /cygdrive/c/Documents\
and\ Settings/$user_name/Application\ Data/Mozilla/Firefox/Profiles/
-iname '*.default')" and then run rsync -av --delete --exclude=Cache
--exclude=lock "$1":"$rpath/" "$lpath".
Christ, what could be easier?
(Oh, and don't forget to install CygWin.)
--
Jason Smith
Proven Corporation
Bangkok, Thailand
http://www.proven-corporation.com
Re: Chance of including CouchDB in Linux distros or desktops?
Posted by Jan Lehnardt <ja...@apache.org>.
On 5 Mar 2009, at 14:31, Noah Slater wrote:
> On Thu, Mar 05, 2009 at 01:15:18PM +0100, Jan Lehnardt wrote:
>> On 5 Mar 2009, at 09:31, Jason Smith wrote:
>>> What are the chances that the free software movers and shakers could
>>> successfully lobby CouchDB to be included in the Freedesktop.org
>>> system?
>>
>> "A couch on every desktop" sounds like a worthy cause and I think it
>> makes sense to spend time on this :)
>
> Come on, let's not make the common mistake of getting distracted
> with technology
> for technology's sake. I would like to see a detailed use case for
> having a
> standard CouchDB database available. What actual problem would this
> solve?
Imagine an Address Book application that doesn't suck (at syncing with
other
devices (mobiles, PDAs) and machines). In fact, imagine a desktop
environment
where all user-data can be syndicated to different endpoints without
much hassle.
Boy would I like to see browser history syncing without going through
"The Cloud".
Yes, some solutions to these problems exist, but a generic, cross
platform
abstraction is missing and CouchDB could be decent technological
foundation.
Cheers
Jan
--
Re: Chance of including CouchDB in Linux distros or desktops?
Posted by Noah Slater <ns...@apache.org>.
On Fri, Mar 06, 2009 at 08:23:21PM +0000, Brian Candler wrote:
> At the moment there's no front-end which makes this process half decent. In
> any case, any "generic" front-end could, at best, show you all the versions
> side by side and ask you to pick the right one by hand, as it has no way of
> knowing which is the best one. It can't even do what Unison does, which is
> to assume that if the document changed at side A but didn't change at side
> B, then side A is the preferred one.
>
> Alternatively, you could try to have automatic resolution. This is
> application-specific, that's to say, specific to the format of the JSON
> documents in question, and what extra fields they may or may not include for
> this purpose (such as timestamps). There's no framework in place where you
> can plug in conflict-resolution rules yet, so everyone has to build their
> own, externally.
This feels like a good place to say "patches welcome" with a wink and a smile.
--
Noah Slater, http://tumbolia.org/nslater
Re: Chance of including CouchDB in Linux distros or desktops?
Posted by Chris Anderson <jc...@apache.org>.
On Fri, Mar 6, 2009 at 12:23 PM, Brian Candler <B....@pobox.com> wrote:
>
> At the moment there's no front-end which makes this process half decent. In
> any case, any "generic" front-end could, at best, show you all the versions
> side by side and ask you to pick the right one by hand, as it has no way of
> knowing which is the best one.
This sounds like it would be an afternoon Futon hack.
> If desktop distros had included 0.8.1, then probably all apps would have to
> stick with the 0.8.1 API for that span of time. The alternative is to
> release 0.9.0 and break all the apps which used it.
>
I agree with you hear, Brian. Building apps against CouchDB trunk is
fine for developers, but unless you want to support out of date
environments with new apps, it's probably better not to encourage
standalone Couch on non-developer desktops except for at releases.
Luckily 0.9.0 is around the corner.
> FWIW, I don't think CouchDB's strengths particularly play to the desktop.
Ha! We'll see about that. ;)
--
Chris Anderson
http://jchris.mfdz.com
Re: Chance of including CouchDB in Linux distros or desktops?
Posted by Brian Candler <B....@pobox.com>.
On Thu, Mar 05, 2009 at 11:26:08PM +0100, Jan Lehnardt wrote:
>> This is all future-talk.
>
> This is what this thread is about.
I interpreted it as "why can't we get desktop distros to start integrating
couchdb *now*?"
(Searches archives) OP said:
"What are the chances that the free software movers and shakers could
successfully lobby CouchDB to be included in the Freedesktop.org system?"
...
"I'm asking if it's desirable that modern distros bundle a document DB
(Couch) for all apps to build from."
This to me sounds like a request for today: "modern" distros rather than
"future" distros.
>> Until couchdb has some half-decent way of resolving conflicts - either
>> a
>> front-end for doing it manually (such as in futon) and/or via
>> application-specific programmatic rules - it doesn't cut. As far as
>> the user
>> is concerned, updated documents vanish at random.
>
> Sorry, but WTF? This is bollocks :)
...
> CouchDB sets _conflics = [list of conflicting revs] in a conflicting doc
> and a map-only-view gives you a list of all conflicting docs and the
> app can easily resolve them.
Yes. And it picks the winner using a completely arbitrary algorithm which
you have no control over. So anyone who was replicating their laptop
(containing the most recent version of their work) to their desktop
(containing an older one) will, with a 50/50 chance, see the laptop revert
to the older version of the file when they next try to open it.
Yes, I *know* the other document is there, on both sides. If you construct a
suitable couchdb query, you can even find it and retrieve it.
At the moment there's no front-end which makes this process half decent. In
any case, any "generic" front-end could, at best, show you all the versions
side by side and ask you to pick the right one by hand, as it has no way of
knowing which is the best one. It can't even do what Unison does, which is
to assume that if the document changed at side A but didn't change at side
B, then side A is the preferred one.
Alternatively, you could try to have automatic resolution. This is
application-specific, that's to say, specific to the format of the JSON
documents in question, and what extra fields they may or may not include for
this purpose (such as timestamps). There's no framework in place where you
can plug in conflict-resolution rules yet, so everyone has to build their
own, externally.
Even if you have sufficient knowledge of your document structure to perform
automatic merging, CouchDB won't give you sufficient history (back to a
common ancestor) if compaction has taken place.
If you want all this, then right now git is a much better tool to build on.
Doesn't have anything like Couch's performance of course.
>> IMO it's also a long way before couchdb becomes stable enough to
>> become an
>> integral part of the desktop (by "stable" I mean "not constantly
>> changing",
>> rather than "working properly")
>
> Oh hey, can I get a few more JIRA's from you?
I think you misunderstand me, or I did not word carefully.
What I meant is CouchDB works pretty well, and has done so for a while - so
it's "stable" by that meaning of the word.
CouchDB is also changing and developing lots. This is also good in many
ways. But it's the last thing you want for integrating into a desktop distro
which may have a support lifecycle of 1-4 years.
If desktop distros had included 0.8.1, then probably all apps would have to
stick with the 0.8.1 API for that span of time. The alternative is to
release 0.9.0 and break all the apps which used it.
> Also, CouchDB has
> been labelled alpha ver clearly. A in-flux API does not mean integration
> into a desktop system can be *started* to be talked about.
But should it be "lobbied" for??
> For that matter, nobody should use CouchDB in the current state, yet
> some have done so since the 0.7. releases (which ran till ~Dec 2008 and
> are now on 0.8)
Of course. If it solves your specific problem today, then go for it. But is
it ready today to be a foundation piece of the O/S itself?
> Sorry if that sounds cranky, but nobody said "CouchDB will make desktops
> more awesome today and immediately". It was a request for discussion.
No problem, I'm happy to discuss.
FWIW, I don't think CouchDB's strengths particularly play to the desktop. If
you're considering it as a gconf replacement, it's overkill (CouchDB
provides huge write/read/index performance, which isn't needed). If you want
to turn your entire filesystem into a CouchDB store, and index all your
documents, that's interesting but I'm not sure that's under discussion. I
also seem to recall that Microsoft tried this a few years ago, and failed...
although maybe the time is right to try again.
But then again, CouchDB semantics don't map particularly well to POSIX-type
filesystems either.
Would it be cool to store my mailbox in CouchDB rather than Maildir?
Perhaps, but unless you also provide an IMAP front-end, I doubt the MUA
writers are going to write Couch-HTTP backends for their code. In any case,
saying "use CouchDB" for this is like saying "use XML" - without a standard
for the format of the JSON metadata and the views, different consumers of
mailboxes won't be able to share data.
So in summary - I don't yet see a need for a kick-ass data warehouse on a
GUI desktop plugged into DBus. And I believe that if and when one is needed,
the application will push adoption of the database, rather than vice versa.
Regards,
Brian.
Re: Chance of including CouchDB in Linux distros or desktops?
Posted by Dean Landolt <de...@deanlandolt.com>.
On Thu, Mar 5, 2009 at 7:16 PM, Jeffrey Melloy <jm...@gmail.com> wrote:
> - Show quoted text -
> On Thu, Mar 5, 2009 at 2:26 PM, Jan Lehnardt <ja...@apache.org> wrote:
> >
> > On 5 Mar 2009, at 18:56, Brian Candler wrote:
> >
> >> On Thu, Mar 05, 2009 at 03:31:43PM +0000, Alan Bell wrote:
> >>>
> >>> Noah Slater wrote:
> >>>>
> >>>> What actual problem would this solve?
> >>>>
> >>> well I think that is the discussion point. It certainly raises a few
> >>> interesting thoughts. One of the suggestions was a couchdb process per
> >>> user. Not quite sure about this one, sounds like it might not scale
> well
> >>> with multiple users. One database per user might well be handy. It
> could
> >>> perhaps replace the gconf database.
> >>>
> >>> If it could do everything for a user, perhaps even with a FUSE
> >>> filesystem pointing at the users database mounted at /home/user then
> the
> >>> replication between machines would be quite cool, particularly for
> >>> laptops.
> >>
> >> This is all future-talk.
> >
> > This is what this thread is about.
> >
> >>
> >> Until couchdb has some half-decent way of resolving conflicts - either a
> >> front-end for doing it manually (such as in futon) and/or via
> >> application-specific programmatic rules - it doesn't cut. As far as the
> >> user
> >> is concerned, updated documents vanish at random.
> >
> > Sorry, but WTF? This is bollocks :)
> >
> >> For now I will stick with unison, which at least tells me there are
> files
> >> with conflicting updates and leaves the state at both ends unchanged.
> >
> > CouchDB sets _conflics = [list of conflicting revs] in a conflicting doc
> > and a map-only-view gives you a list of all conflicting docs and the
> > app can easily resolve them.
> >
> >
> >> IMO it's also a long way before couchdb becomes stable enough to become
> an
> >> integral part of the desktop (by "stable" I mean "not constantly
> >> changing",
> >> rather than "working properly")
> >
> > Oh hey, can I get a few more JIRA's from you? Also, CouchDB has
> > been labelled alpha ver clearly. A in-flux API does not mean integration
> > into a desktop system can be *started* to be talked about.
> >
> > For that matter, nobody should use CouchDB in the current state, yet
> > some have done so since the 0.7. releases (which ran till ~Dec 2008 and
> > are now on 0.8)
> >
> > Sorry if that sounds cranky, but nobody said "CouchDB will make desktops
> > more awesome today and immediately". It was a request for discussion.
> >
> > Cheers
> > Jan
> > --
>
> Off the top of my head, I can think of quite a few situations where
> having a data store that replicates point-to-point would be useful.
>
> Bookmarks have already been mentioned: add a bookmark on one machine,
> sit down at the other and it's automatically there. Browser history,
> too.
>
> Address books are a decent example for replication, but they're a
> pretty good example of a system where you might want to store
> different pieces of information about different people.
>
> Another thought would be something similar to DevonTHINK or Yojimbo.
> If you're unfamiliar, they're sort of desktop databases. Each allows
> you to keep a variety of types and styles of notes (and completely
> rich media) without you having to think about the filesystem. If they
> were backed by couchdb they could be replicated and backed up among
> all of your machines.
Most of these examples are pretty much right in line with what Mozilla
Weave's aiming for. Of course, it'd be awfully sweet if they offered up a
couch backend (I bet it'd beat the shit out of their flakey-ass webdav).
>
> And, lastly, configuration file management. Who's had to say, "What's
> different between these two machines" or have problems in the push
> from development to production or production box #1 to production box
> 40?
I think it's pretty easy to think of dozens of cool uses of couch on the
desktop, but I get the thrust of Noah's argument. In each of these cases, if
couch is the right tool for the job, it can always be installed as needed by
the desktop app.
Of course, it'd be great if there were as simple a couch install story on
windows as on linux and mac, but I think Jan already threw that out there as
an idea for a summer of code project (Antony's even been offering a bounty
for it -- hopefully someone bites).
> My thinking is that if you have a good product and you work to make it
> *ubiquitous*, you'll get people using it in a lot of ways you don't
> expect.
[emphasis mine] -- speaking of Mozilla, another project (already using couch
for their command sharing) that could get a huge boost from having a local
couch as a nountypes store, if you're into that kinda thing...
All I'm saying is the easiest way to get a couch on every desktop, so to
speak, is to not through a Freedesktop spec but as an infrastructure include
in other successful cross-platform open source desktop projects.
Re: Chance of including CouchDB in Linux distros or desktops?
Posted by Jeffrey Melloy <jm...@gmail.com>.
On Thu, Mar 5, 2009 at 2:26 PM, Jan Lehnardt <ja...@apache.org> wrote:
>
> On 5 Mar 2009, at 18:56, Brian Candler wrote:
>
>> On Thu, Mar 05, 2009 at 03:31:43PM +0000, Alan Bell wrote:
>>>
>>> Noah Slater wrote:
>>>>
>>>> What actual problem would this solve?
>>>>
>>> well I think that is the discussion point. It certainly raises a few
>>> interesting thoughts. One of the suggestions was a couchdb process per
>>> user. Not quite sure about this one, sounds like it might not scale well
>>> with multiple users. One database per user might well be handy. It could
>>> perhaps replace the gconf database.
>>>
>>> If it could do everything for a user, perhaps even with a FUSE
>>> filesystem pointing at the users database mounted at /home/user then the
>>> replication between machines would be quite cool, particularly for
>>> laptops.
>>
>> This is all future-talk.
>
> This is what this thread is about.
>
>>
>> Until couchdb has some half-decent way of resolving conflicts - either a
>> front-end for doing it manually (such as in futon) and/or via
>> application-specific programmatic rules - it doesn't cut. As far as the
>> user
>> is concerned, updated documents vanish at random.
>
> Sorry, but WTF? This is bollocks :)
>
>> For now I will stick with unison, which at least tells me there are files
>> with conflicting updates and leaves the state at both ends unchanged.
>
> CouchDB sets _conflics = [list of conflicting revs] in a conflicting doc
> and a map-only-view gives you a list of all conflicting docs and the
> app can easily resolve them.
>
>
>> IMO it's also a long way before couchdb becomes stable enough to become an
>> integral part of the desktop (by "stable" I mean "not constantly
>> changing",
>> rather than "working properly")
>
> Oh hey, can I get a few more JIRA's from you? Also, CouchDB has
> been labelled alpha ver clearly. A in-flux API does not mean integration
> into a desktop system can be *started* to be talked about.
>
> For that matter, nobody should use CouchDB in the current state, yet
> some have done so since the 0.7. releases (which ran till ~Dec 2008 and
> are now on 0.8)
>
> Sorry if that sounds cranky, but nobody said "CouchDB will make desktops
> more awesome today and immediately". It was a request for discussion.
>
> Cheers
> Jan
> --
Off the top of my head, I can think of quite a few situations where
having a data store that replicates point-to-point would be useful.
Bookmarks have already been mentioned: add a bookmark on one machine,
sit down at the other and it's automatically there. Browser history,
too.
Address books are a decent example for replication, but they're a
pretty good example of a system where you might want to store
different pieces of information about different people.
Another thought would be something similar to DevonTHINK or Yojimbo.
If you're unfamiliar, they're sort of desktop databases. Each allows
you to keep a variety of types and styles of notes (and completely
rich media) without you having to think about the filesystem. If they
were backed by couchdb they could be replicated and backed up among
all of your machines.
And, lastly, configuration file management. Who's had to say, "What's
different between these two machines" or have problems in the push
from development to production or production box #1 to production box
40?
My thinking is that if you have a good product and you work to make it
ubiquitous, you'll get people using it in a lot of ways you don't
expect.
-Jeff
Re: Chance of including CouchDB in Linux distros or desktops?
Posted by Jan Lehnardt <ja...@apache.org>.
On 5 Mar 2009, at 18:56, Brian Candler wrote:
> On Thu, Mar 05, 2009 at 03:31:43PM +0000, Alan Bell wrote:
>> Noah Slater wrote:
>>> What actual problem would this solve?
>>>
>> well I think that is the discussion point. It certainly raises a few
>> interesting thoughts. One of the suggestions was a couchdb process
>> per
>> user. Not quite sure about this one, sounds like it might not scale
>> well
>> with multiple users. One database per user might well be handy. It
>> could
>> perhaps replace the gconf database.
>>
>> If it could do everything for a user, perhaps even with a FUSE
>> filesystem pointing at the users database mounted at /home/user
>> then the
>> replication between machines would be quite cool, particularly for
>> laptops.
>
> This is all future-talk.
This is what this thread is about.
>
> Until couchdb has some half-decent way of resolving conflicts -
> either a
> front-end for doing it manually (such as in futon) and/or via
> application-specific programmatic rules - it doesn't cut. As far as
> the user
> is concerned, updated documents vanish at random.
Sorry, but WTF? This is bollocks :)
> For now I will stick with unison, which at least tells me there are
> files
> with conflicting updates and leaves the state at both ends unchanged.
CouchDB sets _conflics = [list of conflicting revs] in a conflicting doc
and a map-only-view gives you a list of all conflicting docs and the
app can easily resolve them.
> IMO it's also a long way before couchdb becomes stable enough to
> become an
> integral part of the desktop (by "stable" I mean "not constantly
> changing",
> rather than "working properly")
Oh hey, can I get a few more JIRA's from you? Also, CouchDB has
been labelled alpha ver clearly. A in-flux API does not mean integration
into a desktop system can be *started* to be talked about.
For that matter, nobody should use CouchDB in the current state, yet
some have done so since the 0.7. releases (which ran till ~Dec 2008 and
are now on 0.8)
Sorry if that sounds cranky, but nobody said "CouchDB will make desktops
more awesome today and immediately". It was a request for discussion.
Cheers
Jan
--
Re: Chance of including CouchDB in Linux distros or desktops?
Posted by Brian Candler <B....@pobox.com>.
On Thu, Mar 05, 2009 at 03:31:43PM +0000, Alan Bell wrote:
> Noah Slater wrote:
>> What actual problem would this solve?
>>
> well I think that is the discussion point. It certainly raises a few
> interesting thoughts. One of the suggestions was a couchdb process per
> user. Not quite sure about this one, sounds like it might not scale well
> with multiple users. One database per user might well be handy. It could
> perhaps replace the gconf database.
>
> If it could do everything for a user, perhaps even with a FUSE
> filesystem pointing at the users database mounted at /home/user then the
> replication between machines would be quite cool, particularly for
> laptops.
This is all future-talk.
Until couchdb has some half-decent way of resolving conflicts - either a
front-end for doing it manually (such as in futon) and/or via
application-specific programmatic rules - it doesn't cut. As far as the user
is concerned, updated documents vanish at random.
For now I will stick with unison, which at least tells me there are files
with conflicting updates and leaves the state at both ends unchanged.
IMO it's also a long way before couchdb becomes stable enough to become an
integral part of the desktop (by "stable" I mean "not constantly changing",
rather than "working properly")
Re: Chance of including CouchDB in Linux distros or desktops?
Posted by Michael McDaniel <co...@autosys.us>.
On Thu, Mar 05, 2009 at 08:38:11PM -0800, Chris Anderson wrote:
> On Thu, Mar 5, 2009 at 8:17 PM, kowsik <ko...@gmail.com> wrote:
> > Speaking of couchlite, I would __love__ to see parallel map/reduce. I
> > got this 100K+ doc database and the instant-gratification-factor for
> > trying out new views just sucks (takes about 15 minutes). I have to
> > reduce the doc count, just so I can try out new ideas in terms of
> > views. And it's on a dual quad-core system. The other 7 cpus badly
> > want to relax on the couch, but can't. :-(
>
> I haven't had a chance yet, but one science experiment that might help
> with identifying bottlenecks would be Michael McDaniel's erlang view
> server:
> http://github.com/mmcdanie/erlview/tree/master
>
> My gut tells me that the bottleneck is index insertion, and not view
> function execution. Erlang views should have almost no overhead, so we
> can guess at the best case for parallel JS view servers. It maybe that
> to get true parallelism we need to parallelize indexing and not just
> generation...
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
I agree with paralleizing indexing, Chris. Also, being able to
have different index/views in different files could speed things
up, especially when every map fun has to run again; e.g. place
the different index/view files on different spindles.
Regarding parallizing views, that is TODO #7 in erlview. If
couch_query_servers:map_docs/2 sent a list of documents, the
erlview:handle_call( {map_doc ... ) could split up the list
and run parallel funs. couch_query_servers could dynamically
determine how many docs to send in the list by how much memory
is currently available.
Do note that early erlview testing has shown it to be two to
ten times faster than javascript views, depending on the
map fun complexity. I attribute this greatly to not having
to do json <-> Erlang term() conversions.
~Michael
>
>
> --
> Chris Anderson
> http://jchris.mfdz.com
--
Michael McDaniel
Portland, Oregon, USA
http://trip.autosys.us
http://autosys.us
Re: Chance of including CouchDB in Linux distros or desktops?
Posted by Chris Anderson <jc...@apache.org>.
On Thu, Mar 5, 2009 at 8:17 PM, kowsik <ko...@gmail.com> wrote:
> Speaking of couchlite, I would __love__ to see parallel map/reduce. I
> got this 100K+ doc database and the instant-gratification-factor for
> trying out new views just sucks (takes about 15 minutes). I have to
> reduce the doc count, just so I can try out new ideas in terms of
> views. And it's on a dual quad-core system. The other 7 cpus badly
> want to relax on the couch, but can't. :-(
I haven't had a chance yet, but one science experiment that might help
with identifying bottlenecks would be Michael McDaniel's erlang view
server:
http://github.com/mmcdanie/erlview/tree/master
My gut tells me that the bottleneck is index insertion, and not view
function execution. Erlang views should have almost no overhead, so we
can guess at the best case for parallel JS view servers. It maybe that
to get true parallelism we need to parallelize indexing and not just
generation...
--
Chris Anderson
http://jchris.mfdz.com
Re: Chance of including CouchDB in Linux distros or desktops?
Posted by kowsik <ko...@gmail.com>.
Speaking of couchlite, I would __love__ to see parallel map/reduce. I
got this 100K+ doc database and the instant-gratification-factor for
trying out new views just sucks (takes about 15 minutes). I have to
reduce the doc count, just so I can try out new ideas in terms of
views. And it's on a dual quad-core system. The other 7 cpus badly
want to relax on the couch, but can't. :-(
If we can get the map/reduce parallelized, I think it would be a huge
win for "personal" couch use. For really simple (but large) data sets,
sqlite is just super __fast__! And it's just a 'gem install sqlite3'
away.
K.
On Thu, Mar 5, 2009 at 7:54 AM, Noah Slater <ns...@apache.org> wrote:
> On Thu, Mar 05, 2009 at 10:44:58PM +0700, Jason Smith wrote:
>> Alan Bell wrote:
>>> Noah Slater wrote:
>>>> What actual problem would this solve?
>>>>
>>> well I think that is the discussion point. It certainly raises a few
>>> interesting thoughts. One of the suggestions was a couchdb process per
>>> user. Not quite sure about this one, sounds like it might not scale
>>> well with multiple users. One database per user might well be handy. It
>>> could perhaps replace the gconf database.
>>
>> Well, gconf has its own API and should probably be "up" even in bad
>> situations like a full disk or when the OOM killer starts spraying
>> bullets into the process space.
>
> Oh man, that's such a great mental image.
>
> Guess it helps when you've had to firefight OOM killer on production machines.
>
>> (To play Devil's advocate, my argument could be made for MySQL too and
>> we don't see that happening. Some might say that it's just a bad idea,
>> but I would still argue that Linux desktop developers are simply
>> uncreative.)
>
> I'm sure someone will disagree with me here, but I think that the problem stems
> from the server/client model. While this architectural constraint is great for
> distributed hypertext systems, which is what we're designing CouchDB for, it's
> not so great when you're trying to re-purpose it for local use. That's the
> difference between MySQL and SQLite.
>
> If someone could adapt CouchDB into CouchLite or something similar...
>
> --
> Noah Slater, http://tumbolia.org/nslater
>
Re: Chance of including CouchDB in Linux distros or desktops?
Posted by Noah Slater <ns...@apache.org>.
On Thu, Mar 05, 2009 at 10:44:58PM +0700, Jason Smith wrote:
> Alan Bell wrote:
>> Noah Slater wrote:
>>> What actual problem would this solve?
>>>
>> well I think that is the discussion point. It certainly raises a few
>> interesting thoughts. One of the suggestions was a couchdb process per
>> user. Not quite sure about this one, sounds like it might not scale
>> well with multiple users. One database per user might well be handy. It
>> could perhaps replace the gconf database.
>
> Well, gconf has its own API and should probably be "up" even in bad
> situations like a full disk or when the OOM killer starts spraying
> bullets into the process space.
Oh man, that's such a great mental image.
Guess it helps when you've had to firefight OOM killer on production machines.
> (To play Devil's advocate, my argument could be made for MySQL too and
> we don't see that happening. Some might say that it's just a bad idea,
> but I would still argue that Linux desktop developers are simply
> uncreative.)
I'm sure someone will disagree with me here, but I think that the problem stems
from the server/client model. While this architectural constraint is great for
distributed hypertext systems, which is what we're designing CouchDB for, it's
not so great when you're trying to re-purpose it for local use. That's the
difference between MySQL and SQLite.
If someone could adapt CouchDB into CouchLite or something similar...
--
Noah Slater, http://tumbolia.org/nslater
Re: Chance of including CouchDB in Linux distros or desktops?
Posted by Jason Smith <jh...@proven-corporation.com>.
Alan Bell wrote:
> Noah Slater wrote:
>> What actual problem would this solve?
>>
> well I think that is the discussion point. It certainly raises a few
> interesting thoughts. One of the suggestions was a couchdb process per
> user. Not quite sure about this one, sounds like it might not scale well
> with multiple users. One database per user might well be handy. It could
> perhaps replace the gconf database.
Well, gconf has its own API and should probably be "up" even in bad
situations like a full disk or when the OOM killer starts spraying
bullets into the process space. But yeah, totally. IMO if CouchDB is
really well-suited for a large class of web apps then it similarly
well-suited for desktop apps, since nowadays desktop apps are
Internet-aware. (The whole point of Adobe AIR is to build web-aware
desktop apps using Internet technologies.)
> If it could do everything for a user, perhaps even with a FUSE
> filesystem pointing at the users database mounted at /home/user then the
> replication between machines would be quite cool, particularly for laptops.
I don't know how that would perform, not to mention in the case of
random-access, but I betcha that there would be fewer files to
synchronize in the first place if more apps stored (at least) their
metadata in a database.
(To play Devil's advocate, my argument could be made for MySQL too and
we don't see that happening. Some might say that it's just a bad idea,
but I would still argue that Linux desktop developers are simply
uncreative.)
--
Jason Smith
Proven Corporation
Bangkok, Thailand
http://www.proven-corporation.com
Re: Chance of including CouchDB in Linux distros or desktops?
Posted by Alan Bell <al...@theopenlearningcentre.com>.
Noah Slater wrote:
>
> What would it offer over and above the gconf database?
>
>
I have no idea. I guess it might hold more complex structures and
perform and scale differently.
>> If it could do everything for a user, perhaps even with a FUSE
>> filesystem pointing at the users database mounted at /home/user then the
>> replication between machines would be quite cool, particularly for
>> laptops.
>>
>
> If you want to replicate a filesystem, why not use rsync?
>
>
I do at the moment it works great, and actually does a really good job
on partial file replication. CouchDB+Fuse just seems like something
interesting to think about.
Re: Chance of including CouchDB in Linux distros or desktops?
Posted by Noah Slater <ns...@apache.org>.
On Thu, Mar 05, 2009 at 03:31:43PM +0000, Alan Bell wrote:
> Noah Slater wrote:
>> What actual problem would this solve?
> well I think that is the discussion point. It certainly raises a few
> interesting thoughts. One of the suggestions was a couchdb process per
> user. Not quite sure about this one, sounds like it might not scale well
> with multiple users. One database per user might well be handy. It could
> perhaps replace the gconf database.
What would it offer over and above the gconf database?
> If it could do everything for a user, perhaps even with a FUSE
> filesystem pointing at the users database mounted at /home/user then the
> replication between machines would be quite cool, particularly for
> laptops.
If you want to replicate a filesystem, why not use rsync?
--
Noah Slater, http://tumbolia.org/nslater
Re: Chance of including CouchDB in Linux distros or desktops?
Posted by Alan Bell <al...@theopenlearningcentre.com>.
Noah Slater wrote:
> What actual problem would this solve?
>
well I think that is the discussion point. It certainly raises a few
interesting thoughts. One of the suggestions was a couchdb process per
user. Not quite sure about this one, sounds like it might not scale well
with multiple users. One database per user might well be handy. It could
perhaps replace the gconf database.
If it could do everything for a user, perhaps even with a FUSE
filesystem pointing at the users database mounted at /home/user then the
replication between machines would be quite cool, particularly for laptops.
Alan.
Re: Chance of including CouchDB in Linux distros or desktops?
Posted by Jason Smith <jh...@proven-corporation.com>.
Noah Slater wrote:
> On Thu, Mar 05, 2009 at 08:56:15PM +0700, Jason Smith wrote:
>>> Come on, let's not make the common mistake of getting distracted with technology
>>> for technology's sake. I would like to see a detailed use case for having a
>>> standard CouchDB database available. What actual problem would this solve?
>> Talk like that is why there is never (let's be kind: "hardly") any
>> innovation on the Linux desktop.
>
> I'm sorry, but this is bullshit.
>
> Read this essay, and then come back and tell me I'm talking out of my arse:
>
> http://inamidst.com/whits/2008/technobunkum
Yes, I'm fond of Spolsky's "Architecture Astronaut" essay myself, which
makes a similar point.
> When you meet someone who wants to advocate the widespread
> adoption of some piece of technology, without any thought whatsoever as to how
> that technology will actually help people in some defined way, you have a pretty
> good indication that you're dealing with technobunkum.
Hear, hear!
--
Jason Smith
Proven Corporation
Bangkok, Thailand
http://www.proven-corporation.com
Re: Chance of including CouchDB in Linux distros or desktops?
Posted by Noah Slater <ns...@apache.org>.
On Thu, Mar 05, 2009 at 08:56:15PM +0700, Jason Smith wrote:
>> Come on, let's not make the common mistake of getting distracted with technology
>> for technology's sake. I would like to see a detailed use case for having a
>> standard CouchDB database available. What actual problem would this solve?
>
> Talk like that is why there is never (let's be kind: "hardly") any
> innovation on the Linux desktop.
I'm sorry, but this is bullshit.
Read this essay, and then come back and tell me I'm talking out of my arse:
http://inamidst.com/whits/2008/technobunkum
People spend way too much time focusing on technology for technology's sake, an
nowhere near enough time thinking about what we can actually accomplish with
that technology. When you meet someone who wants to advocate the widespread
adoption of some piece of technology, without any thought whatsoever as to how
that technology will actually help people in some defined way, you have a pretty
good indication that you're dealing with technobunkum.
--
Noah Slater, http://tumbolia.org/nslater
Re: Chance of including CouchDB in Linux distros or desktops?
Posted by Jason Smith <jh...@proven-corporation.com>.
Noah Slater wrote:
> On Thu, Mar 05, 2009 at 01:15:18PM +0100, Jan Lehnardt wrote:
>> On 5 Mar 2009, at 09:31, Jason Smith wrote:
>>> What are the chances that the free software movers and shakers could
>>> successfully lobby CouchDB to be included in the Freedesktop.org
>>> system?
>> "A couch on every desktop" sounds like a worthy cause and I think it
>> makes sense to spend time on this :)
>
> Come on, let's not make the common mistake of getting distracted with technology
> for technology's sake. I would like to see a detailed use case for having a
> standard CouchDB database available. What actual problem would this solve?
Talk like that is why there is never (let's be kind: "hardly") any
innovation on the Linux desktop.
Anyway, perhaps you're right, but what is the opportunity cost of not
having an advanced data store provided by the OS? IMO CouchDB is
suitable for a large class of desktop apps as well as web apps. It's
synchronization features work well with users' distributed-style usage
(where one user may use a laptop, a desktop which dual-boots, and a
workstation at the office, plus cloud/web apps).
Also, I am inspired by the story of SQLite, which suggests that you
ditch your custom binary file format in place of a structured, queryable
format. For this reason (and others), SQLite has become the most
widely-deployed software package in the world (it's on every Windows,
Mac, and Linux box and a slew of mobile phones).
--
Jason Smith
Proven Corporation
Bangkok, Thailand
http://www.proven-corporation.com
Re: Chance of including CouchDB in Linux distros or desktops?
Posted by Noah Slater <ns...@apache.org>.
On Thu, Mar 05, 2009 at 01:15:18PM +0100, Jan Lehnardt wrote:
> On 5 Mar 2009, at 09:31, Jason Smith wrote:
>> What are the chances that the free software movers and shakers could
>> successfully lobby CouchDB to be included in the Freedesktop.org
>> system?
>
> "A couch on every desktop" sounds like a worthy cause and I think it
> makes sense to spend time on this :)
Come on, let's not make the common mistake of getting distracted with technology
for technology's sake. I would like to see a detailed use case for having a
standard CouchDB database available. What actual problem would this solve?
--
Noah Slater, http://tumbolia.org/nslater
Re: Chance of including CouchDB in Linux distros or desktops?
Posted by Jan Lehnardt <ja...@apache.org>.
On 5 Mar 2009, at 09:31, Jason Smith wrote:
> Hi, all. I had a thought the other day and wanted to share:
>
> What are the chances that the free software movers and shakers could
> successfully lobby CouchDB to be included in the Freedesktop.org
> system?
"A couch on every desktop" sounds like a worthy cause and I think it
makes sense to spend time on this :)
> Consider DBus, which is a mandatory component of Linux (actually,
> Freedesktop) desktops, is now understood by all developers, leading
> to more and more apps talking to each other over DBus. I suggest
> that having a document DB built in to all Linux desktops would be
> true innovation for Linux development (especially since the GNOME
> pundits want to move to "web-aware" desktops).
>
> If there is any chance in Hell that it could gain traction (I'm
> enthusiastic but skeptical--IMHO "Linux desktop innovation" is a
> myth, but I digress), I'd definitely volunteer to write code, as I
> have relevant experience. I'm thinking of two components:
>
> 1. Similar to DBus, you have one CouchDB process per user that runs
> when he logs on and exits when he logs out. (Maybe have a system-
> wide CouchDB too but I'm not sure if there is a need.)
>
> 2. (I'm surprised this doesn't exist already) A DBus CouchDB client
> API, so that nobody has to learn or use HTTP in their code, just the
> well-known DBus.
So DBus is somehow more well known than HTTP? :)
Cheers
Jan
--
>
> Given 1 and 2, any desktop app could just assume a private (for that
> user) DB in the same way they assume a per-user-session DBus bus
> today. So, in summary, I'm asking if it's desirable that modern
> distros bundle a document DB (Couch) for all apps to build from.
>
> --
> Jason Smith
> Proven Corporation
> Bangkok, Thailand
> http://www.proven-corporation.com
>