You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@couchdb.apache.org by "Andrew Stuart (SuperCoders)" <an...@supercoders.com.au> on 2011/07/12 01:18:09 UTC
Building substantial "pure" 2 tier CouchApps - feasible?
I'd like to build a rather substantial application, it would be nice
to build it as a pure 2 tier CouchApp.
How practical is this really? Does anyone have experience building
anything big in this way? Or are 2 tier CouchApps really only
effective for small stuff?
I understand that it very much depends on what the app does, but I'm
trying to get a general sense of it.
If pure 2 tier CouchDB apps is an impractical goal then we'll design
the architecture from the beginning as a 3 tier app and whack in some
sort of server.
as
Re: Building substantial "pure" 2 tier CouchApps - feasible?
Posted by Jonathan Geddes <ge...@gmail.com>.
I think the "pure" 2 layer approach is not going to be feasible, to be
honest with you. I think a more realistic goal is what some have been
calling a 2.1 tier architecture. The 2.1 tier architecture is simply the 2
layers plus some simple utilities *behind* couchdb. What I mean by 'behind'
is that the client doesn't interface directly with the utilities, but with
couchdb directly, just like the 'pure' architechture. The utilities behind
couchdb either:
a) listen on the _changes api for certain events and do some processing in
the db (wait for image uploads and then crop them and put them back in the
database, for example)
b) accept requests that come in through couchdb's reverse proxy capabilities
[0] and perform something that is beyond the scope of couchdb (large joins
over several queries, etc).
c) run as an "external process" [2].
In cases a and b, you can use "external os processes"[2] to make sure that
the other parts of the system are always running (if they crash, couch will
start them again!). In case c, a new process is spawned for each request, so
this can be slow.
The nice thing about this approach is that you can start slow with a simple
couchapp, but know that if needed, your architecture can grow
to accommodate essentially any requirements (at the cost of some of the
deployment flexibility of a chouchapp).
[0]
https://github.com/apache/couchdb/commit/11469d902b15145d361f9f7ec66a09ac3d04757c
[1] http://wiki.apache.org/couchdb/ExternalProcesses
[2]
https://github.com/apache/couchdb/commit/c8785113522b8486fab0ba53f2043d94e9b1507f
disclaimer: I'm currently exploring the effectiveness of this approach, but
I've never seen it to be effective from start to finish! And my app still
hasn't grown beyond the simple couchapp model (no extra utilities yet). It's
surprising how much fits into the couchapp model!
HTH
--Jonathan
On Mon, Jul 11, 2011 at 7:18 PM, Marcello Nuccio
<ma...@gmail.com>wrote:
> I am building a pure couchapp to:
> - insert the orders of the clients;
> - insert the modifications requested by the clients;
> - assign the work to the laboratories;
> - track the warehouse.
> - etc.
>
> Not very big, but quite complex because I have many document types and
> I must exchange data with lot of legacy systems.
>
> It is an hard work if you are used to RDMS. The most challenging problems
> are:
> - how to model the data to keep the couchapp simple, efficient, and
> easily interoperable with other system?
> - access control: in CouchDB is a brand new world. Everything you
> already know on access control is almost useless with CouchDB.
> - you will need to write some external utility to do some tasks, not
> only for maintenance.
>
> just my 2cents...
>
> Marcello
>
> 2011/7/12 Andrew Stuart (SuperCoders) <an...@supercoders.com.au>:
> > I'd like to build a rather substantial application, it would be nice to
> > build it as a pure 2 tier CouchApp.
> >
> > How practical is this really? Does anyone have experience building
> anything
> > big in this way? Or are 2 tier CouchApps really only effective for small
> > stuff?
> >
> > I understand that it very much depends on what the app does, but I'm
> trying
> > to get a general sense of it.
> >
> > If pure 2 tier CouchDB apps is an impractical goal then we'll design the
> > architecture from the beginning as a 3 tier app and whack in some sort of
> > server.
> >
> > as
> >
>
Re: Building substantial "pure" 2 tier CouchApps - feasible?
Posted by Max Ogden <ma...@maxogden.com>.
You can do a lot with the _changes feed:
https://github.com/iriscouch/follow or
https://github.com/mikeal/dbemitter(especially
https://github.com/mikeal/dbemitter/tree/master/tests)
On Mon, Jul 11, 2011 at 6:18 PM, Marcello Nuccio
<ma...@gmail.com>wrote:
> I am building a pure couchapp to:
> - insert the orders of the clients;
> - insert the modifications requested by the clients;
> - assign the work to the laboratories;
> - track the warehouse.
> - etc.
>
> Not very big, but quite complex because I have many document types and
> I must exchange data with lot of legacy systems.
>
> It is an hard work if you are used to RDMS. The most challenging problems
> are:
> - how to model the data to keep the couchapp simple, efficient, and
> easily interoperable with other system?
> - access control: in CouchDB is a brand new world. Everything you
> already know on access control is almost useless with CouchDB.
> - you will need to write some external utility to do some tasks, not
> only for maintenance.
>
> just my 2cents...
>
> Marcello
>
> 2011/7/12 Andrew Stuart (SuperCoders) <an...@supercoders.com.au>:
> > I'd like to build a rather substantial application, it would be nice to
> > build it as a pure 2 tier CouchApp.
> >
> > How practical is this really? Does anyone have experience building
> anything
> > big in this way? Or are 2 tier CouchApps really only effective for small
> > stuff?
> >
> > I understand that it very much depends on what the app does, but I'm
> trying
> > to get a general sense of it.
> >
> > If pure 2 tier CouchDB apps is an impractical goal then we'll design the
> > architecture from the beginning as a 3 tier app and whack in some sort of
> > server.
> >
> > as
> >
>
Re: Building substantial "pure" 2 tier CouchApps - feasible?
Posted by Marcello Nuccio <ma...@gmail.com>.
I am building a pure couchapp to:
- insert the orders of the clients;
- insert the modifications requested by the clients;
- assign the work to the laboratories;
- track the warehouse.
- etc.
Not very big, but quite complex because I have many document types and
I must exchange data with lot of legacy systems.
It is an hard work if you are used to RDMS. The most challenging problems are:
- how to model the data to keep the couchapp simple, efficient, and
easily interoperable with other system?
- access control: in CouchDB is a brand new world. Everything you
already know on access control is almost useless with CouchDB.
- you will need to write some external utility to do some tasks, not
only for maintenance.
just my 2cents...
Marcello
2011/7/12 Andrew Stuart (SuperCoders) <an...@supercoders.com.au>:
> I'd like to build a rather substantial application, it would be nice to
> build it as a pure 2 tier CouchApp.
>
> How practical is this really? Does anyone have experience building anything
> big in this way? Or are 2 tier CouchApps really only effective for small
> stuff?
>
> I understand that it very much depends on what the app does, but I'm trying
> to get a general sense of it.
>
> If pure 2 tier CouchDB apps is an impractical goal then we'll design the
> architecture from the beginning as a 3 tier app and whack in some sort of
> server.
>
> as
>
Re: Building substantial "pure" 2 tier CouchApps - feasible?
Posted by Benoit Chesneau <bc...@gmail.com>.
I still think the couchapp framework miss some features. And I will
try to summarize some points. Side notes for usual people fast to jump
on conclusions, this isn't the couchapp trial at all. I quite love the
couchapp framework like it is. But today afetr 3 years of existence or
so itt's a little time to think more about it. So couchapp framework
miss some features:
1. Even if restish, it's not restfull . Most of the couchapp these
days are using a url fragments for the routing witch breaks the
meaning of URIS. Also it's isn't yet possible to fallback to the
google solution to make these uri searchable [1]. The rewriter helps
but isn't complete and once again isn't restful:
I can do :
GET /db/_design/dname/_show/fun
GET /db/_design/dname/_list/fun
(POST|PUT) /db/_design/dname/_update/fun
but I can't simply do (GET|PUT|POST|DELET|...) of )
/db/_design/dname/_app/somehandler
Of course I can play with the rewriter:
[
{
"from": "somehandler/*",
"to": "_show/showfun"
"method": "GET"
},
{
"from": "somehandler/*",
"to": "_update/updatefun"
"method": "PUT"
},
{
"from": "somehandler/*",
"to": "_update/updatefun"
"method": "POST"
}
]
but this isn't obviously the simplest solution.
2. The rewriter limits
While really powerful, the rewriter need some enhancements to fit all
needs. That something I investigate and I actually works on different
ways to achieve that.
3. Multiple calls on a page
Actually done by ajax, web workers, iframes (put your method) .
Something should be done to allows that. The obvious way to achieve
it would be to all funs to access to the couchdb api. Could be cool,
I'm also working on it (hence couchc [2]) and definitly useful to
extend couchdb api without having to rely on couchdb core. This
something appearing in redis with LUA that I find really useful. It
could aslo possible eventually to add a REST api to allows merging or
other fancy things you would like. Something fluidinfo [3] provides in
its service.
3) sessions
There is no secure way to maintain user infos across functions today.
4) Sandboxing
There are discussions today to allows write only db and such things.
Something that could be done on app level. What about sandboxing
couchapps like android apps are ?
5) About externals (os daemons)
That something really cool, but not really portable. Say I replicate a
couchapp that use these externals. Now my problem will have to install
this external. I've no clear idea to sove that point, maybe exposing
some systems calls like nodejs do ?
- benoit
[1] http://code.google.com/web/ajaxcrawling/docs/specification.html
[2] https://github.com/benoitc/couchc
[3] http://fluidinfo.com/
On Tue, Jul 12, 2011 at 10:00 AM, Benoit Chesneau <bc...@gmail.com> wrote:
> On Tue, Jul 12, 2011 at 4:06 AM, Jason Smith <jh...@iriscouch.com> wrote:
>> tl;dr summary: It is practical using external processes to check in
>> and keep things sane. This is very future-proof, but with a slightly
>> longer time-to-market.
>>
>> On Tue, Jul 12, 2011 at 6:18 AM, Andrew Stuart (SuperCoders)
>> <an...@supercoders.com.au> wrote:
>>> I'd like to build a rather substantial application, it would be nice to
>>> build it as a pure 2 tier CouchApp.
>>
>> Many have tried.
>>
>> It is completely impractical to build a pure 2-tier Couch app.
>> However, two tiers, plus external helpers ("2.1 tier") is very
>> practical.
>>
>> We've got some difficult days ahead. But it doesn't really matter with
>> me now. Because I've been to the mountaintop. I don't mind. Like
>> anybody, I would like to build a full-featured Couch app; scalability
>> has its place. But I'm not concerned about that now. I just want to do
>> God's will. And He's allowed me to go up to the mountain. And I've
>> looked over. And I've seen the Promised Land. I may not get there with
>> you. But I want you to know tonight, that we, as a people, will get to
>> the Promised Land. So I'm happy, tonight. I'm not worried about
>> anything. I'm not fearing any <span>.
>>
>> What I prefer is this: Your programmers develop on and deploy to the
>> Couch. Your users also interact directly with the Couch. However,
>> *external agents* also access the couch and perform administrative
>> tasks as needed.
>>
>> The benefits of two-tier apps are obvious; I will list a few:
>>
>> * Simpler
>> * Reduced sysadmin workload
>> * Excellent future-proofing: multi-datacenter, offline operation, run
>> from an iPhone, etc.
>>
>> Unfortunately, real-world apps are more than a pretty web page. To name a few:
>>
>> * Send emails or other messages (e.g. push notifications)
>> * Help users recover their password
>> * Interface with related services: boot EC2 instances, update a DNS
>> entry, generate an certificate
>> * Notice that somebody is spamming and do the needful
>> * Ops stuff: run backups, rotate logs, monitor services. Maybe that's
>> an external responsibility; but IMO nowadays that is an application's
>> self-hygeine
>> * Catch unexpected errors, reassure the user, alert the staff
>>
>> There is no alternative. You need a third component--admin stuff--to
>> build a comprehensive service.
>
> ANother path would be providing a way to code these features inside
> couch by providing needed API and the way to script it. Exactly like
> it was possible in good old hypercard time.
>
> - benoît
>
Re: Building substantial "pure" 2 tier CouchApps - feasible?
Posted by Benoit Chesneau <bc...@gmail.com>.
On Tue, Jul 12, 2011 at 4:06 AM, Jason Smith <jh...@iriscouch.com> wrote:
> tl;dr summary: It is practical using external processes to check in
> and keep things sane. This is very future-proof, but with a slightly
> longer time-to-market.
>
> On Tue, Jul 12, 2011 at 6:18 AM, Andrew Stuart (SuperCoders)
> <an...@supercoders.com.au> wrote:
>> I'd like to build a rather substantial application, it would be nice to
>> build it as a pure 2 tier CouchApp.
>
> Many have tried.
>
> It is completely impractical to build a pure 2-tier Couch app.
> However, two tiers, plus external helpers ("2.1 tier") is very
> practical.
>
> We've got some difficult days ahead. But it doesn't really matter with
> me now. Because I've been to the mountaintop. I don't mind. Like
> anybody, I would like to build a full-featured Couch app; scalability
> has its place. But I'm not concerned about that now. I just want to do
> God's will. And He's allowed me to go up to the mountain. And I've
> looked over. And I've seen the Promised Land. I may not get there with
> you. But I want you to know tonight, that we, as a people, will get to
> the Promised Land. So I'm happy, tonight. I'm not worried about
> anything. I'm not fearing any <span>.
>
> What I prefer is this: Your programmers develop on and deploy to the
> Couch. Your users also interact directly with the Couch. However,
> *external agents* also access the couch and perform administrative
> tasks as needed.
>
> The benefits of two-tier apps are obvious; I will list a few:
>
> * Simpler
> * Reduced sysadmin workload
> * Excellent future-proofing: multi-datacenter, offline operation, run
> from an iPhone, etc.
>
> Unfortunately, real-world apps are more than a pretty web page. To name a few:
>
> * Send emails or other messages (e.g. push notifications)
> * Help users recover their password
> * Interface with related services: boot EC2 instances, update a DNS
> entry, generate an certificate
> * Notice that somebody is spamming and do the needful
> * Ops stuff: run backups, rotate logs, monitor services. Maybe that's
> an external responsibility; but IMO nowadays that is an application's
> self-hygeine
> * Catch unexpected errors, reassure the user, alert the staff
>
> There is no alternative. You need a third component--admin stuff--to
> build a comprehensive service.
ANother path would be providing a way to code these features inside
couch by providing needed API and the way to script it. Exactly like
it was possible in good old hypercard time.
- benoît
Re: Building substantial "pure" 2 tier CouchApps - feasible?
Posted by Jason Smith <jh...@iriscouch.com>.
tl;dr summary: It is practical using external processes to check in
and keep things sane. This is very future-proof, but with a slightly
longer time-to-market.
On Tue, Jul 12, 2011 at 6:18 AM, Andrew Stuart (SuperCoders)
<an...@supercoders.com.au> wrote:
> I'd like to build a rather substantial application, it would be nice to
> build it as a pure 2 tier CouchApp.
Many have tried.
It is completely impractical to build a pure 2-tier Couch app.
However, two tiers, plus external helpers ("2.1 tier") is very
practical.
We've got some difficult days ahead. But it doesn't really matter with
me now. Because I've been to the mountaintop. I don't mind. Like
anybody, I would like to build a full-featured Couch app; scalability
has its place. But I'm not concerned about that now. I just want to do
God's will. And He's allowed me to go up to the mountain. And I've
looked over. And I've seen the Promised Land. I may not get there with
you. But I want you to know tonight, that we, as a people, will get to
the Promised Land. So I'm happy, tonight. I'm not worried about
anything. I'm not fearing any <span>.
What I prefer is this: Your programmers develop on and deploy to the
Couch. Your users also interact directly with the Couch. However,
*external agents* also access the couch and perform administrative
tasks as needed.
The benefits of two-tier apps are obvious; I will list a few:
* Simpler
* Reduced sysadmin workload
* Excellent future-proofing: multi-datacenter, offline operation, run
from an iPhone, etc.
Unfortunately, real-world apps are more than a pretty web page. To name a few:
* Send emails or other messages (e.g. push notifications)
* Help users recover their password
* Interface with related services: boot EC2 instances, update a DNS
entry, generate an certificate
* Notice that somebody is spamming and do the needful
* Ops stuff: run backups, rotate logs, monitor services. Maybe that's
an external responsibility; but IMO nowadays that is an application's
self-hygeine
* Catch unexpected errors, reassure the user, alert the staff
There is no alternative. You need a third component--admin stuff--to
build a comprehensive service. But should that component be a blockade
between your users and their data? Or should it work behind the
scenes? Finally, note that external agents needn't have the same
quality as middleware. An exception here or crash there is not
devastating to the user experience. (Perhaps emails are delayed, or a
backup is missed. That is better than 500 INTERNAL SERVER ERROR.)
I use NodeJS scripts to this end. For a few weeks, one such agent ran
from my laptop, over DSL! Hey, it's HTTPS to a Couch in EC2. What does
it matter? Similarly, our web site is compiled to static HTML by a
couchapp. That app only runs on Chrome/OSX. I've no idea why, nor do I
care. The "compiled" site supports all browsers and loads rather
quickly (for EC2).
--
Iris Couch