You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@couchdb.apache.org by David Mitchell <mo...@gmail.com> on 2009/04/21 13:16:37 UTC
Building Erlang app around CouchDB
Hello group,
I'm having trouble tracking down info on building Erlang applications
embedding CouchDB, as distinct from simply using CouchDB as a datastore.
I'm reasonably confident it's because I'm just not picking the right
keywords for Google, but any pointers would be highly appreciated.
Here's what I'm trying to achieve...
I want to be able to build an application around CouchDB that lets me load
in XML documents (i.e. converting XML to JSON within Erlang, rather than via
e.g. a Rails application talking to CouchDB). I also want to be able to
extract data from CouchDB as XML documents, again doing all the JSON<->XML
conversion within Erlang rather than a separate language. The whole thing
should work as a Web service (either REST or SOAP would be fine, although I
imagine REST should be easier as CouchDB itself has a REST interface).
The main reason for wanting to stay totally within Erlang is that we'll be
potentially loading in 100s of XML documents per second at peak times, and
it needs to be highly available. Eventual consistency is fine, so I can put
together a cluster of servers with a load balancer out the front, and have
them all replicate with each other. I know how to put together suitable
infrastructure under Erlang; what I don't know is how best to tackle it with
CouchDB in the mix.
Initial questions:
- is there a "best practice" way of sending HTTP PUTs containing XML data to
CouchDB, and have it convert the XML to JSON within CouchDB? For all I
know, it may be as trivial as calling some Javascript library... I'm
guessing I'm not the first to want to do this, so hopefully there's a "known
good" way of doing this
- can I build CouchDB views that output XML directly, or do I need to take
JSON out of the view and then convert it to XML "outside of" CouchDB?
- is there existing documentation on how to embed CouchDB within Erlang
applications, or is the only option to go through the source?
As I said, I suspect this info is already out there, but I can't find the
right combination of Google keywords to track it down.
Thanks in advance
David Mitchell
Re: Building Erlang app around CouchDB
Posted by Robert Dionne <di...@dionne-associates.com>.
I'm not quite sure what it is you are trying to do but it seems like
you don't need the ecouch in the middle if you're already using
MochiWeb.
Anyway I put together an erlang client for some testing[1] that is
implemented as a gen_server and uses either inets or ibrowse. Feel
free to push it along if it's useful to you. It evolved from another
[2], which has been discontinued.
Cheers,
Bob
[1] http://github.com/bdionne/erl-couch/tree/master
[2] http://code.google.com/p/erlcouch/
On Apr 22, 2009, at 4:51 PM, David Mitchell wrote:
> OK, I think I've got two of the key pieces in place:
> - I can use MochiWeb to construct a Web framework pretty easily
> (note to
> self: why haven't I bothered to look at Yaws alternatives before?)
> - I can use eCouch to have MochiWeb interface with CouchDB
>
> In other words,
> Web client --> (REST call) --> MochiWeb --> eCouch --> CouchDB
> and I haven't had to go outside of Erlang
>
> I put together a very quick couple of pages today with this setup
> and it
> worked fine.
>
> What I'm still not seeing is a way to do XML<->JSON conversions within
> Erlang. Are there libraries out there that do this, or do I have
> to resort
> to something like JavaScript (or try to come up with my own Erlang-
> based
> solution)?
>
> Thanks in advance
>
> Dave M.
>
>
>
> 2009/4/21 Noah Slater <ns...@apache.org>
>
>> On Tue, Apr 21, 2009 at 09:16:37PM +1000, David Mitchell wrote:
>>> I want to be able to build an application around CouchDB that
>>> lets me
>> load
>>> in XML documents (i.e. converting XML to JSON within Erlang,
>>> rather than
>> via
>>> e.g. a Rails application talking to CouchDB). I also want to be
>>> able to
>>> extract data from CouchDB as XML documents, again doing all the
>> JSON<->XML
>>> conversion within Erlang rather than a separate language. The whole
>> thing
>>> should work as a Web service (either REST or SOAP would be fine,
>>> although
>> I
>>> imagine REST should be easier as CouchDB itself has a REST
>>> interface).
>>
>> This sounds like a good opportunity to work with the grain of
>> CouchDB and
>> HTTP.
>>
>> By eschewing shared state, and enforcing a uniform interface, we
>> restrict
>> components from “seeing” beyond the context of an individual
>> request. This
>> introduces a “pipe and filter” style property to the system,
>> where access
>> or
>> functionality can be added, modified, or restricted transparently
>> between
>> client and server.
>>
>> Common features such as authentication, authorization, caching,
>> compression,
>> partitioning, proxying, tunneling, encryption or URI rewriting are
>> possible by
>> placing standard applications such as Apache httpd or nginx in
>> front of
>> your
>> server. You’re free to augment, adapt or limit your setup as you
>> see fit.
>> Figure 1-7 shows an example configuration of layered Web
>> architecture.
>>
>> Need some caching? Just throw in a cache between your server and
>> clients.
>> Need
>> some load balancing or authentication? Just throw in a reverse
>> proxy.
>>
>> - Layered System, CouchDB: TDG,
>> http://books.couchdb.org/relax/why-couchdb
>>
>> Need media type conversion? Throw in a proxy!
>>
>> Intermediary components act as both a client and a server in
>> order to
>> forward,
>> with possible translation, requests and responses. A proxy
>> component is an
>> intermediary selected by a client to provide interface
>> encapsulation of
>> other
>> services, data translation, performance enhancement, or security
>> protection.
>>
>> -
>> http://www.ics.uci.edu/~fielding/pubs/dissertation/
>> rest_arch_style.htm<http://www.ics.uci.edu/%7Efielding/pubs/
>> dissertation/rest_arch_style.htm>
>>
>> See Figure 5-10, ibid.
>>
>> Best,
>>
>> --
>> Noah Slater, http://tumbolia.org/nslater
>>
Re: Building Erlang app around CouchDB
Posted by David Mitchell <mo...@gmail.com>.
OK, I think I've got two of the key pieces in place:
- I can use MochiWeb to construct a Web framework pretty easily (note to
self: why haven't I bothered to look at Yaws alternatives before?)
- I can use eCouch to have MochiWeb interface with CouchDB
In other words,
Web client --> (REST call) --> MochiWeb --> eCouch --> CouchDB
and I haven't had to go outside of Erlang
I put together a very quick couple of pages today with this setup and it
worked fine.
What I'm still not seeing is a way to do XML<->JSON conversions within
Erlang. Are there libraries out there that do this, or do I have to resort
to something like JavaScript (or try to come up with my own Erlang-based
solution)?
Thanks in advance
Dave M.
2009/4/21 Noah Slater <ns...@apache.org>
> On Tue, Apr 21, 2009 at 09:16:37PM +1000, David Mitchell wrote:
> > I want to be able to build an application around CouchDB that lets me
> load
> > in XML documents (i.e. converting XML to JSON within Erlang, rather than
> via
> > e.g. a Rails application talking to CouchDB). I also want to be able to
> > extract data from CouchDB as XML documents, again doing all the
> JSON<->XML
> > conversion within Erlang rather than a separate language. The whole
> thing
> > should work as a Web service (either REST or SOAP would be fine, although
> I
> > imagine REST should be easier as CouchDB itself has a REST interface).
>
> This sounds like a good opportunity to work with the grain of CouchDB and
> HTTP.
>
> By eschewing shared state, and enforcing a uniform interface, we restrict
> components from “seeing” beyond the context of an individual request. This
> introduces a “pipe and filter” style property to the system, where access
> or
> functionality can be added, modified, or restricted transparently between
> client and server.
>
> Common features such as authentication, authorization, caching,
> compression,
> partitioning, proxying, tunneling, encryption or URI rewriting are
> possible by
> placing standard applications such as Apache httpd or nginx in front of
> your
> server. You’re free to augment, adapt or limit your setup as you see fit.
> Figure 1-7 shows an example configuration of layered Web architecture.
>
> Need some caching? Just throw in a cache between your server and clients.
> Need
> some load balancing or authentication? Just throw in a reverse proxy.
>
> - Layered System, CouchDB: TDG,
> http://books.couchdb.org/relax/why-couchdb
>
> Need media type conversion? Throw in a proxy!
>
> Intermediary components act as both a client and a server in order to
> forward,
> with possible translation, requests and responses. A proxy component is an
> intermediary selected by a client to provide interface encapsulation of
> other
> services, data translation, performance enhancement, or security
> protection.
>
> -
> http://www.ics.uci.edu/~fielding/pubs/dissertation/rest_arch_style.htm<http://www.ics.uci.edu/%7Efielding/pubs/dissertation/rest_arch_style.htm>
>
> See Figure 5-10, ibid.
>
> Best,
>
> --
> Noah Slater, http://tumbolia.org/nslater
>
Re: Building Erlang app around CouchDB
Posted by Noah Slater <ns...@apache.org>.
On Tue, Apr 21, 2009 at 09:16:37PM +1000, David Mitchell wrote:
> I want to be able to build an application around CouchDB that lets me load
> in XML documents (i.e. converting XML to JSON within Erlang, rather than via
> e.g. a Rails application talking to CouchDB). I also want to be able to
> extract data from CouchDB as XML documents, again doing all the JSON<->XML
> conversion within Erlang rather than a separate language. The whole thing
> should work as a Web service (either REST or SOAP would be fine, although I
> imagine REST should be easier as CouchDB itself has a REST interface).
This sounds like a good opportunity to work with the grain of CouchDB and HTTP.
By eschewing shared state, and enforcing a uniform interface, we restrict
components from “seeing” beyond the context of an individual request. This
introduces a “pipe and filter” style property to the system, where access or
functionality can be added, modified, or restricted transparently between
client and server.
Common features such as authentication, authorization, caching, compression,
partitioning, proxying, tunneling, encryption or URI rewriting are possible by
placing standard applications such as Apache httpd or nginx in front of your
server. You’re free to augment, adapt or limit your setup as you see fit.
Figure 1-7 shows an example configuration of layered Web architecture.
Need some caching? Just throw in a cache between your server and clients. Need
some load balancing or authentication? Just throw in a reverse proxy.
- Layered System, CouchDB: TDG, http://books.couchdb.org/relax/why-couchdb
Need media type conversion? Throw in a proxy!
Intermediary components act as both a client and a server in order to forward,
with possible translation, requests and responses. A proxy component is an
intermediary selected by a client to provide interface encapsulation of other
services, data translation, performance enhancement, or security protection.
- http://www.ics.uci.edu/~fielding/pubs/dissertation/rest_arch_style.htm
See Figure 5-10, ibid.
Best,
--
Noah Slater, http://tumbolia.org/nslater