You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@couchdb.apache.org by Jon Brisbin <jo...@jbrisbin.com> on 2010/08/30 15:15:22 UTC
ANN: CouchDB access via AMQP
I've been working on a plugin for the RabbitMQ AMQP broker that relays messages from the broker to HTTP REST APIs. I call it the "webhooks" plugin.
I've been testing against CouchDB 1.0.1 and I'm able to bulk load/update/delete documents as fast as the server will take them (which is pretty fast ;). I plan to have responses sent back to AMQP reply queues, which means one could have fully-asynchronous access to CouchDB via AMQP consumers/producers. Right now I'm using it to asynchronously update my CouchDB server (comet will come into play here) but I suspect I'll add the ability to see the server response by implementing a "ReplyTo" feature.
It's written in Erlang and uses the lhttpc module for HTTP functions. It runs directly inside the broker as a plugin. If anyone's interested in helping me with this, I'd love to get some feedback/patches that make the CouchDB access easier and more robust. I've also got some so more work to do to make it more configurable.
Like the RabbitMQ source and other plugins (from which I drew significant inspiration), this alpha plugin is MPL-licensed and available on Github:
http://github.com/jbrisbin/rabbitmq-webhooks
Normal caveats apply: this is still experimental, mostly untested code. It will likely crash your broker if a bad error happens (URL is not responsive, bad reply codes, etc...). I'm still working on making it more bullet-proof.
Thanks!
J. Brisbin
http://jbrisbin.com/
Re: ANN: CouchDB access via AMQP
Posted by Russell Branca <ch...@gmail.com>.
Sounds good. I'll let you know any issues I run into.
-Russell
On Wed, Sep 1, 2010 at 11:37 AM, Jon Brisbin <jo...@jbrisbin.com> wrote:
>
> On Sep 1, 2010, at 1:20 PM, Russell Branca wrote:
>
>> Very interesting. Heroku is soon offering an AMQP solution. I'm not
>> familiar with using plugins with AMQP, I wonder if it would be
>> possible to integrate that with heroku? I'm intrigued by the idea of
>> being able to kick off requests from a local heroku instance into
>> AMQP, and then forward them off to a remote CouchDB instance. Been
>> looking at options for doing that. I just started watching your github
>> repo. I'll play around with it this week if I get a chance. Keep us
>> posted, looks very cool!
>>
>
> I'm not sure what the integration between RabbitMQ and Heroku apps is going to look like. I'm not sure what kind of control you'll have over the broker itself. I'm assuming there will be shared brokers, or at least some way to provision a broker for your application's use. I'll get with the RabbitMQ folks and see if I can figure something out.
>
> Please let me know if you encounter issues. I'll be adding some more configuration ability shortly, as well as make the HTTP sending code more robust in case of errors. I've been testing new functionality against CouchDB as the HTTP endpoint, so it should perform the basic operations.
>
> I was having some trouble today with the interaction between the plugin and the 2.0.0 release version of the broker. Seems there are some incompatibilities. Please let me know on success/failure as I'll be looking at it too.
>
> jb
>
>
>>
>> -Russell
>>
>> On Mon, Aug 30, 2010 at 6:15 AM, Jon Brisbin <jo...@jbrisbin.com> wrote:
>>> I've been working on a plugin for the RabbitMQ AMQP broker that relays messages from the broker to HTTP REST APIs. I call it the "webhooks" plugin.
>>>
>>> I've been testing against CouchDB 1.0.1 and I'm able to bulk load/update/delete documents as fast as the server will take them (which is pretty fast ;). I plan to have responses sent back to AMQP reply queues, which means one could have fully-asynchronous access to CouchDB via AMQP consumers/producers. Right now I'm using it to asynchronously update my CouchDB server (comet will come into play here) but I suspect I'll add the ability to see the server response by implementing a "ReplyTo" feature.
>>>
>>> It's written in Erlang and uses the lhttpc module for HTTP functions. It runs directly inside the broker as a plugin. If anyone's interested in helping me with this, I'd love to get some feedback/patches that make the CouchDB access easier and more robust. I've also got some so more work to do to make it more configurable.
>>>
>>> Like the RabbitMQ source and other plugins (from which I drew significant inspiration), this alpha plugin is MPL-licensed and available on Github:
>>>
>>> http://github.com/jbrisbin/rabbitmq-webhooks
>>>
>>> Normal caveats apply: this is still experimental, mostly untested code. It will likely crash your broker if a bad error happens (URL is not responsive, bad reply codes, etc...). I'm still working on making it more bullet-proof.
>>>
>>> Thanks!
>>>
>>> J. Brisbin
>>> http://jbrisbin.com/
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>
>
> Thanks!
>
> J. Brisbin
> http://jbrisbin.com/
>
>
>
>
>
>
>
Re: ANN: CouchDB access via AMQP
Posted by Jon Brisbin <jo...@jbrisbin.com>.
On Sep 1, 2010, at 1:20 PM, Russell Branca wrote:
> Very interesting. Heroku is soon offering an AMQP solution. I'm not
> familiar with using plugins with AMQP, I wonder if it would be
> possible to integrate that with heroku? I'm intrigued by the idea of
> being able to kick off requests from a local heroku instance into
> AMQP, and then forward them off to a remote CouchDB instance. Been
> looking at options for doing that. I just started watching your github
> repo. I'll play around with it this week if I get a chance. Keep us
> posted, looks very cool!
>
I'm not sure what the integration between RabbitMQ and Heroku apps is going to look like. I'm not sure what kind of control you'll have over the broker itself. I'm assuming there will be shared brokers, or at least some way to provision a broker for your application's use. I'll get with the RabbitMQ folks and see if I can figure something out.
Please let me know if you encounter issues. I'll be adding some more configuration ability shortly, as well as make the HTTP sending code more robust in case of errors. I've been testing new functionality against CouchDB as the HTTP endpoint, so it should perform the basic operations.
I was having some trouble today with the interaction between the plugin and the 2.0.0 release version of the broker. Seems there are some incompatibilities. Please let me know on success/failure as I'll be looking at it too.
jb
>
> -Russell
>
> On Mon, Aug 30, 2010 at 6:15 AM, Jon Brisbin <jo...@jbrisbin.com> wrote:
>> I've been working on a plugin for the RabbitMQ AMQP broker that relays messages from the broker to HTTP REST APIs. I call it the "webhooks" plugin.
>>
>> I've been testing against CouchDB 1.0.1 and I'm able to bulk load/update/delete documents as fast as the server will take them (which is pretty fast ;). I plan to have responses sent back to AMQP reply queues, which means one could have fully-asynchronous access to CouchDB via AMQP consumers/producers. Right now I'm using it to asynchronously update my CouchDB server (comet will come into play here) but I suspect I'll add the ability to see the server response by implementing a "ReplyTo" feature.
>>
>> It's written in Erlang and uses the lhttpc module for HTTP functions. It runs directly inside the broker as a plugin. If anyone's interested in helping me with this, I'd love to get some feedback/patches that make the CouchDB access easier and more robust. I've also got some so more work to do to make it more configurable.
>>
>> Like the RabbitMQ source and other plugins (from which I drew significant inspiration), this alpha plugin is MPL-licensed and available on Github:
>>
>> http://github.com/jbrisbin/rabbitmq-webhooks
>>
>> Normal caveats apply: this is still experimental, mostly untested code. It will likely crash your broker if a bad error happens (URL is not responsive, bad reply codes, etc...). I'm still working on making it more bullet-proof.
>>
>> Thanks!
>>
>> J. Brisbin
>> http://jbrisbin.com/
>>
>>
>>
>>
>>
>>
>>
Thanks!
J. Brisbin
http://jbrisbin.com/
Re: ANN: CouchDB access via AMQP
Posted by Russell Branca <ch...@gmail.com>.
Very interesting. Heroku is soon offering an AMQP solution. I'm not
familiar with using plugins with AMQP, I wonder if it would be
possible to integrate that with heroku? I'm intrigued by the idea of
being able to kick off requests from a local heroku instance into
AMQP, and then forward them off to a remote CouchDB instance. Been
looking at options for doing that. I just started watching your github
repo. I'll play around with it this week if I get a chance. Keep us
posted, looks very cool!
-Russell
On Mon, Aug 30, 2010 at 6:15 AM, Jon Brisbin <jo...@jbrisbin.com> wrote:
> I've been working on a plugin for the RabbitMQ AMQP broker that relays messages from the broker to HTTP REST APIs. I call it the "webhooks" plugin.
>
> I've been testing against CouchDB 1.0.1 and I'm able to bulk load/update/delete documents as fast as the server will take them (which is pretty fast ;). I plan to have responses sent back to AMQP reply queues, which means one could have fully-asynchronous access to CouchDB via AMQP consumers/producers. Right now I'm using it to asynchronously update my CouchDB server (comet will come into play here) but I suspect I'll add the ability to see the server response by implementing a "ReplyTo" feature.
>
> It's written in Erlang and uses the lhttpc module for HTTP functions. It runs directly inside the broker as a plugin. If anyone's interested in helping me with this, I'd love to get some feedback/patches that make the CouchDB access easier and more robust. I've also got some so more work to do to make it more configurable.
>
> Like the RabbitMQ source and other plugins (from which I drew significant inspiration), this alpha plugin is MPL-licensed and available on Github:
>
> http://github.com/jbrisbin/rabbitmq-webhooks
>
> Normal caveats apply: this is still experimental, mostly untested code. It will likely crash your broker if a bad error happens (URL is not responsive, bad reply codes, etc...). I'm still working on making it more bullet-proof.
>
> Thanks!
>
> J. Brisbin
> http://jbrisbin.com/
>
>
>
>
>
>
>