Re: [DiSCUSS] Hosted RabbitMQ Server for Airavata

[Suresh Marru <sm...@apache.org>]

GSoC students,

Any volunteers to pick this thread? 

Currently Airavata uses Apache derby as a default database and for quick start tutorials and production usage, we recommend switching to MySQL. Similarly, Embedded Zookeeper is used as default and for production usage starting a zookeeper cluster is recommended. 

Similarly can we use Apache QPid as the embedded messaging implementation and suggest to use RabbitMQ for production usage? We can have this switch in the airavata-server.properties. 

Volunteers please,

Suresh


> On Nov 16, 2014, at 10:34 AM, Suresh Marru <sm...@apache.org> wrote:
> 
> Hi Lahiru,
> 
> If the theory matches practice (QPid and RabbitMQ are swappable), + 1 for this approach. This is analogies to how we use derby for all quick start and mysql for production use. 
> 
> Suresh
> 
> On Nov 15, 2014, at 3:26 PM, Lahiru Gunathilake <gl...@gmail.com> wrote:
> 
>> We can start a qpid server during startup and use Rabbitmq java clients as it is. When we do real production deployment we can deploy rabbitmq server and disable qpid during startup. There is no big difference between rabbitmq server and Qpid server when it comes to simple day to day usage.
>> 
>> Lahiru
>> 
>> On Thu, Nov 13, 2014 at 11:59 AM, Suresh Marru <sm...@apache.org> wrote:
>> Hi All,
>> 
>> As we get ready for the 0.14 release, one thing which always comes up is the installation of RabbitMQ. We addressed this for Zookeper by using embedded server. But thats not a good approach for RabbitMQ server since its in Erlang (and not in Java, unlike rest of Airavata) and forking of an external process on different operating systems will lead to unpredictable errors.
>> 
>> How about we mitigate this pointing the release build to a hosted service? Here are some pros and cons:
>> 
>> * This will alleviate the installation requirements and will go back to one click installation.
>> * Users will not have to worry about downloading and starting up RabbitMQ server. But can change it to local or other installations in properties file.
>> * If a user is trying to use Airavata without having the need for internet connectivity, then they have to have a local installation.
>> * There is a risk of the service being down and the release being pointed to a stale service. This can be mitigated by a persistent CName alias which points to a hosted server.
>> * There are popular rabbitmq hosted services [1], [2], [3] but are often expensive [4] for a community project.
>> * Few of Airavata active developers (along with me) are part of a download airavata project which runs airavata as a service. Within SciGaP project [1] we could run a persistent service like rabbitmq-service.scigap.org atleast for near future.
>> 
>> Given these tradeoff’s and options, any opinions?
>> 
>> Cheers,
>> Suresh
>> 
>> [1] - https://cloud.google.com/solutions/rabbitmq/
>> [2] - http://azure.microsoft.com/en-us/documentation/services/service-bus/ (which claims to interoperate with rabbitmq)
>> [3] - https://www.cloudamqp.com/
>> [4] - https://www.cloudamqp.com/plans.html
>> [5] - http://scigap.org/
>> 
>> 
>> 
>> -- 
>> Research Assistant 
>> Science Gateways Group
>> Indiana University
>