Couchbase Mobile to CouchDB on AWS - Looking for instance sizing advice.

[Douglas Turner <dc...@gmail.com>]

Hello
Let's see if I can articulate this sufficiently.

Some background: I am a one person shop, I wear all the hats and I have 
been learning everything as I go for the last 18-20 months, so I am 
still a bit of a noob.

I have an iOS app created in Titanium using Pegli's ti_couchbase module. 
If the user wants to enable syncing, they enter their requested 
GroupName, Password, eMail address.  The app reaches out to a php 
document that uses php-on-couchdb to check if the GroupName (database 
name) is available, if it is, that database is created with the proper 
authorization/password. This all works great and I am very pleased with it.

I am currently running everything into IrisCouch. I am considering 
changing over to AWS as IrisCouch seems a bit slow, plus last week they 
were down all morning one day. I have everything up and running on an 
AWS micro instance. (Couchdb 1.2). In fact I have two instances, Couch A 
and Couch B. Couch B is my backup server and has a cron job running a 
script to continuously replicate everything on A to B.

The current version of my app has about 3k users. If history is an 
indicator, when this syncing version is released, I expect about 250 
users (world wide) per day to update to the sync version. I expect 80% 
will enable syncing. The numbers I am anticipating are 200 people per 
day creating a database initially syncing 300-600 documents (2-4Meg per 
db). The Couchdb server is for replication only.

After the initial updates I anticipate an average of 15 users per day 
growth, databases created with less than 20 docs to start.

Using Cloudant is not an option at this time and I would rather get away 
from IrisCouch.

Will a Micro instance be sufficient or will I need to go larger?

Thank you for the help and advice!

Re: Couchbase Mobile to CouchDB on AWS - Looking for instance sizing advice.

[Gabriel Mancini <ga...@gmail.com>]

Hi Douglas,

   You can create a doc Json on _replicator base, and when  the couchdb
server restart he will survive...

[]s

On Mon, Jul 2, 2012 at 3:51 PM, Douglas Turner <dc...@gmail.com> wrote:

> Hi Octavian
> There are two halves to the cronjob. First is finding any new databases
> that have been created on Server A, creating them on Server B then starting
> replication. Second, send a replication continuous = true for existing
> databases.This covers server changes/moves, new databases, missing
> replication instances.
>
> My understanding is that I can send the replication request multiple times
> with no ill effects and continuous replication does not survive server
> reboots. Using _replicator has been problematic in my usage and I have yet
> to have create_target=true work, so I have opted to take the KISS approach.
> LOL, at least that is how I see it.
>
> Thanks!
> Doug
>
>
>
>
> On 7/2/12 11:15 AM, Octavian Damiean wrote:
>
>> Hello Douglas,
>>
>> Is there a special reason for using a cron job to continously sync
>> between the two remote databases? CouchDB offers continous replication
>> already.
>>
>> Cheers,
>> Octavian Damiean
>>
>> On Mon, Jul 2, 2012 at 7:28 PM, Douglas Turner<dc...@gmail.com>
>>  wrote:
>>
>>> Hello
>>> Let's see if I can articulate this sufficiently.
>>>
>>> Some background: I am a one person shop, I wear all the hats and I have
>>> been
>>> learning everything as I go for the last 18-20 months, so I am still a
>>> bit
>>> of a noob.
>>>
>>> I have an iOS app created in Titanium using Pegli's ti_couchbase module.
>>> If
>>> the user wants to enable syncing, they enter their requested GroupName,
>>> Password, eMail address.  The app reaches out to a php document that uses
>>> php-on-couchdb to check if the GroupName (database name) is available,
>>> if it
>>> is, that database is created with the proper authorization/password. This
>>> all works great and I am very pleased with it.
>>>
>>> I am currently running everything into IrisCouch. I am considering
>>> changing
>>> over to AWS as IrisCouch seems a bit slow, plus last week they were down
>>> all
>>> morning one day. I have everything up and running on an AWS micro
>>> instance.
>>> (Couchdb 1.2). In fact I have two instances, Couch A and Couch B. Couch
>>> B is
>>> my backup server and has a cron job running a script to continuously
>>> replicate everything on A to B.
>>>
>>> The current version of my app has about 3k users. If history is an
>>> indicator, when this syncing version is released, I expect about 250
>>> users
>>> (world wide) per day to update to the sync version. I expect 80% will
>>> enable
>>> syncing. The numbers I am anticipating are 200 people per day creating a
>>> database initially syncing 300-600 documents (2-4Meg per db). The Couchdb
>>> server is for replication only.
>>>
>>> After the initial updates I anticipate an average of 15 users per day
>>> growth, databases created with less than 20 docs to start.
>>>
>>> Using Cloudant is not an option at this time and I would rather get away
>>> from IrisCouch.
>>>
>>> Will a Micro instance be sufficient or will I need to go larger?
>>>
>>> Thank you for the help and advice!
>>>
>>


-- 
Gabriel Mancini de Campos
Arquiteto de Soluções

+55 (11) 9449-1706
gabriel.mancini@gmail.com
São Paulo - SP - Brasil

Re: Couchbase Mobile to CouchDB on AWS - Looking for instance sizing advice.

[Douglas Turner <dc...@gmail.com>]

Hi Octavian
There are two halves to the cronjob. First is finding any new databases 
that have been created on Server A, creating them on Server B then 
starting replication. Second, send a replication continuous = true for 
existing databases.This covers server changes/moves, new databases, 
missing replication instances.

My understanding is that I can send the replication request multiple 
times with no ill effects and continuous replication does not survive 
server reboots. Using _replicator has been problematic in my usage and I 
have yet to have create_target=true work, so I have opted to take the 
KISS approach. LOL, at least that is how I see it.

Thanks!
Doug



On 7/2/12 11:15 AM, Octavian Damiean wrote:
> Hello Douglas,
>
> Is there a special reason for using a cron job to continously sync
> between the two remote databases? CouchDB offers continous replication
> already.
>
> Cheers,
> Octavian Damiean
>
> On Mon, Jul 2, 2012 at 7:28 PM, Douglas Turner<dc...@gmail.com>  wrote:
>> Hello
>> Let's see if I can articulate this sufficiently.
>>
>> Some background: I am a one person shop, I wear all the hats and I have been
>> learning everything as I go for the last 18-20 months, so I am still a bit
>> of a noob.
>>
>> I have an iOS app created in Titanium using Pegli's ti_couchbase module. If
>> the user wants to enable syncing, they enter their requested GroupName,
>> Password, eMail address.  The app reaches out to a php document that uses
>> php-on-couchdb to check if the GroupName (database name) is available, if it
>> is, that database is created with the proper authorization/password. This
>> all works great and I am very pleased with it.
>>
>> I am currently running everything into IrisCouch. I am considering changing
>> over to AWS as IrisCouch seems a bit slow, plus last week they were down all
>> morning one day. I have everything up and running on an AWS micro instance.
>> (Couchdb 1.2). In fact I have two instances, Couch A and Couch B. Couch B is
>> my backup server and has a cron job running a script to continuously
>> replicate everything on A to B.
>>
>> The current version of my app has about 3k users. If history is an
>> indicator, when this syncing version is released, I expect about 250 users
>> (world wide) per day to update to the sync version. I expect 80% will enable
>> syncing. The numbers I am anticipating are 200 people per day creating a
>> database initially syncing 300-600 documents (2-4Meg per db). The Couchdb
>> server is for replication only.
>>
>> After the initial updates I anticipate an average of 15 users per day
>> growth, databases created with less than 20 docs to start.
>>
>> Using Cloudant is not an option at this time and I would rather get away
>> from IrisCouch.
>>
>> Will a Micro instance be sufficient or will I need to go larger?
>>
>> Thank you for the help and advice!

Re: Couchbase Mobile to CouchDB on AWS - Looking for instance sizing advice.

[Octavian Damiean <ma...@gmail.com>]

Hello Douglas,

Is there a special reason for using a cron job to continously sync
between the two remote databases? CouchDB offers continous replication
already.

Cheers,
Octavian Damiean

On Mon, Jul 2, 2012 at 7:28 PM, Douglas Turner <dc...@gmail.com> wrote:
> Hello
> Let's see if I can articulate this sufficiently.
>
> Some background: I am a one person shop, I wear all the hats and I have been
> learning everything as I go for the last 18-20 months, so I am still a bit
> of a noob.
>
> I have an iOS app created in Titanium using Pegli's ti_couchbase module. If
> the user wants to enable syncing, they enter their requested GroupName,
> Password, eMail address.  The app reaches out to a php document that uses
> php-on-couchdb to check if the GroupName (database name) is available, if it
> is, that database is created with the proper authorization/password. This
> all works great and I am very pleased with it.
>
> I am currently running everything into IrisCouch. I am considering changing
> over to AWS as IrisCouch seems a bit slow, plus last week they were down all
> morning one day. I have everything up and running on an AWS micro instance.
> (Couchdb 1.2). In fact I have two instances, Couch A and Couch B. Couch B is
> my backup server and has a cron job running a script to continuously
> replicate everything on A to B.
>
> The current version of my app has about 3k users. If history is an
> indicator, when this syncing version is released, I expect about 250 users
> (world wide) per day to update to the sync version. I expect 80% will enable
> syncing. The numbers I am anticipating are 200 people per day creating a
> database initially syncing 300-600 documents (2-4Meg per db). The Couchdb
> server is for replication only.
>
> After the initial updates I anticipate an average of 15 users per day
> growth, databases created with less than 20 docs to start.
>
> Using Cloudant is not an option at this time and I would rather get away
> from IrisCouch.
>
> Will a Micro instance be sufficient or will I need to go larger?
>
> Thank you for the help and advice!

Re: Couchbase Mobile to CouchDB on AWS - Looking for instance sizing advice.

[Keith Gable <zi...@ignition-project.com>]

I would use at least a m1.small, but if a service provider is too slow,
either they really suck, or you need much more infrastructure than they
have. I assume they have several servers, and they're probably better than
m1.small... so rolling your own here is probably not going to be cost
effective.

Also, as a person who spent quite a bit of time in Titanium... it's slow. I
would try benchmarking your app using ab and curl. This will help you model
traffic patterns quickly so that you know what needs improvement. I find
that a simple XHR request that takes 20-30ms in curl takes 250-300ms in
Titanium in the iOS simulator. Maybe it's because I'm sending a few
floating point numbers in the request, but I have no idea why it would make
it an order of magnitude slower (besides my default answer regarding
Titanium: Titanium sucks).
On Jul 2, 2012 4:00 PM, "Douglas Turner" <dc...@gmail.com> wrote:

> When testing I have noticed IrisCouch to be slower, but the micro EC2
> instance really isn't experiencing a real load either. Could be any number
> of factors such as PHP EC2 to Couch EC2 micro vs PHP EC2 to IrisCouch.
> Internal network vs external.
>
> A programmer I know with pretty extensive server experience thought a
> micro would be fine as it's just many small IOs, but his experience with
> AWS is minimal and CouchDB is zero. My gut disagrees, but what do I know?
>
> I am trying to sort through all of this, thus me asking for instance
> sizing advice. Thought I might get some feedback from the people with
> experience.
>
> -noob
>
>
> On 7/2/12 1:43 PM, Robert Newson wrote:
>
>> I'd love to hear if IrisCouch is slower an a micro EC2 instance, I'd be
>> astonished. Using a micro for anything serious, like a production database,
>> is "penny wise, pound foolish" imo.
>>
>> B.
>>
>> On 2 Jul 2012, at 18:28, Douglas Turner wrote:
>>
>>  Hello
>>> Let's see if I can articulate this sufficiently.
>>>
>>> Some background: I am a one person shop, I wear all the hats and I have
>>> been learning everything as I go for the last 18-20 months, so I am still a
>>> bit of a noob.
>>>
>>> I have an iOS app created in Titanium using Pegli's ti_couchbase module.
>>> If the user wants to enable syncing, they enter their requested GroupName,
>>> Password, eMail address.  The app reaches out to a php document that uses
>>> php-on-couchdb to check if the GroupName (database name) is available, if
>>> it is, that database is created with the proper authorization/password.
>>> This all works great and I am very pleased with it.
>>>
>>> I am currently running everything into IrisCouch. I am considering
>>> changing over to AWS as IrisCouch seems a bit slow, plus last week they
>>> were down all morning one day. I have everything up and running on an AWS
>>> micro instance. (Couchdb 1.2). In fact I have two instances, Couch A and
>>> Couch B. Couch B is my backup server and has a cron job running a script to
>>> continuously replicate everything on A to B.
>>>
>>> The current version of my app has about 3k users. If history is an
>>> indicator, when this syncing version is released, I expect about 250 users
>>> (world wide) per day to update to the sync version. I expect 80% will
>>> enable syncing. The numbers I am anticipating are 200 people per day
>>> creating a database initially syncing 300-600 documents (2-4Meg per db).
>>> The Couchdb server is for replication only.
>>>
>>> After the initial updates I anticipate an average of 15 users per day
>>> growth, databases created with less than 20 docs to start.
>>>
>>> Using Cloudant is not an option at this time and I would rather get away
>>> from IrisCouch.
>>>
>>> Will a Micro instance be sufficient or will I need to go larger?
>>>
>>> Thank you for the help and advice!
>>>
>>

Re: Couchbase Mobile to CouchDB on AWS - Looking for instance sizing advice.

[Douglas Turner <dc...@gmail.com>]

When testing I have noticed IrisCouch to be slower, but the micro EC2 
instance really isn't experiencing a real load either. Could be any 
number of factors such as PHP EC2 to Couch EC2 micro vs PHP EC2 to 
IrisCouch. Internal network vs external.

A programmer I know with pretty extensive server experience thought a 
micro would be fine as it's just many small IOs, but his experience with 
AWS is minimal and CouchDB is zero. My gut disagrees, but what do I know?

I am trying to sort through all of this, thus me asking for instance 
sizing advice. Thought I might get some feedback from the people with 
experience.

-noob


On 7/2/12 1:43 PM, Robert Newson wrote:
> I'd love to hear if IrisCouch is slower an a micro EC2 instance, I'd be astonished. Using a micro for anything serious, like a production database, is "penny wise, pound foolish" imo.
>
> B.
>
> On 2 Jul 2012, at 18:28, Douglas Turner wrote:
>
>> Hello
>> Let's see if I can articulate this sufficiently.
>>
>> Some background: I am a one person shop, I wear all the hats and I have been learning everything as I go for the last 18-20 months, so I am still a bit of a noob.
>>
>> I have an iOS app created in Titanium using Pegli's ti_couchbase module. If the user wants to enable syncing, they enter their requested GroupName, Password, eMail address.  The app reaches out to a php document that uses php-on-couchdb to check if the GroupName (database name) is available, if it is, that database is created with the proper authorization/password. This all works great and I am very pleased with it.
>>
>> I am currently running everything into IrisCouch. I am considering changing over to AWS as IrisCouch seems a bit slow, plus last week they were down all morning one day. I have everything up and running on an AWS micro instance. (Couchdb 1.2). In fact I have two instances, Couch A and Couch B. Couch B is my backup server and has a cron job running a script to continuously replicate everything on A to B.
>>
>> The current version of my app has about 3k users. If history is an indicator, when this syncing version is released, I expect about 250 users (world wide) per day to update to the sync version. I expect 80% will enable syncing. The numbers I am anticipating are 200 people per day creating a database initially syncing 300-600 documents (2-4Meg per db). The Couchdb server is for replication only.
>>
>> After the initial updates I anticipate an average of 15 users per day growth, databases created with less than 20 docs to start.
>>
>> Using Cloudant is not an option at this time and I would rather get away from IrisCouch.
>>
>> Will a Micro instance be sufficient or will I need to go larger?
>>
>> Thank you for the help and advice!

Re: Couchbase Mobile to CouchDB on AWS - Looking for instance sizing advice.

[Robert Newson <rn...@apache.org>]

I'd love to hear if IrisCouch is slower an a micro EC2 instance, I'd be astonished. Using a micro for anything serious, like a production database, is "penny wise, pound foolish" imo.

B.

On 2 Jul 2012, at 18:28, Douglas Turner wrote:

> Hello
> Let's see if I can articulate this sufficiently.
> 
> Some background: I am a one person shop, I wear all the hats and I have been learning everything as I go for the last 18-20 months, so I am still a bit of a noob.
> 
> I have an iOS app created in Titanium using Pegli's ti_couchbase module. If the user wants to enable syncing, they enter their requested GroupName, Password, eMail address.  The app reaches out to a php document that uses php-on-couchdb to check if the GroupName (database name) is available, if it is, that database is created with the proper authorization/password. This all works great and I am very pleased with it.
> 
> I am currently running everything into IrisCouch. I am considering changing over to AWS as IrisCouch seems a bit slow, plus last week they were down all morning one day. I have everything up and running on an AWS micro instance. (Couchdb 1.2). In fact I have two instances, Couch A and Couch B. Couch B is my backup server and has a cron job running a script to continuously replicate everything on A to B.
> 
> The current version of my app has about 3k users. If history is an indicator, when this syncing version is released, I expect about 250 users (world wide) per day to update to the sync version. I expect 80% will enable syncing. The numbers I am anticipating are 200 people per day creating a database initially syncing 300-600 documents (2-4Meg per db). The Couchdb server is for replication only.
> 
> After the initial updates I anticipate an average of 15 users per day growth, databases created with less than 20 docs to start.
> 
> Using Cloudant is not an option at this time and I would rather get away from IrisCouch.
> 
> Will a Micro instance be sufficient or will I need to go larger?
> 
> Thank you for the help and advice!