How would you implement bulk doc changes happening on their own?

[Metin Akat <ak...@gmail.com>]

I asked this on #couchdb but it seems like the middle of the night is
not the best time for having such discussions there :)

Imagine Farmville, although my use case is not exactly the same.
Players in Farmville plant some crops and they grow up after some
time. Let's say 5 hours.
If we want to implement something like this in couchdb, how would you
do it, in a crash proof way? It's something like a _changes listener,
but well, happening some (long) time after the change has happened.

Here we have some requirements:

1. Even if they don't grow at exactly 5 hours but say at 5 hours +-
several minutes, crops must not stop growing totally for the whole
server. If it happens, this will do major damage to in-game economy,
because the other parts of the economy may still function (like buying
and selling etc), prices of crops will go sky high in half an hour. If
crops stop growing, the whole game world must stop. It's OK if
something (pseudorandomly) misbehaves, but it's not OK if the whole
economy misbehaves.

2. Growing must happen asynchronously for the different players. It
must not be a cron job executed every hour. Maybe every minute could
be OK, but it would cause peaks of server load every minute and I
don't like the idea very much.

3. All state should (if possible) be stored in couchdb. I don't want
to rely on storing anything in the memory of my application. It must
be crash proof. This implies that if couchdb crashes and is back
online after one hour, that one hour must not have affected crop
growth at all. If I plant my crops at 1pm and the game world is
offline from 2pm to 3pm, my crops must be ready for harvesting at 7pm
instead of 6pm.


Of course, I have been thinking of all kinds of weird stuff like
writing a "server is fine" every other minute and then after crash
have a look at when was it OK. Same for crop growth (some monitor to
check if growth is working fine and stop the game world otherwise),
but well, everything so far is at best awkward and doesn't feel good.
I guess there may not be a good solution to my problem (at least not
with couchdb). If so, I'd like to know that :)

Re: How would you implement bulk doc changes happening on their own?

[Ian Hobson <ia...@ianhobson.co.uk>]

On 05/09/2010 03:38, Metin Akat wrote:
> I asked this on #couchdb but it seems like the middle of the night is
> not the best time for having such discussions there :)
>
I can imagine - a few sleepy people, not wanting to get in to a deep 
subject.:)

> Imagine Farmville, although my use case is not exactly the same.
> Players in Farmville plant some crops and they grow up after some
> time. Let's say 5 hours.
> If we want to implement something like this in couchdb, how would you
> do it, in a crash proof way? It's something like a _changes listener,
> but well, happening some (long) time after the change has happened.
>
Your requirements appear to be very close to those of simulation.

I would divide the activities up in to two classes: starts and finishes.

For example your crops start growing only when someone plants. They 
become ready 5 hours/months later and rot in the field unless the player 
has the resources to harvest them before 6 month/hours.

In a real simulation of something like a factory or a road network, your 
starts always require resources. If the resources are available the sim 
starts them. If not the work waits. Later when the processes finish, the 
finishing process release resources. A finish cannot require a resource, 
becasue it may not be available.

In your case you may not need to handle resources at all, because you 
are not waiting to start work, you are starting something when the 
player requests it.

However, you have a takt time (http://en.wikipedia.org/wiki/Takt_time) 
and you may need to progress various processes every clock tick.

I think you need a queue, in couchDB, of when things will happen in the 
future. The time is not real clock time, but game time, so if the server
crashes, all clocks stop.

As part of the click tick event, you check the queue and take all the 
actions queued for this time tick.

For the processes you add the processes to the list of tasks you have to 
do this time tick, until it is finished, and then remove it from the list.

When your player plants, you add two events to the queue. One in 5 
"months" to set the crop ripe, and one in 6 to set the crop rotten, and 
away it goes. When it ripens you mark it ripe. If it is not harvested, 
then the "is rotten" event marks it rotten.

If and when the player starts to harvest the crop, you set a process to 
harvest an amount every tick, until it is done or the crop becomes rotten.

There are many ways to handle the rotting. You can add it at the 
beginning, or when recording the crop as ripe. You can remove it from 
the queue when the crop is harvested, or let it fire and ignore it if 
the crop has gone. Letting it fire and ignoring it is usually more 
reliable.

If harvesting will take some time, and you don't want the overhead of 
acting each tick, you might start the harvest and queue up the end 
harvesting event. Then you must calculate the result of the harvest when 
the first of "harvest end" or "is rotten" events go off, and either 
remove the other or ignore it when it fires.

Regards

Ian

Re: How would you implement bulk doc changes happening on their own?

["Eli Stevens (Gmail)" <wi...@gmail.com>]

On Sat, Sep 4, 2010 at 7:38 PM, Metin Akat <ak...@gmail.com> wrote:
>This implies that if couchdb crashes and is back
> online after one hour, that one hour must not have affected crop
> growth at all.

This statement could be interpreted in two ways; either crop growth
wasn't affected (because the crops kept growing, in spite of server
instability), or crop growth wasn't affected (because game time
doesn't progress while the server is down).  Not sure which you mean.
:/

However, it would probably be best to not use wall-clock time in
either case.  I suggest that your periodic update to the DB write the
current game time (something like the number of seconds since the game
world was first initialized, either including server downtime or not,
as you prefer).  Crop ripening times would be expressed as a
seconds-since-first-init time, rather than anything to do with
wall-clock time (the only thing that wall-clock time would be used for
would be to inform the server of how much time should be added to game
time while the server was running).

A view could show all of the crops in state 'growing', ordered by
ripening time.

The server would start up, read the current game time (initialized to
zero), and begin looping.  The loop would update the current game
time, ripen any crops that are ready, handle any other tasks, and then
look at the growing list to find out how long to sleep for (with a
short max, so that the game time doesn't get too far behind if the
server is killed while sleeping).

Having a setup like this allows you to do fun things, like have
double-time holidays, where all crops ripen twice as quickly as
normal, etc.

HTH,
Eli

On Sat, Sep 4, 2010 at 7:38 PM, Metin Akat <ak...@gmail.com> wrote:
> I asked this on #couchdb but it seems like the middle of the night is
> not the best time for having such discussions there :)
>
> Imagine Farmville, although my use case is not exactly the same.
> Players in Farmville plant some crops and they grow up after some
> time. Let's say 5 hours.
> If we want to implement something like this in couchdb, how would you
> do it, in a crash proof way? It's something like a _changes listener,
> but well, happening some (long) time after the change has happened.
>
> Here we have some requirements:
>
> 1. Even if they don't grow at exactly 5 hours but say at 5 hours +-
> several minutes, crops must not stop growing totally for the whole
> server. If it happens, this will do major damage to in-game economy,
> because the other parts of the economy may still function (like buying
> and selling etc), prices of crops will go sky high in half an hour. If
> crops stop growing, the whole game world must stop. It's OK if
> something (pseudorandomly) misbehaves, but it's not OK if the whole
> economy misbehaves.
>
> 2. Growing must happen asynchronously for the different players. It
> must not be a cron job executed every hour. Maybe every minute could
> be OK, but it would cause peaks of server load every minute and I
> don't like the idea very much.
>
> 3. All state should (if possible) be stored in couchdb. I don't want
> to rely on storing anything in the memory of my application. It must
> be crash proof. This implies that if couchdb crashes and is back
> online after one hour, that one hour must not have affected crop
> growth at all. If I plant my crops at 1pm and the game world is
> offline from 2pm to 3pm, my crops must be ready for harvesting at 7pm
> instead of 6pm.
>
>
> Of course, I have been thinking of all kinds of weird stuff like
> writing a "server is fine" every other minute and then after crash
> have a look at when was it OK. Same for crop growth (some monitor to
> check if growth is working fine and stop the game world otherwise),
> but well, everything so far is at best awkward and doesn't feel good.
> I guess there may not be a good solution to my problem (at least not
> with couchdb). If so, I'd like to know that :)
>



-- 
Eli