Re: memcouchd...

[Tim King <tg...@gmail.com>]

Hi kowsik,

Thanks for sharing your code it looks pretty interesting.

I can think of a cool use for it outside of nodejs. I'm currently writing
some applications using Titanium Appcelerator http://www.appcelerator.com/.
I don't know if you know about it or not. It allows you to write
applications for iPhone/iPad/Android using Javascript which gets converted
into Objective-C or Java respectively and compiled into native applications.

One thing I'd really like to have is couchdb or a compatible API running on
these mobile devices which can be used for offline storage and syncing with
couchdb servers. I'm aware that it is possible to compile couch for Android
but after investigating running it on iOS I think this will be hard and it
would probably not be allowed by Apple anyway due to the Erlang VM.

Anyway I was thinking that using your memcouchd it would be possible to
create an embedded couch compatible db for use in Appclerator based apps.
Appcelerator has partial support for commonjs and I've got your code running
with a few small alterations. I haven't been able to get the test suite
running due to issues with some of the modules from nodejs but it should be
doable with a bit of hacking.

The only things missing that would make this really useful are the ability
to persist the documents on the device and replication. I think that the
documents could potentially be stored using the Properties API which stores
key value pairs. Unfortunately the support for manipulating binary files is
lacking in Appcelerator so I think implementing the couch file format would
be impossible without implementing it in Objective-c and Java as native
extensions which is possible but I'd rather avoid writing platform specific
code if possible.

What are you thoughts about using your code for this purpose? Have you got
any thoughts about serialising the store?

Thanks again for sharing the code!

Tim

On 30 January 2011 12:33, kowsik <ko...@gmail.com> wrote:

> Just something I that threw together in the last couple of days. YMMV
> and I have no idea what you would use this for! :-)
>
> https://github.com/pcapr/memcouchd
>
> K.
> ---
> http://twitter.com/pcapr
> http://labs.mudynamics.com
>

Re: memcouchd...

[kowsik <ko...@gmail.com>]

>> BTW, Just committed a partial btree implementation to memcouchd:
>>
>> https://github.com/pcapr/memcouchd/blob/master/lib/btree.js
>>
>
> You might have a look at nStore or node-dirty. Some work has already
> gone into making these efficient.
> As I recall from talking to Tim Caswell, the major limitation to
> nStore at some time was file size limitations due to js not handling
> big integers with 100% precision.
> It may be that a database size limit is an acceptable limitation.

Just looked at the code and based on a quick glance, both of them seem
to be key/value store's with no collation. For views to work in any
capacity, collation is a prerequisite (think startkey/endkey style of
iteration).

K.
---
http;//twitter.com/pcapr
http://labs.mudynamics.com

Re: memcouchd...

[Randall Leeds <ra...@gmail.com>]

On Tue, Feb 1, 2011 at 14:01, kowsik <ko...@gmail.com> wrote:
>> Anyway I was thinking that using your memcouchd it would be possible to
>> create an embedded couch compatible db for use in Appclerator based apps.
>> Appcelerator has partial support for commonjs and I've got your code running
>> with a few small alterations. I haven't been able to get the test suite
>> running due to issues with some of the modules from nodejs but it should be
>> doable with a bit of hacking.
>>
>> The only things missing that would make this really useful are the ability
>> to persist the documents on the device and replication. I think that the
>> documents could potentially be stored using the Properties API which stores
>> key value pairs. Unfortunately the support for manipulating binary files is
>> lacking in Appcelerator so I think implementing the couch file format would
>> be impossible without implementing it in Objective-c and Java as native
>> extensions which is possible but I'd rather avoid writing platform specific
>> code if possible.
>
> Persistence, replication and _changes are what makes Couch what it is.
> I wasn't thinking persistence at all for memcouchd, though at this
> point (for a 2 day old project), it's anybody's guess where it's going
> to go. If we can implement the _changes feed + replication without the
> persistence, I can see memcouchd being a pure transient store that can
> replicate at-will from the "real" couch.

All that is required in order to produce a semantically correct
replicator is an absolute ordering on updates like CouchDB's sequence
numbers.

>
> BTW, Just committed a partial btree implementation to memcouchd:
>
> https://github.com/pcapr/memcouchd/blob/master/lib/btree.js
>

You might have a look at nStore or node-dirty. Some work has already
gone into making these efficient.
As I recall from talking to Tim Caswell, the major limitation to
nStore at some time was file size limitations due to js not handling
big integers with 100% precision.
It may be that a database size limit is an acceptable limitation.

-Randall

Re: memcouchd...

[kowsik <ko...@gmail.com>]

> Anyway I was thinking that using your memcouchd it would be possible to
> create an embedded couch compatible db for use in Appclerator based apps.
> Appcelerator has partial support for commonjs and I've got your code running
> with a few small alterations. I haven't been able to get the test suite
> running due to issues with some of the modules from nodejs but it should be
> doable with a bit of hacking.
>
> The only things missing that would make this really useful are the ability
> to persist the documents on the device and replication. I think that the
> documents could potentially be stored using the Properties API which stores
> key value pairs. Unfortunately the support for manipulating binary files is
> lacking in Appcelerator so I think implementing the couch file format would
> be impossible without implementing it in Objective-c and Java as native
> extensions which is possible but I'd rather avoid writing platform specific
> code if possible.

Persistence, replication and _changes are what makes Couch what it is.
I wasn't thinking persistence at all for memcouchd, though at this
point (for a 2 day old project), it's anybody's guess where it's going
to go. If we can implement the _changes feed + replication without the
persistence, I can see memcouchd being a pure transient store that can
replicate at-will from the "real" couch.

BTW, Just committed a partial btree implementation to memcouchd:

https://github.com/pcapr/memcouchd/blob/master/lib/btree.js

Working on this during my free cycles, but with an order of 16, I just
saw 1 million [collated, obviously] inserts taking about 3 seconds on
my Mac. So one application I can see is pure number crunching for
in-memory, throw-away data with all the goodness of map/reduce.

K.
---
http://twitter.com/pcapr
http://labs.mudynamics.com/

Re: memcouchd...

[David Pratt <fa...@gmail.com>]

Hi Tim. I am in the same situation and this would be pretty cool. Can
you advise what was required to get it running in appcelerator. I
communicated with mikeal not so long ago on irc and couchdb for iphone
had already been submitted to apple for review. I am not sure what
they will do with it. Hoping apple will allow it. This development is
definitely interesting.

On Tue, Feb 1, 2011 at 5:15 PM, Tim King <tg...@gmail.com> wrote:
> Hi kowsik,
>
> Thanks for sharing your code it looks pretty interesting.
>
> I can think of a cool use for it outside of nodejs. I'm currently writing
> some applications using Titanium Appcelerator http://www.appcelerator.com/.
> I don't know if you know about it or not. It allows you to write
> applications for iPhone/iPad/Android using Javascript which gets converted
> into Objective-C or Java respectively and compiled into native applications.
>
> One thing I'd really like to have is couchdb or a compatible API running on
> these mobile devices which can be used for offline storage and syncing with
> couchdb servers. I'm aware that it is possible to compile couch for Android
> but after investigating running it on iOS I think this will be hard and it
> would probably not be allowed by Apple anyway due to the Erlang VM.
>
> Anyway I was thinking that using your memcouchd it would be possible to
> create an embedded couch compatible db for use in Appclerator based apps.
> Appcelerator has partial support for commonjs and I've got your code running
> with a few small alterations. I haven't been able to get the test suite
> running due to issues with some of the modules from nodejs but it should be
> doable with a bit of hacking.
>
> The only things missing that would make this really useful are the ability
> to persist the documents on the device and replication. I think that the
> documents could potentially be stored using the Properties API which stores
> key value pairs. Unfortunately the support for manipulating binary files is
> lacking in Appcelerator so I think implementing the couch file format would
> be impossible without implementing it in Objective-c and Java as native
> extensions which is possible but I'd rather avoid writing platform specific
> code if possible.
>
> What are you thoughts about using your code for this purpose? Have you got
> any thoughts about serialising the store?
>
> Thanks again for sharing the code!
>
> Tim
>
> On 30 January 2011 12:33, kowsik <ko...@gmail.com> wrote:
>
>> Just something I that threw together in the last couple of days. YMMV
>> and I have no idea what you would use this for! :-)
>>
>> https://github.com/pcapr/memcouchd
>>
>> K.
>> ---
>> http://twitter.com/pcapr
>> http://labs.mudynamics.com
>>
>