DMS, _attachment or path_field?

[Anselmo Silva <an...@gmail.com>]

When Building a document management system, will you consider the  
binary _attachment (as couchdb current feature) or a path_field to a  
server/file_system (protected) enviroment?

its an ongoing question about saving binary into a DB ( even with  
couchDB ). From this we can raise some high-level architecture question:

- Will the binary _attachments affect the rebuild index views ( even  
with append ) ?
- How about replication?( I think it would be hard and intense when  
dealing with higher db size values. )

Thanks

Re: DMS, _attachment or path_field?

[anselmo silva <an...@gmail.com>]

Extremely explanatory! You left my mouth open....

Cheers...

(still open....)


On Tue, Sep 9, 2008 at 10:47 AM, Jan Lehnardt <ja...@apache.org> wrote:

>
> On Sep 9, 2008, at 11:14 , Anselmo Silva wrote:
>
>  When Building a document management system, will you consider the binary
>> _attachment (as couchdb current feature) or a path_field to a
>> server/file_system (protected) enviroment?
>>
>> its an ongoing question about saving binary into a DB ( even with couchDB
>> ). From this we can raise some high-level architecture question:
>>
>> - Will the binary _attachments affect the rebuild index views ( even with
>> append ) ?
>> - How about replication?( I think it would be hard and intense when
>> dealing with higher db size values. )
>>
>
> A couple of notes:
>
> If all your stuff is in the DB, managing said stuff becomes easier.
>
> Not using a database as a blob store is usually recommended because
> data needs to pass the border of user- and kernel-land a few times before
> being sent. The sendfile() syscall helps here, but Erlang developers say
> they don't see a measurable difference. So this looks like a non-issue
> in Erlang-land and hence CouchDB.
>
> If you keep your files external to CouchDB, you need to manage deletes and
> updates and everything.
>
> If you mix in replication, you need to manage replication as well. If that
> is
> easier or harder for you depends on your setup.
>
> Attachments have no impact on view index creation time.
>
> The more data is in a doc, the more resources you need to replicate said
> doc.
> It is also very convenient, see above. Fast, convenient, efficient: pick
> two.
>
> I don't think that is as big of an architectural question as it might
> sound. Start
> by building an app that works. If profiling shows that attachment
> replication is
> your bottleneck, think about solving that in a way that doesn't hurt you.
> If you
> opt for a more complex external solution now, no one can guarantee that
> this
> won't include a bottleneck when it comes to profiling. Ripping out
> attachments
> and do manual handling is not that big a deal (imho), c.f.
> DbUpdateNotifications.
>
> Cheers
> Jan
> --
>
>
>


-- 
Anselmo Silva

Re: DMS, _attachment or path_field?

[Jan Lehnardt <ja...@apache.org>]

On Sep 9, 2008, at 11:14 , Anselmo Silva wrote:

> When Building a document management system, will you consider the  
> binary _attachment (as couchdb current feature) or a path_field to a  
> server/file_system (protected) enviroment?
>
> its an ongoing question about saving binary into a DB ( even with  
> couchDB ). From this we can raise some high-level architecture  
> question:
>
> - Will the binary _attachments affect the rebuild index views ( even  
> with append ) ?
> - How about replication?( I think it would be hard and intense when  
> dealing with higher db size values. )

A couple of notes:

If all your stuff is in the DB, managing said stuff becomes easier.

Not using a database as a blob store is usually recommended because
data needs to pass the border of user- and kernel-land a few times  
before
being sent. The sendfile() syscall helps here, but Erlang developers say
they don't see a measurable difference. So this looks like a non-issue
in Erlang-land and hence CouchDB.

If you keep your files external to CouchDB, you need to manage deletes  
and
updates and everything.

If you mix in replication, you need to manage replication as well. If  
that is
easier or harder for you depends on your setup.

Attachments have no impact on view index creation time.

The more data is in a doc, the more resources you need to replicate  
said doc.
It is also very convenient, see above. Fast, convenient, efficient:  
pick two.

I don't think that is as big of an architectural question as it might  
sound. Start
by building an app that works. If profiling shows that attachment  
replication is
your bottleneck, think about solving that in a way that doesn't hurt  
you. If you
opt for a more complex external solution now, no one can guarantee  
that this
won't include a bottleneck when it comes to profiling. Ripping out  
attachments
and do manual handling is not that big a deal (imho), c.f.  
DbUpdateNotifications.

Cheers
Jan
--