Some questions on large attachments

[Stefan Klein <st...@googlemail.com>]

Hi List,

When I upload a large (100-300MB in my case) file and the document is 
changed while I am uploading the file the upload will fail because the 
revision had changed. So I thought of having separate documents for the 
files and move them to the destination document after the upload is 
complete.

Is it possible to copy (or move) an attachment from one document to an 
other without downloading and re uploading the file?
if it's possible, will it also work between 2 databases on the same 
instance?

If the same file is attached to 2 or more documents, will couchdb hold 
one copy for each or is it possible to tell couchdb these are the same, 
you only have to store it once?

thanks,
Stefan

Re: Some questions on large attachments

[Daniel Truemper <tr...@googlemail.com>]

> Right now I added a field to the podcast doc which holds the mediadoc._id if
> this field is empty i have to create the mediadoc. Now i can check if the
> mediadoc exists and doesn't have an attachment ->  i have to download the
> file.
You could also write a view emitting all mediadocs that don't have an 
attachment. If you query the view with the mediadoc id and get an empty 
result, you have to download it.

Re: Some questions on large attachments

[Stefan Klein <st...@googlemail.com>]

Hi Daniel

2011/4/11 Daniel Truemper <tr...@googlemail.com>

> Hi Stefan,
>
>
[ ... ]


> Maybe you could store the attachment in a separate document from the
> original one and store a link to it?
>
>
For my use case it's i little more complicated to store the attachment(s) in
separate documents, but it'll work (at least for now).

I'm saving podcasts from an rss feed an will have to determine if my helper
script will have to download the file or if it's already there. If the media
file is attached to the pocast document i could just check if there is a url
in the doc an no attachment -> i have to download the file. With a separate
document for the file it's more complicated to determine if i have to
download anything.
Right now I added a field to the podcast doc which holds the mediadoc._id if
this field is empty i have to create the mediadoc. Now i can check if the
mediadoc exists and doesn't have an attachment -> i have to download the
file.


--
Stefan

Re: Some questions on large attachments

[Daniel Truemper <tr...@googlemail.com>]

Hi Stefan,

> Is it possible to copy (or move) an attachment from one document to an
> other without downloading and re uploading the file?
AFAIK, server side copying of attachments is not possible.

> If the same file is attached to 2 or more documents, will couchdb hold
> one copy for each or is it possible to tell couchdb these are the same,
> you only have to store it once?
The same as above, I don't think this is possible right now. It would 
require a lot of server side processing of the database if one 
attachment is equal to any of all other documents and would slow down 
attachment uploading significantly.

Maybe you could store the attachment in a separate document from the 
original one and store a link to it?

Best,
Daniel