Help with setting a fake CouchDB replication endpoint

[Giovanni P <fi...@gmail.com>]

I'm trying to make a fake CouchDB replication endpoint so I can replicate
just one dynamically generated design document to an empty CouchDB database
(so, my fake endpoint wants to be only the source of the replication, never
the target).

I've followed the description of the protocol at
http://docs.couchdb.org/en/latest/replication/protocol.html and wrote views
for each of the paths the CouchDB replicator would have to call, in this
order:

HEAD / -> just says I'm here
GET / -> returns {update_seq: "1-initial", instance_start_time: "0"}
GET /_local/someidentificator -> 404
GET /_changes (I show only my design doc id, with a _rev of "1-initial")
GET /_design/<id>?revs=true&open_revs=all&latest=true

At this point I should return a bizarre multipart/mixed response, which I
do (you can check it by curling or visiting
http://couchapps.alhur.es/fiatjaf/microanalytics/_design/microanalytics?revs=true&open_revs=all&latest=true),
but at this point the CouchDB replicator starts to make repeated requests
to both / and /_design/<id>?revs=true&open_revs=all&latest=true and I don't
know what exactly went wrong.

Is it my multipart/mixed response that is wrong? (probably, because it is
written by hand, but where is it wrong and how should I be doing this?) or
is it something else? Cloudant DBs do not return this multipart/mixed
response no matter how much I try, they return a strange JSON array.

I know this is a bizarre question, but thanks for reading anyway.

Re: Help with setting a fake CouchDB replication endpoint

[Giovanni P <fi...@gmail.com>]

Thank you very much for looking at it, Alexander. I haven't had time to
(try to) fix my implementation, but I will.

On Sun, Feb 22, 2015 at 6:57 PM, Alexander Shorin <kx...@gmail.com> wrote:

> On Mon, Feb 23, 2015 at 12:53 AM, Alexander Shorin <kx...@gmail.com>
> wrote:
> >> Is it my multipart/mixed response that is wrong? (probably, because it
> is
> >> written by hand, but where is it wrong and how should I be doing this?)
> or
> >> is it something else?
> >
> > Yes, it is. For documents with attachments in multipart format,
> > document object MUST contains an _attachments field with the following
> > fields per attachment: content_type, length, but instead of data with
> > base64 encoded content, there should be "follows": true field be which
> > signs that the attachment content goes in next body parts. See
> >
> http://docs.couchdb.org/en/latest/replication/protocol.html#fetch-changed-documents
> > for an example.
>
> More issues:
> - content_type couldn't be an empty string
> - content_length must be integer
> - body field makes no sense, only data is
> - if you'd specified data one, no need to send the body part related to it
> - but if you uses multipart format, base64 encoded data makes no
> sense, so use follows: true and body parts with raw attachments data
>
> --
> ,,,^..^,,,
>

Re: Help with setting a fake CouchDB replication endpoint

[Alexander Shorin <kx...@gmail.com>]

On Mon, Feb 23, 2015 at 12:53 AM, Alexander Shorin <kx...@gmail.com> wrote:
>> Is it my multipart/mixed response that is wrong? (probably, because it is
>> written by hand, but where is it wrong and how should I be doing this?) or
>> is it something else?
>
> Yes, it is. For documents with attachments in multipart format,
> document object MUST contains an _attachments field with the following
> fields per attachment: content_type, length, but instead of data with
> base64 encoded content, there should be "follows": true field be which
> signs that the attachment content goes in next body parts. See
> http://docs.couchdb.org/en/latest/replication/protocol.html#fetch-changed-documents
> for an example.

More issues:
- content_type couldn't be an empty string
- content_length must be integer
- body field makes no sense, only data is
- if you'd specified data one, no need to send the body part related to it
- but if you uses multipart format, base64 encoded data makes no
sense, so use follows: true and body parts with raw attachments data

--
,,,^..^,,,

Re: Help with setting a fake CouchDB replication endpoint

[Alexander Shorin <kx...@gmail.com>]

Hi,

sorry, miss your email.

On Sun, Feb 8, 2015 at 12:35 AM, Giovanni P <fi...@gmail.com> wrote:
> HEAD / -> just says I'm here
> GET / -> returns {update_seq: "1-initial", instance_start_time: "0"}
> GET /_local/someidentificator -> 404
> GET /_changes (I show only my design doc id, with a _rev of "1-initial")
> GET /_design/<id>?revs=true&open_revs=all&latest=true

Actually, you'd miss /db/_revs_diff resource where you should
determine a missing revs list which will be passed as argument for
open_revs query param.
http://docs.couchdb.org/en/latest/replication/protocol.html#locate-changed-documents

> At this point I should return a bizarre multipart/mixed response, which I
> do (you can check it by curling or visiting
> http://couchapps.alhur.es/fiatjaf/microanalytics/_design/microanalytics?revs=true&open_revs=all&latest=true),
> but at this point the CouchDB replicator starts to make repeated requests
> to both / and /_design/<id>?revs=true&open_revs=all&latest=true and I don't
> know what exactly went wrong.
>
> Is it my multipart/mixed response that is wrong? (probably, because it is
> written by hand, but where is it wrong and how should I be doing this?) or
> is it something else?

Yes, it is. For documents with attachments in multipart format,
document object MUST contains an _attachments field with the following
fields per attachment: content_type, length, but instead of data with
base64 encoded content, there should be "follows": true field be which
signs that the attachment content goes in next body parts. See
http://docs.couchdb.org/en/latest/replication/protocol.html#fetch-changed-documents
for an example.

> Cloudant DBs do not return this multipart/mixed
> response no matter how much I try, they return a strange JSON array.

Their response is similar to CouchDB one if you explicitly accept
application/json. However, for handling attachments in this case you
need to additionally pass attachments=true query parameter and get
ready to deal with a huge json blob which could ruin your client.

--
,,,^..^,,,

Re: Help with setting a fake CouchDB replication endpoint

[Giovanni P <fi...@gmail.com>]

So, anyone has any idea about what might be happening?

On Sat, Feb 7, 2015 at 7:35 PM, Giovanni P <fi...@gmail.com> wrote:

> I'm trying to make a fake CouchDB replication endpoint so I can replicate
> just one dynamically generated design document to an empty CouchDB database
> (so, my fake endpoint wants to be only the source of the replication, never
> the target).
>
> I've followed the description of the protocol at
> http://docs.couchdb.org/en/latest/replication/protocol.html and wrote
> views for each of the paths the CouchDB replicator would have to call, in
> this order:
>
> HEAD / -> just says I'm here
> GET / -> returns {update_seq: "1-initial", instance_start_time: "0"}
> GET /_local/someidentificator -> 404
> GET /_changes (I show only my design doc id, with a _rev of "1-initial")
> GET /_design/<id>?revs=true&open_revs=all&latest=true
>
> At this point I should return a bizarre multipart/mixed response, which I
> do (you can check it by curling or visiting
> http://couchapps.alhur.es/fiatjaf/microanalytics/_design/microanalytics?revs=true&open_revs=all&latest=true),
> but at this point the CouchDB replicator starts to make repeated requests
> to both / and /_design/<id>?revs=true&open_revs=all&latest=true and I don't
> know what exactly went wrong.
>
> Is it my multipart/mixed response that is wrong? (probably, because it is
> written by hand, but where is it wrong and how should I be doing this?) or
> is it something else? Cloudant DBs do not return this multipart/mixed
> response no matter how much I try, they return a strange JSON array.
>
> I know this is a bizarre question, but thanks for reading anyway.
>