You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@couchdb.apache.org by "Rutherford, Tim - Exelis" <Ti...@exelisinc.com> on 2015/06/16 18:46:19 UTC
HTTPS Reverse Proxy Replication
I have been struggling for a while now to get couchdb continuous replication to work with a remote database behind an https reverse proxy. I'm not sure if I am missing something, but as far as I can tell, it should work and just isn't.
My setup has a windows machine running couchdb (version 1.5.1) on the default port 5984 and a remote couchdb (version 1.5.1) behind apache httpd using HTTPS.
Apache httpd is configured with: ProxyPass /futon http://localhost:5984
I added a replication document to _replicator that looks like this:
{
"source": {
"url": "https://[remote ip]/futon/iss-local<https://[remote%20ip]/futon/iss-local>",
"headers": {
"Authorization": "Basic [valid base64 user:pass]"
}
},
"target": "iss-remote",
"create_target": true,
"continuous": true,
"user_ctx": {
"roles": ["_admin"]
}
}
The replication document is successfully added and replication_state is set to triggered.
The status page shows the Replication task and the status column shows "Checkpointed source sequence 0, current source sequence 20416, progress 0%".
It never progresses from there. iss-remote is created successfully, but no documents are added. Checking CouchDB's log file shows a series of error messages like this:
[Thu, 28 May 2015 22:04:46 GMT] [error] [<0.22546.4>] Replicator, request GET to "https://[remote ip]/futon/iss-local/1_33333?revs=true&open_revs=%5B%22615-738587c5fb3aa93f573fa5a79d3cf3ec%22%5D&latest=true<https://[remote%20ip]/futon/iss-local/1_33333?revs=true&open_revs=%5B%22615-738587c5fb3aa93f573fa5a79d3cf3ec%22%5D&latest=true>" failed due to error timeout
[Thu, 28 May 2015 22:04:46 GMT] [info] [<0.22544.4>] Retrying GET to https://[remote ip]/futon/iss-local/1_33333?revs=true&open_revs=%5B%22615-738587c5fb3aa93f573fa5a79d3cf3ec%22%5D&latest=true in 1.0 seconds due to error timeout
Couch then waits a bit and tries again and I get the same error message.
Which doesn't make sense to me because that URL works correctly using multiple tools (web browser, curl, and even using an httpc:request() in erlang) from the same machine.
If I expose port 80 on the remote server and use http instead everything works as expected. Documents are replicated and progress goes up to 100%.
As far as I can tell, this is an issue with couchdb when talking to the reverse proxy. I tried exposing couchdb's https port 6984 using the same certificate and it worked fine.
I have installed a new version of apache and only configured the proxypass and ssl and I still get the same issue.
I ran wireshark (with ssl decryption using the certificate from the server) on the machine that I am replicating to and I don't see any issues. I see the GET request from the error log and a corresponding 200 response with data. A little while later, the error message shows up in the log.
Has anyone seen this before? Is there anything I can do to fix it?
Thank you,
Tim Rutherford
________________________________
This e-mail and any files transmitted with it may be proprietary and are intended solely for the use of the individual or entity to whom they are addressed. If you have received this e-mail in error please notify the sender. Please note that any views or opinions presented in this e-mail are solely those of the author and do not necessarily represent those of Exelis Inc. The recipient should check this e-mail and any attachments for the presence of viruses. Exelis Inc. accepts no liability for any damage caused by any virus transmitted by this e-mail.
Re: HTTPS Reverse Proxy Replication
Posted by Franck Eyraud <fr...@nospam.yrnm.net>.
Good morning,
I know this is a very old thread, but it doesn't seem solve, and I met
this very same issue while trying to have replication work through a
Apache HTTPS reverse proxy. All CouchDB versions (up to 1.6.1) tested
failed in replicating.
After several tries to solve this (use of Apache is mandatory in the
infrastructure) I came to use the MSIE hacks to fix some SSL+KeepAlive
issues, that Apache proposes in default ssl configuration. The
combination that works the best for both Apache 2.2 and 2.4 :
> BrowserMatch "CouchDB" ssl-unclean-shutdown downgrade-1.0
This still leaves some "request timed out" or "connection closing" error
messages on the client log, but the replication goes on correctly till
the end.
Is CouchDB client known to have a problem with SSL and KeepAlive ?
Is it possible to find a more accurate setting in Apache (or CouchDB) ?
Is it necessary to file a bug for this ?
Regards,
Franck
On 22 June 2015 at 17:48, Rutherford, Tim - Exelis
<Tim.Rutherford@exelisinc.com <ma...@exelisinc.com>> wrote:
Thanks for the suggestion, but I have been playing with the settings
and so far I have been unsuccessful.
None of the settings I have set seem to help. The X-Forwarded-Proto,
and x_forwarded_ssl settings don't seem to make a difference. I also
played with the keepalive settings and they seem to be working, but
couch is still failing.
I did dig into the wireshark logs a little more and I notice
something. The request that couchdb is reporting a timeout on seems
to return successfully, but the socket seems to be reused for the
next request. The next request seems to be getting a tcp reset
without a response from the server. The next request in my setup is
usually to the root of the server, but I have seen it be other
requests as well.
Thank you,
Tim Rutherford
-----Original Message-----
From: Daniel Holth [mailto:dholth@gmail.com <ma...@gmail.com>]
Sent: Wednesday, June 17, 2015 11:29 AM
To: user@couchdb.apache.org <ma...@couchdb.apache.org>
Subject: Re: HTTPS Reverse Proxy Replication
Check X-Forwarded-Proto
http://docs.couchdb.org/en/latest/config/http.html#httpd/x_forwarded_proto
On Wed, Jun 17, 2015 at 1:09 PM Rutherford, Tim - Exelis <
Tim.Rutherford@exelisinc.com <ma...@exelisinc.com>>
wrote:
> Thank you for the example. Unfortunately, I don't really have the
> option to switch to nginx. I can modify the Apache httpd
configuration
> some, but there are other components that need httpd.
>
> I don't know enough about ProxyPass or nginx to be sure if my config
> does everything yours does. But I have tried copying a pasting the
> config from the page linked below without success on a fresh
Apache install.
>
> I was hoping someone could verify that they had Apache httpd and
couch
> working with https.
>
> I tried following this wiki page
> https://wiki.apache.org/couchdb/Apache_As_a_Reverse_Proxy and I can
> still only get http working. Https still fails.
>
> The strange part to me is that wireshark follows the request and
> response all the way back to couch. From what I can tell, Apache is
> properly serving the requests. Couchdb is reporting a timeout in the
> log even though I see the data being returned from apache.
>
> Tim Rutherford
>
>
> -----Original Message-----
> From: Daniel Holth [mailto:dholth@gmail.com
<ma...@gmail.com>]
> Sent: Tuesday, June 16, 2015 2:48 PM
> To: user@couchdb.apache.org <ma...@couchdb.apache.org>
> Subject: Re: HTTPS Reverse Proxy Replication
>
> I spent a long time struggling with my reverse proxy configuration.
> Here's the relevant part of my nginx config. We rewrite requests to
> CouchDB so it does not see the prefix. Critically we also use
> proxy_redirect to rewrite redirects issued by CouchDB so that they
> point at the correct location "/db" instead of "127.0.0.1:5984
<http://127.0.0.1:5984>".
>
> location /db {
>
> rewrite /db/(.*) /$1 break;
>
> proxy_set_header X-Forwarded-For $remote_addr; proxy_set_header Host
> $proxy_host;
>
> proxy_pass http://127.0.0.1:5984;
> proxy_redirect https://127.0.0.1:5984 https://$host/db;
>
> }
>
>
> On Tue, Jun 16, 2015 at 12:46 PM, Rutherford, Tim - Exelis
> <Tim.Rutherford@exelisinc.com
<ma...@exelisinc.com>> wrote:
> > I have been struggling for a while now to get couchdb continuous
> replication to work with a remote database behind an https
reverse proxy.
> I'm not sure if I am missing something, but as far as I can tell, it
> should work and just isn't.
> >
> > My setup has a windows machine running couchdb (version 1.5.1) on
> > the
> default port 5984 and a remote couchdb (version 1.5.1) behind apache
> httpd using HTTPS.
> >
> > Apache httpd is configured with: ProxyPass /futon
> > http://localhost:5984
> >
> > I added a replication document to _replicator that looks like this:
> > {
> > "source": {
> > "url": "https://[remote ip]/futon/iss-local<https://
> [remote%20ip]/futon/iss-local>",
> > "headers": {
> > "Authorization": "Basic [valid base64 user:pass]"
> > }
> > },
> > "target": "iss-remote",
> > "create_target": true,
> > "continuous": true,
> > "user_ctx": {
> > "roles": ["_admin"]
> > }
> > }
> >
> > The replication document is successfully added and
replication_state
> > is
> set to triggered.
> > The status page shows the Replication task and the status column
> > shows
> "Checkpointed source sequence 0, current source sequence 20416,
> progress 0%".
> >
> > It never progresses from there. iss-remote is created successfully,
> > but
> no documents are added. Checking CouchDB's log file shows a series of
> error messages like this:
> > [Thu, 28 May 2015 22:04:46 GMT] [error] [<0.22546.4>] Replicator,
> request GET to "https://[remote
>
ip]/futon/iss-local/1_33333?revs=true&open_revs=%5B%22615-738587c5fb3aa93f573fa5a79d3cf3ec%22%5D&latest=true<https://[remote%20ip]/futon/iss-local/1_33333?revs=true&open_revs=%5B%22615-738587c5fb3aa93f573fa5a79d3cf3ec%22%5D&latest=true>"
> failed due to error timeout
> > [Thu, 28 May 2015 22:04:46 GMT] [info] [<0.22544.4>] Retrying
GET to
> https://[remote
>
ip]/futon/iss-local/1_33333?revs=true&open_revs=%5B%22615-738587c5fb3a
> a93f573fa5a79d3cf3ec%22%5D&latest=true
> in 1.0 seconds due to error timeout
> >
> > Couch then waits a bit and tries again and I get the same error
message.
> >
> > Which doesn't make sense to me because that URL works correctly
> > using
> multiple tools (web browser, curl, and even using an httpc:request()
> in
> erlang) from the same machine.
> >
> > If I expose port 80 on the remote server and use http instead
> > everything
> works as expected. Documents are replicated and progress goes up
to 100%.
> >
> > As far as I can tell, this is an issue with couchdb when talking to
> > the
> reverse proxy. I tried exposing couchdb's https port 6984 using the
> same certificate and it worked fine.
> >
> > I have installed a new version of apache and only configured the
> proxypass and ssl and I still get the same issue.
> >
> > I ran wireshark (with ssl decryption using the certificate from the
> server) on the machine that I am replicating to and I don't see
any issues.
> I see the GET request from the error log and a corresponding 200
> response with data. A little while later, the error message shows
up in the log.
> >
> > Has anyone seen this before? Is there anything I can do to fix it?
> >
> > Thank you,
> > Tim Rutherford
> >
> >
> > ________________________________
> >
> > This e-mail and any files transmitted with it may be
proprietary and
> > are
> intended solely for the use of the individual or entity to whom they
> are addressed. If you have received this e-mail in error please
notify
> the sender. Please note that any views or opinions presented in this
> e-mail are solely those of the author and do not necessarily
represent
> those of Exelis Inc. The recipient should check this e-mail and any
> attachments for the presence of viruses. Exelis Inc. accepts no
> liability for any damage caused by any virus transmitted by this
e-mail.
>
RE: HTTPS Reverse Proxy Replication
Posted by "Rutherford, Tim - Exelis" <Ti...@exelisinc.com>.
Thanks for the suggestion, but I have been playing with the settings and so far I have been unsuccessful.
None of the settings I have set seem to help. The X-Forwarded-Proto, and x_forwarded_ssl settings don't seem to make a difference. I also played with the keepalive settings and they seem to be working, but couch is still failing.
I did dig into the wireshark logs a little more and I notice something. The request that couchdb is reporting a timeout on seems to return successfully, but the socket seems to be reused for the next request. The next request seems to be getting a tcp reset without a response from the server. The next request in my setup is usually to the root of the server, but I have seen it be other requests as well.
Thank you,
Tim Rutherford
-----Original Message-----
From: Daniel Holth [mailto:dholth@gmail.com]
Sent: Wednesday, June 17, 2015 11:29 AM
To: user@couchdb.apache.org
Subject: Re: HTTPS Reverse Proxy Replication
Check X-Forwarded-Proto
http://docs.couchdb.org/en/latest/config/http.html#httpd/x_forwarded_proto
On Wed, Jun 17, 2015 at 1:09 PM Rutherford, Tim - Exelis < Tim.Rutherford@exelisinc.com> wrote:
> Thank you for the example. Unfortunately, I don't really have the
> option to switch to nginx. I can modify the Apache httpd configuration
> some, but there are other components that need httpd.
>
> I don't know enough about ProxyPass or nginx to be sure if my config
> does everything yours does. But I have tried copying a pasting the
> config from the page linked below without success on a fresh Apache install.
>
> I was hoping someone could verify that they had Apache httpd and couch
> working with https.
>
> I tried following this wiki page
> https://wiki.apache.org/couchdb/Apache_As_a_Reverse_Proxy and I can
> still only get http working. Https still fails.
>
> The strange part to me is that wireshark follows the request and
> response all the way back to couch. From what I can tell, Apache is
> properly serving the requests. Couchdb is reporting a timeout in the
> log even though I see the data being returned from apache.
>
> Tim Rutherford
>
>
> -----Original Message-----
> From: Daniel Holth [mailto:dholth@gmail.com]
> Sent: Tuesday, June 16, 2015 2:48 PM
> To: user@couchdb.apache.org
> Subject: Re: HTTPS Reverse Proxy Replication
>
> I spent a long time struggling with my reverse proxy configuration.
> Here's the relevant part of my nginx config. We rewrite requests to
> CouchDB so it does not see the prefix. Critically we also use
> proxy_redirect to rewrite redirects issued by CouchDB so that they
> point at the correct location "/db" instead of "127.0.0.1:5984".
>
> location /db {
>
> rewrite /db/(.*) /$1 break;
>
> proxy_set_header X-Forwarded-For $remote_addr; proxy_set_header Host
> $proxy_host;
>
> proxy_pass http://127.0.0.1:5984;
> proxy_redirect https://127.0.0.1:5984 https://$host/db;
>
> }
>
>
> On Tue, Jun 16, 2015 at 12:46 PM, Rutherford, Tim - Exelis
> <Ti...@exelisinc.com> wrote:
> > I have been struggling for a while now to get couchdb continuous
> replication to work with a remote database behind an https reverse proxy.
> I'm not sure if I am missing something, but as far as I can tell, it
> should work and just isn't.
> >
> > My setup has a windows machine running couchdb (version 1.5.1) on
> > the
> default port 5984 and a remote couchdb (version 1.5.1) behind apache
> httpd using HTTPS.
> >
> > Apache httpd is configured with: ProxyPass /futon
> > http://localhost:5984
> >
> > I added a replication document to _replicator that looks like this:
> > {
> > "source": {
> > "url": "https://[remote ip]/futon/iss-local<https://
> [remote%20ip]/futon/iss-local>",
> > "headers": {
> > "Authorization": "Basic [valid base64 user:pass]"
> > }
> > },
> > "target": "iss-remote",
> > "create_target": true,
> > "continuous": true,
> > "user_ctx": {
> > "roles": ["_admin"]
> > }
> > }
> >
> > The replication document is successfully added and replication_state
> > is
> set to triggered.
> > The status page shows the Replication task and the status column
> > shows
> "Checkpointed source sequence 0, current source sequence 20416,
> progress 0%".
> >
> > It never progresses from there. iss-remote is created successfully,
> > but
> no documents are added. Checking CouchDB's log file shows a series of
> error messages like this:
> > [Thu, 28 May 2015 22:04:46 GMT] [error] [<0.22546.4>] Replicator,
> request GET to "https://[remote
> ip]/futon/iss-local/1_33333?revs=true&open_revs=%5B%22615-738587c5fb3aa93f573fa5a79d3cf3ec%22%5D&latest=true<https://[remote%20ip]/futon/iss-local/1_33333?revs=true&open_revs=%5B%22615-738587c5fb3aa93f573fa5a79d3cf3ec%22%5D&latest=true>"
> failed due to error timeout
> > [Thu, 28 May 2015 22:04:46 GMT] [info] [<0.22544.4>] Retrying GET to
> https://[remote
> ip]/futon/iss-local/1_33333?revs=true&open_revs=%5B%22615-738587c5fb3a
> a93f573fa5a79d3cf3ec%22%5D&latest=true
> in 1.0 seconds due to error timeout
> >
> > Couch then waits a bit and tries again and I get the same error message.
> >
> > Which doesn't make sense to me because that URL works correctly
> > using
> multiple tools (web browser, curl, and even using an httpc:request()
> in
> erlang) from the same machine.
> >
> > If I expose port 80 on the remote server and use http instead
> > everything
> works as expected. Documents are replicated and progress goes up to 100%.
> >
> > As far as I can tell, this is an issue with couchdb when talking to
> > the
> reverse proxy. I tried exposing couchdb's https port 6984 using the
> same certificate and it worked fine.
> >
> > I have installed a new version of apache and only configured the
> proxypass and ssl and I still get the same issue.
> >
> > I ran wireshark (with ssl decryption using the certificate from the
> server) on the machine that I am replicating to and I don't see any issues.
> I see the GET request from the error log and a corresponding 200
> response with data. A little while later, the error message shows up in the log.
> >
> > Has anyone seen this before? Is there anything I can do to fix it?
> >
> > Thank you,
> > Tim Rutherford
> >
> >
> > ________________________________
> >
> > This e-mail and any files transmitted with it may be proprietary and
> > are
> intended solely for the use of the individual or entity to whom they
> are addressed. If you have received this e-mail in error please notify
> the sender. Please note that any views or opinions presented in this
> e-mail are solely those of the author and do not necessarily represent
> those of Exelis Inc. The recipient should check this e-mail and any
> attachments for the presence of viruses. Exelis Inc. accepts no
> liability for any damage caused by any virus transmitted by this e-mail.
>
Re: HTTPS Reverse Proxy Replication
Posted by Daniel Holth <dh...@gmail.com>.
Check X-Forwarded-Proto
http://docs.couchdb.org/en/latest/config/http.html#httpd/x_forwarded_proto
On Wed, Jun 17, 2015 at 1:09 PM Rutherford, Tim - Exelis <
Tim.Rutherford@exelisinc.com> wrote:
> Thank you for the example. Unfortunately, I don't really have the option
> to switch to nginx. I can modify the Apache httpd configuration some, but
> there are other components that need httpd.
>
> I don't know enough about ProxyPass or nginx to be sure if my config does
> everything yours does. But I have tried copying a pasting the config from
> the page linked below without success on a fresh Apache install.
>
> I was hoping someone could verify that they had Apache httpd and couch
> working with https.
>
> I tried following this wiki page
> https://wiki.apache.org/couchdb/Apache_As_a_Reverse_Proxy and I can still
> only get http working. Https still fails.
>
> The strange part to me is that wireshark follows the request and response
> all the way back to couch. From what I can tell, Apache is properly serving
> the requests. Couchdb is reporting a timeout in the log even though I see
> the data being returned from apache.
>
> Tim Rutherford
>
>
> -----Original Message-----
> From: Daniel Holth [mailto:dholth@gmail.com]
> Sent: Tuesday, June 16, 2015 2:48 PM
> To: user@couchdb.apache.org
> Subject: Re: HTTPS Reverse Proxy Replication
>
> I spent a long time struggling with my reverse proxy configuration.
> Here's the relevant part of my nginx config. We rewrite requests to
> CouchDB so it does not see the prefix. Critically we also use
> proxy_redirect to rewrite redirects issued by CouchDB so that they
> point at the correct location "/db" instead of "127.0.0.1:5984".
>
> location /db {
>
> rewrite /db/(.*) /$1 break;
>
> proxy_set_header X-Forwarded-For $remote_addr;
> proxy_set_header Host $proxy_host;
>
> proxy_pass http://127.0.0.1:5984;
> proxy_redirect https://127.0.0.1:5984 https://$host/db;
>
> }
>
>
> On Tue, Jun 16, 2015 at 12:46 PM, Rutherford, Tim - Exelis
> <Ti...@exelisinc.com> wrote:
> > I have been struggling for a while now to get couchdb continuous
> replication to work with a remote database behind an https reverse proxy.
> I'm not sure if I am missing something, but as far as I can tell, it should
> work and just isn't.
> >
> > My setup has a windows machine running couchdb (version 1.5.1) on the
> default port 5984 and a remote couchdb (version 1.5.1) behind apache httpd
> using HTTPS.
> >
> > Apache httpd is configured with: ProxyPass /futon http://localhost:5984
> >
> > I added a replication document to _replicator that looks like this:
> > {
> > "source": {
> > "url": "https://[remote ip]/futon/iss-local<https://
> [remote%20ip]/futon/iss-local>",
> > "headers": {
> > "Authorization": "Basic [valid base64 user:pass]"
> > }
> > },
> > "target": "iss-remote",
> > "create_target": true,
> > "continuous": true,
> > "user_ctx": {
> > "roles": ["_admin"]
> > }
> > }
> >
> > The replication document is successfully added and replication_state is
> set to triggered.
> > The status page shows the Replication task and the status column shows
> "Checkpointed source sequence 0, current source sequence 20416, progress
> 0%".
> >
> > It never progresses from there. iss-remote is created successfully, but
> no documents are added. Checking CouchDB's log file shows a series of error
> messages like this:
> > [Thu, 28 May 2015 22:04:46 GMT] [error] [<0.22546.4>] Replicator,
> request GET to "https://[remote
> ip]/futon/iss-local/1_33333?revs=true&open_revs=%5B%22615-738587c5fb3aa93f573fa5a79d3cf3ec%22%5D&latest=true<https://[remote%20ip]/futon/iss-local/1_33333?revs=true&open_revs=%5B%22615-738587c5fb3aa93f573fa5a79d3cf3ec%22%5D&latest=true>"
> failed due to error timeout
> > [Thu, 28 May 2015 22:04:46 GMT] [info] [<0.22544.4>] Retrying GET to
> https://[remote
> ip]/futon/iss-local/1_33333?revs=true&open_revs=%5B%22615-738587c5fb3aa93f573fa5a79d3cf3ec%22%5D&latest=true
> in 1.0 seconds due to error timeout
> >
> > Couch then waits a bit and tries again and I get the same error message.
> >
> > Which doesn't make sense to me because that URL works correctly using
> multiple tools (web browser, curl, and even using an httpc:request() in
> erlang) from the same machine.
> >
> > If I expose port 80 on the remote server and use http instead everything
> works as expected. Documents are replicated and progress goes up to 100%.
> >
> > As far as I can tell, this is an issue with couchdb when talking to the
> reverse proxy. I tried exposing couchdb's https port 6984 using the same
> certificate and it worked fine.
> >
> > I have installed a new version of apache and only configured the
> proxypass and ssl and I still get the same issue.
> >
> > I ran wireshark (with ssl decryption using the certificate from the
> server) on the machine that I am replicating to and I don't see any issues.
> I see the GET request from the error log and a corresponding 200 response
> with data. A little while later, the error message shows up in the log.
> >
> > Has anyone seen this before? Is there anything I can do to fix it?
> >
> > Thank you,
> > Tim Rutherford
> >
> >
> > ________________________________
> >
> > This e-mail and any files transmitted with it may be proprietary and are
> intended solely for the use of the individual or entity to whom they are
> addressed. If you have received this e-mail in error please notify the
> sender. Please note that any views or opinions presented in this e-mail are
> solely those of the author and do not necessarily represent those of Exelis
> Inc. The recipient should check this e-mail and any attachments for the
> presence of viruses. Exelis Inc. accepts no liability for any damage caused
> by any virus transmitted by this e-mail.
>
RE: HTTPS Reverse Proxy Replication
Posted by "Rutherford, Tim - Exelis" <Ti...@exelisinc.com>.
Thank you for the example. Unfortunately, I don't really have the option to switch to nginx. I can modify the Apache httpd configuration some, but there are other components that need httpd.
I don't know enough about ProxyPass or nginx to be sure if my config does everything yours does. But I have tried copying a pasting the config from the page linked below without success on a fresh Apache install.
I was hoping someone could verify that they had Apache httpd and couch working with https.
I tried following this wiki page https://wiki.apache.org/couchdb/Apache_As_a_Reverse_Proxy and I can still only get http working. Https still fails.
The strange part to me is that wireshark follows the request and response all the way back to couch. From what I can tell, Apache is properly serving the requests. Couchdb is reporting a timeout in the log even though I see the data being returned from apache.
Tim Rutherford
-----Original Message-----
From: Daniel Holth [mailto:dholth@gmail.com]
Sent: Tuesday, June 16, 2015 2:48 PM
To: user@couchdb.apache.org
Subject: Re: HTTPS Reverse Proxy Replication
I spent a long time struggling with my reverse proxy configuration.
Here's the relevant part of my nginx config. We rewrite requests to
CouchDB so it does not see the prefix. Critically we also use
proxy_redirect to rewrite redirects issued by CouchDB so that they
point at the correct location "/db" instead of "127.0.0.1:5984".
location /db {
rewrite /db/(.*) /$1 break;
proxy_set_header X-Forwarded-For $remote_addr;
proxy_set_header Host $proxy_host;
proxy_pass http://127.0.0.1:5984;
proxy_redirect https://127.0.0.1:5984 https://$host/db;
}
On Tue, Jun 16, 2015 at 12:46 PM, Rutherford, Tim - Exelis
<Ti...@exelisinc.com> wrote:
> I have been struggling for a while now to get couchdb continuous replication to work with a remote database behind an https reverse proxy. I'm not sure if I am missing something, but as far as I can tell, it should work and just isn't.
>
> My setup has a windows machine running couchdb (version 1.5.1) on the default port 5984 and a remote couchdb (version 1.5.1) behind apache httpd using HTTPS.
>
> Apache httpd is configured with: ProxyPass /futon http://localhost:5984
>
> I added a replication document to _replicator that looks like this:
> {
> "source": {
> "url": "https://[remote ip]/futon/iss-local<https://[remote%20ip]/futon/iss-local>",
> "headers": {
> "Authorization": "Basic [valid base64 user:pass]"
> }
> },
> "target": "iss-remote",
> "create_target": true,
> "continuous": true,
> "user_ctx": {
> "roles": ["_admin"]
> }
> }
>
> The replication document is successfully added and replication_state is set to triggered.
> The status page shows the Replication task and the status column shows "Checkpointed source sequence 0, current source sequence 20416, progress 0%".
>
> It never progresses from there. iss-remote is created successfully, but no documents are added. Checking CouchDB's log file shows a series of error messages like this:
> [Thu, 28 May 2015 22:04:46 GMT] [error] [<0.22546.4>] Replicator, request GET to "https://[remote ip]/futon/iss-local/1_33333?revs=true&open_revs=%5B%22615-738587c5fb3aa93f573fa5a79d3cf3ec%22%5D&latest=true<https://[remote%20ip]/futon/iss-local/1_33333?revs=true&open_revs=%5B%22615-738587c5fb3aa93f573fa5a79d3cf3ec%22%5D&latest=true>" failed due to error timeout
> [Thu, 28 May 2015 22:04:46 GMT] [info] [<0.22544.4>] Retrying GET to https://[remote ip]/futon/iss-local/1_33333?revs=true&open_revs=%5B%22615-738587c5fb3aa93f573fa5a79d3cf3ec%22%5D&latest=true in 1.0 seconds due to error timeout
>
> Couch then waits a bit and tries again and I get the same error message.
>
> Which doesn't make sense to me because that URL works correctly using multiple tools (web browser, curl, and even using an httpc:request() in erlang) from the same machine.
>
> If I expose port 80 on the remote server and use http instead everything works as expected. Documents are replicated and progress goes up to 100%.
>
> As far as I can tell, this is an issue with couchdb when talking to the reverse proxy. I tried exposing couchdb's https port 6984 using the same certificate and it worked fine.
>
> I have installed a new version of apache and only configured the proxypass and ssl and I still get the same issue.
>
> I ran wireshark (with ssl decryption using the certificate from the server) on the machine that I am replicating to and I don't see any issues. I see the GET request from the error log and a corresponding 200 response with data. A little while later, the error message shows up in the log.
>
> Has anyone seen this before? Is there anything I can do to fix it?
>
> Thank you,
> Tim Rutherford
>
>
> ________________________________
>
> This e-mail and any files transmitted with it may be proprietary and are intended solely for the use of the individual or entity to whom they are addressed. If you have received this e-mail in error please notify the sender. Please note that any views or opinions presented in this e-mail are solely those of the author and do not necessarily represent those of Exelis Inc. The recipient should check this e-mail and any attachments for the presence of viruses. Exelis Inc. accepts no liability for any damage caused by any virus transmitted by this e-mail.
Re: HTTPS Reverse Proxy Replication
Posted by Daniel Holth <dh...@gmail.com>.
I spent a long time struggling with my reverse proxy configuration.
Here's the relevant part of my nginx config. We rewrite requests to
CouchDB so it does not see the prefix. Critically we also use
proxy_redirect to rewrite redirects issued by CouchDB so that they
point at the correct location "/db" instead of "127.0.0.1:5984".
location /db {
rewrite /db/(.*) /$1 break;
proxy_set_header X-Forwarded-For $remote_addr;
proxy_set_header Host $proxy_host;
proxy_pass http://127.0.0.1:5984;
proxy_redirect https://127.0.0.1:5984 https://$host/db;
}
On Tue, Jun 16, 2015 at 12:46 PM, Rutherford, Tim - Exelis
<Ti...@exelisinc.com> wrote:
> I have been struggling for a while now to get couchdb continuous replication to work with a remote database behind an https reverse proxy. I'm not sure if I am missing something, but as far as I can tell, it should work and just isn't.
>
> My setup has a windows machine running couchdb (version 1.5.1) on the default port 5984 and a remote couchdb (version 1.5.1) behind apache httpd using HTTPS.
>
> Apache httpd is configured with: ProxyPass /futon http://localhost:5984
>
> I added a replication document to _replicator that looks like this:
> {
> "source": {
> "url": "https://[remote ip]/futon/iss-local<https://[remote%20ip]/futon/iss-local>",
> "headers": {
> "Authorization": "Basic [valid base64 user:pass]"
> }
> },
> "target": "iss-remote",
> "create_target": true,
> "continuous": true,
> "user_ctx": {
> "roles": ["_admin"]
> }
> }
>
> The replication document is successfully added and replication_state is set to triggered.
> The status page shows the Replication task and the status column shows "Checkpointed source sequence 0, current source sequence 20416, progress 0%".
>
> It never progresses from there. iss-remote is created successfully, but no documents are added. Checking CouchDB's log file shows a series of error messages like this:
> [Thu, 28 May 2015 22:04:46 GMT] [error] [<0.22546.4>] Replicator, request GET to "https://[remote ip]/futon/iss-local/1_33333?revs=true&open_revs=%5B%22615-738587c5fb3aa93f573fa5a79d3cf3ec%22%5D&latest=true<https://[remote%20ip]/futon/iss-local/1_33333?revs=true&open_revs=%5B%22615-738587c5fb3aa93f573fa5a79d3cf3ec%22%5D&latest=true>" failed due to error timeout
> [Thu, 28 May 2015 22:04:46 GMT] [info] [<0.22544.4>] Retrying GET to https://[remote ip]/futon/iss-local/1_33333?revs=true&open_revs=%5B%22615-738587c5fb3aa93f573fa5a79d3cf3ec%22%5D&latest=true in 1.0 seconds due to error timeout
>
> Couch then waits a bit and tries again and I get the same error message.
>
> Which doesn't make sense to me because that URL works correctly using multiple tools (web browser, curl, and even using an httpc:request() in erlang) from the same machine.
>
> If I expose port 80 on the remote server and use http instead everything works as expected. Documents are replicated and progress goes up to 100%.
>
> As far as I can tell, this is an issue with couchdb when talking to the reverse proxy. I tried exposing couchdb's https port 6984 using the same certificate and it worked fine.
>
> I have installed a new version of apache and only configured the proxypass and ssl and I still get the same issue.
>
> I ran wireshark (with ssl decryption using the certificate from the server) on the machine that I am replicating to and I don't see any issues. I see the GET request from the error log and a corresponding 200 response with data. A little while later, the error message shows up in the log.
>
> Has anyone seen this before? Is there anything I can do to fix it?
>
> Thank you,
> Tim Rutherford
>
>
> ________________________________
>
> This e-mail and any files transmitted with it may be proprietary and are intended solely for the use of the individual or entity to whom they are addressed. If you have received this e-mail in error please notify the sender. Please note that any views or opinions presented in this e-mail are solely those of the author and do not necessarily represent those of Exelis Inc. The recipient should check this e-mail and any attachments for the presence of viruses. Exelis Inc. accepts no liability for any damage caused by any virus transmitted by this e-mail.