You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@couchdb.apache.org by Guby <gu...@gmail.com> on 2008/04/28 01:04:31 UTC
Updating views on save
Hello dear Couchers
I understand that the views are indexed the first time they are
accessed and as far as I know there is no way to turn on view updating
on document save. I really don't understand the reasoning behind this
behavior. The advantage of the pre-populated/indexed views are that
they are blazingly fast to query and access, but that advantage
disappears when the first request after a document update has to
regenerate the view first!
I am currently building a web app where the background processes
perform a lot of writes to the database. The time it takes to write a
document is not critical for me. What is critical though is the time
it takes to load web pages for the end user that require content from
the database.
In some situations the background processes add thousands of documents
to the database within a short period of time, and when the user tries
to access a page after such an update the view querying sometimes
takes minutes and as a consequence of that the browser times out...
Not a recipe for happy customers...
The only solution I can see at the moment is to create a worker that
queries the database whenever it is told that there has been a
document update, but that seems really stupid and unnecessary. And in
my case, running on a smallish VPS, a big waste of resources having an
extra working doing something the database itself could just as well
have done. It also requires a lot of extra coding notifying the worker
whenever I update or create a document throughout my app.
I am sure you have reasons for having implemented the views the way
you have, but I would be really interested to hear why it has been
done this way!
My wishes are for an optional updating of views on save feature! In
some cases that might regenerate a view several times without it
actually being accessed in between, but that is a tradeoff I can live
with, slow views on the other hand is something I can not!
All the best
Sebastian
Re: Updating views on save
Posted by Cortland Klein <me...@pixelcort.com>.
Not sure why your views are timing out, but from my current
understanding views are incrementally updated with modifications but
only incrementally updated on a call to that view.
One hypothesis is that after the thousands of inserts the next view
call is taking a long time doing the incremental updates to the view
since the last time it was called.
Are you using the javascript spodermonkey viewserver(default) or
another one? Check the complexity of the view and possibly minimize
the view's complexity.
As a final thought iirc when querying a view in a design document, all
views in that designdocument are called. Is it possible you might have
unrelated views in the same design document?
I'm not sure, but I think the pattern here is you put views most
likely to be called near to each other in the design document, say
blog summaries view followed by full content view, and have less
related views in a different design document, say for a list of
authors or a tag list.
Hope this helps.
Cortland Klein <me...@pixelcort.com> +1 408 506 9791
http://pixelcort.com/
Sent from my iPhone
On Apr 27, 2008, at 4:04 PM, Guby <gu...@gmail.com> wrote:
> Hello dear Couchers
>
> I understand that the views are indexed the first time they are
> accessed and as far as I know there is no way to turn on view
> updating on document save. I really don't understand the reasoning
> behind this behavior. The advantage of the pre-populated/indexed
> views are that they are blazingly fast to query and access, but that
> advantage disappears when the first request after a document update
> has to regenerate the view first!
>
> I am currently building a web app where the background processes
> perform a lot of writes to the database. The time it takes to write
> a document is not critical for me. What is critical though is the
> time it takes to load web pages for the end user that require
> content from the database.
> In some situations the background processes add thousands of
> documents to the database within a short period of time, and when
> the user tries to access a page after such an update the view
> querying sometimes takes minutes and as a consequence of that the
> browser times out... Not a recipe for happy customers...
>
> The only solution I can see at the moment is to create a worker that
> queries the database whenever it is told that there has been a
> document update, but that seems really stupid and unnecessary. And
> in my case, running on a smallish VPS, a big waste of resources
> having an extra working doing something the database itself could
> just as well have done. It also requires a lot of extra coding
> notifying the worker whenever I update or create a document
> throughout my app.
>
> I am sure you have reasons for having implemented the views the way
> you have, but I would be really interested to hear why it has been
> done this way!
>
> My wishes are for an optional updating of views on save feature! In
> some cases that might regenerate a view several times without it
> actually being accessed in between, but that is a tradeoff I can
> live with, slow views on the other hand is something I can not!
>
> All the best
> Sebastian
Re: Updating views on save
Posted by Benoit Chesneau <bc...@gmail.com>.
On Mon, Apr 28, 2008 at 1:04 AM, Guby <gu...@gmail.com> wrote:
> The only solution I can see at the moment is to create a worker that
> queries the database whenever it is told that there has been a document
> update, but that seems really stupid and unnecessary.
why not using FulltextSearch api or extend it to be notified about
document updates ? With something like a comet server, it may solve
your problem without having another layer that poll the database about
updated. imho it would be best to solve it on couchdb layer.
http://wiki.apache.org/couchdb/FullTextSearch
- benoît
Re: firewall
Posted by Tommy Chheng <tc...@uci.edu>.
I'm on couch 0.7.3a813
I only recently checked it from SVN so it should be fairly new.
On Apr 28, 2008, at 1:32 PM, Paul Davis wrote:
> What version of couchdb are you using?
>
> I'm on the most recent svn and I don't need it. Or do you have some
> other setup that's requiring that?
>
> Also, I'm out of ideas, so maybe someone else can jump in.
>
> On Mon, Apr 28, 2008 at 4:11 PM, Tommy Chheng <tc...@uci.edu> wrote:
>> Apparently the httpd_conf is needed :(
>>
>> httpd_conf: Error while reading config file: httpd_conf: Cannot open
>> couch_httpd.conf
>>
>>
>>
>>
>>
>> On Apr 28, 2008, at 12:04 PM, Paul Davis wrote:
>>
>>
>>> On Mon, Apr 28, 2008 at 2:57 PM, Tommy Chheng <tc...@uci.edu>
>>> wrote:
>>>
>>>> ok, I added the Port to the couch.ini and changed my
>>>> couch_http.conf to
>>>> match Still no luck:
>>>> my couch_http.conf file
>>>> http://pastie.caboo.se/188195
>>>>
>>>> I'm running nginx and mongrel for rails on this server.. I doubt
>>>> these
>> can
>>>> be an issue since they aren't touching 5984 at all and no
>>>> firewall is
>> on...
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Apr 28, 2008, at 11:38 AM, Paul Davis wrote:
>>>>
>>>>
>>>>
>>>>> On Mon, Apr 28, 2008 at 2:30 PM, Tommy Chheng <tc...@uci.edu>
>>>>> wrote:
>>>>>
>>>>>
>>>>>> I added BindAddress so my couch.ini file looks like:
>>>>>> http://pastie.caboo.se/188174
>>>>>>
>>>>>> Still same problem.
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Apr 28, 2008, at 11:22 AM, Paul Davis wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>> On Mon, Apr 28, 2008 at 2:19 PM, Tommy Chheng <tc...@uci.edu>
>> wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> Ok, so i just turned off the firewall temporary
>>>>>>>>
>>>>>>>> /sbin/iptables -L
>>>>>>>> Chain INPUT (policy ACCEPT)
>>>>>>>> target prot opt source destination
>>>>>>>>
>>>>>>>> Chain FORWARD (policy ACCEPT)
>>>>>>>>
>>>>>>>> target prot opt source destination
>>>>>>>>
>>>>>>>> Chain OUTPUT (policy ACCEPT)
>>>>>>>>
>>>>>>>> target prot opt source destination
>>>>>>>>
>>>>>>>> Chain RH-Firewall-1-INPUT (0 references)
>>>>>>>>
>>>>>>>> target prot opt source destination
>>>>>>>>
>>>>>>>> Still no response from outside. It's only working via
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>> localhost:5984.
>>>>
>>>>>
>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> Any ideas how to debug this?
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Apr 28, 2008, at 11:12 AM, Paul Davis wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> /sbin/iptables -A OUTPUT -p tcp -m tcp --dport 5984 -j
>> ACCEPT
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>> Ohhhh. Heh.
>>>>>>>
>>>>>>> /usr/local/etc/couchdb/couch.ini
>>>>>>>
>>>>>>> Change "BindAddresss=127.0.0.1" to "BindAddress=0.0.0.0"
>>>>>>>
>>>>>>> Or your local static ip to only listen on that IP.
>>>>>>>
>>>>>>> Paul
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>> I don't see a port specified. I'd assume that's specified in
>>>>> /usr/local/etc/couchdb/couch_httpd.conf. You might check if you
>>>>> have a
>>>>> conflicting BindAddress in there as well.
>>>>>
>>>>> For reference, my couch.ini looks like this:
>>>>> http://pastie.caboo.se/188178
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>
>>> The other servers are reachable from an external host, right?
>>>
>>> Try commenting out the HttpConf. I've never used that and don't have
>>> much of an idea on what the side effects might be. Could it be that
>>> having the BindAddress and Port specified multiple times is causing
>>> issues?
>>>
>>> Paul
>>>
>>>
>>
>>
>
Re: firewall
Posted by Paul Davis <pa...@gmail.com>.
What version of couchdb are you using?
I'm on the most recent svn and I don't need it. Or do you have some
other setup that's requiring that?
Also, I'm out of ideas, so maybe someone else can jump in.
On Mon, Apr 28, 2008 at 4:11 PM, Tommy Chheng <tc...@uci.edu> wrote:
> Apparently the httpd_conf is needed :(
>
> httpd_conf: Error while reading config file: httpd_conf: Cannot open
> couch_httpd.conf
>
>
>
>
>
> On Apr 28, 2008, at 12:04 PM, Paul Davis wrote:
>
>
> > On Mon, Apr 28, 2008 at 2:57 PM, Tommy Chheng <tc...@uci.edu> wrote:
> >
> > > ok, I added the Port to the couch.ini and changed my couch_http.conf to
> > > match Still no luck:
> > > my couch_http.conf file
> > > http://pastie.caboo.se/188195
> > >
> > > I'm running nginx and mongrel for rails on this server.. I doubt these
> can
> > > be an issue since they aren't touching 5984 at all and no firewall is
> on...
> > >
> > >
> > >
> > >
> > >
> > > On Apr 28, 2008, at 11:38 AM, Paul Davis wrote:
> > >
> > >
> > >
> > > > On Mon, Apr 28, 2008 at 2:30 PM, Tommy Chheng <tc...@uci.edu> wrote:
> > > >
> > > >
> > > > > I added BindAddress so my couch.ini file looks like:
> > > > > http://pastie.caboo.se/188174
> > > > >
> > > > > Still same problem.
> > > > >
> > > > >
> > > > >
> > > > > On Apr 28, 2008, at 11:22 AM, Paul Davis wrote:
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > > On Mon, Apr 28, 2008 at 2:19 PM, Tommy Chheng <tc...@uci.edu>
> wrote:
> > > > > >
> > > > > >
> > > > > >
> > > > > > > Ok, so i just turned off the firewall temporary
> > > > > > >
> > > > > > > /sbin/iptables -L
> > > > > > > Chain INPUT (policy ACCEPT)
> > > > > > > target prot opt source destination
> > > > > > >
> > > > > > > Chain FORWARD (policy ACCEPT)
> > > > > > >
> > > > > > > target prot opt source destination
> > > > > > >
> > > > > > > Chain OUTPUT (policy ACCEPT)
> > > > > > >
> > > > > > > target prot opt source destination
> > > > > > >
> > > > > > > Chain RH-Firewall-1-INPUT (0 references)
> > > > > > >
> > > > > > > target prot opt source destination
> > > > > > >
> > > > > > > Still no response from outside. It's only working via
> > > > > > >
> > > > > >
> > > > >
> > > >
> > > localhost:5984.
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > Any ideas how to debug this?
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > On Apr 28, 2008, at 11:12 AM, Paul Davis wrote:
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > > /sbin/iptables -A OUTPUT -p tcp -m tcp --dport 5984 -j
> ACCEPT
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > > > Ohhhh. Heh.
> > > > > >
> > > > > > /usr/local/etc/couchdb/couch.ini
> > > > > >
> > > > > > Change "BindAddresss=127.0.0.1" to "BindAddress=0.0.0.0"
> > > > > >
> > > > > > Or your local static ip to only listen on that IP.
> > > > > >
> > > > > > Paul
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > >
> > > > I don't see a port specified. I'd assume that's specified in
> > > > /usr/local/etc/couchdb/couch_httpd.conf. You might check if you have a
> > > > conflicting BindAddress in there as well.
> > > >
> > > > For reference, my couch.ini looks like this:
> > > > http://pastie.caboo.se/188178
> > > >
> > > >
> > > >
> > >
> > >
> > >
> >
> > The other servers are reachable from an external host, right?
> >
> > Try commenting out the HttpConf. I've never used that and don't have
> > much of an idea on what the side effects might be. Could it be that
> > having the BindAddress and Port specified multiple times is causing
> > issues?
> >
> > Paul
> >
> >
>
>
Re: firewall
Posted by Paul Davis <pa...@gmail.com>.
I'd start by getting a tcpdump of the traffic to port 5984 on the
couchdb machine.
On Mon, Apr 28, 2008 at 5:59 PM, Tommy Chheng <tc...@uci.edu> wrote:
> Ok, previously I checked out from the google code source.
> Still can't access from external and all iptables rules are turned off.
>
> I tried both
> BindAddress=127.0.0.1
>
> BindAddress=0.0.0.0
>
>
> There's no response from the outside.
> couch 0.7.3a652071 (LogLevel=info)
> Apache CouchDB is starting.
> Apache CouchDB has started. Time to relax.
>
> Only when I access from the same machine do i get info from the couchdb
> output:
> [info] [<0.54.0>] 127.0.0.1 - - "GET /" 200
>
>
> Any thoughts in where to start to debug?
>
> -
> Tommy
>
>
>
> On Apr 28, 2008, at 2:23 PM, Paul Davis wrote:
>
>
> > I might also point out that judging from the version number you
> > checked out of SVN before the move to apache. I'd check out a fresh
> > copy from svn.apache.org and see if that fixes things.
> >
> > Paul
> >
> > On Mon, Apr 28, 2008 at 5:18 PM, Jan Lehnardt <ja...@apache.org> wrote:
> >
> > > With trunk you don't need the couch_httpd.ini anymore. It
> > > was all merged into couch.ini
> > >
> > > Cheers
> > > Jan
> > > --
> > >
> > >
> > > On Apr 28, 2008, at 22:11, Tommy Chheng wrote:
> > >
> > >
> > > > Apparently the httpd_conf is needed :(
> > > >
> > > > httpd_conf: Error while reading config file: httpd_conf: Cannot open
> > > >
> > > couch_httpd.conf
> > >
> > > >
> > > >
> > > >
> > > > On Apr 28, 2008, at 12:04 PM, Paul Davis wrote:
> > > >
> > > >
> > > >
> > > > > On Mon, Apr 28, 2008 at 2:57 PM, Tommy Chheng <tc...@uci.edu>
> wrote:
> > > > >
> > > > >
> > > > > > ok, I added the Port to the couch.ini and changed my
> couch_http.conf
> > > > > >
> > > > >
> > > >
> > > to
> > >
> > > >
> > > > >
> > > > > > match Still no luck:
> > > > > > my couch_http.conf file
> > > > > > http://pastie.caboo.se/188195
> > > > > >
> > > > > > I'm running nginx and mongrel for rails on this server.. I doubt
> > > > > >
> > > > >
> > > >
> > > these can
> > >
> > > >
> > > > >
> > > > > > be an issue since they aren't touching 5984 at all and no firewall
> is
> > > > > >
> > > > >
> > > >
> > > on...
> > >
> > > >
> > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Apr 28, 2008, at 11:38 AM, Paul Davis wrote:
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > > On Mon, Apr 28, 2008 at 2:30 PM, Tommy Chheng <tc...@uci.edu>
> > > > > > >
> > > > > >
> > > > >
> > > >
> > > wrote:
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > > I added BindAddress so my couch.ini file looks like:
> > > > > > > > http://pastie.caboo.se/188174
> > > > > > > >
> > > > > > > > Still same problem.
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > On Apr 28, 2008, at 11:22 AM, Paul Davis wrote:
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > > On Mon, Apr 28, 2008 at 2:19 PM, Tommy Chheng
> <tc...@uci.edu>
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > > wrote:
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > > Ok, so i just turned off the firewall temporary
> > > > > > > > > >
> > > > > > > > > > /sbin/iptables -L
> > > > > > > > > > Chain INPUT (policy ACCEPT)
> > > > > > > > > > target prot opt source destination
> > > > > > > > > >
> > > > > > > > > > Chain FORWARD (policy ACCEPT)
> > > > > > > > > >
> > > > > > > > > > target prot opt source destination
> > > > > > > > > >
> > > > > > > > > > Chain OUTPUT (policy ACCEPT)
> > > > > > > > > >
> > > > > > > > > > target prot opt source destination
> > > > > > > > > >
> > > > > > > > > > Chain RH-Firewall-1-INPUT (0 references)
> > > > > > > > > >
> > > > > > > > > > target prot opt source destination
> > > > > > > > > >
> > > > > > > > > > Still no response from outside. It's only working via
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > localhost:5984.
> > > > > >
> > > > > >
> > > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Any ideas how to debug this?
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > On Apr 28, 2008, at 11:12 AM, Paul Davis wrote:
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > > /sbin/iptables -A OUTPUT -p tcp -m tcp --dport 5984 -j
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > > ACCEPT
> > >
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > > Ohhhh. Heh.
> > > > > > > > >
> > > > > > > > > /usr/local/etc/couchdb/couch.ini
> > > > > > > > >
> > > > > > > > > Change "BindAddresss=127.0.0.1" to "BindAddress=0.0.0.0"
> > > > > > > > >
> > > > > > > > > Or your local static ip to only listen on that IP.
> > > > > > > > >
> > > > > > > > > Paul
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > > I don't see a port specified. I'd assume that's specified in
> > > > > > > /usr/local/etc/couchdb/couch_httpd.conf. You might check if you
> have
> > > > > > >
> > > > > >
> > > > >
> > > >
> > > a
> > >
> > > >
> > > > >
> > > > > >
> > > > > > > conflicting BindAddress in there as well.
> > > > > > >
> > > > > > > For reference, my couch.ini looks like this:
> > > > > > > http://pastie.caboo.se/188178
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > > The other servers are reachable from an external host, right?
> > > > >
> > > > > Try commenting out the HttpConf. I've never used that and don't have
> > > > > much of an idea on what the side effects might be. Could it be that
> > > > > having the BindAddress and Port specified multiple times is causing
> > > > > issues?
> > > > >
> > > > > Paul
> > > > >
> > > > >
> > > > >
> > > >
> > > >
> > > >
> > > >
> > >
> > >
> > >
> >
> >
>
>
Re: firewall
Posted by Tommy Chheng <tc...@uci.edu>.
Thanks for the tip. The problem was my host provider only allowed port
80/21 at the router level.
On Apr 28, 2008, at 3:12 PM, Anthony Mills wrote:
> Have you tried checking your router? Some of them can block ports.
> If your comfortable in command line you can try querying the server
> directly:
>
> telnet <ip> 5984
> GET /
>
> You need to hit return twice, usually.
>
> If you do not get a connection, then your connection is misrouted,
> or your port is blocked. If it works, but your still not able to
> connect with a web browser your proxy or socks server is in the way.
>
> Hope it helps,
>
> Anthony
>
>
> On Apr 28, 2008, at 4:59 PM, Tommy Chheng wrote:
>
>> Ok, previously I checked out from the google code source.
>> Still can't access from external and all iptables rules are turned
>> off.
>>
>> I tried both
>> BindAddress=127.0.0.1
>> BindAddress=0.0.0.0
>>
>>
>> There's no response from the outside.
>> couch 0.7.3a652071 (LogLevel=info)
>> Apache CouchDB is starting.
>> Apache CouchDB has started. Time to relax.
>>
>> Only when I access from the same machine do i get info from the
>> couchdb output:
>> [info] [<0.54.0>] 127.0.0.1 - - "GET /" 200
>>
>>
>> Any thoughts in where to start to debug?
>>
>> -
>> Tommy
>>
>> On Apr 28, 2008, at 2:23 PM, Paul Davis wrote:
>>
>>> I might also point out that judging from the version number you
>>> checked out of SVN before the move to apache. I'd check out a fresh
>>> copy from svn.apache.org and see if that fixes things.
>>>
>>> Paul
>>>
>>> On Mon, Apr 28, 2008 at 5:18 PM, Jan Lehnardt <ja...@apache.org>
>>> wrote:
>>>> With trunk you don't need the couch_httpd.ini anymore. It
>>>> was all merged into couch.ini
>>>>
>>>> Cheers
>>>> Jan
>>>> --
>>>>
>>>>
>>>> On Apr 28, 2008, at 22:11, Tommy Chheng wrote:
>>>>
>>>>> Apparently the httpd_conf is needed :(
>>>>>
>>>>> httpd_conf: Error while reading config file: httpd_conf: Cannot
>>>>> open
>>>> couch_httpd.conf
>>>>>
>>>>>
>>>>>
>>>>> On Apr 28, 2008, at 12:04 PM, Paul Davis wrote:
>>>>>
>>>>>
>>>>>> On Mon, Apr 28, 2008 at 2:57 PM, Tommy Chheng <tc...@uci.edu>
>>>>>> wrote:
>>>>>>
>>>>>>> ok, I added the Port to the couch.ini and changed my
>>>>>>> couch_http.conf
>>>> to
>>>>>>> match Still no luck:
>>>>>>> my couch_http.conf file
>>>>>>> http://pastie.caboo.se/188195
>>>>>>>
>>>>>>> I'm running nginx and mongrel for rails on this server.. I
>>>>>>> doubt
>>>> these can
>>>>>>> be an issue since they aren't touching 5984 at all and no
>>>>>>> firewall is
>>>> on...
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Apr 28, 2008, at 11:38 AM, Paul Davis wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> On Mon, Apr 28, 2008 at 2:30 PM, Tommy Chheng <tc...@uci.edu>
>>>> wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>> I added BindAddress so my couch.ini file looks like:
>>>>>>>>> http://pastie.caboo.se/188174
>>>>>>>>>
>>>>>>>>> Still same problem.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Apr 28, 2008, at 11:22 AM, Paul Davis wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> On Mon, Apr 28, 2008 at 2:19 PM, Tommy Chheng <tchheng@uci.edu
>>>>>>>>>> >
>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> Ok, so i just turned off the firewall temporary
>>>>>>>>>>>
>>>>>>>>>>> /sbin/iptables -L
>>>>>>>>>>> Chain INPUT (policy ACCEPT)
>>>>>>>>>>> target prot opt source destination
>>>>>>>>>>>
>>>>>>>>>>> Chain FORWARD (policy ACCEPT)
>>>>>>>>>>>
>>>>>>>>>>> target prot opt source destination
>>>>>>>>>>>
>>>>>>>>>>> Chain OUTPUT (policy ACCEPT)
>>>>>>>>>>>
>>>>>>>>>>> target prot opt source destination
>>>>>>>>>>>
>>>>>>>>>>> Chain RH-Firewall-1-INPUT (0 references)
>>>>>>>>>>>
>>>>>>>>>>> target prot opt source destination
>>>>>>>>>>>
>>>>>>>>>>> Still no response from outside. It's only working via
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>> localhost:5984.
>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Any ideas how to debug this?
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Apr 28, 2008, at 11:12 AM, Paul Davis wrote:
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>> /sbin/iptables -A OUTPUT -p tcp -m tcp --dport 5984 -j
>>>> ACCEPT
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Ohhhh. Heh.
>>>>>>>>>>
>>>>>>>>>> /usr/local/etc/couchdb/couch.ini
>>>>>>>>>>
>>>>>>>>>> Change "BindAddresss=127.0.0.1" to "BindAddress=0.0.0.0"
>>>>>>>>>>
>>>>>>>>>> Or your local static ip to only listen on that IP.
>>>>>>>>>>
>>>>>>>>>> Paul
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>> I don't see a port specified. I'd assume that's specified in
>>>>>>>> /usr/local/etc/couchdb/couch_httpd.conf. You might check if
>>>>>>>> you have
>>>> a
>>>>>>>> conflicting BindAddress in there as well.
>>>>>>>>
>>>>>>>> For reference, my couch.ini looks like this:
>>>>>>>> http://pastie.caboo.se/188178
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> The other servers are reachable from an external host, right?
>>>>>>
>>>>>> Try commenting out the HttpConf. I've never used that and don't
>>>>>> have
>>>>>> much of an idea on what the side effects might be. Could it be
>>>>>> that
>>>>>> having the BindAddress and Port specified multiple times is
>>>>>> causing
>>>>>> issues?
>>>>>>
>>>>>> Paul
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>
>
Re: firewall
Posted by Anthony Mills <am...@gascard.net>.
Have you tried checking your router? Some of them can block ports.
If your comfortable in command line you can try querying the server
directly:
telnet <ip> 5984
GET /
You need to hit return twice, usually.
If you do not get a connection, then your connection is misrouted, or
your port is blocked. If it works, but your still not able to connect
with a web browser your proxy or socks server is in the way.
Hope it helps,
Anthony
On Apr 28, 2008, at 4:59 PM, Tommy Chheng wrote:
> Ok, previously I checked out from the google code source.
> Still can't access from external and all iptables rules are turned
> off.
>
> I tried both
> BindAddress=127.0.0.1
> BindAddress=0.0.0.0
>
>
> There's no response from the outside.
> couch 0.7.3a652071 (LogLevel=info)
> Apache CouchDB is starting.
> Apache CouchDB has started. Time to relax.
>
> Only when I access from the same machine do i get info from the
> couchdb output:
> [info] [<0.54.0>] 127.0.0.1 - - "GET /" 200
>
>
> Any thoughts in where to start to debug?
>
> -
> Tommy
>
> On Apr 28, 2008, at 2:23 PM, Paul Davis wrote:
>
>> I might also point out that judging from the version number you
>> checked out of SVN before the move to apache. I'd check out a fresh
>> copy from svn.apache.org and see if that fixes things.
>>
>> Paul
>>
>> On Mon, Apr 28, 2008 at 5:18 PM, Jan Lehnardt <ja...@apache.org> wrote:
>>> With trunk you don't need the couch_httpd.ini anymore. It
>>> was all merged into couch.ini
>>>
>>> Cheers
>>> Jan
>>> --
>>>
>>>
>>> On Apr 28, 2008, at 22:11, Tommy Chheng wrote:
>>>
>>>> Apparently the httpd_conf is needed :(
>>>>
>>>> httpd_conf: Error while reading config file: httpd_conf: Cannot
>>>> open
>>> couch_httpd.conf
>>>>
>>>>
>>>>
>>>> On Apr 28, 2008, at 12:04 PM, Paul Davis wrote:
>>>>
>>>>
>>>>> On Mon, Apr 28, 2008 at 2:57 PM, Tommy Chheng <tc...@uci.edu>
>>>>> wrote:
>>>>>
>>>>>> ok, I added the Port to the couch.ini and changed my
>>>>>> couch_http.conf
>>> to
>>>>>> match Still no luck:
>>>>>> my couch_http.conf file
>>>>>> http://pastie.caboo.se/188195
>>>>>>
>>>>>> I'm running nginx and mongrel for rails on this server.. I doubt
>>> these can
>>>>>> be an issue since they aren't touching 5984 at all and no
>>>>>> firewall is
>>> on...
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Apr 28, 2008, at 11:38 AM, Paul Davis wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>> On Mon, Apr 28, 2008 at 2:30 PM, Tommy Chheng <tc...@uci.edu>
>>> wrote:
>>>>>>>
>>>>>>>
>>>>>>>> I added BindAddress so my couch.ini file looks like:
>>>>>>>> http://pastie.caboo.se/188174
>>>>>>>>
>>>>>>>> Still same problem.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Apr 28, 2008, at 11:22 AM, Paul Davis wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> On Mon, Apr 28, 2008 at 2:19 PM, Tommy Chheng
>>>>>>>>> <tc...@uci.edu>
>>> wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> Ok, so i just turned off the firewall temporary
>>>>>>>>>>
>>>>>>>>>> /sbin/iptables -L
>>>>>>>>>> Chain INPUT (policy ACCEPT)
>>>>>>>>>> target prot opt source destination
>>>>>>>>>>
>>>>>>>>>> Chain FORWARD (policy ACCEPT)
>>>>>>>>>>
>>>>>>>>>> target prot opt source destination
>>>>>>>>>>
>>>>>>>>>> Chain OUTPUT (policy ACCEPT)
>>>>>>>>>>
>>>>>>>>>> target prot opt source destination
>>>>>>>>>>
>>>>>>>>>> Chain RH-Firewall-1-INPUT (0 references)
>>>>>>>>>>
>>>>>>>>>> target prot opt source destination
>>>>>>>>>>
>>>>>>>>>> Still no response from outside. It's only working via
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>> localhost:5984.
>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Any ideas how to debug this?
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Apr 28, 2008, at 11:12 AM, Paul Davis wrote:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> /sbin/iptables -A OUTPUT -p tcp -m tcp --dport 5984 -j
>>> ACCEPT
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Ohhhh. Heh.
>>>>>>>>>
>>>>>>>>> /usr/local/etc/couchdb/couch.ini
>>>>>>>>>
>>>>>>>>> Change "BindAddresss=127.0.0.1" to "BindAddress=0.0.0.0"
>>>>>>>>>
>>>>>>>>> Or your local static ip to only listen on that IP.
>>>>>>>>>
>>>>>>>>> Paul
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>> I don't see a port specified. I'd assume that's specified in
>>>>>>> /usr/local/etc/couchdb/couch_httpd.conf. You might check if
>>>>>>> you have
>>> a
>>>>>>> conflicting BindAddress in there as well.
>>>>>>>
>>>>>>> For reference, my couch.ini looks like this:
>>>>>>> http://pastie.caboo.se/188178
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>> The other servers are reachable from an external host, right?
>>>>>
>>>>> Try commenting out the HttpConf. I've never used that and don't
>>>>> have
>>>>> much of an idea on what the side effects might be. Could it be
>>>>> that
>>>>> having the BindAddress and Port specified multiple times is
>>>>> causing
>>>>> issues?
>>>>>
>>>>> Paul
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>
>
Re: firewall
Posted by Tommy Chheng <tc...@uci.edu>.
Ok, previously I checked out from the google code source.
Still can't access from external and all iptables rules are turned off.
I tried both
BindAddress=127.0.0.1
BindAddress=0.0.0.0
There's no response from the outside.
couch 0.7.3a652071 (LogLevel=info)
Apache CouchDB is starting.
Apache CouchDB has started. Time to relax.
Only when I access from the same machine do i get info from the
couchdb output:
[info] [<0.54.0>] 127.0.0.1 - - "GET /" 200
Any thoughts in where to start to debug?
-
Tommy
On Apr 28, 2008, at 2:23 PM, Paul Davis wrote:
> I might also point out that judging from the version number you
> checked out of SVN before the move to apache. I'd check out a fresh
> copy from svn.apache.org and see if that fixes things.
>
> Paul
>
> On Mon, Apr 28, 2008 at 5:18 PM, Jan Lehnardt <ja...@apache.org> wrote:
>> With trunk you don't need the couch_httpd.ini anymore. It
>> was all merged into couch.ini
>>
>> Cheers
>> Jan
>> --
>>
>>
>> On Apr 28, 2008, at 22:11, Tommy Chheng wrote:
>>
>>> Apparently the httpd_conf is needed :(
>>>
>>> httpd_conf: Error while reading config file: httpd_conf: Cannot open
>> couch_httpd.conf
>>>
>>>
>>>
>>> On Apr 28, 2008, at 12:04 PM, Paul Davis wrote:
>>>
>>>
>>>> On Mon, Apr 28, 2008 at 2:57 PM, Tommy Chheng <tc...@uci.edu>
>>>> wrote:
>>>>
>>>>> ok, I added the Port to the couch.ini and changed my
>>>>> couch_http.conf
>> to
>>>>> match Still no luck:
>>>>> my couch_http.conf file
>>>>> http://pastie.caboo.se/188195
>>>>>
>>>>> I'm running nginx and mongrel for rails on this server.. I doubt
>> these can
>>>>> be an issue since they aren't touching 5984 at all and no
>>>>> firewall is
>> on...
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Apr 28, 2008, at 11:38 AM, Paul Davis wrote:
>>>>>
>>>>>
>>>>>
>>>>>> On Mon, Apr 28, 2008 at 2:30 PM, Tommy Chheng <tc...@uci.edu>
>> wrote:
>>>>>>
>>>>>>
>>>>>>> I added BindAddress so my couch.ini file looks like:
>>>>>>> http://pastie.caboo.se/188174
>>>>>>>
>>>>>>> Still same problem.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Apr 28, 2008, at 11:22 AM, Paul Davis wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> On Mon, Apr 28, 2008 at 2:19 PM, Tommy Chheng <tc...@uci.edu>
>> wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> Ok, so i just turned off the firewall temporary
>>>>>>>>>
>>>>>>>>> /sbin/iptables -L
>>>>>>>>> Chain INPUT (policy ACCEPT)
>>>>>>>>> target prot opt source destination
>>>>>>>>>
>>>>>>>>> Chain FORWARD (policy ACCEPT)
>>>>>>>>>
>>>>>>>>> target prot opt source destination
>>>>>>>>>
>>>>>>>>> Chain OUTPUT (policy ACCEPT)
>>>>>>>>>
>>>>>>>>> target prot opt source destination
>>>>>>>>>
>>>>>>>>> Chain RH-Firewall-1-INPUT (0 references)
>>>>>>>>>
>>>>>>>>> target prot opt source destination
>>>>>>>>>
>>>>>>>>> Still no response from outside. It's only working via
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>> localhost:5984.
>>>>>
>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>> Any ideas how to debug this?
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Apr 28, 2008, at 11:12 AM, Paul Davis wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> /sbin/iptables -A OUTPUT -p tcp -m tcp --dport 5984 -j
>> ACCEPT
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>> Ohhhh. Heh.
>>>>>>>>
>>>>>>>> /usr/local/etc/couchdb/couch.ini
>>>>>>>>
>>>>>>>> Change "BindAddresss=127.0.0.1" to "BindAddress=0.0.0.0"
>>>>>>>>
>>>>>>>> Or your local static ip to only listen on that IP.
>>>>>>>>
>>>>>>>> Paul
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> I don't see a port specified. I'd assume that's specified in
>>>>>> /usr/local/etc/couchdb/couch_httpd.conf. You might check if you
>>>>>> have
>> a
>>>>>> conflicting BindAddress in there as well.
>>>>>>
>>>>>> For reference, my couch.ini looks like this:
>>>>>> http://pastie.caboo.se/188178
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>> The other servers are reachable from an external host, right?
>>>>
>>>> Try commenting out the HttpConf. I've never used that and don't
>>>> have
>>>> much of an idea on what the side effects might be. Could it be that
>>>> having the BindAddress and Port specified multiple times is causing
>>>> issues?
>>>>
>>>> Paul
>>>>
>>>>
>>>
>>>
>>>
>>
>>
>
Re: firewall
Posted by Paul Davis <pa...@gmail.com>.
I might also point out that judging from the version number you
checked out of SVN before the move to apache. I'd check out a fresh
copy from svn.apache.org and see if that fixes things.
Paul
On Mon, Apr 28, 2008 at 5:18 PM, Jan Lehnardt <ja...@apache.org> wrote:
> With trunk you don't need the couch_httpd.ini anymore. It
> was all merged into couch.ini
>
> Cheers
> Jan
> --
>
>
> On Apr 28, 2008, at 22:11, Tommy Chheng wrote:
>
> > Apparently the httpd_conf is needed :(
> >
> > httpd_conf: Error while reading config file: httpd_conf: Cannot open
> couch_httpd.conf
> >
> >
> >
> > On Apr 28, 2008, at 12:04 PM, Paul Davis wrote:
> >
> >
> > > On Mon, Apr 28, 2008 at 2:57 PM, Tommy Chheng <tc...@uci.edu> wrote:
> > >
> > > > ok, I added the Port to the couch.ini and changed my couch_http.conf
> to
> > > > match Still no luck:
> > > > my couch_http.conf file
> > > > http://pastie.caboo.se/188195
> > > >
> > > > I'm running nginx and mongrel for rails on this server.. I doubt
> these can
> > > > be an issue since they aren't touching 5984 at all and no firewall is
> on...
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > On Apr 28, 2008, at 11:38 AM, Paul Davis wrote:
> > > >
> > > >
> > > >
> > > > > On Mon, Apr 28, 2008 at 2:30 PM, Tommy Chheng <tc...@uci.edu>
> wrote:
> > > > >
> > > > >
> > > > > > I added BindAddress so my couch.ini file looks like:
> > > > > > http://pastie.caboo.se/188174
> > > > > >
> > > > > > Still same problem.
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Apr 28, 2008, at 11:22 AM, Paul Davis wrote:
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > > On Mon, Apr 28, 2008 at 2:19 PM, Tommy Chheng <tc...@uci.edu>
> wrote:
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > > Ok, so i just turned off the firewall temporary
> > > > > > > >
> > > > > > > > /sbin/iptables -L
> > > > > > > > Chain INPUT (policy ACCEPT)
> > > > > > > > target prot opt source destination
> > > > > > > >
> > > > > > > > Chain FORWARD (policy ACCEPT)
> > > > > > > >
> > > > > > > > target prot opt source destination
> > > > > > > >
> > > > > > > > Chain OUTPUT (policy ACCEPT)
> > > > > > > >
> > > > > > > > target prot opt source destination
> > > > > > > >
> > > > > > > > Chain RH-Firewall-1-INPUT (0 references)
> > > > > > > >
> > > > > > > > target prot opt source destination
> > > > > > > >
> > > > > > > > Still no response from outside. It's only working via
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > > localhost:5984.
> > > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > Any ideas how to debug this?
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > On Apr 28, 2008, at 11:12 AM, Paul Davis wrote:
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > > /sbin/iptables -A OUTPUT -p tcp -m tcp --dport 5984 -j
> ACCEPT
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > > Ohhhh. Heh.
> > > > > > >
> > > > > > > /usr/local/etc/couchdb/couch.ini
> > > > > > >
> > > > > > > Change "BindAddresss=127.0.0.1" to "BindAddress=0.0.0.0"
> > > > > > >
> > > > > > > Or your local static ip to only listen on that IP.
> > > > > > >
> > > > > > > Paul
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > > I don't see a port specified. I'd assume that's specified in
> > > > > /usr/local/etc/couchdb/couch_httpd.conf. You might check if you have
> a
> > > > > conflicting BindAddress in there as well.
> > > > >
> > > > > For reference, my couch.ini looks like this:
> > > > > http://pastie.caboo.se/188178
> > > > >
> > > > >
> > > > >
> > > >
> > > >
> > > >
> > >
> > > The other servers are reachable from an external host, right?
> > >
> > > Try commenting out the HttpConf. I've never used that and don't have
> > > much of an idea on what the side effects might be. Could it be that
> > > having the BindAddress and Port specified multiple times is causing
> > > issues?
> > >
> > > Paul
> > >
> > >
> >
> >
> >
>
>
Re: firewall
Posted by Jan Lehnardt <ja...@apache.org>.
With trunk you don't need the couch_httpd.ini anymore. It
was all merged into couch.ini
Cheers
Jan
--
On Apr 28, 2008, at 22:11, Tommy Chheng wrote:
> Apparently the httpd_conf is needed :(
>
> httpd_conf: Error while reading config file: httpd_conf: Cannot open
> couch_httpd.conf
>
>
>
> On Apr 28, 2008, at 12:04 PM, Paul Davis wrote:
>
>> On Mon, Apr 28, 2008 at 2:57 PM, Tommy Chheng <tc...@uci.edu>
>> wrote:
>>> ok, I added the Port to the couch.ini and changed my
>>> couch_http.conf to
>>> match Still no luck:
>>> my couch_http.conf file
>>> http://pastie.caboo.se/188195
>>>
>>> I'm running nginx and mongrel for rails on this server.. I doubt
>>> these can
>>> be an issue since they aren't touching 5984 at all and no firewall
>>> is on...
>>>
>>>
>>>
>>>
>>>
>>> On Apr 28, 2008, at 11:38 AM, Paul Davis wrote:
>>>
>>>
>>>> On Mon, Apr 28, 2008 at 2:30 PM, Tommy Chheng <tc...@uci.edu>
>>>> wrote:
>>>>
>>>>> I added BindAddress so my couch.ini file looks like:
>>>>> http://pastie.caboo.se/188174
>>>>>
>>>>> Still same problem.
>>>>>
>>>>>
>>>>>
>>>>> On Apr 28, 2008, at 11:22 AM, Paul Davis wrote:
>>>>>
>>>>>
>>>>>
>>>>>> On Mon, Apr 28, 2008 at 2:19 PM, Tommy Chheng <tc...@uci.edu>
>>>>>> wrote:
>>>>>>
>>>>>>
>>>>>>> Ok, so i just turned off the firewall temporary
>>>>>>>
>>>>>>> /sbin/iptables -L
>>>>>>> Chain INPUT (policy ACCEPT)
>>>>>>> target prot opt source destination
>>>>>>>
>>>>>>> Chain FORWARD (policy ACCEPT)
>>>>>>>
>>>>>>> target prot opt source destination
>>>>>>>
>>>>>>> Chain OUTPUT (policy ACCEPT)
>>>>>>>
>>>>>>> target prot opt source destination
>>>>>>>
>>>>>>> Chain RH-Firewall-1-INPUT (0 references)
>>>>>>>
>>>>>>> target prot opt source destination
>>>>>>>
>>>>>>> Still no response from outside. It's only working via
>>> localhost:5984.
>>>>>>>
>>>>>>> Any ideas how to debug this?
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Apr 28, 2008, at 11:12 AM, Paul Davis wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> /sbin/iptables -A OUTPUT -p tcp -m tcp --dport 5984 -j ACCEPT
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> Ohhhh. Heh.
>>>>>>
>>>>>> /usr/local/etc/couchdb/couch.ini
>>>>>>
>>>>>> Change "BindAddresss=127.0.0.1" to "BindAddress=0.0.0.0"
>>>>>>
>>>>>> Or your local static ip to only listen on that IP.
>>>>>>
>>>>>> Paul
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>> I don't see a port specified. I'd assume that's specified in
>>>> /usr/local/etc/couchdb/couch_httpd.conf. You might check if you
>>>> have a
>>>> conflicting BindAddress in there as well.
>>>>
>>>> For reference, my couch.ini looks like this:
>>>> http://pastie.caboo.se/188178
>>>>
>>>>
>>>
>>>
>>
>> The other servers are reachable from an external host, right?
>>
>> Try commenting out the HttpConf. I've never used that and don't have
>> much of an idea on what the side effects might be. Could it be that
>> having the BindAddress and Port specified multiple times is causing
>> issues?
>>
>> Paul
>>
>
>
Re: firewall
Posted by Tommy Chheng <tc...@uci.edu>.
Apparently the httpd_conf is needed :(
httpd_conf: Error while reading config file: httpd_conf: Cannot open
couch_httpd.conf
On Apr 28, 2008, at 12:04 PM, Paul Davis wrote:
> On Mon, Apr 28, 2008 at 2:57 PM, Tommy Chheng <tc...@uci.edu> wrote:
>> ok, I added the Port to the couch.ini and changed my
>> couch_http.conf to
>> match Still no luck:
>> my couch_http.conf file
>> http://pastie.caboo.se/188195
>>
>> I'm running nginx and mongrel for rails on this server.. I doubt
>> these can
>> be an issue since they aren't touching 5984 at all and no firewall
>> is on...
>>
>>
>>
>>
>>
>> On Apr 28, 2008, at 11:38 AM, Paul Davis wrote:
>>
>>
>>> On Mon, Apr 28, 2008 at 2:30 PM, Tommy Chheng <tc...@uci.edu>
>>> wrote:
>>>
>>>> I added BindAddress so my couch.ini file looks like:
>>>> http://pastie.caboo.se/188174
>>>>
>>>> Still same problem.
>>>>
>>>>
>>>>
>>>> On Apr 28, 2008, at 11:22 AM, Paul Davis wrote:
>>>>
>>>>
>>>>
>>>>> On Mon, Apr 28, 2008 at 2:19 PM, Tommy Chheng <tc...@uci.edu>
>>>>> wrote:
>>>>>
>>>>>
>>>>>> Ok, so i just turned off the firewall temporary
>>>>>>
>>>>>> /sbin/iptables -L
>>>>>> Chain INPUT (policy ACCEPT)
>>>>>> target prot opt source destination
>>>>>>
>>>>>> Chain FORWARD (policy ACCEPT)
>>>>>>
>>>>>> target prot opt source destination
>>>>>>
>>>>>> Chain OUTPUT (policy ACCEPT)
>>>>>>
>>>>>> target prot opt source destination
>>>>>>
>>>>>> Chain RH-Firewall-1-INPUT (0 references)
>>>>>>
>>>>>> target prot opt source destination
>>>>>>
>>>>>> Still no response from outside. It's only working via
>> localhost:5984.
>>>>>>
>>>>>> Any ideas how to debug this?
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Apr 28, 2008, at 11:12 AM, Paul Davis wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> /sbin/iptables -A OUTPUT -p tcp -m tcp --dport 5984 -j ACCEPT
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>> Ohhhh. Heh.
>>>>>
>>>>> /usr/local/etc/couchdb/couch.ini
>>>>>
>>>>> Change "BindAddresss=127.0.0.1" to "BindAddress=0.0.0.0"
>>>>>
>>>>> Or your local static ip to only listen on that IP.
>>>>>
>>>>> Paul
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>
>>> I don't see a port specified. I'd assume that's specified in
>>> /usr/local/etc/couchdb/couch_httpd.conf. You might check if you
>>> have a
>>> conflicting BindAddress in there as well.
>>>
>>> For reference, my couch.ini looks like this:
>>> http://pastie.caboo.se/188178
>>>
>>>
>>
>>
>
> The other servers are reachable from an external host, right?
>
> Try commenting out the HttpConf. I've never used that and don't have
> much of an idea on what the side effects might be. Could it be that
> having the BindAddress and Port specified multiple times is causing
> issues?
>
> Paul
>
Re: firewall
Posted by Paul Davis <pa...@gmail.com>.
On Mon, Apr 28, 2008 at 2:57 PM, Tommy Chheng <tc...@uci.edu> wrote:
> ok, I added the Port to the couch.ini and changed my couch_http.conf to
> match Still no luck:
> my couch_http.conf file
> http://pastie.caboo.se/188195
>
> I'm running nginx and mongrel for rails on this server.. I doubt these can
> be an issue since they aren't touching 5984 at all and no firewall is on...
>
>
>
>
>
> On Apr 28, 2008, at 11:38 AM, Paul Davis wrote:
>
>
> > On Mon, Apr 28, 2008 at 2:30 PM, Tommy Chheng <tc...@uci.edu> wrote:
> >
> > > I added BindAddress so my couch.ini file looks like:
> > > http://pastie.caboo.se/188174
> > >
> > > Still same problem.
> > >
> > >
> > >
> > > On Apr 28, 2008, at 11:22 AM, Paul Davis wrote:
> > >
> > >
> > >
> > > > On Mon, Apr 28, 2008 at 2:19 PM, Tommy Chheng <tc...@uci.edu> wrote:
> > > >
> > > >
> > > > > Ok, so i just turned off the firewall temporary
> > > > >
> > > > > /sbin/iptables -L
> > > > > Chain INPUT (policy ACCEPT)
> > > > > target prot opt source destination
> > > > >
> > > > > Chain FORWARD (policy ACCEPT)
> > > > >
> > > > > target prot opt source destination
> > > > >
> > > > > Chain OUTPUT (policy ACCEPT)
> > > > >
> > > > > target prot opt source destination
> > > > >
> > > > > Chain RH-Firewall-1-INPUT (0 references)
> > > > >
> > > > > target prot opt source destination
> > > > >
> > > > > Still no response from outside. It's only working via
> localhost:5984.
> > > > >
> > > > > Any ideas how to debug this?
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > On Apr 28, 2008, at 11:12 AM, Paul Davis wrote:
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > > /sbin/iptables -A OUTPUT -p tcp -m tcp --dport 5984 -j ACCEPT
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > >
> > > > Ohhhh. Heh.
> > > >
> > > > /usr/local/etc/couchdb/couch.ini
> > > >
> > > > Change "BindAddresss=127.0.0.1" to "BindAddress=0.0.0.0"
> > > >
> > > > Or your local static ip to only listen on that IP.
> > > >
> > > > Paul
> > > >
> > > >
> > > >
> > >
> > >
> > >
> >
> > I don't see a port specified. I'd assume that's specified in
> > /usr/local/etc/couchdb/couch_httpd.conf. You might check if you have a
> > conflicting BindAddress in there as well.
> >
> > For reference, my couch.ini looks like this:
> > http://pastie.caboo.se/188178
> >
> >
>
>
The other servers are reachable from an external host, right?
Try commenting out the HttpConf. I've never used that and don't have
much of an idea on what the side effects might be. Could it be that
having the BindAddress and Port specified multiple times is causing
issues?
Paul
Re: firewall
Posted by Tommy Chheng <tc...@uci.edu>.
ok, I added the Port to the couch.ini and changed my couch_http.conf
to match Still no luck:
my couch_http.conf file
http://pastie.caboo.se/188195
I'm running nginx and mongrel for rails on this server.. I doubt
these can be an issue since they aren't touching 5984 at all and no
firewall is on...
On Apr 28, 2008, at 11:38 AM, Paul Davis wrote:
> On Mon, Apr 28, 2008 at 2:30 PM, Tommy Chheng <tc...@uci.edu> wrote:
>> I added BindAddress so my couch.ini file looks like:
>> http://pastie.caboo.se/188174
>>
>> Still same problem.
>>
>>
>>
>> On Apr 28, 2008, at 11:22 AM, Paul Davis wrote:
>>
>>
>>> On Mon, Apr 28, 2008 at 2:19 PM, Tommy Chheng <tc...@uci.edu>
>>> wrote:
>>>
>>>> Ok, so i just turned off the firewall temporary
>>>>
>>>> /sbin/iptables -L
>>>> Chain INPUT (policy ACCEPT)
>>>> target prot opt source destination
>>>>
>>>> Chain FORWARD (policy ACCEPT)
>>>>
>>>> target prot opt source destination
>>>>
>>>> Chain OUTPUT (policy ACCEPT)
>>>>
>>>> target prot opt source destination
>>>>
>>>> Chain RH-Firewall-1-INPUT (0 references)
>>>>
>>>> target prot opt source destination
>>>>
>>>> Still no response from outside. It's only working via localhost:
>>>> 5984.
>>>>
>>>> Any ideas how to debug this?
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Apr 28, 2008, at 11:12 AM, Paul Davis wrote:
>>>>
>>>>
>>>>
>>>>>
>>>>>
>>>>>> /sbin/iptables -A OUTPUT -p tcp -m tcp --dport 5984 -j ACCEPT
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>
>>> Ohhhh. Heh.
>>>
>>> /usr/local/etc/couchdb/couch.ini
>>>
>>> Change "BindAddresss=127.0.0.1" to "BindAddress=0.0.0.0"
>>>
>>> Or your local static ip to only listen on that IP.
>>>
>>> Paul
>>>
>>>
>>
>>
>
> I don't see a port specified. I'd assume that's specified in
> /usr/local/etc/couchdb/couch_httpd.conf. You might check if you have a
> conflicting BindAddress in there as well.
>
> For reference, my couch.ini looks like this:
> http://pastie.caboo.se/188178
>
Re: firewall
Posted by Paul Davis <pa...@gmail.com>.
On Mon, Apr 28, 2008 at 2:30 PM, Tommy Chheng <tc...@uci.edu> wrote:
> I added BindAddress so my couch.ini file looks like:
> http://pastie.caboo.se/188174
>
> Still same problem.
>
>
>
> On Apr 28, 2008, at 11:22 AM, Paul Davis wrote:
>
>
> > On Mon, Apr 28, 2008 at 2:19 PM, Tommy Chheng <tc...@uci.edu> wrote:
> >
> > > Ok, so i just turned off the firewall temporary
> > >
> > > /sbin/iptables -L
> > > Chain INPUT (policy ACCEPT)
> > > target prot opt source destination
> > >
> > > Chain FORWARD (policy ACCEPT)
> > >
> > > target prot opt source destination
> > >
> > > Chain OUTPUT (policy ACCEPT)
> > >
> > > target prot opt source destination
> > >
> > > Chain RH-Firewall-1-INPUT (0 references)
> > >
> > > target prot opt source destination
> > >
> > > Still no response from outside. It's only working via localhost:5984.
> > >
> > > Any ideas how to debug this?
> > >
> > >
> > >
> > >
> > >
> > > On Apr 28, 2008, at 11:12 AM, Paul Davis wrote:
> > >
> > >
> > >
> > > >
> > > >
> > > > > /sbin/iptables -A OUTPUT -p tcp -m tcp --dport 5984 -j ACCEPT
> > > > >
> > > > >
> > > >
> > > >
> > >
> > >
> > >
> >
> > Ohhhh. Heh.
> >
> > /usr/local/etc/couchdb/couch.ini
> >
> > Change "BindAddresss=127.0.0.1" to "BindAddress=0.0.0.0"
> >
> > Or your local static ip to only listen on that IP.
> >
> > Paul
> >
> >
>
>
I don't see a port specified. I'd assume that's specified in
/usr/local/etc/couchdb/couch_httpd.conf. You might check if you have a
conflicting BindAddress in there as well.
For reference, my couch.ini looks like this:
http://pastie.caboo.se/188178
Re: firewall
Posted by Tommy Chheng <tc...@uci.edu>.
I added BindAddress so my couch.ini file looks like:
http://pastie.caboo.se/188174
Still same problem.
On Apr 28, 2008, at 11:22 AM, Paul Davis wrote:
> On Mon, Apr 28, 2008 at 2:19 PM, Tommy Chheng <tc...@uci.edu> wrote:
>> Ok, so i just turned off the firewall temporary
>>
>> /sbin/iptables -L
>> Chain INPUT (policy ACCEPT)
>> target prot opt source destination
>>
>> Chain FORWARD (policy ACCEPT)
>>
>> target prot opt source destination
>>
>> Chain OUTPUT (policy ACCEPT)
>>
>> target prot opt source destination
>>
>> Chain RH-Firewall-1-INPUT (0 references)
>>
>> target prot opt source destination
>>
>> Still no response from outside. It's only working via localhost:5984.
>>
>> Any ideas how to debug this?
>>
>>
>>
>>
>>
>> On Apr 28, 2008, at 11:12 AM, Paul Davis wrote:
>>
>>
>>>
>>>> /sbin/iptables -A OUTPUT -p tcp -m tcp --dport 5984 -j ACCEPT
>>>>
>>>
>>
>>
>
> Ohhhh. Heh.
>
> /usr/local/etc/couchdb/couch.ini
>
> Change "BindAddresss=127.0.0.1" to "BindAddress=0.0.0.0"
>
> Or your local static ip to only listen on that IP.
>
> Paul
>
Re: firewall
Posted by Paul Davis <pa...@gmail.com>.
On Mon, Apr 28, 2008 at 2:19 PM, Tommy Chheng <tc...@uci.edu> wrote:
> Ok, so i just turned off the firewall temporary
>
> /sbin/iptables -L
> Chain INPUT (policy ACCEPT)
> target prot opt source destination
>
> Chain FORWARD (policy ACCEPT)
>
> target prot opt source destination
>
> Chain OUTPUT (policy ACCEPT)
>
> target prot opt source destination
>
> Chain RH-Firewall-1-INPUT (0 references)
>
> target prot opt source destination
>
> Still no response from outside. It's only working via localhost:5984.
>
> Any ideas how to debug this?
>
>
>
>
>
> On Apr 28, 2008, at 11:12 AM, Paul Davis wrote:
>
>
> >
> > > /sbin/iptables -A OUTPUT -p tcp -m tcp --dport 5984 -j ACCEPT
> > >
> >
>
>
Ohhhh. Heh.
/usr/local/etc/couchdb/couch.ini
Change "BindAddresss=127.0.0.1" to "BindAddress=0.0.0.0"
Or your local static ip to only listen on that IP.
Paul
Re: firewall
Posted by Tommy Chheng <tc...@uci.edu>.
Ok, so i just turned off the firewall temporary
/sbin/iptables -L
Chain INPUT (policy ACCEPT)
target prot opt source destination
Chain FORWARD (policy ACCEPT)
target prot opt source destination
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
Chain RH-Firewall-1-INPUT (0 references)
target prot opt source destination
Still no response from outside. It's only working via localhost:5984.
Any ideas how to debug this?
On Apr 28, 2008, at 11:12 AM, Paul Davis wrote:
>> /sbin/iptables -A OUTPUT -p tcp -m tcp --dport 5984 -j ACCEPT
Re: firewall
Posted by Paul Davis <pa...@gmail.com>.
On Mon, Apr 28, 2008 at 2:03 PM, Tommy Chheng <tc...@uci.edu> wrote:
> Hi,
> I'm trying to get CouchDB to work as a server. I got it install and
> everything works from the same machine, ie, wget localhost:5984 gives a
> response.
>
> but i want to access this server over the internet. I set an allow in
> iptables:
> /sbin/iptables -A INPUT -p tcp -m tcp --sport 5984 -j ACCEPT
> /sbin/iptables -A OUTPUT -p tcp -m tcp --dport 5984 -j ACCEPT
>
> /sbin/iptables -L
> Chain INPUT (policy ACCEPT)
> target prot opt source destination
> RH-Firewall-1-INPUT all -- anywhere anywhere
> RH-Firewall-1-INPUT all -- anywhere anywhere
> ACCEPT tcp -- anywhere anywhere tcp dpt:5984
> ACCEPT tcp -- anywhere anywhere tcp spt:5984
>
>
> I'm able to connect to my webserver on this same machine fine. Just no
> response on 5984.
>
> Any ideas what's wrong?
>
> thanks,
> tommy
I think this is your problem:
-A OUTPUT -p tcp -m tcp --dport 5984 -j ACCEPT
Try chaning the --dport to --sport. Or, also, you could just not
filter outbound traffic. Unless for some reason you don't trust code
running behind your firewall.
HTH,
Paul
firewall
Posted by Tommy Chheng <tc...@uci.edu>.
Hi,
I'm trying to get CouchDB to work as a server. I got it install and
everything works from the same machine, ie, wget localhost:5984 gives
a response.
but i want to access this server over the internet. I set an allow in
iptables:
/sbin/iptables -A INPUT -p tcp -m tcp --sport 5984 -j ACCEPT
/sbin/iptables -A OUTPUT -p tcp -m tcp --dport 5984 -j ACCEPT
/sbin/iptables -L
Chain INPUT (policy ACCEPT)
target prot opt source destination
RH-Firewall-1-INPUT all -- anywhere anywhere
RH-Firewall-1-INPUT all -- anywhere anywhere
ACCEPT tcp -- anywhere anywhere tcp dpt:
5984
ACCEPT tcp -- anywhere anywhere tcp spt:
5984
I'm able to connect to my webserver on this same machine fine. Just no
response on 5984.
Any ideas what's wrong?
thanks,
tommy
Re: Updating views on save
Posted by Guby <gu...@gmail.com>.
Now this morning there are 42 of them... crazy...
I don't think they receive any updates from the server.
They are not zoombie processes either, and clearly still belong to
couchdb, because when I quit couchDB the processes ended too!
But it seems it somehow starts a lot of new processes for some reason!
I actually remember that was sometimes the case on my local
development machine as well using the shell script you sent me!
Best regards
Sebastian
On Apr 30, 2008, at 8:08 AM, Jan Lehnardt wrote:
>
> On Apr 29, 2008, at 23:13, Guby wrote:
>> Hm...
>> On my server I just noticed CouchDB had started 14 instances of the
>> DbUpdateNotificationProcess!
>> Is this normal, and in case, why so many? And why do they all keep
>> running when the CouchDB is somewhat idle and doesn't need them at
>> all?
>
> CouchDB should only start one instance. It will kill and spawn a new
> one in case of a problem. Maybe you have old instances of the
> daemon that decided not to die and CouchDB happily spawned
> fresh copies. What's the state of all the processes? Do they all
> receive updates?
>
> Cheers
> Jan
> --
>
>>
>>
>> Best regards
>> Sebastian
>>
>>
>>
>> On Apr 28, 2008, at 1:55 PM, Guby wrote:
>>
>>> Added to wiki: http://wiki.apache.org/couchdb/RegeneratingViewsOnUpdate
>>> Edit as you see fit.
>>>
>>> S
>>>
>>>
>>> On Apr 28, 2008, at 1:14 PM, Guby wrote:
>>>
>>>> Sure!
>>>> That makes more sense than having it here in the mailinglist :)
>>>>
>>>> S
>>>>
>>>> On Apr 28, 2008, at 12:58 PM, Jan Lehnardt wrote:
>>>>
>>>>>
>>>>> On Apr 28, 2008, at 17:18, Guby wrote:
>>>>>> I am very happy with the way this discussion turned out!
>>>>>> Now I have written a tiny ruby script that updates the views
>>>>>> for every tenth document update and at most once every second
>>>>>> under heavy updates.
>>>>>> Here on my development machine it uses about 1.2 mb of ram, and
>>>>>> I can live with that.
>>>>>>
>>>>>> I'll include it here in case somebody reads this discussion in
>>>>>> the future and needs a similar script.
>>>>>
>>>>> Would you mind putting it on the Wiki? For the obvious benefits :)
>>>>>
>>>>> Good work there,
>>>>>
>>>>> Cheers
>>>>> Jan
>>>>> --
>>>>>
>>>>>
>>>>>>
>>>>>>
>>>>>> Thanks again for all the help and for a great database!
>>>>>>
>>>>>> Best regards
>>>>>> Sebastian
>>>>>>
>>>>>>
>>>>>> **********************************
>>>>>>
>>>>>> CONFIG: couch.ini:
>>>>>> add line:
>>>>>> DbUpdateNotificationProcess=/PATH/TO/view_updater.rb
>>>>>>
>>>>>> **********************************
>>>>>>
>>>>>> SCRIPT: view_updater.rb
>>>>>>
>>>>>> #!/usr/bin/ruby
>>>>>>
>>>>>> ###
>>>>>> # CONF
>>>>>> ###
>>>>>>
>>>>>> # The smallest amount of changed documents before the views are
>>>>>> updated
>>>>>> MIN_NUM_OF_CHANGED_DOCS = 10
>>>>>>
>>>>>> # URL to the DB on the CouchDB server
>>>>>> # Only supports one database at the moment as that is what I need
>>>>>> # Should be easy to improve for several databases if needed
>>>>>> URL = "http://localhost:5984/kleio"
>>>>>>
>>>>>> # Set the minimum pause between calls to the database
>>>>>> PAUSE = 1 # seconds
>>>>>>
>>>>>> # One for each design document
>>>>>> VIEWS = ["feed/to_check",
>>>>>> "feed_entries/list_for_user_by_feed",
>>>>>> "subscription/number_of_subscriptions",
>>>>>> "subscription_requests/request",
>>>>>> "user/by_email",
>>>>>> "couch_object_has_many_relations/related_documents"]
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> ###
>>>>>> # "MAIN LOOP"
>>>>>> ###
>>>>>>
>>>>>> run = true
>>>>>> number_of_changed_docs = 0
>>>>>>
>>>>>> threads = []
>>>>>>
>>>>>> # Updates the views
>>>>>> threads << Thread.new do
>>>>>>
>>>>>> while run do
>>>>>>
>>>>>> if number_of_changed_docs >= MIN_NUM_OF_CHANGED_DOCS
>>>>>>
>>>>>> number_of_changed_docs = 0
>>>>>>
>>>>>> VIEWS.each do |view|
>>>>>> `curl #{URL}/_view/#{view}?count=0`
>>>>>> end
>>>>>>
>>>>>> end
>>>>>>
>>>>>> sleep PAUSE
>>>>>>
>>>>>> end
>>>>>>
>>>>>> end
>>>>>>
>>>>>> # Receives the update notification from CouchDB
>>>>>> threads << Thread.new do
>>>>>>
>>>>>> while run do
>>>>>>
>>>>>> update_call = gets
>>>>>>
>>>>>> # When CouchDB exits the script gets called with
>>>>>> # a never ending series of nil
>>>>>> if update_call == ni
Re: Updating views on save
Posted by Jan Lehnardt <ja...@apache.org>.
On Apr 29, 2008, at 23:13, Guby wrote:
> Hm...
> On my server I just noticed CouchDB had started 14 instances of the
> DbUpdateNotificationProcess!
> Is this normal, and in case, why so many? And why do they all keep
> running when the CouchDB is somewhat idle and doesn't need them at
> all?
CouchDB should only start one instance. It will kill and spawn a new
one in case of a problem. Maybe you have old instances of the
daemon that decided not to die and CouchDB happily spawned
fresh copies. What's the state of all the processes? Do they all
receive updates?
Cheers
Jan
--
>
>
> Best regards
> Sebastian
>
>
>
> On Apr 28, 2008, at 1:55 PM, Guby wrote:
>
>> Added to wiki: http://wiki.apache.org/couchdb/RegeneratingViewsOnUpdate
>> Edit as you see fit.
>>
>> S
>>
>>
>> On Apr 28, 2008, at 1:14 PM, Guby wrote:
>>
>>> Sure!
>>> That makes more sense than having it here in the mailinglist :)
>>>
>>> S
>>>
>>> On Apr 28, 2008, at 12:58 PM, Jan Lehnardt wrote:
>>>
>>>>
>>>> On Apr 28, 2008, at 17:18, Guby wrote:
>>>>> I am very happy with the way this discussion turned out!
>>>>> Now I have written a tiny ruby script that updates the views for
>>>>> every tenth document update and at most once every second under
>>>>> heavy updates.
>>>>> Here on my development machine it uses about 1.2 mb of ram, and
>>>>> I can live with that.
>>>>>
>>>>> I'll include it here in case somebody reads this discussion in
>>>>> the future and needs a similar script.
>>>>
>>>> Would you mind putting it on the Wiki? For the obvious benefits :)
>>>>
>>>> Good work there,
>>>>
>>>> Cheers
>>>> Jan
>>>> --
>>>>
>>>>
>>>>>
>>>>>
>>>>> Thanks again for all the help and for a great database!
>>>>>
>>>>> Best regards
>>>>> Sebastian
>>>>>
>>>>>
>>>>> **********************************
>>>>>
>>>>> CONFIG: couch.ini:
>>>>> add line:
>>>>> DbUpdateNotificationProcess=/PATH/TO/view_updater.rb
>>>>>
>>>>> **********************************
>>>>>
>>>>> SCRIPT: view_updater.rb
>>>>>
>>>>> #!/usr/bin/ruby
>>>>>
>>>>> ###
>>>>> # CONF
>>>>> ###
>>>>>
>>>>> # The smallest amount of changed documents before the views are
>>>>> updated
>>>>> MIN_NUM_OF_CHANGED_DOCS = 10
>>>>>
>>>>> # URL to the DB on the CouchDB server
>>>>> # Only supports one database at the moment as that is what I need
>>>>> # Should be easy to improve for several databases if needed
>>>>> URL = "http://localhost:5984/kleio"
>>>>>
>>>>> # Set the minimum pause between calls to the database
>>>>> PAUSE = 1 # seconds
>>>>>
>>>>> # One for each design document
>>>>> VIEWS = ["feed/to_check",
>>>>> "feed_entries/list_for_user_by_feed",
>>>>> "subscription/number_of_subscriptions",
>>>>> "subscription_requests/request",
>>>>> "user/by_email",
>>>>> "couch_object_has_many_relations/related_documents"]
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> ###
>>>>> # "MAIN LOOP"
>>>>> ###
>>>>>
>>>>> run = true
>>>>> number_of_changed_docs = 0
>>>>>
>>>>> threads = []
>>>>>
>>>>> # Updates the views
>>>>> threads << Thread.new do
>>>>>
>>>>> while run do
>>>>>
>>>>> if number_of_changed_docs >= MIN_NUM_OF_CHANGED_DOCS
>>>>>
>>>>> number_of_changed_docs = 0
>>>>>
>>>>> VIEWS.each do |view|
>>>>> `curl #{URL}/_view/#{view}?count=0`
>>>>> end
>>>>>
>>>>> end
>>>>>
>>>>> sleep PAUSE
>>>>>
>>>>> end
>>>>>
>>>>> end
>>>>>
>>>>> # Receives the update notification from CouchDB
>>>>> threads << Thread.new do
>>>>>
>>>>> while run do
>>>>>
>>>>> update_call = gets
>>>>>
>>>>> # When CouchDB exits the script gets called with
>>>>> # a never ending series of nil
>>>>> if update_call == nil
>>>>> run = false
>>>>> else
>>>>> number_of_changed_docs += 1
>>>>> end
>>>>>
>>>>> end
>>>>>
>>>>> end
>>>>>
>>>>> # Good bye
>>>>> threads.each {|thr| thr.join}
>>>>>
>>>>>
>>>>
>>>
>>
>
Re: Updating views on save
Posted by Guby <gu...@gmail.com>.
Hm...
On my server I just noticed CouchDB had started 14 instances of the
DbUpdateNotificationProcess!
Is this normal, and in case, why so many? And why do they all keep
running when the CouchDB is somewhat idle and doesn't need them at all?
Best regards
Sebastian
On Apr 28, 2008, at 1:55 PM, Guby wrote:
> Added to wiki: http://wiki.apache.org/couchdb/
> RegeneratingViewsOnUpdate
> Edit as you see fit.
>
> S
>
>
> On Apr 28, 2008, at 1:14 PM, Guby wrote:
>
>> Sure!
>> That makes more sense than having it here in the mailinglist :)
>>
>> S
>>
>> On Apr 28, 2008, at 12:58 PM, Jan Lehnardt wrote:
>>
>>>
>>> On Apr 28, 2008, at 17:18, Guby wrote:
>>>> I am very happy with the way this discussion turned out!
>>>> Now I have written a tiny ruby script that updates the views for
>>>> every tenth document update and at most once every second under
>>>> heavy updates.
>>>> Here on my development machine it uses about 1.2 mb of ram, and I
>>>> can live with that.
>>>>
>>>> I'll include it here in case somebody reads this discussion in
>>>> the future and needs a similar script.
>>>
>>> Would you mind putting it on the Wiki? For the obvious benefits :)
>>>
>>> Good work there,
>>>
>>> Cheers
>>> Jan
>>> --
>>>
>>>
>>>>
>>>>
>>>> Thanks again for all the help and for a great database!
>>>>
>>>> Best regards
>>>> Sebastian
>>>>
>>>>
>>>> **********************************
>>>>
>>>> CONFIG: couch.ini:
>>>> add line:
>>>> DbUpdateNotificationProcess=/PATH/TO/view_updater.rb
>>>>
>>>> **********************************
>>>>
>>>> SCRIPT: view_updater.rb
>>>>
>>>> #!/usr/bin/ruby
>>>>
>>>> ###
>>>> # CONF
>>>> ###
>>>>
>>>> # The smallest amount of changed documents before the views are
>>>> updated
>>>> MIN_NUM_OF_CHANGED_DOCS = 10
>>>>
>>>> # URL to the DB on the CouchDB server
>>>> # Only supports one database at the moment as that is what I need
>>>> # Should be easy to improve for several databases if needed
>>>> URL = "http://localhost:5984/kleio"
>>>>
>>>> # Set the minimum pause between calls to the database
>>>> PAUSE = 1 # seconds
>>>>
>>>> # One for each design document
>>>> VIEWS = ["feed/to_check",
>>>> "feed_entries/list_for_user_by_feed",
>>>> "subscription/number_of_subscriptions",
>>>> "subscription_requests/request",
>>>> "user/by_email",
>>>> "couch_object_has_many_relations/related_documents"]
>>>>
>>>>
>>>>
>>>>
>>>> ###
>>>> # "MAIN LOOP"
>>>> ###
>>>>
>>>> run = true
>>>> number_of_changed_docs = 0
>>>>
>>>> threads = []
>>>>
>>>> # Updates the views
>>>> threads << Thread.new do
>>>>
>>>> while run do
>>>>
>>>> if number_of_changed_docs >= MIN_NUM_OF_CHANGED_DOCS
>>>>
>>>> number_of_changed_docs = 0
>>>>
>>>> VIEWS.each do |view|
>>>> `curl #{URL}/_view/#{view}?count=0`
>>>> end
>>>>
>>>> end
>>>>
>>>> sleep PAUSE
>>>>
>>>> end
>>>>
>>>> end
>>>>
>>>> # Receives the update notification from CouchDB
>>>> threads << Thread.new do
>>>>
>>>> while run do
>>>>
>>>> update_call = gets
>>>>
>>>> # When CouchDB exits the script gets called with
>>>> # a never ending series of nil
>>>> if update_call == nil
>>>> run = false
>>>> else
>>>> number_of_changed_docs += 1
>>>> end
>>>>
>>>> end
>>>>
>>>> end
>>>>
>>>> # Good bye
>>>> threads.each {|thr| thr.join}
>>>>
>>>>
>>>
>>
>
Re: Updating views on save
Posted by Guby <gu...@gmail.com>.
Added to wiki: http://wiki.apache.org/couchdb/RegeneratingViewsOnUpdate
Edit as you see fit.
S
On Apr 28, 2008, at 1:14 PM, Guby wrote:
> Sure!
> That makes more sense than having it here in the mailinglist :)
>
> S
>
> On Apr 28, 2008, at 12:58 PM, Jan Lehnardt wrote:
>
>>
>> On Apr 28, 2008, at 17:18, Guby wrote:
>>> I am very happy with the way this discussion turned out!
>>> Now I have written a tiny ruby script that updates the views for
>>> every tenth document update and at most once every second under
>>> heavy updates.
>>> Here on my development machine it uses about 1.2 mb of ram, and I
>>> can live with that.
>>>
>>> I'll include it here in case somebody reads this discussion in the
>>> future and needs a similar script.
>>
>> Would you mind putting it on the Wiki? For the obvious benefits :)
>>
>> Good work there,
>>
>> Cheers
>> Jan
>> --
>>
>>
>>>
>>>
>>> Thanks again for all the help and for a great database!
>>>
>>> Best regards
>>> Sebastian
>>>
>>>
>>> **********************************
>>>
>>> CONFIG: couch.ini:
>>> add line:
>>> DbUpdateNotificationProcess=/PATH/TO/view_updater.rb
>>>
>>> **********************************
>>>
>>> SCRIPT: view_updater.rb
>>>
>>> #!/usr/bin/ruby
>>>
>>> ###
>>> # CONF
>>> ###
>>>
>>> # The smallest amount of changed documents before the views are
>>> updated
>>> MIN_NUM_OF_CHANGED_DOCS = 10
>>>
>>> # URL to the DB on the CouchDB server
>>> # Only supports one database at the moment as that is what I need
>>> # Should be easy to improve for several databases if needed
>>> URL = "http://localhost:5984/kleio"
>>>
>>> # Set the minimum pause between calls to the database
>>> PAUSE = 1 # seconds
>>>
>>> # One for each design document
>>> VIEWS = ["feed/to_check",
>>> "feed_entries/list_for_user_by_feed",
>>> "subscription/number_of_subscriptions",
>>> "subscription_requests/request",
>>> "user/by_email",
>>> "couch_object_has_many_relations/related_documents"]
>>>
>>>
>>>
>>>
>>> ###
>>> # "MAIN LOOP"
>>> ###
>>>
>>> run = true
>>> number_of_changed_docs = 0
>>>
>>> threads = []
>>>
>>> # Updates the views
>>> threads << Thread.new do
>>>
>>> while run do
>>>
>>> if number_of_changed_docs >= MIN_NUM_OF_CHANGED_DOCS
>>>
>>> number_of_changed_docs = 0
>>>
>>> VIEWS.each do |view|
>>> `curl #{URL}/_view/#{view}?count=0`
>>> end
>>>
>>> end
>>>
>>> sleep PAUSE
>>>
>>> end
>>>
>>> end
>>>
>>> # Receives the update notification from CouchDB
>>> threads << Thread.new do
>>>
>>> while run do
>>>
>>> update_call = gets
>>>
>>> # When CouchDB exits the script gets called with
>>> # a never ending series of nil
>>> if update_call == nil
>>> run = false
>>> else
>>> number_of_changed_docs += 1
>>> end
>>>
>>> end
>>>
>>> end
>>>
>>> # Good bye
>>> threads.each {|thr| thr.join}
>>>
>>>
>>
>
Re: Updating views on save
Posted by Guby <gu...@gmail.com>.
Sure!
That makes more sense than having it here in the mailinglist :)
S
On Apr 28, 2008, at 12:58 PM, Jan Lehnardt wrote:
>
> On Apr 28, 2008, at 17:18, Guby wrote:
>> I am very happy with the way this discussion turned out!
>> Now I have written a tiny ruby script that updates the views for
>> every tenth document update and at most once every second under
>> heavy updates.
>> Here on my development machine it uses about 1.2 mb of ram, and I
>> can live with that.
>>
>> I'll include it here in case somebody reads this discussion in the
>> future and needs a similar script.
>
> Would you mind putting it on the Wiki? For the obvious benefits :)
>
> Good work there,
>
> Cheers
> Jan
> --
>
>
>>
>>
>> Thanks again for all the help and for a great database!
>>
>> Best regards
>> Sebastian
>>
>>
>> **********************************
>>
>> CONFIG: couch.ini:
>> add line:
>> DbUpdateNotificationProcess=/PATH/TO/view_updater.rb
>>
>> **********************************
>>
>> SCRIPT: view_updater.rb
>>
>> #!/usr/bin/ruby
>>
>> ###
>> # CONF
>> ###
>>
>> # The smallest amount of changed documents before the views are
>> updated
>> MIN_NUM_OF_CHANGED_DOCS = 10
>>
>> # URL to the DB on the CouchDB server
>> # Only supports one database at the moment as that is what I need
>> # Should be easy to improve for several databases if needed
>> URL = "http://localhost:5984/kleio"
>>
>> # Set the minimum pause between calls to the database
>> PAUSE = 1 # seconds
>>
>> # One for each design document
>> VIEWS = ["feed/to_check",
>> "feed_entries/list_for_user_by_feed",
>> "subscription/number_of_subscriptions",
>> "subscription_requests/request",
>> "user/by_email",
>> "couch_object_has_many_relations/related_documents"]
>>
>>
>>
>>
>> ###
>> # "MAIN LOOP"
>> ###
>>
>> run = true
>> number_of_changed_docs = 0
>>
>> threads = []
>>
>> # Updates the views
>> threads << Thread.new do
>>
>> while run do
>>
>> if number_of_changed_docs >= MIN_NUM_OF_CHANGED_DOCS
>>
>> number_of_changed_docs = 0
>>
>> VIEWS.each do |view|
>> `curl #{URL}/_view/#{view}?count=0`
>> end
>>
>> end
>>
>> sleep PAUSE
>>
>> end
>>
>> end
>>
>> # Receives the update notification from CouchDB
>> threads << Thread.new do
>>
>> while run do
>>
>> update_call = gets
>>
>> # When CouchDB exits the script gets called with
>> # a never ending series of nil
>> if update_call == nil
>> run = false
>> else
>> number_of_changed_docs += 1
>> end
>>
>> end
>>
>> end
>>
>> # Good bye
>> threads.each {|thr| thr.join}
>>
>>
>
Re: Updating views on save
Posted by Jan Lehnardt <ja...@apache.org>.
On Apr 28, 2008, at 17:18, Guby wrote:
> I am very happy with the way this discussion turned out!
> Now I have written a tiny ruby script that updates the views for
> every tenth document update and at most once every second under
> heavy updates.
> Here on my development machine it uses about 1.2 mb of ram, and I
> can live with that.
>
> I'll include it here in case somebody reads this discussion in the
> future and needs a similar script.
Would you mind putting it on the Wiki? For the obvious benefits :)
Good work there,
Cheers
Jan
--
>
>
> Thanks again for all the help and for a great database!
>
> Best regards
> Sebastian
>
>
> **********************************
>
> CONFIG: couch.ini:
> add line:
> DbUpdateNotificationProcess=/PATH/TO/view_updater.rb
>
> **********************************
>
> SCRIPT: view_updater.rb
>
> #!/usr/bin/ruby
>
> ###
> # CONF
> ###
>
> # The smallest amount of changed documents before the views are
> updated
> MIN_NUM_OF_CHANGED_DOCS = 10
>
> # URL to the DB on the CouchDB server
> # Only supports one database at the moment as that is what I need
> # Should be easy to improve for several databases if needed
> URL = "http://localhost:5984/kleio"
>
> # Set the minimum pause between calls to the database
> PAUSE = 1 # seconds
>
> # One for each design document
> VIEWS = ["feed/to_check",
> "feed_entries/list_for_user_by_feed",
> "subscription/number_of_subscriptions",
> "subscription_requests/request",
> "user/by_email",
> "couch_object_has_many_relations/related_documents"]
>
>
>
>
> ###
> # "MAIN LOOP"
> ###
>
> run = true
> number_of_changed_docs = 0
>
> threads = []
>
> # Updates the views
> threads << Thread.new do
>
> while run do
>
> if number_of_changed_docs >= MIN_NUM_OF_CHANGED_DOCS
>
> number_of_changed_docs = 0
>
> VIEWS.each do |view|
> `curl #{URL}/_view/#{view}?count=0`
> end
>
> end
>
> sleep PAUSE
>
> end
>
> end
>
> # Receives the update notification from CouchDB
> threads << Thread.new do
>
> while run do
>
> update_call = gets
>
> # When CouchDB exits the script gets called with
> # a never ending series of nil
> if update_call == nil
> run = false
> else
> number_of_changed_docs += 1
> end
>
> end
>
> end
>
> # Good bye
> threads.each {|thr| thr.join}
>
>
Re: Updating views on save
Posted by Guby <gu...@gmail.com>.
I am very happy with the way this discussion turned out!
Now I have written a tiny ruby script that updates the views for every
tenth document update and at most once every second under heavy updates.
Here on my development machine it uses about 1.2 mb of ram, and I can
live with that.
I'll include it here in case somebody reads this discussion in the
future and needs a similar script.
Thanks again for all the help and for a great database!
Best regards
Sebastian
**********************************
CONFIG: couch.ini:
add line:
DbUpdateNotificationProcess=/PATH/TO/view_updater.rb
**********************************
SCRIPT: view_updater.rb
#!/usr/bin/ruby
###
# CONF
###
# The smallest amount of changed documents before the views are updated
MIN_NUM_OF_CHANGED_DOCS = 10
# URL to the DB on the CouchDB server
# Only supports one database at the moment as that is what I need
# Should be easy to improve for several databases if needed
URL = "http://localhost:5984/kleio"
# Set the minimum pause between calls to the database
PAUSE = 1 # seconds
# One for each design document
VIEWS = ["feed/to_check",
"feed_entries/list_for_user_by_feed",
"subscription/number_of_subscriptions",
"subscription_requests/request",
"user/by_email",
"couch_object_has_many_relations/related_documents"]
###
# "MAIN LOOP"
###
run = true
number_of_changed_docs = 0
threads = []
# Updates the views
threads << Thread.new do
while run do
if number_of_changed_docs >= MIN_NUM_OF_CHANGED_DOCS
number_of_changed_docs = 0
VIEWS.each do |view|
`curl #{URL}/_view/#{view}?count=0`
end
end
sleep PAUSE
end
end
# Receives the update notification from CouchDB
threads << Thread.new do
while run do
update_call = gets
# When CouchDB exits the script gets called with
# a never ending series of nil
if update_call == nil
run = false
else
number_of_changed_docs += 1
end
end
end
# Good bye
threads.each {|thr| thr.join}
Re: Updating views on save
Posted by Guby <gu...@gmail.com>.
> I am wondering, why is there a loop here? Isn't the shell script
> called once every time the database receives an update? That is what
> I can read out of the documentation for the
> DbUpdateNotificationProcess in the wiki...
> Couldn't it just as well be written like this?
>
>> #!/bin/sh
>>
>> read database
>>
>> `curl http://server:5984/$database/_view/one_for_each/view?count=0`
>> `curl http://server:5984/$database/_view/other_view/jadijadd?count=0`
>>
>
> What is it I am not understanding?
A little testing made me understand why it can't be written like this...
The script is called and just runs in the background waiting for input
from the user... so the loop is obviously there to keep it alive...
And the counter to ensure the script only gets called once for every
hundred changed documents... well... now it makes sense to me :)
Best regards
Sebastian
Re: Updating views on save
Posted by Guby <gu...@gmail.com>.
Thank you for all the answers!
Benoit:
good idea, and I believe that is what Jan used here as well? I didn't
know about this before! Great sollution!
Kristopher:
>In the current situation, if I write 10 times and read 100 times,
the index may only be processed once if the 100 reads come after the
10 writes -- in a more real-world situation though, we're only >going
to be updating the index as many times as we write -- but keep in mind
that given the right circumstances, the index only has to be
regenerated once.
Thanks for your comments!
In my case it is actually more like 100 writes for each 10 reads... So
then having to wait for a substantial view update takes quite a bit of
time.
>Also, please realize that the index does not have to be /completely/
regenerated upon reindex -- only the documents that have been added/
modified.
That is why I thought regenerating the view for changed documents on
save wouldn't be that much of a performance hit... but it turns out I
was wrong. I didn't think of low level stuff like the byte layout
mentioned by Jan. In my case updating on save is still going to be the
best solution I believe, because I can't afford waiting long for view
updating when I am requesting the views from the front end.
>Lastly, I would recommend that you try to optimize your view code to
make things faster, as well.
This is also a good idea! Although my view code is really straight
forward. I have read all the wikis as well, but if you guys know of
any "best view practice" resource, or have tips for view optimizations
handy please let me know!
Cortland:
>Not sure why your views are timing out, but from my current
understanding views are incrementally updated with modifications but
only incrementally updated on a call to that view.
My bad for not being clear. The views themselves do not time out. The
web page generation for the end user times out because regenerating
the view takes so much time.
>Are you using the javascript spodermonkey viewserver(default) or
another one? Check the complexity of the view and possibly minimize
the view's complexity.
I am using the standard built in spidermonkey view server. I believe
the views to be pretty clean, but then again I have a lot to learn
about best practices for document based databases!
>I'm not sure, but I think the pattern here is you put views most
likely to be called near to each other in the design document, say
blog summaries view followed by full content view, and have less
>related views in a different design document, say for a list of
authors or a tag list.
Smart approach. I had currently grouped my views by the datatypes they
contain... Different views for feeds in one design documents and views
for feed entries in another and one design document for users and so
on... I'll see how much I can change it around for the better!
Jan:
Thanks for all the comments!
I didn't know about the DbUpdateNotificationProcess functionality
before you and Benoit mentioned it! Just what I need!
> #!/bin/sh
>
> counter=0
> max_docs=100
>
> while true
> do
> read database
>
> counter=`expr $counter + 1`
>
> if [ $counter -ge $max_docs ]; then
> `curl http://server:5984/$database/_view/name?count=0`
> counter=0
> fi
> done
I am wondering, why is there a loop here? Isn't the shell script
called once every time the database receives an update? That is what I
can read out of the documentation for the DbUpdateNotificationProcess
in the wiki...
Couldn't it just as well be written like this?
> #!/bin/sh
>
> read database
>
> `curl http://server:5984/$database/_view/one_for_each/view?count=0`
> `curl http://server:5984/$database/_view/other_view/jadijadd?count=0`
>
What is it I am not understanding?
Thanks for all the answers and your time!
Best regards
Sebastian
On Apr 28, 2008, at 5:22 AM, Jan Lehnardt wrote:
> Heya Sebastian,
> it seems you feel rather strongly about this issue. But that's
> nothing a little engineering can solve for you, read on :)
>
> On Apr 28, 2008, at 01:04, Guby wrote:
>> Hello dear Couchers
>>
>> I understand that the views are indexed the first time they are
>> accessed and as far as I know there is no way to turn on view
>> updating on document save. I really don't understand the reasoning
>> behind this behavior. The advantage of the pre-populated/indexed
>> views are that they are blazingly fast to query and access, but
>> that advantage disappears when the first request after a document
>> update has to regenerate the view first!
>> I am currently building a web app where the background processes
>> perform a lot of writes to the database. The time it takes to write
>> a document is not critical for me. What is critical though is the
>> time it takes to load web pages for the end user that require
>> content from the database.
>> In some situations the background processes add thousands of
>> documents to the database within a short period of time, and when
>> the user tries to access a page after such an update the view
>> querying sometimes takes minutes and as a consequence of that the
>> browser times out... Not a recipe for happy customers...
>>
>> The only solution I can see at the moment is to create a worker
>> that queries the database whenever it is told that there has been a
>> document update, but that seems really stupid and unnecessary. And
>> in my case, running on a smallish VPS, a big waste of resources
>> having an extra working doing something the database itself could
>> just as well have done. It also requires a lot of extra coding
>> notifying the worker whenever I update or create a document
>> throughout my app.
>
> That would be a rather extreme solution. Why not, for
> example, trigger a view update from your document-
> insertion code, every N (N = 10, 30, 60?) seconds?
>
>
>> I am sure you have reasons for having implemented the views the way
>> you have, but I would be really interested to hear why it has been
>> done this way!
>
> 1) To not have a 'write penalty' for all views when
> documents are added. We expect you to have
> quite a few of views and updating all of them on-write
> seems silly. The data is generated when needed,
> saving resources by 2) not clogging them up when
> needed elsewhere and 3) processing large quantities
> of data in batches. and finally 4) The very layout of the
> bytes that make up documents on disk and the way they
> are read are optimised for super-fast index creation. This
> is expected to be a common operation. I still understand
> that this leaves things to be desired for you.
>
>
>> My wishes are for an optional updating of views on save feature! In
>> some cases that might regenerate a view several times without it
>> actually being accessed in between, but that is a tradeoff I can
>> live with, slow views on the other hand is something I can not!
>
> Put this in a shell script called view_trigger.sh
>
> #!/bin/sh
>
> counter=0
> max_docs=100
>
> while true
> do
> read database
>
> counter=`expr $counter + 1`
>
> if [ $counter -ge $max_docs ]; then
> `curl http://server:5984/$database/_view/name?count=0`
> counter=0
> fi
> done
>
> and add view_trigger.sh to our couch.ini as a
> DbUpdateNotificationProcess
>
> voilá :)
>
> Yes, this is extra work externally, but this is still a sensible
> solution. From our perspective, we do not need to change
> the core server behaviour to get you what you need and
> you still benefit from the batching of index creation.
>
> Also, I'd like to second what Cortland said: All views in a
> design document get updated if you query one of them.
> Be aware of that :)
>
> And on a final note: Thanks for writing in. Don't be
> discouraged by the replies. If there are other things that
> you would love to see in CouchDB, please let us know.
>
> Also, if enough users request a feature, we will consider
> putting it in, even on-
Re: Updating views on save
Posted by Damien Katz <da...@gmail.com>.
On Apr 28, 2008, at 7:44 AM, David Zülke wrote:
> Just curious... does the view generation have some kind of stampede
> protection / locking mechanism? So multiple calls to the same view
> at the same time result in the first call generating the view and
> others waiting or serving old info until the view index has been
> updated?
>
> David
>
Yes, if a view index is already being updated for someone else,
additional view readers wait for the same update to complete.
Re: Updating views on save
Posted by David Zülke <dz...@bitxtender.com>.
Just curious... does the view generation have some kind of stampede
protection / locking mechanism? So multiple calls to the same view at
the same time result in the first call generating the view and others
waiting or serving old info until the view index has been updated?
David
Am 28.04.2008 um 10:22 schrieb Jan Lehnardt:
> Heya Sebastian,
> it seems you feel rather strongly about this issue. But that's
> nothing a little engineering can solve for you, read on :)
>
> On Apr 28, 2008, at 01:04, Guby wrote:
>> Hello dear Couchers
>>
>> I understand that the views are indexed the first time they are
>> accessed and as far as I know there is no way to turn on view
>> updating on document save. I really don't understand the reasoning
>> behind this behavior. The advantage of the pre-populated/indexed
>> views are that they are blazingly fast to query and access, but
>> that advantage disappears when the first request after a document
>> update has to regenerate the view first!
>> I am currently building a web app where the background processes
>> perform a lot of writes to the database. The time it takes to write
>> a document is not critical for me. What is critical though is the
>> time it takes to load web pages for the end user that require
>> content from the database.
>> In some situations the background processes add thousands of
>> documents to the database within a short period of time, and when
>> the user tries to access a page after such an update the view
>> querying sometimes takes minutes and as a consequence of that the
>> browser times out... Not a recipe for happy customers...
>>
>> The only solution I can see at the moment is to create a worker
>> that queries the database whenever it is told that there has been a
>> document update, but that seems really stupid and unnecessary. And
>> in my case, running on a smallish VPS, a big waste of resources
>> having an extra working doing something the database itself could
>> just as well have done. It also requires a lot of extra coding
>> notifying the worker whenever I update or create a document
>> throughout my app.
>
> That would be a rather extreme solution. Why not, for
> example, trigger a view update from your document-
> insertion code, every N (N = 10, 30, 60?) seconds?
>
>
>> I am sure you have reasons for having implemented the views the way
>> you have, but I would be really interested to hear why it has been
>> done this way!
>
> 1) To not have a 'write penalty' for all views when
> documents are added. We expect you to have
> quite a few of views and updating all of them on-write
> seems silly. The data is generated when needed,
> saving resources by 2) not clogging them up when
> needed elsewhere and 3) processing large quantities
> of data in batches. and finally 4) The very layout of the
> bytes that make up documents on disk and the way they
> are read are optimised for super-fast index creation. This
> is expected to be a common operation. I still understand
> that this leaves things to be desired for you.
>
>
>> My wishes are for an optional updating of views on save feature! In
>> some cases that might regenerate a view several times without it
>> actually being accessed in between, but that is a tradeoff I can
>> live with, slow views on the other hand is something I can not!
>
> Put this in a shell script called view_trigger.sh
>
> #!/bin/sh
>
> counter=0
> max_docs=100
>
> while true
> do
> read database
>
> counter=`expr $counter + 1`
>
> if [ $counter -ge $max_docs ]; then
> `curl http://server:5984/$database/_view/name?count=0`
> counter=0
> fi
> done
>
> and add view_trigger.sh to our couch.ini as a
> DbUpdateNotificationProcess
>
> voilá :)
>
> Yes, this is extra work externally, but this is still a sensible
> solution. From our perspective, we do not need to change
> the core server behaviour to get you what you need and
> you still benefit from the batching of index creation.
>
> Also, I'd like to second what Cortland said: All views in a
> design document get updated if you query one of them.
> Be aware of that :)
>
> And on a final note: Thanks for writing in. Don't be
> discouraged by the replies. If there are other things that
> you would love to see in CouchDB, please let us know.
>
> Also, if enough users request a feature, we will consider
> putting it in, even on-write view updates, but do not expect
> that to happen anytime soon or at all. There are a lot
> of things that need our attention first.
>
> Cheers
> Jan
> --
Re: Updating views on save
Posted by Jan Lehnardt <ja...@apache.org>.
Heya Sebastian,
it seems you feel rather strongly about this issue. But that's
nothing a little engineering can solve for you, read on :)
On Apr 28, 2008, at 01:04, Guby wrote:
> Hello dear Couchers
>
> I understand that the views are indexed the first time they are
> accessed and as far as I know there is no way to turn on view
> updating on document save. I really don't understand the reasoning
> behind this behavior. The advantage of the pre-populated/indexed
> views are that they are blazingly fast to query and access, but that
> advantage disappears when the first request after a document update
> has to regenerate the view first!
> I am currently building a web app where the background processes
> perform a lot of writes to the database. The time it takes to write
> a document is not critical for me. What is critical though is the
> time it takes to load web pages for the end user that require
> content from the database.
> In some situations the background processes add thousands of
> documents to the database within a short period of time, and when
> the user tries to access a page after such an update the view
> querying sometimes takes minutes and as a consequence of that the
> browser times out... Not a recipe for happy customers...
>
> The only solution I can see at the moment is to create a worker that
> queries the database whenever it is told that there has been a
> document update, but that seems really stupid and unnecessary. And
> in my case, running on a smallish VPS, a big waste of resources
> having an extra working doing something the database itself could
> just as well have done. It also requires a lot of extra coding
> notifying the worker whenever I update or create a document
> throughout my app.
That would be a rather extreme solution. Why not, for
example, trigger a view update from your document-
insertion code, every N (N = 10, 30, 60?) seconds?
> I am sure you have reasons for having implemented the views the way
> you have, but I would be really interested to hear why it has been
> done this way!
1) To not have a 'write penalty' for all views when
documents are added. We expect you to have
quite a few of views and updating all of them on-write
seems silly. The data is generated when needed,
saving resources by 2) not clogging them up when
needed elsewhere and 3) processing large quantities
of data in batches. and finally 4) The very layout of the
bytes that make up documents on disk and the way they
are read are optimised for super-fast index creation. This
is expected to be a common operation. I still understand
that this leaves things to be desired for you.
> My wishes are for an optional updating of views on save feature! In
> some cases that might regenerate a view several times without it
> actually being accessed in between, but that is a tradeoff I can
> live with, slow views on the other hand is something I can not!
Put this in a shell script called view_trigger.sh
#!/bin/sh
counter=0
max_docs=100
while true
do
read database
counter=`expr $counter + 1`
if [ $counter -ge $max_docs ]; then
`curl http://server:5984/$database/_view/name?count=0`
counter=0
fi
done
and add view_trigger.sh to our couch.ini as a
DbUpdateNotificationProcess
voilá :)
Yes, this is extra work externally, but this is still a sensible
solution. From our perspective, we do not need to change
the core server behaviour to get you what you need and
you still benefit from the batching of index creation.
Also, I'd like to second what Cortland said: All views in a
design document get updated if you query one of them.
Be aware of that :)
And on a final note: Thanks for writing in. Don't be
discouraged by the replies. If there are other things that
you would love to see in CouchDB, please let us know.
Also, if enough users request a feature, we will consider
putting it in, even on-write view updates, but do not expect
that to happen anytime soon or at all. There are a lot
of things that need our attention first.
Cheers
Jan
--
Re: Updating views on save
Posted by Kristopher Tate <kr...@bluebridge.jp>.
Sebastian, at the moment I am following developments just like you and
starting to study more of the codebase.
This may or may not be correct, but I believe that the benefits of
having views update on the first time they are viewed far outweighs
updating the index/view each time data gets stored into CouchDB.
In the current situation, if I write 10 times and read 100 times, the
index may only be processed once if the 100 reads come after the 10
writes -- in a more real-world situation though, we're only going to
be updating the index as many times as we write -- but keep in mind
that given the right circumstances, the index only has to be
regenerated once.
If the DB indexed for each write, the number of indexes is the same as
writes, which may incur more IO penalties, beyond a simple write
(EmitReduce, etc.)
Also, please realize that the index does not have to be /completely/
regenerated upon reindex -- only the documents that have been added/
modified.
Optional updating (as you mention) could be a compromise, but I think
that resources should be spent making reindexing views faster.
Lastly, I would recommend that you try to optimize your view code to
make things faster, as well.
kristopher
On 2008/04/28, at 8:04, Guby wrote:
> Hello dear Couchers
>
> I understand that the views are indexed the first time they are
> accessed and as far as I know there is no way to turn on view
> updating on document save. I really don't understand the reasoning
> behind this behavior. The advantage of the pre-populated/indexed
> views are that they are blazingly fast to query and access, but that
> advantage disappears when the first request after a document update
> has to regenerate the view first!
>
> I am currently building a web app where the background processes
> perform a lot of writes to the database. The time it takes to write
> a document is not critical for me. What is critical though is the
> time it takes to load web pages for the end user that require
> content from the database.
> In some situations the background processes add thousands of
> documents to the database within a short period of time, and when
> the user tries to access a page after such an update the view
> querying sometimes takes minutes and as a consequence of that the
> browser times out... Not a recipe for happy customers...
>
> The only solution I can see at the moment is to create a worker that
> queries the database whenever it is told that there has been a
> document update, but that seems really stupid and unnecessary. And
> in my case, running on a smallish VPS, a big waste of resources
> having an extra working doing something the database itself could
> just as well have done. It also requires a lot of extra coding
> notifying the worker whenever I update or create a document
> throughout my app.
>
> I am sure you have reasons for having implemented the views the way
> you have, but I would be really interested to hear why it has been
> done this way!
>
> My wishes are for an optional updating of views on save feature! In
> some cases that might regenerate a view several times without it
> actually being accessed in between, but that is a tradeoff I can
> live with, slow views on the other hand is something I can not!
>
> All the best
> Sebastian