You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@couchdb.apache.org by robertkowalski <gi...@git.apache.org> on 2015/06/03 00:41:13 UTC

[GitHub] couchdb-couch pull request: perf: increase performance of http han...

GitHub user robertkowalski opened a pull request:

    https://github.com/apache/couchdb-couch/pull/57

    perf: increase performance of http handlers

    we are avoiding lagers formatting here which happens before it
    asynchronously dispatches the event.
    
    for a /get on a document this increases perfomance ~4% and should
    apply to all http requests.
    
    Benchmarking and profiling
    ===
    
    Sadly GitHub does not allow svg uploads
    
    Before
    ---
    ![flame--second--unpatched](https://cloud.githubusercontent.com/assets/298166/7948941/0c55302c-0989-11e5-8266-8a6fc574f4c4.png)
    
    After
    ---
    ![flame--second--patched](https://cloud.githubusercontent.com/assets/298166/7948937/0751fbb4-0989-11e5-998c-cc34b723ee0e.png)
    
    Benchmark data
    ---
    http://robert-kowalski.de/assets/data/2015-06-01-erlang-perf-2/results-siege.txt


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/robertkowalski/couchdb-couch increse-perf-2

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/couchdb-couch/pull/57.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #57
    
----
commit d2b5fce890863f7bbbf887e2c80fcf98a16230e6
Author: Robert Kowalski <ro...@kowalski.gd>
Date:   2015-06-02T20:56:20Z

    perf: increase performance of http handlers
    
    we are avoiding lagers formatting here which happens before it
    asynchronously dispatches the event.
    
    for a /get on a document this increases perfomance ~4% and should
    apply to all http requests.

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] couchdb-couch pull request: perf: increase performance of http han...

Posted by kxepal <gi...@git.apache.org>.
Github user kxepal commented on the pull request:

    https://github.com/apache/couchdb-couch/pull/57#issuecomment-113901460
  
    @robertkowalski 
    
    >  the problem is the formatting in lager, which is slow
    
    Shouldn't we propose fix for lager instead? Optimizing one log call in one popular place is good, but is that the only place we have? So far, your benchmark covered only tiny part of CouchDB usage.
    
    > I don't see many people working on a new log format and given that the tickets are around 4 years old 
    
    Because they didn't have such opportunity for all these 4 years, because they all being waited for better logging system that supports this feature?
    
    > I am implementing for my needs now, not the ones I might have in the future
    
    Not sure how introducing one special optimization for one single case which cuts every 100ms down to 96ms may be useful. But for sure, first person that will have to touch formatting will be very upset by such special cases.
    
    Suggestion: let's not rush now with optimizations. Let's first make solid reproduceable benchmark suite that covers at least all the basic CouchDB usage and then think about performance improvements. Without tracking performance in time such optimizations makes no sense and only harms: show arbitrary developer your changes and first his intention will be to replace it with couch_log, because DRY and Single responsibility principle are broken here.  


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] couchdb-couch pull request: perf: increase performance of http han...

Posted by robertkowalski <gi...@git.apache.org>.
GitHub user robertkowalski reopened a pull request:

    https://github.com/apache/couchdb-couch/pull/57

    perf: increase performance of http handlers

    we are avoiding lagers formatting here which happens before it
    asynchronously dispatches the event.
    
    for a /get on a document this increases perfomance ~4% and should
    apply to all http requests.
    
    PRs:
    https://github.com/apache/couchdb-couch-log/pull/2
    https://github.com/apache/couchdb-couch/pull/57
    
    Benchmarking and profiling
    ===
    
    Sadly GitHub does not allow svg uploads
    
    Before
    ---
    ![flame--second--unpatched](https://cloud.githubusercontent.com/assets/298166/7948941/0c55302c-0989-11e5-8266-8a6fc574f4c4.png)
    
    After
    ---
    ![flame--second--patched](https://cloud.githubusercontent.com/assets/298166/7948937/0751fbb4-0989-11e5-998c-cc34b723ee0e.png)
    
    Benchmark data
    ---
    http://robert-kowalski.de/assets/data/2015-06-01-erlang-perf-2/results-siege.txt


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/robertkowalski/couchdb-couch increse-perf-2

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/couchdb-couch/pull/57.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #57
    
----
commit e06a810ff8f88cdd52555a1b1aade22b53a7b98b
Author: Robert Kowalski <ro...@kowalski.gd>
Date:   2015-06-02T20:56:20Z

    perf: increase performance of http handlers
    
    we are avoiding lagers formatting here which happens before it
    asynchronously dispatches the event.
    
    for a /get on a document this increases perfomance ~4% and should
    apply to all http requests.
    
    PRs:
    https://github.com/apache/couchdb-couch-log/pull/2
    https://github.com/apache/couchdb-couch/pull/57

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] couchdb-couch pull request: perf: increase performance of http han...

Posted by robertkowalski <gi...@git.apache.org>.
Github user robertkowalski closed the pull request at:

    https://github.com/apache/couchdb-couch/pull/57


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] couchdb-couch pull request: perf: increase performance of http han...

Posted by kxepal <gi...@git.apache.org>.
Github user kxepal commented on the pull request:

    https://github.com/apache/couchdb-couch/pull/57#issuecomment-114844084
  
    @robertkowalski My position here about is that all this work on this is like a fixing bugs without tests: it's good, it works for now, it becomes useless tomorrow because of unknown status. So the final decision is up to you, my duty here is to warn about any possible problems and make sure we all understand them and ready to accept them in {short|mid|long}-term.
    
    P.S. I see that you're cares about performance a lot and also there are Jan, Tony. Imho, quite good company for start. 
    
    P.P.S. I do like lager-patch way here (thanks for raising issue for them!) since it may be applied not only for this particular case, but also to the others, which was not covered by your script, with lesser code modifications. Need only to make sure that there is no side effect from those changes that we do not want to have.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] couchdb-couch pull request: perf: increase performance of http han...

Posted by kxepal <gi...@git.apache.org>.
Github user kxepal commented on the pull request:

    https://github.com/apache/couchdb-couch/pull/57#issuecomment-108122239
  
    4% speedup isn't significant gain as CouchDB 2.0 now is about 7 times slower than 1.6.1 due to cluster routines, even with a single node.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] couchdb-couch pull request: perf: increase performance of http han...

Posted by robertkowalski <gi...@git.apache.org>.
Github user robertkowalski commented on the pull request:

    https://github.com/apache/couchdb-couch/pull/57#issuecomment-113898189
  
    @janl the problem is the formatting in lager, which is slow (see https://github.com/apache/couchdb-lager/blob/215c425d091aec61c33f9920aa76d035733b1cde/src/lager.erl#L350-353) -- you can also see in the flamegraph that the time is spent on `lager:safe_format/4` which calls `lager:trunc_io_format/4`
    
    @kxepal @janl When I grep the sourcecode for `couch_log:notice` every call uses an own custom formatting and `couch_log:notice` then delegates to `lager` which decides if it uses its own formatter or not. I don't see why the patch blocks https://issues.apache.org/jira/browse/COUCHDB-1278 and https://issues.apache.org/jira/browse/COUCHDB-1401 -- given someone really wants to implement it at some point, but to be honest I don't see many people working on that or even talking about it.
    
    Conclusion:
    
     - I think 4% speed improvement is quite a big deal.
     - I don't see why it blocks another log format, can you elaborate with constructive feedback please?
     - I am implementing for my needs **now**, not the ones **I might have in the future**. I don't see many people working on a new log format and given that the tickets are around 4 years old they *might* not have any priority *any more* or at least *right now* -- but to repeat: I don't see why it would block potential work and I am looking forward to your constructive feedback
    



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] couchdb-couch pull request: perf: increase performance of http han...

Posted by janl <gi...@git.apache.org>.
Github user janl commented on the pull request:

    https://github.com/apache/couchdb-couch/pull/57#issuecomment-108573545
  
    @robertkowalski keep this open please :)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] couchdb-couch pull request: perf: increase performance of http han...

Posted by robertkowalski <gi...@git.apache.org>.
Github user robertkowalski closed the pull request at:

    https://github.com/apache/couchdb-couch/pull/57


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] couchdb-couch pull request: perf: increase performance of http han...

Posted by robertkowalski <gi...@git.apache.org>.
Github user robertkowalski commented on the pull request:

    https://github.com/apache/couchdb-couch/pull/57#issuecomment-108517032
  
    oh! i don't want to block potential work on 3.5-4.1 old issues


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] couchdb-couch pull request: perf: increase performance of http han...

Posted by janl <gi...@git.apache.org>.
Github user janl commented on the pull request:

    https://github.com/apache/couchdb-couch/pull/57#issuecomment-114069858
  
    > Suggestion: let's not rush now with optimizations. Let's first make solid reproduceable benchmark suite that covers at least all the basic CouchDB
    
    We’ve been wanting this for years. We can’t block optimisations because we don’t have it. I agree we should have it, and anyone working on this is more than welcome to, but blocking major (imho > 1%) is not a good idea.
    
    > > I don't see many people working on a new log format and given that the tickets are around 4 years old
    
    > Because they didn't have such opportunity for all these 4 years, because they all being waited for better logging system that supports this feature?
    
    I’m with @robertkowalski here, let’s not block this. These issues have never had much interest. IIRC, I just filed this, because I had one person in a workshop asking for this. It has never been a substantial problem.
    
    > Shouldn't we propose fix for lager instead?
    
    There are multiple types of ways to address issues. @robertkowalski is proposing a short-term fix. @kxepal suggest a long-term fix. That is, if we get lager to speed up, that would be desirable. In the meantime, if we can get a 4% speed increase by being smarter with logging, that’s a win in my book.
    
    >  It has no effect on public :5984 port since there chttpd is being requested, not couch_httpd
    
    There are plenty of APIs where chttpd calls through to couch_httpd. Although I don’t know how that affects logging, we certainly don’t get two entries. We should check if this optimisation applies.
    
    Generally, the observation about `5984` vs. `5986` is a good one, though. We should look at optimising the former, not the latter.



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] couchdb-couch pull request: perf: increase performance of http han...

Posted by kxepal <gi...@git.apache.org>.
Github user kxepal commented on the pull request:

    https://github.com/apache/couchdb-couch/pull/57#issuecomment-108121847
  
    -1 This change blocks the way to solve the following issues:
    - https://issues.apache.org/jira/browse/COUCHDB-1278
    - https://issues.apache.org/jira/browse/COUCHDB-1401
    
    There was a hope that we could provide ability to customize logging format with lagger, but you're going to kill it right before the start.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] couchdb-couch pull request: perf: increase performance of http han...

Posted by kxepal <gi...@git.apache.org>.
Github user kxepal commented on the pull request:

    https://github.com/apache/couchdb-couch/pull/57#issuecomment-113911030
  
    @robertkowalski Few more moments about this change:
    1) It has no effect on public :5984 port since there chttpd is being requested, not couch_httpd
    2) :5986 port usage for everyday tasks is discouraged and we want to deprecate it in order to have one single HTTP port to run all the operations. 
    3) We are already have log formatting inconsistency between logged requests against :5984 and :5986 :
    
    ```
    2015-06-21 17:40:14.408 [notice] node1@127.0.0.1 <0.482.0> a683d574 127.0.0.1 localhost:15984 GET /_users 200 ok 1
    2015-06-21 17:42:33.407 [notice] node1@127.0.0.1 <0.235.0> 127.0.0.1 - - GET /_users 200
    ```
    
    First line was about http://localhost:5984/_users request, second is for http://localhost:5986/_users
    
    So we're fixing wrong problem here.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] couchdb-couch pull request: perf: increase performance of http han...

Posted by robertkowalski <gi...@git.apache.org>.
Github user robertkowalski commented on the pull request:

    https://github.com/apache/couchdb-couch/pull/57#issuecomment-114837039
  
    I am working with the lager team to improve the performance of lager itself now.
    
    @kxepal I don't want to discourage you and see your good intentions regarding building a full featured benchmark suite **before** doing any further performance work. 
    
    I think a automatic testsuite will evolve automatically as soon as enough people in the project **care** about performance. We can build the best monitoring/benchmarking whatever tool and throw it in front of the community - as long as nobody **cares** about performance, they won't use it. One example for that is https://github.com/apache/couchdb/tree/master/test/bench which already provides some performance tests, a few moths ago I deleted half of them because they were broken since years and nobody noticed, as nobody uses them. I am not sure what the future for another testsuite is, if I build it right now.
    
    Just to clarify: I am not saying it is a bad idea to have a full featured benchmark suite, but I think it will appear automatically on a way to more performance where more and more people care about performance and do performance related work and working on that testsuite should not block work from people that care about performance.
    
    For me that means working further on low-hanging fruits where a performance improvement is easily observable even with a small microbenchmark. I hope that more and more people in the project see how much we can CouchDB's performance and jump onto that wagon. Throwing a testsuite in front of them probably does not motivate most people to start to care about performance, as it is an additional tool that can help people that are working on performance improvements.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] couchdb-couch pull request: perf: increase performance of http han...

Posted by janl <gi...@git.apache.org>.
Github user janl commented on the pull request:

    https://github.com/apache/couchdb-couch/pull/57#issuecomment-108402056
  
    I’m sure we can find a way that adds flexibility (maybe just hand the atoms/binaries to lager) and let it do all the formatting, while retaining the speedup?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---