You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@allura.apache.org by Dave Brondsema <da...@brondsema.net> on 2015/04/17 23:24:00 UTC

[allura:tickets] #7873 Git branch & tag speedups



---

** [tickets:#7873] Git branch & tag speedups**

**Status:** open
**Milestone:** unreleased
**Labels:** performance 
**Created:** Fri Apr 17, 2015 09:23 PM UTC by Dave Brondsema
**Last Updated:** Fri Apr 17, 2015 09:23 PM UTC
**Owner:** nobody

I saw that git pages on forge-allura.apache.org were slow, so I looked at stats.log and saw that the sidebar was the slowest part.  I did some additional digging and found 2 specific areas for improvement:

* in `git_main.py` change `default_branch_name` to a `@LazyProperty` since it is called many times inside a loop in `RepositoryApp.sidebar_menu`
* Since `sidebar_menu` only requests a certain number of branches, pass that "limit" all the way through to `git_repo.py`'s `branches` method so that `is_valid()` is only called a minimum number of times needed.
    * make sure the default branch logic to put it at the top of the list still works (e.g. always put it at the top first)

In addition to those changes, generalize and apply the same approach to the tags.  And also check ForgeHg to see if mercurial can benefit the same way.


---

Sent from forge-allura.apache.org because dev@allura.apache.org is subscribed to https://forge-allura.apache.org/p/allura/tickets/

To unsubscribe from further messages, a project admin can change settings at https://forge-allura.apache.org/p/allura/admin/tickets/options.  Or, if this is a mailing list, you can unsubscribe from the mailing list.

[allura:tickets] #7873 Git branch & tag speedups

Posted by Dave Brondsema <da...@brondsema.net>.

- **labels**: performance --> performance, sf-current



---

** [tickets:#7873] Git branch & tag speedups**

**Status:** open
**Milestone:** unreleased
**Labels:** performance sf-current 
**Created:** Fri Apr 17, 2015 09:23 PM UTC by Dave Brondsema
**Last Updated:** Fri Apr 17, 2015 09:23 PM UTC
**Owner:** nobody

I saw that git pages on forge-allura.apache.org were slow, so I looked at stats.log and saw that the sidebar was the slowest part.  I did some additional digging and found 2 specific areas for improvement:

* in `git_main.py` change `default_branch_name` to a `@LazyProperty` since it is called many times inside a loop in `RepositoryApp.sidebar_menu`
* Since `sidebar_menu` only requests a certain number of branches, pass that "limit" all the way through to `git_repo.py`'s `branches` method so that `is_valid()` is only called a minimum number of times needed.
    * make sure the default branch logic to put it at the top of the list still works (e.g. always put it at the top first)

In addition to those changes, generalize and apply the same approach to the tags.  And also check ForgeHg to see if mercurial can benefit the same way.


---

Sent from forge-allura.apache.org because dev@allura.apache.org is subscribed to https://forge-allura.apache.org/p/allura/tickets/

To unsubscribe from further messages, a project admin can change settings at https://forge-allura.apache.org/p/allura/admin/tickets/options.  Or, if this is a mailing list, you can unsubscribe from the mailing list.

[allura:tickets] Re: #7873 Git branch & tag speedups

Posted by Igor Bondarenko <je...@gmail.com>.

I thought we can expand list comprehension in the `branches/tags` to be a full loop and wrap `Object` construction in the `try..except` block and that it will be cheaper than calling `is_valid` on every branch/tag, since exception handling logic will run only for repos with integrity problems, which is fairly rare. I'm not sure, though.

We can also cache results of branches/tags and update cache on repo change with proper `is_valid` checks, but we're moving from keeping repo metadata in mongo to direct access, so don't sure if it worth it.

`git fsck` approach looks like overkill to me, we'll need to parse it's output to determine what's wrong and also save the results somewhere to skip invalid branches. Correct me if I'm mistaken.


---

** [tickets:#7873] Git branch & tag speedups**

**Status:** in-progress
**Milestone:** unreleased
**Labels:** performance sf-current sf-4 
**Created:** Fri Apr 17, 2015 09:23 PM UTC by Dave Brondsema
**Last Updated:** Tue Jun 09, 2015 09:57 PM UTC
**Owner:** Heith Seewald

I saw that git pages on forge-allura.apache.org were slow, so I looked at stats.log and saw that the sidebar was the slowest part.  I did some additional digging and found 2 specific areas for improvement:

* in `git_main.py` change `default_branch_name` to a `@LazyProperty` since it is called many times inside a loop in `RepositoryApp.sidebar_menu`
* Since `sidebar_menu` only requests a certain number of branches, pass that "limit" all the way through to `git_repo.py`'s `branches` method so that `is_valid()` is only called a minimum number of times needed.
    * make sure the default branch logic to put it at the top of the list still works (e.g. always put it at the top first)

In addition to those changes, generalize and apply the same approach to the tags.  And also check ForgeHg to see if mercurial can benefit the same way.


---

Sent from forge-allura.apache.org because dev@allura.apache.org is subscribed to https://forge-allura.apache.org/p/allura/tickets/

To unsubscribe from further messages, a project admin can change settings at https://forge-allura.apache.org/p/allura/admin/tickets/options.  Or, if this is a mailing list, you can unsubscribe from the mailing list.

[allura:tickets] #7873 Git branch & tag speedups

Posted by Heith Seewald <hs...@slashdotmedia.com>.

Yeah, I see what you mean. My thought was that if they had a mismatched sha, then there are probably bigger data integrity problems to deal with. But you're right -- we should be handle it to be safe.

I tried catching the **BadObject** exception in a few places -- but quickly realized that the existing location is the best place for validation (inside the branch/tags list).

So here's another angle:

What if we used **git fsck** to check for invalid refs on repo change (via hooks maybe)? A single check on when something changes would be much more performant than checking on every page view... the only question would be how to handle the invalid refs.

What are your thoughts on that?

---

** [tickets:#7873] Git branch & tag speedups**

**Status:** in-progress
**Milestone:** unreleased
**Labels:** performance sf-current sf-4
**Created:** Fri Apr 17, 2015 09:23 PM UTC by Dave Brondsema
**Last Updated:** Tue Jun 09, 2015 03:40 PM UTC
**Owner:** Heith Seewald

I saw that git pages on forge-allura.apache.org were slow, so I looked at stats.log and saw that the sidebar was the slowest part. I did some additional digging and found 2 specific areas for improvement:

* in `git_main.py` change `default_branch_name` to a `@LazyProperty` since it is called many times inside a loop in `RepositoryApp.sidebar_menu`
* Since `sidebar_menu` only requests a certain number of branches, pass that "limit" all the way through to `git_repo.py`'s `branches` method so that `is_valid()` is only called a minimum number of times needed.
* make sure the default branch logic to put it at the top of the list still works (e.g. always put it at the top first)

In addition to those changes, generalize and apply the same approach to the tags. And also check ForgeHg to see if mercurial can benefit the same way.

---

Sent from forge-allura.apache.org because dev@allura.apache.org is subscribed to https://forge-allura.apache.org/p/allura/tickets/

To unsubscribe from further messages, a project admin can change settings at https://forge-allura.apache.org/p/allura/admin/tickets/options. Or, if this is a mailing list, you can unsubscribe from the mailing list.

[allura:tickets] #7873 Git branch & tag speedups

Posted by Dave Brondsema <da...@brondsema.net>.

- **labels**: performance, sf-current --> performance, sf-current, sf-4



---

** [tickets:#7873] Git branch & tag speedups**

**Status:** open
**Milestone:** unreleased
**Labels:** performance sf-current sf-4 
**Created:** Fri Apr 17, 2015 09:23 PM UTC by Dave Brondsema
**Last Updated:** Thu Apr 30, 2015 02:54 PM UTC
**Owner:** nobody

I saw that git pages on forge-allura.apache.org were slow, so I looked at stats.log and saw that the sidebar was the slowest part.  I did some additional digging and found 2 specific areas for improvement:

* in `git_main.py` change `default_branch_name` to a `@LazyProperty` since it is called many times inside a loop in `RepositoryApp.sidebar_menu`
* Since `sidebar_menu` only requests a certain number of branches, pass that "limit" all the way through to `git_repo.py`'s `branches` method so that `is_valid()` is only called a minimum number of times needed.
    * make sure the default branch logic to put it at the top of the list still works (e.g. always put it at the top first)

In addition to those changes, generalize and apply the same approach to the tags.  And also check ForgeHg to see if mercurial can benefit the same way.


---

Sent from forge-allura.apache.org because dev@allura.apache.org is subscribed to https://forge-allura.apache.org/p/allura/tickets/

To unsubscribe from further messages, a project admin can change settings at https://forge-allura.apache.org/p/allura/admin/tickets/options.  Or, if this is a mailing list, you can unsubscribe from the mailing list.

[allura:tickets] #7873 Git branch & tag speedups

Posted by Heith Seewald <hs...@slashdotmedia.com>.

- Attachments has changed:

Diff:

~~~~

--- old
+++ new
@@ -1 +0,0 @@
-new.png (63.3 kB; image/png)

~~~~




---

** [tickets:#7873] Git branch & tag speedups**

**Status:** review
**Milestone:** unreleased
**Labels:** performance sf-current sf-4 
**Created:** Fri Apr 17, 2015 09:23 PM UTC by Dave Brondsema
**Last Updated:** Tue May 05, 2015 02:22 PM UTC
**Owner:** Heith Seewald

I saw that git pages on forge-allura.apache.org were slow, so I looked at stats.log and saw that the sidebar was the slowest part.  I did some additional digging and found 2 specific areas for improvement:

* in `git_main.py` change `default_branch_name` to a `@LazyProperty` since it is called many times inside a loop in `RepositoryApp.sidebar_menu`
* Since `sidebar_menu` only requests a certain number of branches, pass that "limit" all the way through to `git_repo.py`'s `branches` method so that `is_valid()` is only called a minimum number of times needed.
    * make sure the default branch logic to put it at the top of the list still works (e.g. always put it at the top first)

In addition to those changes, generalize and apply the same approach to the tags.  And also check ForgeHg to see if mercurial can benefit the same way.


---

Sent from forge-allura.apache.org because dev@allura.apache.org is subscribed to https://forge-allura.apache.org/p/allura/tickets/

To unsubscribe from further messages, a project admin can change settings at https://forge-allura.apache.org/p/allura/admin/tickets/options.  Or, if this is a mailing list, you can unsubscribe from the mailing list.

[allura:tickets] #7873 Git branch & tag speedups

Posted by Igor Bondarenko <je...@gmail.com>.

- **status**: review --> in-progress
- **Comment**:

Ok, I see some timing improvements. TTFB mean: 9.7s -> 8.3s. It is not very much, but a lot of time takes other stuff. E.g. if I change `RepositoryApp.sidebar_menu` to always return empty list, TTFB is still pretty high (6-7s on a branch and 7-9 on a master).

That said, `b.commit.hexsha` can raise an error if ref is not valid, so we should handle it gracefully for the (hopefully rare) cases like this.

ValueError: Failed to parse reference information from u'refs/heads/a-fake'
You can reproduce it like this:

echo 'bad commit' > /git/p/t/tee/test/code.git/refs/heads/a-fake

and then refresh the page.

---

** [tickets:#7873] Git branch & tag speedups**

**Status:** in-progress
**Milestone:** unreleased
**Labels:** performance sf-current sf-4
**Created:** Fri Apr 17, 2015 09:23 PM UTC by Dave Brondsema
**Last Updated:** Fri Jun 05, 2015 07:17 PM UTC
**Owner:** Heith Seewald

In addition to those changes, generalize and apply the same approach to the tags. And also check ForgeHg to see if mercurial can benefit the same way.

---

Sent from forge-allura.apache.org because dev@allura.apache.org is subscribed to https://forge-allura.apache.org/p/allura/tickets/

[allura:tickets] #7873 Git branch & tag speedups

Posted by Heith Seewald <hs...@slashdotmedia.com>.

- **status**: in-progress --> review
- **Comment**:

Implemented a cache system that is cleared on repo_refresh.  Much faster overall.   Thanks for the tips :)



---

** [tickets:#7873] Git branch & tag speedups**

**Status:** review
**Milestone:** unreleased
**Labels:** performance sf-current sf-4 
**Created:** Fri Apr 17, 2015 09:23 PM UTC by Dave Brondsema
**Last Updated:** Wed Jun 10, 2015 05:20 PM UTC
**Owner:** Heith Seewald

I saw that git pages on forge-allura.apache.org were slow, so I looked at stats.log and saw that the sidebar was the slowest part.  I did some additional digging and found 2 specific areas for improvement:

* in `git_main.py` change `default_branch_name` to a `@LazyProperty` since it is called many times inside a loop in `RepositoryApp.sidebar_menu`
* Since `sidebar_menu` only requests a certain number of branches, pass that "limit" all the way through to `git_repo.py`'s `branches` method so that `is_valid()` is only called a minimum number of times needed.
    * make sure the default branch logic to put it at the top of the list still works (e.g. always put it at the top first)

In addition to those changes, generalize and apply the same approach to the tags.  And also check ForgeHg to see if mercurial can benefit the same way.


---

Sent from forge-allura.apache.org because dev@allura.apache.org is subscribed to https://forge-allura.apache.org/p/allura/tickets/

To unsubscribe from further messages, a project admin can change settings at https://forge-allura.apache.org/p/allura/admin/tickets/options.  Or, if this is a mailing list, you can unsubscribe from the mailing list.

[allura:tickets] Re: #7873 Git branch & tag speedups

Posted by Igor Bondarenko <je...@gmail.com>.

I tested with 1224 branches and 588 tags.

I've looked into TTFB, for me it's ~7.6 sec and stable across page reloads on master, and on the branch it spiking just like stats: ~7sec, then ~13sec, then ~8.

It's strange that I don't see even tiny performance improvement, in your case it's almost 5 times faster. Could you try it with more branches and maybe some tags too?


---

** [tickets:#7873] Git branch & tag speedups**

**Status:** in-progress
**Milestone:** unreleased
**Labels:** performance sf-current sf-4 
**Created:** Fri Apr 17, 2015 09:23 PM UTC by Dave Brondsema
**Last Updated:** Fri May 08, 2015 03:28 PM UTC
**Owner:** Heith Seewald

I saw that git pages on forge-allura.apache.org were slow, so I looked at stats.log and saw that the sidebar was the slowest part.  I did some additional digging and found 2 specific areas for improvement:

* in `git_main.py` change `default_branch_name` to a `@LazyProperty` since it is called many times inside a loop in `RepositoryApp.sidebar_menu`
* Since `sidebar_menu` only requests a certain number of branches, pass that "limit" all the way through to `git_repo.py`'s `branches` method so that `is_valid()` is only called a minimum number of times needed.
    * make sure the default branch logic to put it at the top of the list still works (e.g. always put it at the top first)

In addition to those changes, generalize and apply the same approach to the tags.  And also check ForgeHg to see if mercurial can benefit the same way.


---

Sent from forge-allura.apache.org because dev@allura.apache.org is subscribed to https://forge-allura.apache.org/p/allura/tickets/

To unsubscribe from further messages, a project admin can change settings at https://forge-allura.apache.org/p/allura/admin/tickets/options.  Or, if this is a mailing list, you can unsubscribe from the mailing list.

[allura:tickets] #7873 Git branch & tag speedups

Posted by Heith Seewald <hs...@slashdotmedia.com>.

These are great notes. 

My performance improvements were based on the *"time to first byte"* from the [chrome dev tools](https://developer.chrome.com/devtools/docs/network). *I attached two files of the before and after based on my tests (using ~200 branches).*
   

TTFB, while not the best metric for gauging specific functions, it does give an indication of the perception of latency.


I'm going to re-run the tests and check out the stats.log.  

How many branches were your performance tests run with?


Attachment: new.png (69.8 kB; image/png)  old.png (69.4 kB; image/png) 


---

** [tickets:#7873] Git branch & tag speedups**

**Status:** in-progress
**Milestone:** unreleased
**Labels:** performance sf-current sf-4 
**Created:** Fri Apr 17, 2015 09:23 PM UTC by Dave Brondsema
**Last Updated:** Fri May 08, 2015 08:49 AM UTC
**Owner:** Heith Seewald

I saw that git pages on forge-allura.apache.org were slow, so I looked at stats.log and saw that the sidebar was the slowest part.  I did some additional digging and found 2 specific areas for improvement:

* in `git_main.py` change `default_branch_name` to a `@LazyProperty` since it is called many times inside a loop in `RepositoryApp.sidebar_menu`
* Since `sidebar_menu` only requests a certain number of branches, pass that "limit" all the way through to `git_repo.py`'s `branches` method so that `is_valid()` is only called a minimum number of times needed.
    * make sure the default branch logic to put it at the top of the list still works (e.g. always put it at the top first)

In addition to those changes, generalize and apply the same approach to the tags.  And also check ForgeHg to see if mercurial can benefit the same way.


---

Sent from forge-allura.apache.org because dev@allura.apache.org is subscribed to https://forge-allura.apache.org/p/allura/tickets/

To unsubscribe from further messages, a project admin can change settings at https://forge-allura.apache.org/p/allura/admin/tickets/options.  Or, if this is a mailing list, you can unsubscribe from the mailing list.

[allura:tickets] #7873 Git branch & tag speedups -- NEEDS INI

Posted by Dave Brondsema <da...@brondsema.net>.

- **labels**: performance, sf-current, sf-4 --> performance, sf-4



---

** [tickets:#7873] Git branch & tag speedups  -- NEEDS INI**

**Status:** closed
**Milestone:** unreleased
**Labels:** performance sf-4 
**Created:** Fri Apr 17, 2015 09:23 PM UTC by Dave Brondsema
**Last Updated:** Thu Jun 25, 2015 04:30 PM UTC
**Owner:** Heith Seewald

I saw that git pages on forge-allura.apache.org were slow, so I looked at stats.log and saw that the sidebar was the slowest part.  I did some additional digging and found 2 specific areas for improvement:

* in `git_main.py` change `default_branch_name` to a `@LazyProperty` since it is called many times inside a loop in `RepositoryApp.sidebar_menu`
* Since `sidebar_menu` only requests a certain number of branches, pass that "limit" all the way through to `git_repo.py`'s `branches` method so that `is_valid()` is only called a minimum number of times needed.
    * make sure the default branch logic to put it at the top of the list still works (e.g. always put it at the top first)

In addition to those changes, generalize and apply the same approach to the tags.  And also check ForgeHg to see if mercurial can benefit the same way.


---

Sent from forge-allura.apache.org because dev@allura.apache.org is subscribed to https://forge-allura.apache.org/p/allura/tickets/

To unsubscribe from further messages, a project admin can change settings at https://forge-allura.apache.org/p/allura/admin/tickets/options.  Or, if this is a mailing list, you can unsubscribe from the mailing list.

[allura:tickets] #7873 Git branch & tag speedups

Posted by Heith Seewald <hs...@slashdotmedia.com>.

- **status**: open --> in-progress
- **assigned_to**: Heith Seewald



---

** [tickets:#7873] Git branch & tag speedups**

**Status:** in-progress
**Milestone:** unreleased
**Labels:** performance sf-current sf-4 
**Created:** Fri Apr 17, 2015 09:23 PM UTC by Dave Brondsema
**Last Updated:** Thu Apr 30, 2015 03:36 PM UTC
**Owner:** Heith Seewald

I saw that git pages on forge-allura.apache.org were slow, so I looked at stats.log and saw that the sidebar was the slowest part.  I did some additional digging and found 2 specific areas for improvement:

* in `git_main.py` change `default_branch_name` to a `@LazyProperty` since it is called many times inside a loop in `RepositoryApp.sidebar_menu`
* Since `sidebar_menu` only requests a certain number of branches, pass that "limit" all the way through to `git_repo.py`'s `branches` method so that `is_valid()` is only called a minimum number of times needed.
    * make sure the default branch logic to put it at the top of the list still works (e.g. always put it at the top first)

In addition to those changes, generalize and apply the same approach to the tags.  And also check ForgeHg to see if mercurial can benefit the same way.


---

Sent from forge-allura.apache.org because dev@allura.apache.org is subscribed to https://forge-allura.apache.org/p/allura/tickets/

To unsubscribe from further messages, a project admin can change settings at https://forge-allura.apache.org/p/allura/admin/tickets/options.  Or, if this is a mailing list, you can unsubscribe from the mailing list.

[allura:tickets] #7873 Git branch & tag speedups

Posted by Heith Seewald <hs...@slashdotmedia.com>.

So the Object construction in the branches/tags context is actually just a ming Object (aka a dict with object-like attr access). And the ['**is_valid**'](https://github.com/gitpython-developers/GitPython/blob/master/git/refs/symbolic.py#L330) is just doing a simple try/except itself -- so I don't think expanding the list comprehension for our own try/except be any cheaper **unless**...

We use git fsck as the try/except and only check **is_valid** for the very rare cases that it finds an issue.  I attached a profile of running **`self._git.git.fsck(full=True)`** and it's very cheap even with 1000s of branches.


~~~~~
    @LazyProperty
    def branches(self):
        try:
            self._git.git.fsck(full=True)
        except git.GitCommandError as gce:
            log.debug(u"Found inconsistency in {}: {}".format(self._repo.name, gce.stderr))
            return [Object(name=b.name, object_id=b.commit.hexsha) for b in self._git.branches if b.is_valid()]
        return [Object(name=b.name, object_id=b.commit.hexsha) for b in self._git.branches]
~~~~~

This looks like it's a pretty good trade-off between handling invalid refs and performance.  It would have the added benefit of informing us of issues instead of just silently returning valid results.

I pushed an optimized version of this idea if you want to take a look.  **hs/7873**


Attachment: Screen Shot 2015-06-10 at 9.56.42 AM.png (552.9 kB; image/png) 


---

** [tickets:#7873] Git branch & tag speedups**

**Status:** in-progress
**Milestone:** unreleased
**Labels:** performance sf-current sf-4 
**Created:** Fri Apr 17, 2015 09:23 PM UTC by Dave Brondsema
**Last Updated:** Tue Jun 09, 2015 09:57 PM UTC
**Owner:** Heith Seewald

I saw that git pages on forge-allura.apache.org were slow, so I looked at stats.log and saw that the sidebar was the slowest part.  I did some additional digging and found 2 specific areas for improvement:

* in `git_main.py` change `default_branch_name` to a `@LazyProperty` since it is called many times inside a loop in `RepositoryApp.sidebar_menu`
* Since `sidebar_menu` only requests a certain number of branches, pass that "limit" all the way through to `git_repo.py`'s `branches` method so that `is_valid()` is only called a minimum number of times needed.
    * make sure the default branch logic to put it at the top of the list still works (e.g. always put it at the top first)

In addition to those changes, generalize and apply the same approach to the tags.  And also check ForgeHg to see if mercurial can benefit the same way.


---

Sent from forge-allura.apache.org because dev@allura.apache.org is subscribed to https://forge-allura.apache.org/p/allura/tickets/

To unsubscribe from further messages, a project admin can change settings at https://forge-allura.apache.org/p/allura/admin/tickets/options.  Or, if this is a mailing list, you can unsubscribe from the mailing list.

[allura:tickets] #7873 Git branch & tag speedups

Posted by Igor Bondarenko <je...@gmail.com>.

- **Comment**:

I think this is finally done :)

However I got new test failure

~~~~
======================================================================
FAIL: forgegit.tests.functional.test_controllers.TestFork.test_merge_request_detail_view
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/var/local/env-allura/lib/python2.7/site-packages/nose-1.3.4-py2.7.egg/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/home/ibondarenko/ibondarenko-1154/forge/ForgeGit/forgegit/tests/functional/test_controllers.py", line 599, in test_merge_request_detail_view
    assert_equal(rev_links[0].get('href'), '/p/test2/code/ci/%s/' % c_id)
AssertionError: u'/p/test2/code/ci/5f781900aac5e36cc9a48228df2e5c125d8a9765/' != '/p/test2/code/ci/1e146e67985dcd71c74de79613719bef7bddca4a/'
-------------------- >> begin captured stdout << ---------------------
~~~~

 
Is caching enabled in tests?  I suggest we disable it in tests by default and enable only whenever needed.

Other than that looks/works good.

Regarding configtree, I meant we should made changes to the "configtree" repo (`/var/local/configtree` on a sandbox). I've made the changed and pushed them to `ib/7873` already, so don't worry about that.

AFAIK yes, we should add "NEEDS CONFIGTREE" to the ticket description after we close the ticket, so that it will be immediately visible that ticket requires configuration changes. It helps when we organize notes for the release and also for siteops, I guess.

You can got ahead and merge this (both allura and configtree), whenever test is fixed.



---

** [tickets:#7873] Git branch & tag speedups**

**Status:** in-progress
**Milestone:** unreleased
**Labels:** performance sf-current sf-4 
**Created:** Fri Apr 17, 2015 09:23 PM UTC by Dave Brondsema
**Last Updated:** Tue Jun 16, 2015 06:28 PM UTC
**Owner:** Heith Seewald

I saw that git pages on forge-allura.apache.org were slow, so I looked at stats.log and saw that the sidebar was the slowest part.  I did some additional digging and found 2 specific areas for improvement:

* in `git_main.py` change `default_branch_name` to a `@LazyProperty` since it is called many times inside a loop in `RepositoryApp.sidebar_menu`
* Since `sidebar_menu` only requests a certain number of branches, pass that "limit" all the way through to `git_repo.py`'s `branches` method so that `is_valid()` is only called a minimum number of times needed.
    * make sure the default branch logic to put it at the top of the list still works (e.g. always put it at the top first)

In addition to those changes, generalize and apply the same approach to the tags.  And also check ForgeHg to see if mercurial can benefit the same way.


---

Sent from forge-allura.apache.org because dev@allura.apache.org is subscribed to https://forge-allura.apache.org/p/allura/tickets/

To unsubscribe from further messages, a project admin can change settings at https://forge-allura.apache.org/p/allura/admin/tickets/options.  Or, if this is a mailing list, you can unsubscribe from the mailing list.

[allura:tickets] #7873 Git branch & tag speedups

Posted by Heith Seewald <hs...@slashdotmedia.com>.

- **status**: in-progress --> review
- **Comment**:

You were right Igor, when adding thousands of branches/tags, things became tricky.

The number of branches and tags were high enough that their initial loading was killing the performance. Due to the nature of git and symbolic references, it was necessary to loop through each tag/branch like we had been doing... But by removing the validation for each branch, the number of calls can be significantly reduced.

~~~~~
[Object(name=b.name, object_id=b.commit.hexsha) for b in self._git.branches]
~~~~~

*I attached two images of the cprofile breakdown of this loop*

Notice the cumtime is significantly higher when checking validation. Given the probability of a symbolic ref existing that doesn't point to a commit, this seems like a more than fair trade-off. Even if it does occur, it would have no major impact.

What do you think?

---

** [tickets:#7873] Git branch & tag speedups**

**Status:** review
**Milestone:** unreleased
**Labels:** performance sf-current sf-4
**Created:** Fri Apr 17, 2015 09:23 PM UTC by Dave Brondsema
**Last Updated:** Fri May 08, 2015 03:28 PM UTC
**Owner:** Heith Seewald

In addition to those changes, generalize and apply the same approach to the tags. And also check ForgeHg to see if mercurial can benefit the same way.

---

Sent from forge-allura.apache.org because dev@allura.apache.org is subscribed to https://forge-allura.apache.org/p/allura/tickets/

[allura:tickets] Re: #7873 Git branch & tag speedups

Posted by Dave Brondsema <da...@brondsema.net>.

I am still skeptical about fsck. Can you time it on a large repo? The
Allura repo might be a good start but there are much much bigger ones out
there too.

In principle I don't like doing extra work to handle a corner case when
try/except could too. But if it is fast all the time it could be ok.

--
Dave Brondsema
Principal Software Engineer - sourceforge.net

---

** [tickets:#7873] Git branch & tag speedups**

**Status:** in-progress
**Milestone:** unreleased
**Labels:** performance sf-current sf-4
**Created:** Fri Apr 17, 2015 09:23 PM UTC by Dave Brondsema
**Last Updated:** Wed Jun 10, 2015 05:20 PM UTC
**Owner:** Heith Seewald

In addition to those changes, generalize and apply the same approach to the tags. And also check ForgeHg to see if mercurial can benefit the same way.

---

Sent from forge-allura.apache.org because dev@allura.apache.org is subscribed to https://forge-allura.apache.org/p/allura/tickets/

[allura:tickets] Re: #7873 Git branch & tag speedups

Posted by Dave Brondsema <da...@brondsema.net>.

No is_valid/fsck would be even faster too right?

I am still skeptical about fsck. Can you time it on a large repo? The Allura repo might be a good start but there are much much bigger ones out there too.

In principle I don't like doing extra work to handle a corner case when try/except could too. But if it is really fast all the time it could be ok.

---

** [tickets:#7873] Git branch & tag speedups**

In addition to those changes, generalize and apply the same approach to the tags. And also check ForgeHg to see if mercurial can benefit the same way.

---

Sent from forge-allura.apache.org because dev@allura.apache.org is subscribed to https://forge-allura.apache.org/p/allura/tickets/

[allura:tickets] Re: #7873 Git branch & tag speedups

Posted by Heith Seewald <hs...@slashdotmedia.com>.

Checkout these simple timings:

~~~~~
        _start = time()
        a = self._get_refs(self._git.branches)
        print("Time using fsck : {}".format(time() - _start))


        _start = time()
        b = [Object(name=b.name, object_id=b.commit.hexsha) for b in self._git.branches if b.is_valid()]
        print("Validation Time (the old way): {}".format(time() - _start))

        print(a == b)
## Results
# Time using fsck : 1.61762309074
# Validation Time (the old way): 2.93375897408
# True

~~~~~

Of the time it took to get the list with `self._get_refs(self._git.branches)`,
fsck only accounted for: **0.121250049591** -- In that time, it checked 256 directories and 3498 objects (the result of which is cached via lazyprop so tags can make use of it too).


The only time the fsck method will be slower is the case where there is an actual error -- then it's the sum of both fsck and is_valid times.  But that is supposedly an edge case anyways, right?

If you guys think it'll cause issues down the road, I have no problem going another route, but it seems to scale better than our existing implementation.


Also, I really like the idea of using mongo as a 'smart cache' and invalidating via  repo_refresh!


---

** [tickets:#7873] Git branch & tag speedups**

**Status:** in-progress
**Milestone:** unreleased
**Labels:** performance sf-current sf-4 
**Created:** Fri Apr 17, 2015 09:23 PM UTC by Dave Brondsema
**Last Updated:** Wed Jun 10, 2015 05:20 PM UTC
**Owner:** Heith Seewald

I saw that git pages on forge-allura.apache.org were slow, so I looked at stats.log and saw that the sidebar was the slowest part.  I did some additional digging and found 2 specific areas for improvement:

* in `git_main.py` change `default_branch_name` to a `@LazyProperty` since it is called many times inside a loop in `RepositoryApp.sidebar_menu`
* Since `sidebar_menu` only requests a certain number of branches, pass that "limit" all the way through to `git_repo.py`'s `branches` method so that `is_valid()` is only called a minimum number of times needed.
    * make sure the default branch logic to put it at the top of the list still works (e.g. always put it at the top first)

In addition to those changes, generalize and apply the same approach to the tags.  And also check ForgeHg to see if mercurial can benefit the same way.


---

Sent from forge-allura.apache.org because dev@allura.apache.org is subscribed to https://forge-allura.apache.org/p/allura/tickets/

To unsubscribe from further messages, a project admin can change settings at https://forge-allura.apache.org/p/allura/admin/tickets/options.  Or, if this is a mailing list, you can unsubscribe from the mailing list.

[allura:tickets] #7873 Git branch & tag speedups

Posted by Dave Brondsema <da...@brondsema.net>.

The `fsck` operation checks all objects in a repo, so I think it's going to scale with the size of the repo overall, not just the number of branches. It seems overkill to me too, with the potential for being slow on big repos. (I also wonder how `fsck` could be faster than the `is_valid` checks since they'd both be going through and checking each ref)

I think a try/catch in the loop would be worth trying. I'd say change it from a list comprehension to a regular loop so you can put the try/catch around just the `ref.commit.hexsha` bit that triggers the exception).

Igor's suggestion of caching the results is also interesting. As he says, we're trying to do direct repo access rather than caching in mongo, but it would be much faster. I'm ok with using mongo storage if we treat it as a smart cache, not as a definitive datastore that must be populated by the "repo refresh" logic before repo webpages are usable. For example: the `branches` property would know how to get the info from the git repo and would cache the result in mongo and use that cache. Cache invalidation would happen whenever new changes are pushed. That'd be in the "repo refresh" functions, but would be fast and simple.

---

** [tickets:#7873] Git branch & tag speedups**

In addition to those changes, generalize and apply the same approach to the tags. And also check ForgeHg to see if mercurial can benefit the same way.

---

Sent from forge-allura.apache.org because dev@allura.apache.org is subscribed to https://forge-allura.apache.org/p/allura/tickets/

[allura:tickets] #7873 Git branch & tag speedups -- NEEDS INI

Posted by Heith Seewald <hs...@slashdotmedia.com>.

- **summary**: Git branch & tag speedups --> Git branch & tag speedups  -- NEEDS INI
- **status**: in-progress --> closed



---

** [tickets:#7873] Git branch & tag speedups  -- NEEDS INI**

**Status:** closed
**Milestone:** unreleased
**Labels:** performance sf-current sf-4 
**Created:** Fri Apr 17, 2015 09:23 PM UTC by Dave Brondsema
**Last Updated:** Wed Jun 17, 2015 03:13 PM UTC
**Owner:** Heith Seewald

I saw that git pages on forge-allura.apache.org were slow, so I looked at stats.log and saw that the sidebar was the slowest part.  I did some additional digging and found 2 specific areas for improvement:

* in `git_main.py` change `default_branch_name` to a `@LazyProperty` since it is called many times inside a loop in `RepositoryApp.sidebar_menu`
* Since `sidebar_menu` only requests a certain number of branches, pass that "limit" all the way through to `git_repo.py`'s `branches` method so that `is_valid()` is only called a minimum number of times needed.
    * make sure the default branch logic to put it at the top of the list still works (e.g. always put it at the top first)

In addition to those changes, generalize and apply the same approach to the tags.  And also check ForgeHg to see if mercurial can benefit the same way.


---

Sent from forge-allura.apache.org because dev@allura.apache.org is subscribed to https://forge-allura.apache.org/p/allura/tickets/

To unsubscribe from further messages, a project admin can change settings at https://forge-allura.apache.org/p/allura/admin/tickets/options.  Or, if this is a mailing list, you can unsubscribe from the mailing list.

[allura:tickets] #7873 Git branch & tag speedups

Posted by Heith Seewald <hs...@slashdotmedia.com>.

- **status**: in-progress --> review
- Attachments has changed:

Diff:

~~~~

--- old
+++ new
@@ -0,0 +1,2 @@
+old.png (61.1 kB; image/png)
+new.png (63.3 kB; image/png)

~~~~

- **Comment**:

Review at: **hs/7873**

TTFB was significantly reduced by removing the logic to locate the '`default_branch_name`'.  Time complexity went from **O(n)** to **O(1)**.

It turns out that the issue was unique to the branches processing -- `tags` and `ForgeHG` are unaffected.



---

** [tickets:#7873] Git branch & tag speedups**

**Status:** review
**Milestone:** unreleased
**Labels:** performance sf-current sf-4 
**Created:** Fri Apr 17, 2015 09:23 PM UTC by Dave Brondsema
**Last Updated:** Mon May 04, 2015 10:25 PM UTC
**Owner:** Heith Seewald

I saw that git pages on forge-allura.apache.org were slow, so I looked at stats.log and saw that the sidebar was the slowest part.  I did some additional digging and found 2 specific areas for improvement:

* in `git_main.py` change `default_branch_name` to a `@LazyProperty` since it is called many times inside a loop in `RepositoryApp.sidebar_menu`
* Since `sidebar_menu` only requests a certain number of branches, pass that "limit" all the way through to `git_repo.py`'s `branches` method so that `is_valid()` is only called a minimum number of times needed.
    * make sure the default branch logic to put it at the top of the list still works (e.g. always put it at the top first)

In addition to those changes, generalize and apply the same approach to the tags.  And also check ForgeHg to see if mercurial can benefit the same way.


---

Sent from forge-allura.apache.org because dev@allura.apache.org is subscribed to https://forge-allura.apache.org/p/allura/tickets/

To unsubscribe from further messages, a project admin can change settings at https://forge-allura.apache.org/p/allura/admin/tickets/options.  Or, if this is a mailing list, you can unsubscribe from the mailing list.

[allura:tickets] #7873 Git branch & tag speedups

Posted by Heith Seewald <hs...@slashdotmedia.com>.

Great notes -- The initial load is much quicker now :)  

Updated @ **hs/7873**

As for the configtree... do I just append CONFIGTREE to the ticket name?


---

** [tickets:#7873] Git branch & tag speedups**

**Status:** in-progress
**Milestone:** unreleased
**Labels:** performance sf-current sf-4 
**Created:** Fri Apr 17, 2015 09:23 PM UTC by Dave Brondsema
**Last Updated:** Tue Jun 16, 2015 11:32 AM UTC
**Owner:** Heith Seewald

I saw that git pages on forge-allura.apache.org were slow, so I looked at stats.log and saw that the sidebar was the slowest part.  I did some additional digging and found 2 specific areas for improvement:

* in `git_main.py` change `default_branch_name` to a `@LazyProperty` since it is called many times inside a loop in `RepositoryApp.sidebar_menu`
* Since `sidebar_menu` only requests a certain number of branches, pass that "limit" all the way through to `git_repo.py`'s `branches` method so that `is_valid()` is only called a minimum number of times needed.
    * make sure the default branch logic to put it at the top of the list still works (e.g. always put it at the top first)

In addition to those changes, generalize and apply the same approach to the tags.  And also check ForgeHg to see if mercurial can benefit the same way.


---

Sent from forge-allura.apache.org because dev@allura.apache.org is subscribed to https://forge-allura.apache.org/p/allura/tickets/

To unsubscribe from further messages, a project admin can change settings at https://forge-allura.apache.org/p/allura/admin/tickets/options.  Or, if this is a mailing list, you can unsubscribe from the mailing list.

[allura:tickets] #7873 Git branch & tag speedups

Posted by Igor Bondarenko <je...@gmail.com>.

- **status**: review --> in-progress
- **Comment**:

Looking good!

Overall timings are much better with caching enabled. First page hit still can be slow, though.  What do you think about updating the cache in `repo_refresh` also, not just clearing it up?

Minor issues:

- `print` statements in tests should be removed
- [test failures](https://forge-allura.apache.org/p/allura/pastebin/558005106d19cd113afb11b5)

 
SF-specific: I guess we need to add `repo_refs_cache_threshold = .5` to `configtree` also.



---

** [tickets:#7873] Git branch & tag speedups**

**Status:** in-progress
**Milestone:** unreleased
**Labels:** performance sf-current sf-4 
**Created:** Fri Apr 17, 2015 09:23 PM UTC by Dave Brondsema
**Last Updated:** Mon Jun 15, 2015 12:29 PM UTC
**Owner:** Heith Seewald

I saw that git pages on forge-allura.apache.org were slow, so I looked at stats.log and saw that the sidebar was the slowest part.  I did some additional digging and found 2 specific areas for improvement:

* in `git_main.py` change `default_branch_name` to a `@LazyProperty` since it is called many times inside a loop in `RepositoryApp.sidebar_menu`
* Since `sidebar_menu` only requests a certain number of branches, pass that "limit" all the way through to `git_repo.py`'s `branches` method so that `is_valid()` is only called a minimum number of times needed.
    * make sure the default branch logic to put it at the top of the list still works (e.g. always put it at the top first)

In addition to those changes, generalize and apply the same approach to the tags.  And also check ForgeHg to see if mercurial can benefit the same way.


---

Sent from forge-allura.apache.org because dev@allura.apache.org is subscribed to https://forge-allura.apache.org/p/allura/tickets/

To unsubscribe from further messages, a project admin can change settings at https://forge-allura.apache.org/p/allura/admin/tickets/options.  Or, if this is a mailing list, you can unsubscribe from the mailing list.

[allura:tickets] #7873 Git branch & tag speedups

Posted by Igor Bondarenko <je...@gmail.com>.

- **Reviewer**: Igor Bondarenko



---

** [tickets:#7873] Git branch & tag speedups**

**Status:** review
**Milestone:** unreleased
**Labels:** performance sf-current sf-4 
**Created:** Fri Apr 17, 2015 09:23 PM UTC by Dave Brondsema
**Last Updated:** Tue May 05, 2015 02:22 PM UTC
**Owner:** Heith Seewald

I saw that git pages on forge-allura.apache.org were slow, so I looked at stats.log and saw that the sidebar was the slowest part.  I did some additional digging and found 2 specific areas for improvement:

* in `git_main.py` change `default_branch_name` to a `@LazyProperty` since it is called many times inside a loop in `RepositoryApp.sidebar_menu`
* Since `sidebar_menu` only requests a certain number of branches, pass that "limit" all the way through to `git_repo.py`'s `branches` method so that `is_valid()` is only called a minimum number of times needed.
    * make sure the default branch logic to put it at the top of the list still works (e.g. always put it at the top first)

In addition to those changes, generalize and apply the same approach to the tags.  And also check ForgeHg to see if mercurial can benefit the same way.


---

Sent from forge-allura.apache.org because dev@allura.apache.org is subscribed to https://forge-allura.apache.org/p/allura/tickets/

To unsubscribe from further messages, a project admin can change settings at https://forge-allura.apache.org/p/allura/admin/tickets/options.  Or, if this is a mailing list, you can unsubscribe from the mailing list.

[allura:tickets] #7873 Git branch & tag speedups

Posted by Heith Seewald <hs...@slashdotmedia.com>.

- Attachments has changed:

Diff:

~~~~

--- old
+++ new
@@ -1,2 +1 @@
-old.png (61.1 kB; image/png)
 new.png (63.3 kB; image/png)

~~~~




---

** [tickets:#7873] Git branch & tag speedups**

**Status:** review
**Milestone:** unreleased
**Labels:** performance sf-current sf-4 
**Created:** Fri Apr 17, 2015 09:23 PM UTC by Dave Brondsema
**Last Updated:** Tue May 05, 2015 02:22 PM UTC
**Owner:** Heith Seewald

I saw that git pages on forge-allura.apache.org were slow, so I looked at stats.log and saw that the sidebar was the slowest part.  I did some additional digging and found 2 specific areas for improvement:

* in `git_main.py` change `default_branch_name` to a `@LazyProperty` since it is called many times inside a loop in `RepositoryApp.sidebar_menu`
* Since `sidebar_menu` only requests a certain number of branches, pass that "limit" all the way through to `git_repo.py`'s `branches` method so that `is_valid()` is only called a minimum number of times needed.
    * make sure the default branch logic to put it at the top of the list still works (e.g. always put it at the top first)

In addition to those changes, generalize and apply the same approach to the tags.  And also check ForgeHg to see if mercurial can benefit the same way.


---

Sent from forge-allura.apache.org because dev@allura.apache.org is subscribed to https://forge-allura.apache.org/p/allura/tickets/

To unsubscribe from further messages, a project admin can change settings at https://forge-allura.apache.org/p/allura/admin/tickets/options.  Or, if this is a mailing list, you can unsubscribe from the mailing list.

[allura:tickets] #7873 Git branch & tag speedups

Posted by Igor Bondarenko <je...@gmail.com>.

- **status**: review --> in-progress
- **Comment**:

Found one small issue. If default branch is in first 10 branches returned by `self.repo.get_branches()` you'll see two buttons for it in the sidebar (this isn't true when you have only one branch)

------


And a test failure:

~~~~
======================================================================
FAIL: forgegit.tests.functional.test_controllers.TestFork.test_merge_request_list_view
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/var/local/env-allura/lib/python2.7/site-packages/nose-1.3.4-py2.7.egg/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/home/ibondarenko/ibondarenko-1154/forge/ForgeGit/forgegit/tests/functional/test_controllers.py", line 585, in test_merge_request_list_view
    assert_equal(r.html.findAll('span')[8].getText(), 'less than 1 minute ago')
AssertionError: u'foo' != 'less than 1 minute ago'
-------------------- >> begin captured stdout << ---------------------
Running setup_app() from allura.websetup
http://localhost/p/test2/code/do_request_merge
 ==> 302 ==> http://localhost/p/test/src-git/merge-requests/1/
~~~~

------

Also, I don't sure there are any improvements on performance. I've imported allura repo itself for testing. I'm looking at `stats.log` and don't see any improvement in `sidebar` timings.

On master (timings fluctuate around 4200 in most cases):

~~~~
            "sidebar": 5145, 
            "sidebar": 4145, 
            "sidebar": 4211, 
~~~~

On a branch (sometimes even slightly worse, seems like it randomly spikes. 4000 then almost ~6700):

~~~~
            "sidebar": 6514, 
            "sidebar": 3916, 
            "sidebar": 6730, 
~~~~

(Hitting `/p/allura/allura/ci/master/tree/` page)

Could you describe how did you measure it? Maybe I'm missing something?



---

** [tickets:#7873] Git branch & tag speedups**

**Status:** in-progress
**Milestone:** unreleased
**Labels:** performance sf-current sf-4 
**Created:** Fri Apr 17, 2015 09:23 PM UTC by Dave Brondsema
**Last Updated:** Thu May 07, 2015 02:22 PM UTC
**Owner:** Heith Seewald

I saw that git pages on forge-allura.apache.org were slow, so I looked at stats.log and saw that the sidebar was the slowest part.  I did some additional digging and found 2 specific areas for improvement:

* in `git_main.py` change `default_branch_name` to a `@LazyProperty` since it is called many times inside a loop in `RepositoryApp.sidebar_menu`
* Since `sidebar_menu` only requests a certain number of branches, pass that "limit" all the way through to `git_repo.py`'s `branches` method so that `is_valid()` is only called a minimum number of times needed.
    * make sure the default branch logic to put it at the top of the list still works (e.g. always put it at the top first)

In addition to those changes, generalize and apply the same approach to the tags.  And also check ForgeHg to see if mercurial can benefit the same way.


---

Sent from forge-allura.apache.org because dev@allura.apache.org is subscribed to https://forge-allura.apache.org/p/allura/tickets/

To unsubscribe from further messages, a project admin can change settings at https://forge-allura.apache.org/p/allura/admin/tickets/options.  Or, if this is a mailing list, you can unsubscribe from the mailing list.

[allura:tickets] Re: #7873 Git branch & tag speedups

Posted by Heith Seewald <hs...@slashdotmedia.com>.

Haha, looks like it,  Here you go.


---

** [tickets:#7873] Git branch & tag speedups**

**Status:** review
**Milestone:** unreleased
**Labels:** performance sf-current sf-4 
**Created:** Fri Apr 17, 2015 09:23 PM UTC by Dave Brondsema
**Last Updated:** Fri Jun 05, 2015 07:17 PM UTC
**Owner:** Heith Seewald

I saw that git pages on forge-allura.apache.org were slow, so I looked at stats.log and saw that the sidebar was the slowest part.  I did some additional digging and found 2 specific areas for improvement:

* in `git_main.py` change `default_branch_name` to a `@LazyProperty` since it is called many times inside a loop in `RepositoryApp.sidebar_menu`
* Since `sidebar_menu` only requests a certain number of branches, pass that "limit" all the way through to `git_repo.py`'s `branches` method so that `is_valid()` is only called a minimum number of times needed.
    * make sure the default branch logic to put it at the top of the list still works (e.g. always put it at the top first)

In addition to those changes, generalize and apply the same approach to the tags.  And also check ForgeHg to see if mercurial can benefit the same way.


---

Sent from forge-allura.apache.org because dev@allura.apache.org is subscribed to https://forge-allura.apache.org/p/allura/tickets/

To unsubscribe from further messages, a project admin can change settings at https://forge-allura.apache.org/p/allura/admin/tickets/options.  Or, if this is a mailing list, you can unsubscribe from the mailing list.

[allura:tickets] Re: #7873 Git branch & tag speedups

Posted by Igor Bondarenko <je...@gmail.com>.

I can't see the attachments. Did you forget to add those? :)

---

** [tickets:#7873] Git branch & tag speedups**

**Status:** review
**Milestone:** unreleased
**Labels:** performance sf-current sf-4
**Created:** Fri Apr 17, 2015 09:23 PM UTC by Dave Brondsema
**Last Updated:** Fri Jun 05, 2015 07:17 PM UTC
**Owner:** Heith Seewald

In addition to those changes, generalize and apply the same approach to the tags. And also check ForgeHg to see if mercurial can benefit the same way.

---

Sent from forge-allura.apache.org because dev@allura.apache.org is subscribed to https://forge-allura.apache.org/p/allura/tickets/

[allura:tickets] Re: #7873 Git branch & tag speedups

Posted by Dave Brondsema <da...@brondsema.net>.

Actually I think "NEEDS INI" would be better on the ticket title since "configtree" is an internal thing to sourceforge.  (SF people can infer that INI means check configtree)


---

** [tickets:#7873] Git branch & tag speedups**

**Status:** in-progress
**Milestone:** unreleased
**Labels:** performance sf-current sf-4 
**Created:** Fri Apr 17, 2015 09:23 PM UTC by Dave Brondsema
**Last Updated:** Wed Jun 17, 2015 03:13 PM UTC
**Owner:** Heith Seewald

I saw that git pages on forge-allura.apache.org were slow, so I looked at stats.log and saw that the sidebar was the slowest part.  I did some additional digging and found 2 specific areas for improvement:

* in `git_main.py` change `default_branch_name` to a `@LazyProperty` since it is called many times inside a loop in `RepositoryApp.sidebar_menu`
* Since `sidebar_menu` only requests a certain number of branches, pass that "limit" all the way through to `git_repo.py`'s `branches` method so that `is_valid()` is only called a minimum number of times needed.
    * make sure the default branch logic to put it at the top of the list still works (e.g. always put it at the top first)

In addition to those changes, generalize and apply the same approach to the tags.  And also check ForgeHg to see if mercurial can benefit the same way.


---

Sent from forge-allura.apache.org because dev@allura.apache.org is subscribed to https://forge-allura.apache.org/p/allura/tickets/

To unsubscribe from further messages, a project admin can change settings at https://forge-allura.apache.org/p/allura/admin/tickets/options.  Or, if this is a mailing list, you can unsubscribe from the mailing list.