You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@allura.apache.org by Dave Brondsema <da...@brondsema.net> on 2017/10/17 20:40:24 UTC
[allura:tickets] #8168 Remove TreesDoc usage
---
** [tickets:#8168] Remove TreesDoc usage**
**Status:** open
**Milestone:** unreleased
**Labels:** indexless
**Created:** Tue Oct 17, 2017 08:40 PM UTC by Dave Brondsema
**Last Updated:** Tue Oct 17, 2017 08:40 PM UTC
**Owner:** nobody
I think we can remove the TreesDoc model and its mongo collection which can be quite large. [#7828] did some prior analysis. My current analysis is that some tests & scripts will have to be updated, but generally everything should keep working. Proof of concept is in branch `db/remove_treestuff`.
The TreesDoc model is used in some LastCommitDoc "refresh" logic currently. Should be able to remove that and/or replace it with calls to the lazy computation of LastCommitDocs. The only sticking point, I think, is that TreesDoc provides a mapping of commit_id to tree_id so that some cleanup knows what TreeDoc records can be deleted. So those would likely never be able to be cleaned up properly. Maybe ok, removing them is very infrequent - only for doing full repo refreshes, not even done when projects are removed (esp. since tree info can be shared across repos/forks)
---
Sent from forge-allura.apache.org because dev@allura.apache.org is subscribed to https://forge-allura.apache.org/p/allura/tickets/
To unsubscribe from further messages, a project admin can change settings at https://forge-allura.apache.org/p/allura/admin/tickets/options. Or, if this is a mailing list, you can unsubscribe from the mailing list.
[allura:tickets] #8168 Remove TreesDoc usage
Posted by Kenton Taylor <kt...@slashdotmedia.com>.
- **status**: review --> closed
---
** [tickets:#8168] Remove TreesDoc usage**
**Status:** closed
**Milestone:** unreleased
**Labels:** indexless
**Created:** Tue Oct 17, 2017 08:40 PM UTC by Dave Brondsema
**Last Updated:** Mon Oct 23, 2017 10:01 PM UTC
**Owner:** Dave Brondsema
I think we can remove the TreesDoc model and its mongo collection which can be quite large. [#7828] did some prior analysis. My current analysis is that some tests & scripts will have to be updated, but generally everything should keep working. Proof of concept is in branch `db/remove_treestuff`.
The TreesDoc model is used in some LastCommitDoc "refresh" logic currently. Should be able to remove that and/or replace it with calls to the lazy computation of LastCommitDocs. The only sticking point, I think, is that TreesDoc provides a mapping of commit_id to tree_id so that some cleanup knows what TreeDoc records can be deleted. So those would likely never be able to be cleaned up properly. Maybe ok, removing them is very infrequent - only for doing full repo refreshes, not even done when projects are removed (esp. since tree info can be shared across repos/forks)
---
Sent from forge-allura.apache.org because dev@allura.apache.org is subscribed to https://forge-allura.apache.org/p/allura/tickets/
To unsubscribe from further messages, a project admin can change settings at https://forge-allura.apache.org/p/allura/admin/tickets/options. Or, if this is a mailing list, you can unsubscribe from the mailing list.
[allura:tickets] #8168 Remove TreesDoc usage
Posted by Dave Brondsema <da...@brondsema.net>.
- **status**: open --> review
- **assigned_to**: Dave Brondsema
- **Comment**:
{allura,forgehg}:db/8168
To test this, use Git, SVN and Hg. If you have existing test repos, rename the `repo_trees` collection so it isn't used at all. Then push commits, and browse through the repo - in different dirs and at different commits/branches. Commit views and history views should be unaffected, don't need to test those much really - just the file/folder listings. Forked repos, and also merge commits might be good situations to test too.
Refresh a full repo like: `paster script conf.ini allura/scripts/refreshrepo.py -- --nbhd /p/ --project test --clean --all --mount-point foobar`
And this script rebuilds the "last commit" docs, although it isn't really used for anything, so not too important. `paster script conf.ini allura/scripts/refresh_last_commits.py -- --nbhd /p/ --project test --mount-point git2 --clean`
---
** [tickets:#8168] Remove TreesDoc usage**
**Status:** review
**Milestone:** unreleased
**Labels:** indexless
**Created:** Tue Oct 17, 2017 08:40 PM UTC by Dave Brondsema
**Last Updated:** Tue Oct 17, 2017 08:40 PM UTC
**Owner:** Dave Brondsema
I think we can remove the TreesDoc model and its mongo collection which can be quite large. [#7828] did some prior analysis. My current analysis is that some tests & scripts will have to be updated, but generally everything should keep working. Proof of concept is in branch `db/remove_treestuff`.
The TreesDoc model is used in some LastCommitDoc "refresh" logic currently. Should be able to remove that and/or replace it with calls to the lazy computation of LastCommitDocs. The only sticking point, I think, is that TreesDoc provides a mapping of commit_id to tree_id so that some cleanup knows what TreeDoc records can be deleted. So those would likely never be able to be cleaned up properly. Maybe ok, removing them is very infrequent - only for doing full repo refreshes, not even done when projects are removed (esp. since tree info can be shared across repos/forks)
---
Sent from forge-allura.apache.org because dev@allura.apache.org is subscribed to https://forge-allura.apache.org/p/allura/tickets/
To unsubscribe from further messages, a project admin can change settings at https://forge-allura.apache.org/p/allura/admin/tickets/options. Or, if this is a mailing list, you can unsubscribe from the mailing list.
[allura:tickets] #8168 Remove TreesDoc usage
Posted by Dave Brondsema <da...@brondsema.net>.
- **Milestone**: unreleased --> v1.8.0
---
** [tickets:#8168] Remove TreesDoc usage**
**Status:** closed
**Milestone:** v1.8.0
**Labels:** indexless
**Created:** Tue Oct 17, 2017 08:40 PM UTC by Dave Brondsema
**Last Updated:** Wed Oct 25, 2017 04:52 PM UTC
**Owner:** Dave Brondsema
I think we can remove the TreesDoc model and its mongo collection which can be quite large. [#7828] did some prior analysis. My current analysis is that some tests & scripts will have to be updated, but generally everything should keep working. Proof of concept is in branch `db/remove_treestuff`.
The TreesDoc model is used in some LastCommitDoc "refresh" logic currently. Should be able to remove that and/or replace it with calls to the lazy computation of LastCommitDocs. The only sticking point, I think, is that TreesDoc provides a mapping of commit_id to tree_id so that some cleanup knows what TreeDoc records can be deleted. So those would likely never be able to be cleaned up properly. Maybe ok, removing them is very infrequent - only for doing full repo refreshes, not even done when projects are removed (esp. since tree info can be shared across repos/forks)
---
Sent from forge-allura.apache.org because dev@allura.apache.org is subscribed to https://forge-allura.apache.org/p/allura/tickets/
To unsubscribe from further messages, a project admin can change settings at https://forge-allura.apache.org/p/allura/admin/tickets/options. Or, if this is a mailing list, you can unsubscribe from the mailing list.