Performance

[Ingo Hornberger <in...@gmx.net>]

Hi!
We just migrated a great amount of user data of an existing forum into an
Allura Forge instance. I have to say, that there are reasons for this move.
We really enjoy Allura and the possible workflows (especially E-Mail
subscriptions and answers).

But we encountered a performance issue. Some requests are taking below 1s,
others are taking 4-8s.

I did a trace, and most of the time is lost in "ming".
Then I digged a bit deeper and found out, that the query of the subscribed
status for mailboxes is a big performance killer:

    @expose('jinja:forgediscussion:templates/index.html')
    @validate(dict(page=validators.Int(if_empty=0, if_invalid=0),
                   limit=validators.Int(if_empty=None, if_invalid=None)))
    def index(self, threads=None, limit=None, page=0, count=0, **kw):
        if self.discussion.deleted:
            redirect(self.discussion.url() + 'deleted')
        limit, page, start = g.handle_paging(limit, page)
        if not c.user.is_anonymous():
           * c.subscribed = M.Mailbox.subscribed(artifact=self.discussion)*

Do you have any hints how this can be improved?
Maybe manually in my db, or by patching the forum app?

BR,
Ingo

Re: Performance

[Dave Brondsema <da...@brondsema.net>]

On 3/3/20 10:08 AM, Ingo Hornberger wrote:
> Hi!
> We just migrated a great amount of user data of an existing forum into an
> Allura Forge instance. I have to say, that there are reasons for this move.
> We really enjoy Allura and the possible workflows (especially E-Mail
> subscriptions and answers).
> 
> But we encountered a performance issue. Some requests are taking below 1s,
> others are taking 4-8s.
> 
> I did a trace, and most of the time is lost in "ming".
> Then I digged a bit deeper and found out, that the query of the subscribed
> status for mailboxes is a big performance killer:
> 
>     @expose('jinja:forgediscussion:templates/index.html')
>     @validate(dict(page=validators.Int(if_empty=0, if_invalid=0),
>                    limit=validators.Int(if_empty=None, if_invalid=None)))
>     def index(self, threads=None, limit=None, page=0, count=0, **kw):
>         if self.discussion.deleted:
>             redirect(self.discussion.url() + 'deleted')
>         limit, page, start = g.handle_paging(limit, page)
>         if not c.user.is_anonymous():
>            * c.subscribed = M.Mailbox.subscribed(artifact=self.discussion)*
> 
> Do you have any hints how this can be improved?
> Maybe manually in my db, or by patching the forum app?
> 
> BR,
> Ingo
> 


I see that Mailbox.subscribed does a query on user_id, project_id,
app_config_id, artifact_index_id.  And Mailbox declares an index on
('project_id', 'artifact_index_id') and also ('project_id', 'app_config_id',
'artifact_index_id', 'topic') both of which overlap with the query fields and
can be used for that query.  In mongo do you see those indexes?  "use allura;
db.mailbox.getIndexes()"  If not, be sure to run the ensure_index command
https://forge-allura.apache.org/docs/getting_started/administration.html#ensure-index

In mongo you can also look at its slow query log.  Check the docs but I think
there are options to control what gets logged, and you can look at them in a log
file and also an internal slow queries collection.  Occasional slow queries
happen but if you're seeing the same type over and over its definitely something
to work on.  You can try running the query manually with the "explain" option
and see what the results are.  You can also try adding an even more specific
index to the "mailbox" collection if the existing indexes which partially match
the query are not working good enough.


-- 
Dave Brondsema : dave@brondsema.net
http://www.brondsema.net : personal
http://www.splike.com : programming
              <><