You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by Sam Toliman <iv...@gmail.com> on 2019/09/11 16:34:38 UTC
Authz perf regression 1.9 -> 1.10
Greetings,
TLDR since 1.10 subversion loading acl file ~15x times slower, which leads
in performance regression in tunnel mode used with svn+ssh scheme.
https://subversion.apache.org/docs/release-notes/1.10#authzperf says that
performance was improved, but we have discovered regression using large
access files (>2Mb) (depersonalized:
https://drive.google.com/file/d/1PP5n9BJkCs1Ssn47mHkI9dADPbo7DasJ/view).
Problem can be localized and reproduced using svnauthz-validate
(/usr/bin/time -v):
stats for https://github.com/apache/subversion/archive/1.9.12.tar.gz:
Command being timed:
"subversion-1.9.12/tools/server-side/svnauthz-validate acl.anon"
User time (seconds): 0.02
System time (seconds): 0.00
Percent of CPU this job got: 80%
Elapsed (wall clock) time (h:mm:ss or m:ss): *0:00.02*
Maximum resident set size (kbytes): *7656*
Minor (reclaiming a frame) page faults: *2812*
Voluntary context switches: 32
Involuntary context switches: 7
stats for https://github.com/apache/subversion/archive/1.10.6.tar.gz:
Command being timed:
"subversion-1.10.6/tools/server-side/svnauthz-validate acl.anon"
User time (seconds): 0.50
System time (seconds): 0.02
Percent of CPU this job got: 98%
Elapsed (wall clock) time (h:mm:ss or m:ss): *0:00.53*
Maximum resident set size (kbytes): *48704*
Minor (reclaiming a frame) page faults: *13103*
Voluntary context switches: 31
Involuntary context switches: 9
Last release still contains regression.
stats for https://github.com/apache/subversion/archive/1.12.2.tar.gz
Command being timed:
"subversion-1.12.2/tools/server-side/svnauthz-validate acl.anon"
User time (seconds): 0.48
System time (seconds): 0.02
Percent of CPU this job got: 98%
Elapsed (wall clock) time (h:mm:ss or m:ss): *0:00.52*
Maximum resident set size (kbytes): *49004*
Minor (reclaiming a frame) page faults: *13095*
Voluntary context switches: 32
Involuntary context switches: 11
Most of the time svnauthz-validate spends in the function svn_authz__parse (
https://github.com/apache/subversion/blob/f91051c82d38a732b546a84f4ead491514e81010/subversion/libsvn_repos/authz_parse.c#L1369).
It seems that in the tunnel mode redundantly parse all groups and all users
- it should be done only for tunnel_user.
Some host info:
>> lsb_release -r
Release: 16.04
>> uname -a
Linux host 4.4.171-70.1 #1 SMP Thu Jan 31 10:46:27 UTC 2019 x86_64 x86_64
x86_64 GNU/Linux
>> gcc --version
gcc (Ubuntu 5.4.0-6ubuntu1~16.04.11) 5.4.0 20160609
How subversion was built:
>> ./autogen.sh
>> ./configure
>> make
Thanks
Re: SVN-4832: Authz perf regression 1.9 -> 1.10
Posted by Julian Foad <ju...@apache.org>.
Bump. Is there anything further to report on this issue?
- Julian
Julian Foad wrote on 2019-09-25:
> Sam,
>
> Thank you for your help in identifying and resolving this problem.
>
> As this is a significant issue in an LTS release, I have filed it in the
> issue tracker:
>
> https://issues.apache.org/jira/browse/SVN-4832
> "Authz perf regression 1.9 -> 1.10"
Re: SVN-4832: Authz perf regression 1.9 -> 1.10
Posted by Julian Foad <ju...@apache.org>.
Sam,
Thank you for your help in identifying and resolving this problem.
As this is a significant issue in an LTS release, I have filed it in the
issue tracker:
https://issues.apache.org/jira/browse/SVN-4832
"Authz perf regression 1.9 -> 1.10"
and updated the email subject to contain the issue number.
- Julian
Re: Authz perf regression 1.9 -> 1.10
Posted by Sam Toliman <iv...@gmail.com>.
Many thanks for feedback!
It was rather a quick example to confirm the possibility to optimize this
function in my case.
Re: Authz perf regression 1.9 -> 1.10
Posted by Branko Čibej <br...@apache.org>.
On 12.09.2019 13:30, Sam Toliman wrote:
> >> (~3500k in my case)
> > You have more than 3 million groups?
> Misprinted. There are ~3500 groups in my case.
>
> > The best thing you can do is prepare a patch, with tests, that
> > implements your suggestions. Considering of course that correctness is
> > more important than performance. :)
> I will try to do it properly, in accordance with the algorithm
> described above.
>
> I have played with code base and ended up with minimal ad hoc diff
> required to speed up svn_authz__parse() [1]. It's certainly not a
> patch candidate, but can you please verify that this user filter
> doesn't break any implicit contracts or semantic agreements in the
> auth logic (see prepare_global_rights())?
Your change might break the semantics of the '*', '$anonymous' and
'$authenticated' ACE selectors. Any username that's not explicitly
mentioned in the authz file and group definitions is still subject to
those access entries. I'd have to re-read the code ... IIRC we do have
precooked global rights for those categories and check those first
before user-specific rights, but it's worth double-checking.
The other thing I can think of is that, even if this is only used in
tunnel mode, you should still make sure that the parsed internal
representation isn't cached. It would be very wrong indeed if the
user-filtered representation was reused for another user.
On that point, I'd suggest that the parser itself is *not* made aware
that it's running in some kind of "tunnel mode" -- that's far too much
information at too low a level, i.e., a layering violation. Renaming
'tunnel_user' to something else (e.g., 'only_user' or 'constrain_user'
or similar) would do the trick, as well as moving comments about tunnel
mode to the place that's actually aware of it and what it means.
> It gave some performance improvements (best of 10 runs):
> 1.10:
> time SVN_SSH='/tmp/svnserve1.10.sh <http://svnserve1.10.sh>' svn info
> svn+ssh://{server}/arc >/dev/null
> real 0m0,501s
> user 0m0,009s
> sys 0m0,008s
>
> 1.10 with patch:
> time SVN_SSH='/tmp/svnserve1.10_patch.sh
> <http://svnserve1.10_patch.sh>' svn info svn+ssh://{server}/arc >/dev/null
> real 0m0,134s
> user 0m0,001s
> sys 0m0,014s
That's a very nice result indeed, if only we can be certain of
correctness. (Hint: we don't have nearly enough tests for authz, and the
current ones almost certainly don't exercise tunnel mode).
(More comments below.)
> Index: subversion/libsvn_repos/authz_parse.c
> ===================================================================
> --- subversion/libsvn_repos/authz_parse.c (revision 5646686)
> +++ subversion/libsvn_repos/authz_parse.c (working copy)
> @@ -133,6 +133,10 @@
>
> N.B.: The result pool is AUTHZ->pool. */
> apr_pool_t *parser_pool;
> +
> + /* Stores tunnel_user to speed up ACL loading in tunnel mode.
> + For more info see ARCADIA-1621. */
> + const char *tunnel_user;
> } ctor_baton_t;
Please keep the 'parser_pool' member last in this structure. It's not a
public type, being defined in the implementation file, so reordering
members is OK. And I'm sure you'll come up with more generic wording for
the comment. :)
> @@ -1298,10 +1307,11 @@
> svn_stream_t *rules,
> svn_stream_t *groups,
> apr_pool_t *result_pool,
> - apr_pool_t *scratch_pool)
> + apr_pool_t *scratch_pool,
> + const char *tunnel_user)
Similarly, we have a convention that pool parameters are always last in
an argument list. I won't repeat this observation for the other cases in
the diff.
> {
> ctor_baton_t *const cb = create_ctor_baton(result_pool, scratch_pool);
> -
> + cb->tunnel_user = tunnel_user;
Anything that's stored in the constructor baton should be allocated in
its pool, so use something like this instead:
cb->tunnel_user = apr_pstrdup(cb->parser_pool, tunnel_user);
(Also leave the empty line.)
> Index: subversion/include/svn_repos.h
> ===================================================================
> --- subversion/include/svn_repos.h (revision 5646686)
> +++ subversion/include/svn_repos.h (working copy)
> @@ -4170,7 +4170,8 @@
> svn_boolean_t must_exist,
> svn_repos_t *repos_hint,
> apr_pool_t *result_pool,
> - apr_pool_t *scratch_pool);
> + apr_pool_t *scratch_pool,
> + const char *tunnel_user);
We need an API revision here, you can't just add parameters to an
existing public function. You'd have to create a new function,
'svn_repos_authz_read5', with the new parameter, then deprecate and
reimplement 'svn_repos_authz_read4' in deprecated.c as a wrapper for
'svn_repos_authz_read4' (there are several examples of how that's done
right there).
-- Brane
Re: Authz perf regression 1.9 -> 1.10
Posted by Sam Toliman <iv...@gmail.com>.
>> (~3500k in my case)
> You have more than 3 million groups?
Misprinted. There are ~3500 groups in my case.
> The best thing you can do is prepare a patch, with tests, that
> implements your suggestions. Considering of course that correctness is
> more important than performance. :)
I will try to do it properly, in accordance with the algorithm described
above.
I have played with code base and ended up with minimal ad hoc diff required
to speed up svn_authz__parse() [1]. It's certainly not a patch candidate,
but can you please verify that this user filter doesn't break any implicit
contracts or semantic agreements in the auth logic (see
prepare_global_rights())?
It gave some performance improvements (best of 10 runs):
1.10:
time SVN_SSH='/tmp/svnserve1.10.sh' svn info svn+ssh://{server}/arc
>/dev/null
real 0m0,501s
user 0m0,009s
sys 0m0,008s
1.10 with patch:
time SVN_SSH='/tmp/svnserve1.10_patch.sh' svn info svn+ssh://{server}/arc
>/dev/null
real 0m0,134s
user 0m0,001s
sys 0m0,014s
1.9:
time SVN_SSH='/tmp/svnserve1.9.sh' svn info svn+ssh://{server}/arc
>/dev/null
real 0m0,114s
user 0m0,001s
sys 0m0,016s
Tests passed:
make check...
Summary of test results:
2415 tests PASSED
94 tests SKIPPED
57 tests XFAILED (1 WORK-IN-PROGRESS)
SUMMARY: All tests successful.
/tmp/svnserve*.sh content:
#!/bin/bash
{path_to_svnserve} --memory-cache-size 128 --compression 0 -t -r
/storage/rep/ --config-file=/storage/rep/svnserve.conf --log-file /dev/null
[1] diff applied to 1.10:
Index: subversion/libsvn_repos/authz_parse.c
===================================================================
--- subversion/libsvn_repos/authz_parse.c (revision 5646686)
+++ subversion/libsvn_repos/authz_parse.c (working copy)
@@ -133,6 +133,10 @@
N.B.: The result pool is AUTHZ->pool. */
apr_pool_t *parser_pool;
+
+ /* Stores tunnel_user to speed up ACL loading in tunnel mode.
+ For more info see ARCADIA-1621. */
+ const char *tunnel_user;
} ctor_baton_t;
@@ -242,6 +246,11 @@
static void
prepare_global_rights(ctor_baton_t *cb, const char *user)
{
+ if (cb->tunnel_user)
+ {
+ if (0 != strcmp(cb->tunnel_user, user))
+ return;
+ }
authz_global_rights_t *gr = svn_hash_gets(cb->authz->user_rights, user);
if (!gr)
{
@@ -1298,10 +1307,11 @@
svn_stream_t *rules,
svn_stream_t *groups,
apr_pool_t *result_pool,
- apr_pool_t *scratch_pool)
+ apr_pool_t *scratch_pool,
+ const char *tunnel_user)
{
ctor_baton_t *const cb = create_ctor_baton(result_pool, scratch_pool);
-
+ cb->tunnel_user = tunnel_user;
/*
* Pass 1: Parse the authz file.
*/
Index: subversion/include/svn_repos.h
===================================================================
--- subversion/include/svn_repos.h (revision 5646686)
+++ subversion/include/svn_repos.h (working copy)
@@ -4170,7 +4170,8 @@
svn_boolean_t must_exist,
svn_repos_t *repos_hint,
apr_pool_t *result_pool,
- apr_pool_t *scratch_pool);
+ apr_pool_t *scratch_pool,
+ const char *tunnel_user);
/**
* Similar to svn_repos_authz_read3(), but with @a repos_hint set to @c
NULL.
Index: subversion/libsvn_repos/authz.c
===================================================================
--- subversion/libsvn_repos/authz.c (revision 5646686)
+++ subversion/libsvn_repos/authz.c (working copy)
@@ -1549,7 +1549,8 @@
svn_boolean_t must_exist,
svn_repos_t *repos_hint,
apr_pool_t *result_pool,
- apr_pool_t *scratch_pool)
+ apr_pool_t *scratch_pool,
+ const char *tunnel_user)
{
svn_error_t* err = NULL;
svn_stream_t *rules_stream = NULL;
@@ -1587,7 +1588,7 @@
/* Parse the configuration(s) and construct the full authz model
* from it. */
err = svn_authz__parse(authz_p, rules_stream, groups_stream,
- item_pool, scratch_pool);
+ item_pool, scratch_pool, tunnel_user);
if (err != SVN_NO_ERROR)
{
/* That pool would otherwise never get destroyed. */
@@ -1613,7 +1614,7 @@
* it. */
err = svn_error_quick_wrapf(svn_authz__parse(authz_p, rules_stream,
groups_stream,
- result_pool,
scratch_pool),
+ result_pool,
scratch_pool, tunnel_user),
"Error while parsing authz file: '%s':",
path);
}
@@ -1634,13 +1635,14 @@
svn_boolean_t must_exist,
svn_repos_t *repos_hint,
apr_pool_t *result_pool,
- apr_pool_t *scratch_pool)
+ apr_pool_t *scratch_pool,
+ const char *tunnel_user)
{
svn_authz_t *authz = apr_pcalloc(result_pool, sizeof(*authz));
authz->pool = result_pool;
SVN_ERR(authz_read(&authz->full, &authz->authz_id, path, groups_path,
- must_exist, repos_hint, result_pool, scratch_pool));
+ must_exist, repos_hint, result_pool, scratch_pool,
tunnel_user));
*authz_p = authz;
return SVN_NO_ERROR;
@@ -1657,7 +1659,7 @@
/* Parse the configuration and construct the full authz model from it. */
SVN_ERR(svn_authz__parse(&authz->full, stream, groups_stream, pool,
- scratch_pool));
+ scratch_pool, NULL));
svn_pool_destroy(scratch_pool);
Index: subversion/libsvn_repos/authz.h
===================================================================
--- subversion/libsvn_repos/authz.h (revision 5646686)
+++ subversion/libsvn_repos/authz.h (working copy)
@@ -304,7 +304,8 @@
svn_stream_t *rules,
svn_stream_t *groups,
apr_pool_t *result_pool,
- apr_pool_t *scratch_pool);
+ apr_pool_t *scratch_pool,
+ const char *tunnel_user);
/* Reverse a STRING of length LEN in place. */
Index: subversion/libsvn_repos/deprecated.c
===================================================================
--- subversion/libsvn_repos/deprecated.c (revision 5646686)
+++ subversion/libsvn_repos/deprecated.c (working copy)
@@ -1282,7 +1282,7 @@
apr_pool_t *scratch_pool = svn_pool_create(pool);
svn_error_t *err = svn_repos_authz_read3(authz_p, path, groups_path,
must_exist, NULL,
- pool, scratch_pool);
+ pool, scratch_pool, NULL);
svn_pool_destroy(scratch_pool);
return svn_error_trace(err);
Index: subversion/mod_authz_svn/mod_authz_svn.c
===================================================================
--- subversion/mod_authz_svn/mod_authz_svn.c (revision 5646686)
+++ subversion/mod_authz_svn/mod_authz_svn.c (working copy)
@@ -475,7 +475,7 @@
svn_err = svn_repos_authz_read3(&access_conf, access_file,
groups_file, TRUE, NULL,
r->connection->pool,
- scratch_pool);
+ scratch_pool, NULL);
if (svn_err)
{
Index: subversion/svnserve/serve.c
===================================================================
--- subversion/svnserve/serve.c (revision 5646686)
+++ subversion/svnserve/serve.c (working copy)
@@ -294,7 +294,8 @@
load_authz_config(repository_t *repository,
const char *repos_root,
svn_config_t *cfg,
- apr_pool_t *pool)
+ apr_pool_t *pool,
+ const char *tunnel_user)
{
const char *authzdb_path;
const char *groupsdb_path;
@@ -323,7 +324,7 @@
if (!err)
err = svn_repos_authz_read3(&repository->authzdb, authzdb_path,
groupsdb_path, TRUE, repository->repos,
- pool, pool);
+ pool, pool, tunnel_user);
if (err)
return svn_error_create(SVN_ERR_AUTHZ_INVALID_CONFIG, err, NULL);
@@ -3794,7 +3795,8 @@
svn_repos__config_pool_t *config_pool,
apr_hash_t *fs_config,
apr_pool_t *result_pool,
- apr_pool_t *scratch_pool)
+ apr_pool_t *scratch_pool,
+ const char *tunnel_user)
{
const char *path, *full_path, *fs_path, *hooks_env;
svn_stringbuf_t *url_buf;
@@ -3870,7 +3872,7 @@
SVN_ERR(load_pwdb_config(repository, cfg, config_pool, result_pool));
SVN_ERR(load_authz_config(repository, repository->repos_root, cfg,
- result_pool));
+ result_pool, tunnel_user));
/* Should we use Cyrus SASL? */
SVN_ERR(svn_config_get_bool(cfg, &sasl_requested,
@@ -4218,7 +4220,8 @@
b->read_only, params->cfg,
b->repository, params->config_pool,
params->fs_config,
- conn_pool, scratch_pool),
+ conn_pool, scratch_pool,
+ b->client_info->tunnel_user),
b);
if (!err)
{
Index: tools/server-side/svnauthz.c
===================================================================
--- tools/server-side/svnauthz.c (revision 5646686)
+++ tools/server-side/svnauthz.c (working copy)
@@ -277,7 +277,7 @@
/* Else */
return svn_repos_authz_read3(authz, opt_state->authz_file,
opt_state->groups_file,
- TRUE, NULL, pool, pool);
+ TRUE, NULL, pool, pool, NULL);
}
static svn_error_t *
Re: Authz perf regression 1.9 -> 1.10
Posted by Branko Čibej <br...@apache.org>.
On 12.09.2019 10:44, Sam Toliman wrote:
> > How do you propose that the parser should know, in advance, which parts
> > of the authz and groups files it can discard unless it parses all of
> them?
>
> By this point
> <https://github.com/apache/subversion/blob/6fb211a2f84c9fe69025e16bc564d2f8dc343249/subversion/svnserve/serve.c#L4261>
> b->client_info->tunnel_user is properly set only in the tunnel mode. I
> think it can be passed to the svn_authz__parse(), which should become
> slightly more intelligent passing stream and tunnel_user to specific
> callbacks for the [groups] and the [aliases] sections. By default
> there are no discovered groups. Surfing the group section parser
> should skip all users except tunnel_user and already discovered
> groups, otherwise add such group with relevant data to the set of
> discovered. Same for aliases.
> Thus we will have parsed only groups related to a particular user and
> svn_authz__parse() won't need to to expand all irrelevant groups
> <https://github.com/apache/subversion/blob/f91051c82d38a732b546a84f4ead491514e81010/subversion/libsvn_repos/authz_parse.c#L1417>
> (~3500k in my case)
You have more than 3 million groups?
> and expand every acl (~100 in my case) applying access rules to all
> irrelevant users
> <https://github.com/apache/subversion/blob/f91051c82d38a732b546a84f4ead491514e81010/subversion/libsvn_repos/authz_parse.c#L1452>
> (~35k in my case).
>
> Please correct me if I am naive somewhere.
The best thing you can do is prepare a patch, with tests, that
implements your suggestions. Considering of course that correctness is
more important than performance. :)
-- Brane
Re: Authz perf regression 1.9 -> 1.10
Posted by Sam Toliman <iv...@gmail.com>.
> How do you propose that the parser should know, in advance, which parts
> of the authz and groups files it can discard unless it parses all of them?
By this point
<https://github.com/apache/subversion/blob/6fb211a2f84c9fe69025e16bc564d2f8dc343249/subversion/svnserve/serve.c#L4261>
b->client_info->tunnel_user is properly set only in the tunnel mode. I
think it can be passed to the svn_authz__parse(), which should become
slightly more intelligent passing stream and tunnel_user to specific
callbacks for the [groups] and the [aliases] sections. By default there are
no discovered groups. Surfing the group section parser should skip all
users except tunnel_user and already discovered groups, otherwise add such
group with relevant data to the set of discovered. Same for aliases.
Thus we will have parsed only groups related to a particular user and
svn_authz__parse() won't need to to expand all irrelevant groups
<https://github.com/apache/subversion/blob/f91051c82d38a732b546a84f4ead491514e81010/subversion/libsvn_repos/authz_parse.c#L1417>
(~3500k in my case) and expand every acl (~100 in my case) applying access
rules to all irrelevant users
<https://github.com/apache/subversion/blob/f91051c82d38a732b546a84f4ead491514e81010/subversion/libsvn_repos/authz_parse.c#L1452>
(~35k in my case).
Please correct me if I am naive somewhere.
ср, 11 сент. 2019 г. в 19:34, Sam Toliman <iv...@gmail.com>:
> Greetings,
>
> TLDR since 1.10 subversion loading acl file ~15x times slower, which leads
> in performance regression in tunnel mode used with svn+ssh scheme.
>
> https://subversion.apache.org/docs/release-notes/1.10#authzperf says that
> performance was improved, but we have discovered regression using large
> access files (>2Mb) (depersonalized:
> https://drive.google.com/file/d/1PP5n9BJkCs1Ssn47mHkI9dADPbo7DasJ/view).
>
> Problem can be localized and reproduced using svnauthz-validate
> (/usr/bin/time -v):
>
> stats for https://github.com/apache/subversion/archive/1.9.12.tar.gz:
> Command being timed:
> "subversion-1.9.12/tools/server-side/svnauthz-validate acl.anon"
> User time (seconds): 0.02
> System time (seconds): 0.00
> Percent of CPU this job got: 80%
> Elapsed (wall clock) time (h:mm:ss or m:ss): *0:00.02*
> Maximum resident set size (kbytes): *7656*
> Minor (reclaiming a frame) page faults: *2812*
> Voluntary context switches: 32
> Involuntary context switches: 7
>
> stats for https://github.com/apache/subversion/archive/1.10.6.tar.gz:
> Command being timed:
> "subversion-1.10.6/tools/server-side/svnauthz-validate acl.anon"
> User time (seconds): 0.50
> System time (seconds): 0.02
> Percent of CPU this job got: 98%
> Elapsed (wall clock) time (h:mm:ss or m:ss): *0:00.53*
> Maximum resident set size (kbytes): *48704*
> Minor (reclaiming a frame) page faults: *13103*
> Voluntary context switches: 31
> Involuntary context switches: 9
>
> Last release still contains regression.
> stats for https://github.com/apache/subversion/archive/1.12.2.tar.gz
> Command being timed:
> "subversion-1.12.2/tools/server-side/svnauthz-validate acl.anon"
> User time (seconds): 0.48
> System time (seconds): 0.02
> Percent of CPU this job got: 98%
> Elapsed (wall clock) time (h:mm:ss or m:ss): *0:00.52*
> Maximum resident set size (kbytes): *49004*
> Minor (reclaiming a frame) page faults: *13095*
> Voluntary context switches: 32
> Involuntary context switches: 11
>
> Most of the time svnauthz-validate spends in the function svn_authz__parse
> (
> https://github.com/apache/subversion/blob/f91051c82d38a732b546a84f4ead491514e81010/subversion/libsvn_repos/authz_parse.c#L1369).
>
> It seems that in the tunnel mode redundantly parse all groups and all
> users - it should be done only for tunnel_user.
>
> Some host info:
> >> lsb_release -r
> Release: 16.04
> >> uname -a
> Linux host 4.4.171-70.1 #1 SMP Thu Jan 31 10:46:27 UTC 2019 x86_64 x86_64
> x86_64 GNU/Linux
> >> gcc --version
> gcc (Ubuntu 5.4.0-6ubuntu1~16.04.11) 5.4.0 20160609
>
> How subversion was built:
> >> ./autogen.sh
> >> ./configure
> >> make
>
> Thanks
>
Re: Authz perf regression 1.9 -> 1.10
Posted by Branko Čibej <br...@apache.org>.
On 11.09.2019 18:34, Sam Toliman wrote:
>
> Greetings,
>
> TLDR since 1.10 subversion loading acl file ~15x times slower, which
> leads in performance regression in tunnel mode used with svn+ssh scheme.
>
> https://subversion.apache.org/docs/release-notes/1.10#authzperf says
> that performance was improved, but we have discovered regression using
> large access files (>2Mb) (depersonalized:
> https://drive.google.com/file/d/1PP5n9BJkCs1Ssn47mHkI9dADPbo7DasJ/view).
>
The performance of authorization lookups was improved, by a couple
orders of magnitude. At the cost of making parsing to the internal
representation more complex. Not really surprising. That's why we added
a cache of the parsed results, but that only helps if there's a
long-running server process that can take advantage of the cache, which
you don't get in tunnel mode.
> It seems that in the tunnel mode redundantly parse all groups and all
> users - it should be done only for tunnel_user.
>
How do you propose that the parser should know, in advance, which parts
of the authz and groups files it can discard unless it parses all of them?
-- Brane