Using mod_rewrite in my authorization module. Need advice.

[Kevac Marko <ma...@kevac.org>]

Hello.

I am porting our authorization module to mpm_worker from mpm_prefork.
This authorization module was using some ugly, but nevertheless
working hack which messes with mod_rewrite configuration.

This authorization module check user permissions for current url A and
rewrites url to AA or AB according to permission check result. Because
I didn't want to reimplement mod_rewrite and because mod_rewrite don't
have any API for url rewriting, this ugly hack was made. I am changing
mod_rewrite configuration on the fly, putting A->AA or A->AB
RewriteRule to it.

I hope you understood that. Ok. This works. But only in mpm_prefork,
where only one execution thread exists in apache process.

Now I need same functionality, but in mpm_worker. And I don't have
idea how to do it easily.

Only two ideas came into my mind:
1) Reimplement needed mod_rewrite functionality in out authorization module.
2) Patch or fork mod_rewrite module. Implement some API (like
rewrite(from, to);) which could be used for url rewriting with full
mod_rewrite power.
3) Patch mod_rewrite and enclose RewriteRules configuration with
locks. Sick! Ugly!

What do you think? I am looking forward for comments and advices. Thanks.

-- 
Marko Kevac
Sent from Moscow, Mow, Russia

Re: Using mod_rewrite in my authorization module. Need advice.

[Rainer Jung <ra...@kippdata.de>]

On 04.02.2010 08:56, Kevac Marko wrote:
> On Thu, Feb 4, 2010 at 10:52 AM, Sorin Manolache<so...@gmail.com>  wrote:
>> Try to set an apache request note in the authentication instead of
>> dynamically changing the configuration of mod_rewrite.c.
>>
>> Thus, you'll have something like
>>
>> RewriteRule /url %{ENV:destination}
>>
>> The configuration would be always the same, just the value of the
>> "destination" request note changes dynamically, depending on the
>> authentication.
>
> Hmmmm... Interesting. Are you sure that this works?
>
> Can I do something like RewriteRule %{ENV:source} %{ENV:destination} ?

You can check environment variables in RewriteCond and set them in 
RewriteRule.

Regards,

Rainer

Re: Using mod_rewrite in my authorization module. Need advice.

[Kevac Marko <ma...@kevac.org>]

On Thu, Feb 4, 2010 at 2:05 PM, Sorin Manolache <so...@gmail.com> wrote:
> No, you cannot. The expansion does not work in the pattern (the second
> argument).
>
> But the second argument can be a regular expression. Hopefully you can
> write regexps for all your cases.

Unfortunately I cannot. These should be in DB also, not in httpd.conf.

Right now I am rewriting basic rewrite functionality with some
mod_rewrite features like [QCA]. I hope this will be enough.



-- 
Marko Kevac
Sent from Moscow, Mow, Russia

Re: Using mod_rewrite in my authorization module. Need advice.

[Sorin Manolache <so...@gmail.com>]

On Thu, Feb 4, 2010 at 08:56, Kevac Marko <ma...@kevac.org> wrote:
> On Thu, Feb 4, 2010 at 10:52 AM, Sorin Manolache <so...@gmail.com> wrote:
>> Try to set an apache request note in the authentication instead of
>> dynamically changing the configuration of mod_rewrite.c.
>>
>> Thus, you'll have something like
>>
>> RewriteRule /url %{ENV:destination}
>>
>> The configuration would be always the same, just the value of the
>> "destination" request note changes dynamically, depending on the
>> authentication.
>
> Hmmmm... Interesting. Are you sure that this works?

The expansion of %{ENV:...} works in the replacement (last argument).

> Can I do something like RewriteRule %{ENV:source} %{ENV:destination} ?

No, you cannot. The expansion does not work in the pattern (the second
argument).

But the second argument can be a regular expression. Hopefully you can
write regexps for all your cases.

S

Re: Using mod_rewrite in my authorization module. Need advice.

[Kevac Marko <ma...@kevac.org>]

On Thu, Feb 4, 2010 at 10:52 AM, Sorin Manolache <so...@gmail.com> wrote:
> Try to set an apache request note in the authentication instead of
> dynamically changing the configuration of mod_rewrite.c.
>
> Thus, you'll have something like
>
> RewriteRule /url %{ENV:destination}
>
> The configuration would be always the same, just the value of the
> "destination" request note changes dynamically, depending on the
> authentication.

Hmmmm... Interesting. Are you sure that this works?

Can I do something like RewriteRule %{ENV:source} %{ENV:destination} ?

-- 
Marko Kevac
Sent from Moscow, Mow, Russia

Re: Using mod_rewrite in my authorization module. Need advice.

[Sorin Manolache <so...@gmail.com>]

On Thu, Feb 4, 2010 at 08:20, Kevac Marko <ma...@kevac.org> wrote:
> Thanks a lot for your comments.
>
> I'll try to describe requirements for our authorization module and
> current implementation a little bit more.
>
> Requirements:
> 1. Authorization must happen before mod_cache (also a little bit fixed
> for our needs), so it must be executed in 'quick_handler'. This is
> because in our architecture, almost every request is cached. Normal
> requests and authorized requests. All of them.
> 2. Authorization configuration should be taken from DB, because it is
> changed a lot and restarting Apache every time is stupid thing to do
> in this case.
> 3. Authorization rules should consist of regular expression, which
> matches some url and two substitution strings. One for 'granted' and
> one for 'denied'. This is because sometimes, when user don't have
> appropriate rights, we should send him same data, but maybe slightly
> different. For example real time data and 20 minutes delayed data.
> Sometimes we should send him to 401, sometimes to 403. It depends on
> which url user is accessing.
>
> Current implementation:
> 1. We keep authorization rules (consisting of url regexp and two
> substitution strings) in DB.
> 2. In apache.conf there are stub RewriteRule, which is dynamically
> substituted with new one. New one, which consist of url regexp A and
> substitute string AA or AB according to authorization result.
> 3. User Request -> Our Authorization module (authorization check +
> mod_rewrite configuration dynamically changed) -> mod_cache ->
> mod_rewrite substitution with our dynamic rule -> mod_proxy or
> anything else

Try to set an apache request note in the authentication instead of
dynamically changing the configuration of mod_rewrite.c.

Thus, you'll have something like

RewriteRule /url %{ENV:destination}

The configuration would be always the same, just the value of the
"destination" request note changes dynamically, depending on the
authentication.

S


>
> This is working great in mpm_prefork where only one thread at a time
> reads mod_rewrite configuration, but will not work in mpm_worker,
> where many threads read mod_rewrite configuration at the same time.
>
> I hope it's more clear now. Our architecture is NOT standard. So many
> things can be unusual. Let's not dig too deeply here :-)
> Most non standardiness is in the way we are caching things. We are
> caching mostly everything, including authorized requests. So
> authorization must happen before mod_cache.
>
> Now i'll try to answer on each comment.
>
> On Thu, Feb 4, 2010 at 12:44 AM, Sorin Manolache <so...@gmail.com> wrote:
>> I think it would be clearer and we could help you better if you pasted
>> some snippets of your code and hack.
>
> I hope it is clearer now, but i'll paste snippets if anything is not
> enough clear.
>
>> I didn't really understand what you wanted to do. And I don't really
>> understand why mono-threading would break your hack. Normally a
>> request is processed by a single thread anyway.
>
> Mono threading (mpm_prefork) will not break it. Multi threading
> (mpm_worker) will, because more than one threat at a time reads
> mod_rewrite configuration, which I am messing up with.
>
>> But look what you could do in your authentication module: You could
>> set an apache request note (or an environment variable) depending on
>> the authentication. And then call ap_run_translate_name that
>> re-invokes the rewrite rules.
>>
>> Something like
>>
>> determine if the user is priviledged or not
>> apr_table_set(r->notes, "auth_status", priviledged_user ? "yes" : "no");
>> ap_run_translate_name(r);
>> return OK;
>>
>> and then in the configuration file you put
>>
>> RewriteCond  %{ENV:auth_status} yes
>> RewriteRule url /destination_priviledged
>>
>> RewriteCond  %{ENV:auth_status} no
>> RewriteRule url /destination_ordinary
>>
>> The first time the translate_name is called, the auth_status note is
>> not defined. Thus no rewrite rule is applied and the URL is unchanged.
>> Next apache executes the request authentication hook from where you
>> invoke again the translate_name callback.
>
> I could do that if our rules are kept in apache.conf, but they are
> not. They are kept in database.
> I cant keep them in apache.conf, because each change will require
> apache restart.
>
>> By the way. The order in which the callbacks are called is
>> translate_name, then authentication/authorisation and last fixups.
>
> Our authorization module is not standard authorization module, so our
> order of execution is another.
> -> authorization -> cache -> translate_name -> fixups
>
> --
> Marko Kevac
>



-- 
A: Because it reverses the logical flow of conversation.
Q: Why is top-posting frowned upon?
A: Top-posting.
Q: What is the most annoying thing in e-mail?

Re: Using mod_rewrite in my authorization module. Need advice.

[Kevac Marko <ma...@kevac.org>]

Thanks a lot for your comments.

I'll try to describe requirements for our authorization module and
current implementation a little bit more.

Requirements:
1. Authorization must happen before mod_cache (also a little bit fixed
for our needs), so it must be executed in 'quick_handler'. This is
because in our architecture, almost every request is cached. Normal
requests and authorized requests. All of them.
2. Authorization configuration should be taken from DB, because it is
changed a lot and restarting Apache every time is stupid thing to do
in this case.
3. Authorization rules should consist of regular expression, which
matches some url and two substitution strings. One for 'granted' and
one for 'denied'. This is because sometimes, when user don't have
appropriate rights, we should send him same data, but maybe slightly
different. For example real time data and 20 minutes delayed data.
Sometimes we should send him to 401, sometimes to 403. It depends on
which url user is accessing.

Current implementation:
1. We keep authorization rules (consisting of url regexp and two
substitution strings) in DB.
2. In apache.conf there are stub RewriteRule, which is dynamically
substituted with new one. New one, which consist of url regexp A and
substitute string AA or AB according to authorization result.
3. User Request -> Our Authorization module (authorization check +
mod_rewrite configuration dynamically changed) -> mod_cache ->
mod_rewrite substitution with our dynamic rule -> mod_proxy or
anything else

This is working great in mpm_prefork where only one thread at a time
reads mod_rewrite configuration, but will not work in mpm_worker,
where many threads read mod_rewrite configuration at the same time.

I hope it's more clear now. Our architecture is NOT standard. So many
things can be unusual. Let's not dig too deeply here :-)
Most non standardiness is in the way we are caching things. We are
caching mostly everything, including authorized requests. So
authorization must happen before mod_cache.

Now i'll try to answer on each comment.

On Thu, Feb 4, 2010 at 12:44 AM, Sorin Manolache <so...@gmail.com> wrote:
> I think it would be clearer and we could help you better if you pasted
> some snippets of your code and hack.

I hope it is clearer now, but i'll paste snippets if anything is not
enough clear.

> I didn't really understand what you wanted to do. And I don't really
> understand why mono-threading would break your hack. Normally a
> request is processed by a single thread anyway.

Mono threading (mpm_prefork) will not break it. Multi threading
(mpm_worker) will, because more than one threat at a time reads
mod_rewrite configuration, which I am messing up with.

> But look what you could do in your authentication module: You could
> set an apache request note (or an environment variable) depending on
> the authentication. And then call ap_run_translate_name that
> re-invokes the rewrite rules.
>
> Something like
>
> determine if the user is priviledged or not
> apr_table_set(r->notes, "auth_status", priviledged_user ? "yes" : "no");
> ap_run_translate_name(r);
> return OK;
>
> and then in the configuration file you put
>
> RewriteCond  %{ENV:auth_status} yes
> RewriteRule url /destination_priviledged
>
> RewriteCond  %{ENV:auth_status} no
> RewriteRule url /destination_ordinary
>
> The first time the translate_name is called, the auth_status note is
> not defined. Thus no rewrite rule is applied and the URL is unchanged.
> Next apache executes the request authentication hook from where you
> invoke again the translate_name callback.

I could do that if our rules are kept in apache.conf, but they are
not. They are kept in database.
I cant keep them in apache.conf, because each change will require
apache restart.

> By the way. The order in which the callbacks are called is
> translate_name, then authentication/authorisation and last fixups.

Our authorization module is not standard authorization module, so our
order of execution is another.
-> authorization -> cache -> translate_name -> fixups

-- 
Marko Kevac

Re: Using mod_rewrite in my authorization module. Need advice.

[Sorin Manolache <so...@gmail.com>]

On Wed, Feb 3, 2010 at 22:24, Kevac Marko <ma...@kevac.org> wrote:
> Hello.
>
> I am porting our authorization module to mpm_worker from mpm_prefork.
> This authorization module was using some ugly, but nevertheless
> working hack which messes with mod_rewrite configuration.
>
> This authorization module check user permissions for current url A and
> rewrites url to AA or AB according to permission check result. Because
> I didn't want to reimplement mod_rewrite and because mod_rewrite don't
> have any API for url rewriting, this ugly hack was made. I am changing
> mod_rewrite configuration on the fly, putting A->AA or A->AB
> RewriteRule to it.
>
> I hope you understood that. Ok. This works. But only in mpm_prefork,
> where only one execution thread exists in apache process.
>
> Now I need same functionality, but in mpm_worker. And I don't have
> idea how to do it easily.
>
> Only two ideas came into my mind:
> 1) Reimplement needed mod_rewrite functionality in out authorization module.
> 2) Patch or fork mod_rewrite module. Implement some API (like
> rewrite(from, to);) which could be used for url rewriting with full
> mod_rewrite power.
> 3) Patch mod_rewrite and enclose RewriteRules configuration with
> locks. Sick! Ugly!
>
> What do you think? I am looking forward for comments and advices. Thanks.
>

I think it would be clearer and we could help you better if you pasted
some snippets of your code and hack.

I didn't really understand what you wanted to do. And I don't really
understand why mono-threading would break your hack. Normally a
request is processed by a single thread anyway.

But look what you could do in your authentication module: You could
set an apache request note (or an environment variable) depending on
the authentication. And then call ap_run_translate_name that
re-invokes the rewrite rules.

Something like

determine if the user is priviledged or not
apr_table_set(r->notes, "auth_status", priviledged_user ? "yes" : "no");
ap_run_translate_name(r);
return OK;

and then in the configuration file you put

RewriteCond  %{ENV:auth_status} yes
RewriteRule url /destination_priviledged

RewriteCond  %{ENV:auth_status} no
RewriteRule url /destination_ordinary

The first time the translate_name is called, the auth_status note is
not defined. Thus no rewrite rule is applied and the URL is unchanged.
Next apache executes the request authentication hook from where you
invoke again the translate_name callback.

S

Re: Using mod_rewrite in my authorization module. Need advice.

[Sorin Manolache <so...@gmail.com>]

By the way. The order in which the callbacks are called is
translate_name, then authentication/authorisation and last fixups.

If you put the RewriteRule at server level then it is applied during
the translate_name callback.

If you put it inside a <Location> directive, it is applied during the
fixups callback (i.e. _after_ the authentication/authorisation!)

S


On Wed, Feb 3, 2010 at 22:24, Kevac Marko <ma...@kevac.org> wrote:
> Hello.
>
> I am porting our authorization module to mpm_worker from mpm_prefork.
> This authorization module was using some ugly, but nevertheless
> working hack which messes with mod_rewrite configuration.
>
> This authorization module check user permissions for current url A and
> rewrites url to AA or AB according to permission check result. Because
> I didn't want to reimplement mod_rewrite and because mod_rewrite don't
> have any API for url rewriting, this ugly hack was made. I am changing
> mod_rewrite configuration on the fly, putting A->AA or A->AB
> RewriteRule to it.
>
> I hope you understood that. Ok. This works. But only in mpm_prefork,
> where only one execution thread exists in apache process.
>
> Now I need same functionality, but in mpm_worker. And I don't have
> idea how to do it easily.
>
> Only two ideas came into my mind:
> 1) Reimplement needed mod_rewrite functionality in out authorization module.
> 2) Patch or fork mod_rewrite module. Implement some API (like
> rewrite(from, to);) which could be used for url rewriting with full
> mod_rewrite power.
> 3) Patch mod_rewrite and enclose RewriteRules configuration with
> locks. Sick! Ugly!
>
> What do you think? I am looking forward for comments and advices. Thanks.
>
> --
> Marko Kevac
> Sent from Moscow, Mow, Russia
>



-- 
A: Because it reverses the logical flow of conversation.
Q: Why is top-posting frowned upon?
A: Top-posting.
Q: What is the most annoying thing in e-mail?

Re: Using mod_rewrite in my authorization module. Need advice.

[Kevac Marko <ma...@kevac.org>]

On Thu, Feb 4, 2010 at 1:12 AM, Ray Morris <su...@bettercgi.com> wrote:
>  That's three ideas. :)  Seriously, I suggest you back up
> a step or three.  You said "Reimplement needed mod_rewrite
> functionality in out authorization module.".  98% of mod_rewrite
> consists of handling it's very flexible configuration
> syntax. It sounds like you're not making use of any of that.
> You simply want to rewrite (or redirect) the URL based on
> some condition.  Either of those can be done with exactly
> TWO lines of code in your own module - no need to hijack
> another module and muck around where you have no business
> mucking around.
>
>   To REDIRECT to another URL:
>
> apr_table_setn(r->headers_out, "Location", r->filename);
> return HTTP_MOVED_TEMPORARILY;
>
>   To REWRITE the URL so some other URL:
>
> r->uri = apr_pstrdup(r->pool, newurl);
> return DECLINED;
>
>    That's probably all of the "re-implementing" you'd
> need - two lines of code.

Well, pretty much you are right, but it's not that simple. We need
some of the things that are put in square brackets in RewriteRule
configuration.
Probably re-implementing only needed functionality will be easier, yes...

>
>   That's backing up a step or two, but really it probably
> makes sense to back up another level:
>
>> This authorization module check user permissions for current
>> url A and rewrites url to AA or AB according to permission
>> check result.
>
>   You don't see "ErrorDocument 401" implemented in any
> of the standard authorization modules for a reason - because
> an authorization module is not the right place to do that.
> An authorization module should do one thing - authorization.
> Were to send them if they aren't authorized is set very nicely
> with "ErrorDocument 401".  That reduces your entire "ugly, but
> nevertheless (use to be) working hack which messes with mod_rewrite
> configuration" to exactly one line:
>
> return HTTP_UNAUTHORIZED;
>
>  You'll notice that's what all of the authorization modules
> written by the experts do when the client is not authorized -
> they return UNAUTHORIZED.  That then let's all of the other
> modules, the clients, and the monitoring systems do the right
> thing.  For example, the request will be correctly logged as
> unauthorized.  If your module incorrectly changes it to a 200
> code and displays a different page, it's then incorrectly logged
> as a succesful, fully authorized request.  Logging is one example -
> there are a thousand different things which may break when one
> module does the wrong thing and lies to everyone else about what's
> really going on.  As another example consider a security module
> or external system which watches for a large number of unauthorized
> responses to a single IP - detecting a brute force attack.
> By behaving correctly and simply returning HTTP_UNAUTHORIZED,
> your module is perfectly compatible with any other systems
> like that.
>
>   I understand that I've challenged some basic design decisions
> while attempting to be helpful to you.  Good programmers are almost
> always arrogant - arrogance seems to make us better programmers,
> so we tend to get defensive when someone suggests we look at something
> froma different perspective.  Still, I encourage you to consider how
> many times you've seen someone ask "what is the right way to do (the
> wrong thing)".   We all do that sometimes, so I would encourage you
> to consider taking a different perpective.

Maybe after reading my previous email with slightly detailed
description, you will understand why I have done all this non standard
things.

UNAUTHORIZED in our architecture is not so strict. Unauthorized user
sometimes can get same data, but slightly delayed version of them. So,
instead of 401 or 403, he will get 200 with delayed data. Sometimes
(for example when delayed data doesn't have sense), he must get 403,
sometimes he must get 401. It must be configurable.

-- 
Marko Kevac
Sent from Moscow, Mow, Russia