How to register additional operators or functions for use in expressions?

[Waldemar Klein <wa...@googlemail.com>]

Hi all,

looking at http://httpd.apache.org/docs/2.4/expr.html, i found things like
"Modules may register additional unary operators." or
"Modules may register additional functions."
but not the slightest hint how to actually do this, also googling for
some time didn't get me anything useful either.

To be more precise, i am _not_ looking to write functions to extend
the API and use in other modules (which is what all the Google
searches for "apache module functions" find). What i am looking for is
extending functions or operators for the expression parser to use in
httpd.conf. Also, i am not looking for a general description how to
write modules, I already wrote, compiled and ran simple modules (like
the usual "helloworld" handler). I am just looking for the specifics
on how to register functions/operators and maybe what else is needed
(probably something similar to the check "is it for me?" in a content
handler).


Some background if anyone is interested:
I am trying to write a function or (unary or binary) operator (either
would work for me) to check a URL against a list read from a file,
which i can use in an <If ..> block. I am not worried about writing
the actual doing in the module, like reading the file or matching the
URL, not because i am that good but the task is quite simple and there
is good documentation around :). Anyhow i don't know how to get
started with registring this function, and if any other checks have to
be done (see above).

I already tried this with a small list directly in a <If " ...
regexmatch here ...">, but i am afraid if this list gets really large
(thinking in the range of millions of URLs) there will be either a
limit to it where it doesn't work, or at least get very slow.

If anyone knows a simple method to do this with existing features, i
would be glad to hear about it, but i doubt it's possible. I know
there is a function "file" that reads a file, and there are functions
-strmatch -in etc., but they are not quite enough to actually match a
rather complex URL pattern (for example i would have something like
"www1.example.com" in my list, "server.www1.example.com" should match
this, but not "www2.example.com/index.html#www1.example.com").


I'm not 100% sure this is the right place to ask, it might also be a
request for the documentation mailinglist, since i wouldn't have asked
if there was at least some documentation about it.

Re: How to register additional operators or functions for use in expressions?

[Waldemar Klein <wa...@googlemail.com>]

Solved.

/* I'm not very experienced with mailinglists, but i think it's not
good to change the subject and add something like [SOLVED], because it
won't be listed together with the other posts in the archives and by
mail client filters, or did i miss something? */


First, thanks for the replies, i already did have a short look at
those files mentioned, but after the answers i looked a lot harder,
cause i was more sure i am on the right path :)

After lots of reading and experimenting i found out how this works,
i'll try to explain it for those interested. That is, mostly i won't
explain, but point to concrete examples in the sources :)

The version of Apache i was working with is 2.4.1, so the line numbers
i mention refer to this version.

To clarify what i'm talking about, i use
(C-)function: means a function inside a C file, in the meaning of a
procedure or what a method would be in C++. It's not an entirly
accurate description but i hope it's clear what i mean.
(expr-)function: means what is used in the expression parser in the
sense of a mathematical function, usually used in a similar context to
operator. something in the lines of function ::= funcname "(" word ")"
A (C-)function that i refer to as "does the work" would be one in
which i'd actually write something like "a+b", if i'd declare a new
operator like "...ADD..."


The files to look at are:

include/ap_expr.h
the interesting parts here are
- the struct ap_expr_eval_ctx_t (lines 104-135): this will be filled
and passed to every call of an actual (C-)funcion that implements a
(expr-)function or operator.
- the struct ap_expor_lookup_parms (lines 254-281): this will be
filled an passed to the lookup hook (C-)function
- the typedefs (line 204+), which don't have to be actually used, but
show how a (C-)function that actually does the work has to be declared

modules/ssl/mod_ssl.c and modules/ssl/ssl_engine_vars.c
An actual example how to register additional variables and a
string-list valued (expr-)function. Operators and string-valued
functions work very similarly.


To add an operator or (expr-)function, 4 parts are needed:

(1) Module declaration. This is the same as any other module, some
register_hooks (C-)function must be called (or passed to the macro,
don't know exactly). Example in modules/ssl/mod_ssl.c (line 585-593)

(2) register hooks function: in this, also similar to other modules,
macros or somethings (not really sure what it is but it's not
important at this point) are called that define the lookup functions
which will be called in (3). Example is in modules/ssl/mod_ssl.c,
ssl_register_hooks, (line544-583) which in turn calls
ssl_var_register() (line 568) in modules/ssl/ssl_engine_vars.c (lines
185-129), where ap_hook_expr_lookup (line 128) is called. This step
would normally be done in one place (that is, call ap_hook_expr_lookup
inside ssl_register_hooks), but for some reason this was split in the
ssl module.

(3) ssl_expr_lookup (line 79-102) is given as parameter to
ap_hook_expr_lookup, and this one is called when reading httpd.conf,
when a operator (or function etc.) is encountered. Here there has to
be a check for the type (variable, unary or binary operator, function,
etc.) and name (that is if we want to register a unary op "-X" we
check for the name "X"). Then the struct ap_expr_lookup_parms has to
be filled with at least the (C-)function  pointer (line 96 for
"PeerExtList") that actually will perform the work behind the
(expr-)function or operator.

(4) the actual "working" (C-)function (in the example for
"PeerExtList" the function is expr_peer_ext_list_fn (line 66-71))
should be defined like in the according typedef from ap_expr.h. It
will recieve a ap_expr_eval_ctx_t, some optional data which we could
have passed in step (3) and the actual arguments from the httpd.conf.
Number and types of the arguments depends on type of the
(expr-)function (i.e. one for unary op's, two for binary op's, etc.).
This one will be executed when apache is running, i.e. when taking a
request for example. Those arguments will not be exactly what is
written in httpd.conf, but rather what they evaluate to during a
request, connection, etc., for example %{REMOTE_HOST} in httpd.conf
means, our function gets the actual value of the variable at the time,
not the string "%{REMOTE_HOST}".

Note that step (3) and (4) are usually just one in other modules, like
a handler or filter. This was one of the parts that confused me most
:).


I hope writing this all down will help someone in the future, it
already did help me to understand it better.


P.S. I am german, so there might be lots of grammar errors in here. If
you find any, you may keep them all !!

Re: How to register additional operators or functions for use in expressions?

[Rainer Jung <ra...@kippdata.de>]

On 27.03.2012 02:01, Nick Kew wrote:
> On Tue, 27 Mar 2012 00:42:43 +0200
> Waldemar Klein<wa...@googlemail.com>  wrote:
>
>> Hi all,
>>
>> looking at http://httpd.apache.org/docs/2.4/expr.html, i found things like
>> "Modules may register additional unary operators." or
>> "Modules may register additional functions."
>> but not the slightest hint how to actually do this, also googling for
>> some time didn't get me anything useful either.
>
> You register your own functions for relevant parts of expression parsing.
>
> To get started, I suggest you read the extensive comments in the header
> file include/ap_expr.h.

I think it would also be useful to search for "expr_lookup" in 
modules/filters/mod_include.c and modules/ssl/ssl_engine_vars.c.

Regards,

Rainer

Re: How to register additional operators or functions for use in expressions?

[Nick Kew <ni...@apache.org>]

On Tue, 27 Mar 2012 00:42:43 +0200
Waldemar Klein <wa...@googlemail.com> wrote:

> Hi all,
> 
> looking at http://httpd.apache.org/docs/2.4/expr.html, i found things like
> "Modules may register additional unary operators." or
> "Modules may register additional functions."
> but not the slightest hint how to actually do this, also googling for
> some time didn't get me anything useful either.

You register your own functions for relevant parts of expression parsing.

To get started, I suggest you read the extensive comments in the header
file include/ap_expr.h.

-- 
Nick Kew