RE: [PATCH] Apache2 mod_jk2

[Mladen Turk <mt...@mappingsoft.com>]

> -----Original Message-----
> From: costinm@covalent.net [mailto:costinm@covalent.net] 
> Sent: 30. svibanj 2002 16:56
> To: Mladen Turk
> Cc: 'Ignacio J. Ortega'
> Subject: RE: [PATCH] Apache2 mod_jk2
> 
> 
> On Thu, 30 May 2002, Mladen Turk wrote:
> 
> > For start I would like to implement the Apache2 servlet input and 
> > output filter. And yes, my general idea is to make that 
> something like 
> > mod_perl, to allow writing apache modules using java.
> > The second thing is IMO beyond tomcat scope, probably a whole new
> > project.
> 
> :-)
> 
> I can't say how happy I am to find there are other people 
> interested in 
> 'writing apache modules in java' or mixing apache modules and 
> java servlets.
> 
> That was one of the reasons for me getting involved with 
> jserv and tomcat ( many years ago ). After few years I still 
> didn't get this to work - so again, be warned, it's not going 
> to be easy !
> 
> If you want to start with Apache2 ( C ) based servlet i/o 
> filters, I sugest you start with something like a compression 
> filter, that can be a proof of concept.
> 
> I assume you'll need extra callbacks in mod_jk. Probably some 
> special java-side filter that will be configured in web.xml 
> and will communicate directly with mod_jk, telling it to add 
> the right filter. At least that's what I would do, but as I 
> said my results so far are a failure ( I 
> can claim lack of time, or underestimating the complexity :-).
> 
> In any case, it may be a good idea moving this discussion to 
> tomcat-dev. Let me know if/how I can help ( if I can't do it 
> at least I can try to help :-).
> 
> Costin
> 
Hi,

Well, I've put that on paper (look at the attachment), and something
'crude' could be finished relatively quickly :)
I was planning to use the existing mod_jk2 and make the needed
extensions.

There will be four classes dealing with particular Apache hook 
1. Handler
2. Input Filter
3. Output Filter
4. Protocol

Each of the hooks will have unlimited (well, if you have unlimited
memory) numbers of unique named modules.
So, something like in httpd.conf for handlers:

JavaHandler someHandler "/home/somehandler.jar"
JavaHandler anotherHandler "/home/anotherhandler.jar"
...etc
<Location /some-handler>
    SetJavaHandler someHandler
</Location>

<Files *.jsxx>
    SetJavaHandler anotherHandler
</Files>

1. The directive JavaHandler will load the handlers in the handlers
table
2. The per-directory SetJavaHandler will associate the location with
particular handler
3. On first invocation will create the JVM and call the Initialize
method from someHandler class
4. AjkHandler will load the callbacks from mod_jk2.
5. rock'n'roll

Each module will be ran inside its own JVM for now, this can be wrapped
with the JVM pool, and using the inactive one.
I've done that for my experimental Mozilla JavaScript module for
js_runtime (suppose it can be done with JVM too).


All that is for the pure Java modules without Tomcat at all.
It can be used inside the Tomcat itself, but only using jni or writing
new protocol stack (something like RPC), but I'm not sure what would be
the purpose of that. I'll be glad to hear some thoughts on that.

Comments will be more then welcome.

MT.

RE: [PATCH] Apache2 mod_jk2

[co...@covalent.net]

On Tue, 4 Jun 2002, Mladen Turk wrote:

> We are talking _apache_ modules true?
> They extends the _apache_ server, so one writing one should be familiar
> with the .conf I think.
> In theory some functionality could be reused on other servers like
> authentication, and I'm not sure I want to do that, but you never know
> :)


Support for other servers is already built into jk, and even if you 
( or me ) are mainly using Apache, there are people who
have to support other servers.

I think an auth module would be far more usefull if it would work on
any server - probably not as good as in apache, but it would allow 
people to migrate or support existing infrastructures.
Besides, many java programmers are not that familiar with the .conf
file.


Please note that the current jk2 is designed very much like tomcat -
a collection of components ( beans ), configurable at runtime, 
the config file is abstracted, etc. It shouldn't matter how you
configure - what's important is to follow some patterns
( JMX-like if you want - named components with attributes ).


Runtime reconfiguration is much more important - while gracefull
restart works great, a servlet environment will have too many
 changes ( webapps deployed/undeployed, lb workers going up and 
down, etc ) - and in apache2 ( with multithreaded mpms ) the 
price for restart may be higher ( since it may lose state
or require in-process java VMs to restart and lose sessions ).


> I'll try to work-out something this week.
> For start I'd like to reorganize the mod_jk2 by putting needed includes
> in the jk_apache2.h and adding
> external module AP_MODULE_DECLARE_DATA jk2_module in it.
> If that's ok with you.
> The reason for that is that I would like to put the extended
> functionality in the separate file.

That's fine. 
You can also start with a different dir - and merge later. 

Costin


--
To unsubscribe, e-mail:   <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>

RE: [PATCH] Apache2 mod_jk2

[Mladen Turk <mt...@mappingsoft.com>]

> -----Original Message-----
> From: costinm@covalent.net [mailto:costinm@covalent.net] 
> Sent: 4. lipanj 2002 22:00
> To: Mladen Turk
> Cc: 'List Tomcat-Dev'
> Subject: RE: [PATCH] Apache2 mod_jk2
> 
> 
> > 
> > That's my goal too. The Java side will have all the 
> interface, and C 
> > side will just find the paticular hook, and implement the missing 
> > interface.
> 
> Finding the hook is already implemented, all you need is a 
> worker that 
> will forward the information that you need ( as oposed to the current 
> worker_ajp that forwards the full request ).

I'll check if I can reuse that.
 
> 
> The reasons we use workers2.properties ( and I think you 
> should use it too, even if using only httpd.conf is possible ):
> 
> - same format for all servers ( IIS, I hope soon others )
> - easier automated generation ( i.e. I hope we'll be able to 
> use JMX and java to generate and maintain the file )
> - reloading ( for some options ).
> - easier for people who are not familiar with httpd.conf
> 
> On the other side using httd.conf:
> - is faster ( since <Location> will be used, and the internal apache 
> mapper )
> - familiar for apache users
> - single config file

We are talking _apache_ modules true?
They extends the _apache_ server, so one writing one should be familiar
with the .conf I think.
In theory some functionality could be reused on other servers like
authentication, and I'm not sure I want to do that, but you never know
:)

I'll try to work-out something this week.
For start I'd like to reorganize the mod_jk2 by putting needed includes
in the jk_apache2.h and adding
external module AP_MODULE_DECLARE_DATA jk2_module in it.
If that's ok with you.
The reason for that is that I would like to put the extended
functionality in the separate file.


MT.


--
To unsubscribe, e-mail:   <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>

RE: [PATCH] Apache2 mod_jk2

[co...@covalent.net]

On Tue, 4 Jun 2002, Mladen Turk wrote:

> > We can add a 'handler' property to reflect the java-side 
> > handler that will be executed ( or one for the handler type 
> > and one for the instance).
> 
> I can extend the JkUriSet, but I need the the table of handlers, input
> filters, output filter and protocols that I'll go through, for each
> particular hook.
> The SetJavaHandler is only example for handler modules.
> SetJavaInputFilter and SetJavaOutputFilter are other two.
> I'm afraid that JkUriSet doesn't allow that. Of course it can be
> extended to keep the config directive count low,
> like 'JkUriSet handler someHandler', and will be confusing.


Jk uses jk_uriEnv_t to represent a location. There are various properties
you can associate with it, and various components will use this structure.

I'm more or less ok with adding new directives - like SetJavaInputFilter.
But the directive should set options on jk_uriEnv.

The problem with new directive - JkUriSet works the same as 
workers2.properties ( i.e. associate name=value for the location
'object' ). 
I think this model - of having different 'objects' with 'properties'
is pretty flexible, and with workers2.properties you'll have it
working for any web server. 

<Location /foo>
JkUriSet inputFilter MyFilter
</Location> 
or

[uri:/foo]
inputFilter=MyFilter 

are doing the same thing, call setAttribute( "inputFilter", "MyFilter" ).

It's ok to add a 
 JkInputFilter MyFilter
or 
 SetInputFilter MyFilter,
but I don't think it's much more intuitive than the current solution.

> > Again, the goal is to keep the C-Java interface to a minimum 
> > that can be optimized, and do more work on the java side.
> > 
> 
> That's my goal too. The Java side will have all the interface, and C
> side will just find the paticular hook, and implement the missing
> interface.

Finding the hook is already implemented, all you need is a worker that 
will forward the information that you need ( as oposed to the current 
worker_ajp that forwards the full request ).


> Again I need four special data structures that consists of synced
> apr_table, and apr_hash_table,
> bassicaly the extended jk_map_aprtable.
> 
> I'd like to minimize the various configuration files. Modules will be
> the part of Apache server and IMO their configuration should be inside
> the server configuration file.

I partially agree.

The reasons we use workers2.properties ( and I think you should use
it too, even if using only httpd.conf is possible ):

- same format for all servers ( IIS, I hope soon others )
- easier automated generation ( i.e. I hope we'll be able to use JMX
and java to generate and maintain the file )
- reloading ( for some options ).
- easier for people who are not familiar with httpd.conf

On the other side using httd.conf:
- is faster ( since <Location> will be used, and the internal apache 
mapper )
- familiar for apache users
- single config file

Both are valid - and each user can choose, but we should support both.
The 'java bean' style of using setters/getters for configuration
makes this reasonably easy.

Costin


--
To unsubscribe, e-mail:   <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>

RE: [PATCH] Apache2 mod_jk2

[Mladen Turk <mt...@mappingsoft.com>]

> -----Original Message-----
> From: costinm@covalent.net [mailto:costinm@covalent.net] 
> Sent: 4. lipanj 2002 20:34
> To: Mladen Turk
> Cc: List Tomcat-Dev
> Subject: RE: [PATCH] Apache2 mod_jk2
> 
> 
> On Tue, 4 Jun 2002, Mladen Turk wrote:
> 
> > There will be four classes dealing with particular Apache hook
> > 1. Handler
> > 2. Input Filter
> > 3. Output Filter
> > 4. Protocol
> 
> For Handler - you should use JkHandler. We can add some more 
> methods if you need, but it would be better to minimize the number of 
> interfaces/concepts. 

Fine with me.
> 
> > JavaHandler someHandler "/home/somehandler.jar"
> > JavaHandler anotherHandler "/home/anotherhandler.jar"
> 
> I assume you want the jar files loaded in the VM ? It's much 
> easier to just put them in the lib/common ( or common/lib ). 
> It is also possible to have some defined in web applications 
> ( WEB-INF/lib ), but it's much easier to deal with this in 
> the java side. ( and keep the C code simpler ).
> 

Again fine with me.

> 
> > ...etc
> > <Location /some-handler>
> >     SetJavaHandler someHandler
> > </Location>
> 
> That's already done and works fine. It's "JkUriSet name value", 
> you can do:
>   JkUriSet group lb
>   JkUriSet debug 1
>   JkUriSet worker status
> 
> We can add a 'handler' property to reflect the java-side 
> handler that will be executed ( or one for the handler type 
> and one for the instance).

I can extend the JkUriSet, but I need the the table of handlers, input
filters, output filter and protocols that I'll go through, for each
particular hook.
The SetJavaHandler is only example for handler modules.
SetJavaInputFilter and SetJavaOutputFilter are other two.
I'm afraid that JkUriSet doesn't allow that. Of course it can be
extended to keep the config directive count low,
like 'JkUriSet handler someHandler', and will be confusing.
 
> Again, the goal is to keep the C-Java interface to a minimum 
> that can be optimized, and do more work on the java side.
> 

That's my goal too. The Java side will have all the interface, and C
side will just find the paticular hook, and implement the missing
interface.

> 
> > 1. The directive JavaHandler will load the handlers in the handlers
> > table
> 
> jk2.properties is doing that ( but on java side ), 
> workers.properties can add the C side for the bridge.
> 

Again I need four special data structures that consists of synced
apr_table, and apr_hash_table,
bassicaly the extended jk_map_aprtable.

I'd like to minimize the various configuration files. Modules will be
the part of Apache server and IMO their configuration should be inside
the server configuration file.

> > 2. The per-directory SetJavaHandler will associate the 
> location with 
> > particular handler
> 
> I think the current system will work, if you need extensions 
> its fine but I wouldn't go with a different path of execution 
> for that ( if you 
> think there are serious problems with the current mapper 
> system - we can either fix or replace, but I really want to 
> reuse the same code )
> 
> 
> > 3. On first invocation will create the JVM and call the Initialize 
> > method from someHandler class
> 
> Again, that's part of the channeljni - and should be the same 
> code as for normal requests.

Fine with me.
 
> > 4. AjkHandler will load the callbacks from mod_jk2.
> 
> Are you talking about Java->C ? The JniHandler is able to 
> call any jk component ( that's what we use for shm and mutex 
> etc ), and is quite 
> optimized. 
> 
> 
> > Each module will be ran inside its own JVM for now, this can be 
> > wrapped with the JVM pool, and using the inactive one.
> 
> It's a bit inefficient, it's much better to have a single VM 
> and maybe use different classloaders. Keep the complexity in 
> java, it's much easier.
> 

Agree! Will check.

> You don't need full tomcat - but the java side of Jk2. The 
> 'RPC-like' mechansim is already in place to allow request 
> forwarding and callbacks and general Java-C communication. 
> 
> It is not very sophisticated ( i.e. simple enough to be 
> optimized ), and may be extended ( I haven't implmented the 
> signals yet, and few other things that may be needed in 
> future versions of jk2 ).
> 
> 
> Costin



Thanks.

MT.


--
To unsubscribe, e-mail:   <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>

RE: [PATCH] Apache2 mod_jk2

[co...@covalent.net]

On Tue, 4 Jun 2002, Mladen Turk wrote:

> There will be four classes dealing with particular Apache hook 
> 1. Handler
> 2. Input Filter
> 3. Output Filter
> 4. Protocol

For Handler - you should use JkHandler. We can add some more methods
if you need, but it would be better to minimize the number of 
interfaces/concepts. 

> JavaHandler someHandler "/home/somehandler.jar"
> JavaHandler anotherHandler "/home/anotherhandler.jar"

I assume you want the jar files loaded in the VM ? It's much easier
to just put them in the lib/common ( or common/lib ). It is also
possible to have some defined in web applications ( WEB-INF/lib ),
but it's much easier to deal with this in the java side.
( and keep the C code simpler ).


> ...etc
> <Location /some-handler>
>     SetJavaHandler someHandler
> </Location>

That's already done and works fine. It's "JkUriSet name value", 
you can do:
  JkUriSet group lb
  JkUriSet debug 1
  JkUriSet worker status

We can add a 'handler' property to reflect the java-side handler that
will be executed ( or one for the handler type and one for the instance).

Again, the goal is to keep the C-Java interface to a minimum that can
be optimized, and do more work on the java side.


> 1. The directive JavaHandler will load the handlers in the handlers
> table

jk2.properties is doing that ( but on java side ), workers.properties can
add the C side for the bridge.

> 2. The per-directory SetJavaHandler will associate the location with
> particular handler

I think the current system will work, if you need extensions its fine
but I wouldn't go with a different path of execution for that ( if you 
think there are serious problems with the current mapper system - we
can either fix or replace, but I really want to reuse the same code )


> 3. On first invocation will create the JVM and call the Initialize
> method from someHandler class

Again, that's part of the channeljni - and should be the same code as for
normal requests.

> 4. AjkHandler will load the callbacks from mod_jk2.

Are you talking about Java->C ? The JniHandler is able to call any jk
component ( that's what we use for shm and mutex etc ), and is quite 
optimized. 


> Each module will be ran inside its own JVM for now, this can be wrapped
> with the JVM pool, and using the inactive one.

It's a bit inefficient, it's much better to have a single VM and maybe
use different classloaders. Keep the complexity in java, it's much easier.


> All that is for the pure Java modules without Tomcat at all.
> It can be used inside the Tomcat itself, but only using jni or writing
> new protocol stack (something like RPC), but I'm not sure what would be
> the purpose of that. I'll be glad to hear some thoughts on that.

You don't need full tomcat - but the java side of Jk2. The 'RPC-like'
mechansim is already in place to allow request forwarding and callbacks
and general Java-C communication. 

It is not very sophisticated ( i.e. simple enough to be optimized ), and
may be extended ( I haven't implmented the signals yet, and few other
things that may be needed in future versions of jk2 ).


Costin


--
To unsubscribe, e-mail:   <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>