RE: [JSR-96-Members] JSR-096 (Java Daemon API) critique...

[Kruszewski Marek <ma...@dialogika.de>]

Hallo,

First of all I would like to ask for excuse those recipients of this mail
who are not directly involved in the specification of java daemons. The
discussion has run somehow out of control. I don't know what kind of
personal or professional problems Pier tries to solve or excuse this way,
but I can guarantee you that the member of this JSR has nothing common with
his problems (if any). I hope, that in the future our discussion will stay
within this expert group.

Pier, please find my comments below.

>  -----Original Message-----
> From: 	Pier P. Fumagalli [mailto:pier@betaversion.org] 
> Sent:	Thursday, 28 June, 2001 18:21
> To:	jsr-96-members@dialogika.de; Thomas.Kopp@dialogika.de
> Cc:	tomcat-dev@jakarta.apache.org; jcp@apache.org; members@apache.org;
jim.driscoll@sun.com
> Subject:	[JSR-96-Members] JSR-096 (Java Daemon API) critique...
> 
> What is a daemon? Historically, a daemon has been defined by the CTSS
group
> in 1963 as a "Disk And Execution MONitor", and has been adopted into the
> Unix world as a program that is not invoked explicitly, but lies dormant
> waiting for some condition(s) to occur.
> 
> What are those conditions? Two classes, basically: an unspecified I/O
> condition that triggers a stage of the daemon execution process, and a
> service request; while we cannot schematize and abstract the first class
of
> conditions, as these are dependant from daemon to daemon and determine
what
> a daemon serves (the LPR daemon is different from the HTTP daemon),
history
> already determined the second class of conditions as signals.
> 
> For instance, it is assumed (and specified), that in response to a TERM
> signal the daemon should terminate its operation and exit its process, in
> response to a HANGUP signal it should hangup a connection, or more
recently
> reload its configurations and restart.

You try to present a specification, which doesn't exist. On Unix there are
no two daemons that behave according to a specific rule. Of course the TERM
signal will terminate most of the daemons because this the default reaction
on this signal, when it is not caught. Have you try to send HANGUP to
different daemons? Most of them NEITHER will hang-up, NOR will reload, they
simply will be killed! How about other signals (SIGINT, SIGSTP, SIGUSR1,
...)? What is the meaning of these signals for daemons? There is no one!
This is so because there is no standard behavior defined for such
situations. There doesn't exist a generic interface to control daemons in
Unix at all. Signals are used in Unix to send some control information
(whatever it is) to processes. There are some common (historical) meanings
for some signals, but generally there is no rule.
Concerning signals, there is one important thing in common for all daemons
(and other programs too) in Unix. All of them have a SINGLE control entry
point (not speaking about the main function) - the signal handler. This
handler gets control on any signal and according to the signal NUMBER
performs an appropriate action (which can be default or defined by the
program). Please note these two capitalized words in this paragraph - we
will need them below.

> 
> So, we can now abstract a more "generic" definition for daemon: a daemon
is
> a dormant program (meaning detached from its parent process, and not
> interrupting its flow of execution), which serves I/O requests for its
> clients (from here the symmetry daemon=server) and interacts with the
> underlying operating system with a defined set of signals or events.

NOT, if we stay by Unix! The set of signals is defined, but their meaning
not! And please note, that this is not a "generic" definition for daemons
because it doesn't define any behavior of these daemons for control signals.

> 
> In the Win32 world, the term daemon is substituted with the term
"service",
> and signals (which do not exist in that environment) have been changed
with
> specific calls to certain declared functions, but on the overall the
design
> remains the same.

Actually, there is a method to send a sort of signals to services in Win32 -
you can send messages to any process. This method is even better then
signals in Unix, because it doesn't limit the set of messages (and their
semantic) you can use.
On the other side, the set of entry points in Win32 services changes from
version to version. It means, that there is no compatible control interface
for services in different versions of Windows. Further, you cannot simply
get a service from WinNT (not speaking about Win95) and let it running on
Win2K - most probably it will crash. At the other side, some of the newer
services will even not compile for older versions of the system.

> 
> In any modern multi-user operating system there is a concept of "deamon",
> "service", "server", and this concept has been associated with "a dormant
> program serving I/O requests for its clients and interacting with the
> underlying operating system with a defined set of signals or events"

So, what do we have for the moment? A daemon is "a dormant program serving
I/O requests for its clients and interacting with the underlying operating
system" with an UNDEFINED set of signals or events. Additionally, the
behavior of the program for these signals or events is UNDEFINED.

> 
> So far, the Java Platform lacked of a way to build an operating system
> daemon using the Java language: Java applications are -all- interactive
> programs, as the current invocation mechanism lacks of a way to create a
> dormant program, and lacks of a way to interact with the underlying
> operating system. Notably, see JavaSoft's Bug # 4323062: the java.exe
> application invocator is used as a "deamon" and it fails. The current fix
is
> a hack, and the platform still lacks of a way thru which an operating
system
> daemon written in java can receive events triggered by the OS (we can
easily
> launch "java myClass &" in Unix, but it dies inexorably when a TERM signal
> is delivered to its process).

This is CORRECT! The TERM signal is one of these few Unix signals that have
well defined semantic. It should be used to gracefully terminate affected
process.
Java doesn't understand Unix signals - they simply don't belong to the Java
environment. If you like to make use of Unix signals in Java programs, the
only thing you can do is to implement a native wrapper that catches signals
and forwards them somehow to the Java program. This can work ONLY on a Unix
machine and has nothing common with Java. And consequently with Java
daemons!

> 
> Those are the problems I wanted to solve when I joined the JSR-096 expert
> group, those are the issues that in the proposal listed on the JCP site we
> are supposed to solve, and this is what the Java community expect us to
> deliver.

These problems are some from many different problems that should be solved
by this expert group.
But I'm afraid, that you didn't understand the real problem this expert
group should solve. The problem is NOT an implementation of native daemons
on Unix (or any other specific) system in Java. Some native means would be
more suitable in this case. This would also conflict with the spirit of
Java.
The problem is NOT to find a solution for your particular projects. This is
your business.

The problem is how to specify a platform independent model of a daemon in
Java. A daemon that doesn't use any native constructions of any particular
operating system and can be embedded in any of these systems.

> 
> And today (incredible timing, I would say) I received the last copy of the
> specification. And I'm disappointed to see that even in this last copy
there
> is no reference to the behavior of the JVM process related to its
supporting
> operating system. This specification does not solve my problem, is not the
> reason why I joined this group.

If this specification does not solve your problem, then this is your fault.
Because this was YOUR job to find a solution for this problem. The other
members of the group didn't even know that you had a problem.

> 
> I see this version of the specification as a description on how to build
> Java daemons for the Java Platform (one tier up), not a Java daemon for a
> generic underlying operating system. This idea is corroborated by the fact
> that its test Reference Implementation delivered with the specification is
> based on JMX, doesn't touch at all the problems that we need to face when>

> integrating a generic Java daemon (from the definition above) within the
> scope the underlying operating system. In my opinion, this specification
is
> redundant, JMX and JSR-111 are already a great way to address these
problems
> (I believe this came out also from our meeting in London last month).

If you compare the numbers of JSR-111 and JSR-96 you can easily see what JSR
is redundant.
The test Reference Implementation uses JMX as an example. Your job was to
implement a test Reference Implementation for Unix.
Indeed the test implementation IS a description on how to build Java daemons
for the Java Platform. This is exactly what we had to specify! A daemon that
runs on any Java platform no matter what is the host operating system. It is
NOT our job to define a "generic underlying operating system". Even when
something like this would be possible, we would need another group of
experts, who have more experience on this subject.

Once again: we need to build Java daemons for Java platforms. This
specification should be accompanied by reference implementations for
different systems (Unix, Win32, and so on). Moreover, these implementations
are only examples - they don't belong to the specification.

> 
> And that's why last week, on my own, silently, I came up with an
> implementation that addresses the problems I needed to solve, and were
> needed to be solved by my team (the Apache Tomcat team) to deal with how a
> daemon written in Java can deal with the operating system.
> <http://www.apache.org/websrc/viewcvs.cgi/jakarta-tomcat-4.0/service/>
> 
> Being responsible for solving these problems in the Tomcat world, his
> implementation is what I feel I can safely deliver to my team and to the
> community, while I don't feel that adding another layer of cumbersomeness
to
> the current specifications will help in any possible way.
> 
> And this is why as an Apache member, and its current representative in
this
> expert group, as a Sun Microsystems employee working on the Tomcat
project,
> I will not support the current specification until the problems I face,
the
> problems I endorsed when joining this effort, the problems we were
supposed
> to solve will be addressed.
> 
> My implementation doesn't try to be a full 360 degrees solution to the
> problems of daemons, and does not try to be a fancy JCP specification, but
> it solves those problems that we, software developers, need to face
everyday
> when developing daemons in the Java Language and with the help of the
> community, will be improved and enhanced.
> 
> I believe that this expert group went too far, forgetting its roots, and
the
> real problems that we needed to solve.

So, recall our roots:
Our goal is to specify an interface that could be used by every java daemon
on the Java platform on any host operating system that supports Java.
I don't want to polemize with your solution at this point. As long as we
cannot find a common basic level of understanding it doesn't make much sense
to discuss about details.

> 
> Also, another problem I faced while working with this group was the way in
> which the discussion was held. Our mailing list rarely held technical
> discussions, if not triggered by one of us in the Apache Software
Foundation
> (thanks especially to Peter Donald) or few others and the whole
> specification was developed internally at Dialogika. I've been thru
several
> specifications in the Servlet/JSP and XML lands and being a member of the
> expert group usually ment to know _what_ was going on, not just listening
to
> decisions taken inside a company by few and "blessing" them.
> 
> In your last email, Thomas, I quote "As a late reaction of Peter's critics
I
> did some brainstorming and discussed the control code model once more with
> my colleagues". WE, Thomas, are your colleagues on this specification, not
> whoever works at Dialogika, my name is on that specification, and apart
from
> the one in London I never had a brainstorming session with you.
> 
> I'm sorry I didn't come to JavaONE, but my current economic situation
didn't
> allow me to get there, I would have told you this in person, but I feel
left
> out, and right now I should have to bless a specification that I don't
agree
> with and that doesn't include any of my thoughts on the matter.
> 
> Apart from this my vote on JSR-096 is a veto, a -1, for technical reasons,
> we don't solve those problems we were supposed to.
> 
> Sincerely,
> 
>     Pier Fumagalli
>     - Apache Software Foundation (member, JSR-096 representative)
> 
> (CCing to the Tomcat Developers mailing list, the Apache JCP mailing list,
> the Apache Members mailing list and my manager at Sun Microsystems)
> 
> 

What do we have for now?

1) A specification for a generic daemon class which says that this class
contains the SINGLE control entry point, which is invoked to perform an
action according to the argument specified which is a NUMBER (cf. with the
corresponding capitalized words at the beginning of this text).
For the moment, we don't say anything about what these actions are. We don't
say how many actions there are. We only say that a daemon itself (actually
its developer) defines what the legal actions are and what their meaning is.
Having that we are able to write almost any daemon - Unix demons fit in this
schema. 
Additionally, the specification defines a generic daemon information object,
which is used by the daemon to report its important information to anybody,
who wants to hear it. The daemon (developer) decides what information (and
to whom) it wants to publish. 

2) Now we would like to have a possibility to control these daemons.

First of all we probably need to fix a minimal but well defined set of
actions that every daemon should understand and define reactions on actions
that the daemon not understands. So, what are these actions? Initialize,
destroy, reload, start, stop, ... (cf. with signals SIGTERM, SIGHUP,
SIGCONT, SIGSTOP). Anything more? Perhaps too many?
We reserve for these actions distinct numbers (one could even use the
corresponding signal numbers). Of course we will define some symbolic
constants for these numbers.

If we have already a set of well-defined actions, it could be practicable
(at least many people like that) to have a separate entry point for each
known action. Nothing easier: we simple build a wrapping object around our
daemon and define the specific entry points, which are probably just
redirectors to the generic entry point for the corresponding action numbers.
At this point, we could also precise what the daemon information object
could be, to be able to understand its messages. The wrapper is responsible
for supplying the daemon information in the requested form.

Why do we build a wrapper instead just to subclass the daemon object itself?
There are at least two reasons to do this:
- We need to write our daemon only once - if it doesn't include platform
specific constructions, everything what we need to port it to another
environment is just to port its wrapper - the daemon stays intact.
- Through disjoining the daemon's control interface from the daemon itself
we make a firewall around the daemon. Nobody gets a reference to the daemon;
nobody can get its private data. This should be enough to keep hackers away
from our daemon.

Now we have a convention: all daemons that conform to the specification of
our entry points and the information object behave more or less "reasonable"
so we could control them.

3) And now we would really like to control our daemons!

There two several scenarios, which could be implemented alternatively or
parallel.

- We could have a central control unit for our daemons. This would be most
likely a Java program, which would be responsible for starting, stopping,
monitoring (and so on) daemons conforming to a given specification. This
solution is ideal when we need a unique environment where we can run our
daemons. We don't need to worry about portability. Even the control center
would be independent of any platform. This solution reminds on the Service
Manager on Win2K.
- For every daemon we could have a native wrapper that would be responsible
for embedding the daemon in the host environment (cf. discussion about
catching signals in java daemons above). This solution fit rather in the
Unix environment, but can be also easily implemented in Win32. This solution
from its nature is somehow dirty - it uses constructions, which are strange
to Java - but makes java daemons native to the host operating system anyway.

Please note, that neither of these implementations belong to our goal in
this JSR. Everything what we need is to supply at least a reference
implementation of one of these scenarios.

I hope that this description will help you to understand the main aspects of
the current specification of Java daemons. If you feel that it still doesn't
address your needs, or you see conceptual defects, please say us where you
see problems and we can try to solve them together.

Sincerely,
Marek Kruszewski
DIaLOGIKa GmbH

Re: [JSR-96-Members] JSR-096 (Java Daemon API) critique...

["Pier P. Fumagalli" <pi...@betaversion.org>]

Kruszewski Marek at marek@dialogika.de wrote:

> First of all I would like to ask for excuse those recipients of this mail
> who are not directly involved in the specification of java daemons. The
> discussion has run somehow out of control. I don't know what kind of
> personal or professional problems Pier tries to solve or excuse this way,
> but I can guarantee you that the member of this JSR has nothing common with
> his problems (if any). I hope, that in the future our discussion will stay
> within this expert group.

The discussion has definitely not ran out of control. My email was targeted
to a specific group of people:

- The JSR-096 members (for obvious reasons).
- The Apache members and JCP committee at Apache: I am the official
  representative of the Apache Software Foundation in this JSR, and if I
  am voting against it, I want to inform my community on _WHY_ I'm doing it.
- Tomcat Developers: my Service code has been posted under the Tomcat 4.0
  project CVS. People looking at what I was doing in Tomcat land and at the
  outcomes of this JSR were wondering why my approach was different, and
  it's (I believe) in my rights to tell them _why_.
- Jim Driscoll: he is my manager (and the person who is responsible for my
  paychecks). I believe it's only fair for him to know what I'm doing.

My further replies were addressed only to the JSR-096 list (apart from an
intense traffic of emails and phone calls that went thru this week-end
privately between me and other members of this JSR), and in the future,
since the specification is not yet in Public Review, all my replies will be
only addressed only to the experts mailing list.

Anyway, it's so nice to know that we have another expert on Daemons from
Dialogika, Marek, as this is the very first email you posted on this topic
(to me or to the experts mailing list) since February the 7th, 2001 (date
when I joined): Hi, my name is Pier Fumagalli and I'm a developer of the
Apache Tomcat project, nice to meet you...

    Pier

(Further replies are on the JSR-096 experts mailing list)