You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lenya.apache.org by Andreas Hartmann <an...@apache.org> on 2008/02/06 16:45:36 UTC
[1.2] lenya.sh stores PID?
Hi Lenya devs,
I don't know if this behaviour is intended:
When I start my Lenya 1.2.x, the process ID seems to be written to a
file. Ctrl+C doesn't shutdown Jetty, but the next ./lenya.sh kills the
process with the stored ID.
Is there any special reason for this behaviour? IMO storing PIDs is
dangerous - on the next day, the ID could be assigned to another process
which will be killed when you startup Lenya ... Or is this not possible?
-- Andreas
--
Andreas Hartmann, CTO
BeCompany GmbH
http://www.becompany.ch
Tel.: +41 (0) 43 818 57 01
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lenya.apache.org
For additional commands, e-mail: dev-help@lenya.apache.org
Re: [1.2] lenya.sh stores PID?
Posted by Andreas Hartmann <an...@apache.org>.
solprovider, Jörn, Richard,
thanks for clarifying! This sounds reasonable.
-- Andreas
Richard Frovarp schrieb:
> solprovider@apache.org wrote:
>> On 2/6/08, Jörn Nettingsmeier <ne...@apache.org> wrote:
>>
>>> Andreas Hartmann wrote:
>>> > I don't know if this behaviour is intended:
>>> > When I start my Lenya 1.2.x, the process ID seems to be written to a
>>> > file. Ctrl+C doesn't shutdown Jetty, but the next ./lenya.sh kills
>>> the
>>> > process with the stored ID.
>>> >
>>> > Is there any special reason for this behaviour? IMO storing PIDs is
>>> > dangerous - on the next day, the ID could be assigned to another
>>> process
>>> > which will be killed when you startup Lenya ... Or is this not
>>> possible?
>>> only if you own it. if you run lenya as a special user (which you really
>>> really should), then there's no danger at all. if you run lenya as
>>> root,
>>> you get what you deserve :)
>>>
>>> storing PIDs is standard behaviour for daemons that are singletons.
>>> check the /var/run directory of most unix systems for an overview of
>>> how
>>> many programs do it...
>>> Jörn Nettingsmeier
>>>
>>
>> I stored the PID to add the "stop" option. As Jorn states, this is
>> standard *nix behavior. The PID being stored is returned from the
>> java command, not the process calling lenya.sh. The PID is not stored
>> when starting in "cli" mode to prevent interfering with a known user
>> process. To allow hijacking through reuse of the PID, a user would
>> need to kill the process without using lenya.sh and find a method to
>> start a new process on that PID. I thought the OS always assigns the
>> PID and has enough PID numbers that I have never seen a rollover.
>>
> It does assign the PID and they do rollover. There's only 32K of them,
> which isn't too many on some systems. My incoming email systems rollover
> several times a day, but those are short lived sendmail and various
> scanning processes.
>
>> Can an OS assign a new Java/Jetty/Lenya process a PID that is already
>> in use?
>>
> No, it shouldn't be able to.
--
Andreas Hartmann, CTO
BeCompany GmbH
http://www.becompany.ch
Tel.: +41 (0) 43 818 57 01
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lenya.apache.org
For additional commands, e-mail: dev-help@lenya.apache.org
Re: [1.2] lenya.sh stores PID?
Posted by Jörn Nettingsmeier <ne...@apache.org>.
solprovider@apache.org wrote:
> My understanding is we write for sh from 1978 because every *nix
> understands the code. Zones and other Sun servers default to that
> shell. Every admin I know changes the shell on Sun servers, but code
> written for that ancient sh works on bash, csh, and ksh so the purpose
> is more about being generic than the zones server.
if i'm feeling masochistic, i boot up my 4-way sparcserver 20 and enjoy
some old solaris without gnu tools for old times' sake. if i want to get
a job done, i let dead and obsolete systems rest in peace and upgrade to
something from the current century.
i'm all for backwards compatibility where it can be accomplished without
ugliness, but coding for a 1978 shell even if it leads to horrible hacks
nobody understands... i don't think that's worth it.
if nobody objects, i'll try and change our zones shell to something
reasonable. i'm not advocating BASHisms, just that we shouldn't waste
time catering to shells no sane admin would ever use for real work.
--
Jörn Nettingsmeier
"One of my most productive days was throwing away 1000 lines of code."
- Ken Thompson.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lenya.apache.org
For additional commands, e-mail: dev-help@lenya.apache.org
Re: [1.2] lenya.sh stores PID?
Posted by so...@apache.org.
On 2/6/08, Joern Nettingsmeier <ne...@folkwang-hochschule.de> wrote:
> solprovider@apache.org wrote:
> > This works in bash. The case format for string matching is used
> > because it does work in ancient sh.
>
> can't log into zones atm, but iirc there is a current bash on that
> system - it may not be our default login shell, but we could change that
> if there is consensus. i did modify a few things in our profile a few
> days ago - meaningful prompt with hostname and path, and a decent pager
> ("less" instead of "more"), and iirc there was a bash as well...
> jörn nettingsmeier
My understanding is we write for sh from 1978 because every *nix
understands the code. Zones and other Sun servers default to that
shell. Every admin I know changes the shell on Sun servers, but code
written for that ancient sh works on bash, csh, and ksh so the purpose
is more about being generic than the zones server.
solprovider
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lenya.apache.org
For additional commands, e-mail: dev-help@lenya.apache.org
Re: [1.2] lenya.sh stores PID?
Posted by Joern Nettingsmeier <ne...@folkwang-hochschule.de>.
solprovider@apache.org wrote:
> This works in bash. The case format for string matching is used
> because it does work in ancient sh.
can't log into zones atm, but iirc there is a current bash on that
system - it may not be our default login shell, but we could change that
if there is consensus. i did modify a few things in our profile a few
days ago - meaningful prompt with hostname and path, and a decent pager
("less" instead of "more"), and iirc there was a bash as well...
--
jörn nettingsmeier
home://germany/45128 essen/lortzingstr. 11/
http://spunk.dnsalias.org
phone://+49/201/491621
Kurt is up in Heaven now.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lenya.apache.org
For additional commands, e-mail: dev-help@lenya.apache.org
Re: [OT] Sendmail PIDs
Posted by Richard Frovarp <Ri...@sendit.nodak.edu>.
solprovider@apache.org wrote:
> On 2/6/08, Richard Frovarp <ri...@sendit.nodak.edu> wrote:
>
>> solprovider@apache.org wrote:
>> > On 2/6/08, Richard Frovarp <ri...@sendit.nodak.edu> wrote:
>> >> It does assign the PID and they do rollover. There's only 32K of them,
>> >> which isn't too many on some systems. My incoming email systems rollover
>> >> several times a day, but those are short lived sendmail and various
>> >> scanning processes.
>> >
>> > Most programs needing many short-lived processes use a pool to avoid
>> > the performance hit of creating many new processes. The performance
>> > hit depends on the threading model; some are optimized so a new
>> > process is faster than a new thread. I am unaware of a modern MTA
>> > that creates a new process for each message.
>> > [Richard: This is an RFI. I hate ignorance, especially my own. What
>> > MTA is used?]
>> Sendmail 8.13. Here's my PID list for sendmail at the moment. Granted
>> some are the parents, queue runner, some are hung open, etc. But it most
>> certainly isn't using a pool. And it would be for each connection, as
>> multiple messages can be sent through a single connection.
>>
>
> Thank you. Wow. My server is running the same version. Either mine
> uses a different configuration or my server handles a tiny portion of
> the quantity of messages that your server receives. My email server
> is basically a spam trap; most messages to my server are forwarded
> into another program so the "write message to filesystem" is never
> called. We receive a few thousand spam messages per day and maybe 40
> real messages per year.
>
> solprovider
>
Yeah, that one handled 207,065 messages yesterday. Granted only 23,874
were counted as clean, the rest as junk. I have two more just like it.
And one that only handles internal mail (and ASF mail, hey as the admin
I can open the firewall up to who I want). If it weren't for spam I'd
need only one piece of hardware instead of 3 plus a VM.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lenya.apache.org
For additional commands, e-mail: dev-help@lenya.apache.org
Re: [OT] Sendmail PIDs
Posted by so...@apache.org.
On 2/6/08, Richard Frovarp <ri...@sendit.nodak.edu> wrote:
> solprovider@apache.org wrote:
> > On 2/6/08, Richard Frovarp <ri...@sendit.nodak.edu> wrote:
> >> It does assign the PID and they do rollover. There's only 32K of them,
> >> which isn't too many on some systems. My incoming email systems rollover
> >> several times a day, but those are short lived sendmail and various
> >> scanning processes.
> >
> > Most programs needing many short-lived processes use a pool to avoid
> > the performance hit of creating many new processes. The performance
> > hit depends on the threading model; some are optimized so a new
> > process is faster than a new thread. I am unaware of a modern MTA
> > that creates a new process for each message.
> > [Richard: This is an RFI. I hate ignorance, especially my own. What
> > MTA is used?]
> Sendmail 8.13. Here's my PID list for sendmail at the moment. Granted
> some are the parents, queue runner, some are hung open, etc. But it most
> certainly isn't using a pool. And it would be for each connection, as
> multiple messages can be sent through a single connection.
Thank you. Wow. My server is running the same version. Either mine
uses a different configuration or my server handles a tiny portion of
the quantity of messages that your server receives. My email server
is basically a spam trap; most messages to my server are forwarded
into another program so the "write message to filesystem" is never
called. We receive a few thousand spam messages per day and maybe 40
real messages per year.
solprovider
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lenya.apache.org
For additional commands, e-mail: dev-help@lenya.apache.org
Re: [OT] Sendmail PIDs
Posted by Jörn Nettingsmeier <ne...@apache.org>.
Richard Frovarp wrote:
> solprovider@apache.org wrote:
>> On 2/6/08, Richard Frovarp <ri...@sendit.nodak.edu> wrote:
>>
>>> It does assign the PID and they do rollover. There's only 32K of them,
>>> which isn't too many on some systems. My incoming email systems
>>> rollover
>>> several times a day, but those are short lived sendmail and various
>>> scanning processes.
>>>
>>
>> Most programs needing many short-lived processes use a pool to avoid
>> the performance hit of creating many new processes. The performance
>> hit depends on the threading model; some are optimized so a new
>> process is faster than a new thread. I am unaware of a modern MTA
>> that creates a new process for each message.
>> [Richard: This is an RFI. I hate ignorance, especially my own. What
>> MTA is used?]
>>
> Sendmail 8.13. Here's my PID list for sendmail at the moment. Granted
> some are the parents, queue runner, some are hung open, etc. But it most
> certainly isn't using a pool. And it would be for each connection, as
> multiple messages can be sent through a single connection.
on linux systems, process creation is cheap. iiuc, it's just as
effective as the creation of a new thread in most cases.
so your sendmail seems to do the Right Thing.
otoh, under nt, such a behaviour would be insane.
--
Jörn Nettingsmeier
"One of my most productive days was throwing away 1000 lines of code."
- Ken Thompson.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lenya.apache.org
For additional commands, e-mail: dev-help@lenya.apache.org
[OT] Sendmail PIDs
Posted by Richard Frovarp <ri...@sendit.nodak.edu>.
solprovider@apache.org wrote:
> On 2/6/08, Richard Frovarp <ri...@sendit.nodak.edu> wrote:
>
>> It does assign the PID and they do rollover. There's only 32K of them,
>> which isn't too many on some systems. My incoming email systems rollover
>> several times a day, but those are short lived sendmail and various
>> scanning processes.
>>
>
> Most programs needing many short-lived processes use a pool to avoid
> the performance hit of creating many new processes. The performance
> hit depends on the threading model; some are optimized so a new
> process is faster than a new thread. I am unaware of a modern MTA
> that creates a new process for each message.
> [Richard: This is an RFI. I hate ignorance, especially my own. What
> MTA is used?]
>
Sendmail 8.13. Here's my PID list for sendmail at the moment. Granted
some are the parents, queue runner, some are hung open, etc. But it most
certainly isn't using a pool. And it would be for each connection, as
multiple messages can be sent through a single connection.
1447 sendmail
1969 sendmail
2504 sendmail
2508 sendmail
2512 sendmail
2811 sendmail
3509 sendmail
4091 sendmail
5725 sendmail
5855 sendmail
6509 sendmail
6780 sendmail
7552 sendmail
7633 sendmail
8796 sendmail
9418 sendmail
9721 sendmail
11312 sendmail
12722 sendmail
14756 sendmail
15116 sendmail
15570 sendmail
16438 sendmail
16786 sendmail
16826 sendmail
16843 sendmail
16983 sendmail
17062 sendmail
17123 sendmail
17192 sendmail
17229 sendmail
17280 sendmail
17300 sendmail
17311 sendmail
17314 sendmail
17338 sendmail
17348 sendmail
17358 sendmail
17361 sendmail
17365 sendmail
17366 sendmail
17367 sendmail
25972 sendmail
26314 sendmail
28220 sendmail
29040 sendmail
29659 sendmail
30858 sendmail
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lenya.apache.org
For additional commands, e-mail: dev-help@lenya.apache.org
Re: [1.2] lenya.sh stores PID?
Posted by so...@apache.org.
On 2/6/08, Richard Frovarp <ri...@sendit.nodak.edu> wrote:
> solprovider@apache.org wrote:
> > On 2/6/08, Jörn Nettingsmeier <ne...@apache.org> wrote:
> >> Andreas Hartmann wrote:
> >> > I don't know if this behaviour is intended:
> >> > When I start my Lenya 1.2.x, the process ID seems to be written to a
> >> > file. Ctrl+C doesn't shutdown Jetty, but the next ./lenya.sh kills the
> >> > process with the stored ID.
> >> >
> >> > Is there any special reason for this behaviour? IMO storing PIDs is
> >> > dangerous - on the next day, the ID could be assigned to another process
> >> > which will be killed when you startup Lenya ... Or is this not possible?
> >> only if you own it. if you run lenya as a special user (which you really
> >> really should), then there's no danger at all. if you run lenya as root,
> >> you get what you deserve :)
> >>
> >> storing PIDs is standard behaviour for daemons that are singletons.
> >> check the /var/run directory of most unix systems for an overview of how
> >> many programs do it...
> >> Jörn Nettingsmeier
> >
> > I stored the PID to add the "stop" option. As Jorn states, this is
> > standard *nix behavior. The PID being stored is returned from the
> > java command, not the process calling lenya.sh. The PID is not stored
> > when starting in "cli" mode to prevent interfering with a known user
> > process. To allow hijacking through reuse of the PID, a user would
> > need to kill the process without using lenya.sh and find a method to
> > start a new process on that PID. I thought the OS always assigns the
> > PID and has enough PID numbers that I have never seen a rollover.
Clarification: The PID being stored is returned by the java command
starting Java/Jetty/Lenya as a BACKGROUND process.
> It does assign the PID and they do rollover. There's only 32K of them,
> which isn't too many on some systems. My incoming email systems rollover
> several times a day, but those are short lived sendmail and various
> scanning processes.
Most programs needing many short-lived processes use a pool to avoid
the performance hit of creating many new processes. The performance
hit depends on the threading model; some are optimized so a new
process is faster than a new thread. I am unaware of a modern MTA
that creates a new process for each message.
[Richard: This is an RFI. I hate ignorance, especially my own. What
MTA is used?]
> > Can an OS assign a new Java/Jetty/Lenya process a PID that is already in use?
> No, it shouldn't be able to.
So Lenya killing the process can be a problem if
- The Lenya server process is killed without using lenya.sh or
deleting the PID file.
- Another process is assigned that PID and is still running when
lenya.sh is run again.
- lenya.sh does not verify the PID still belongs to the Lenya server
in that directory.
POTENTIAL SOLUTION: Check if the process about to be killed was
started from this directory. This code checks that the PID still
refers to a process started from this directory:
LENYA_PID=`cat $LENYA_DIR/lenya.pid`
PID_CMD=`ps -p $LENYA_PID -o cmd --cols 999 --no-headers`
case $PID_CMD in
*$LENYA_DIR* ) echo "The PID from lenya.pid is for this instance of Lenya."
esac
This works in bash. The case format for string matching is used
because it does work in ancient sh. I am having difficulty making a
version that will work on zones with its ancient sh because ps does
not support::
"-o cmd" (Could parse for the CMD column.)
"--cols 999" or "--columns 999" (Uses environment variable COLUMNS.)
"--no-headers" (Use loop to check multiple lines?)
[Just saw Andreas' response so stopping this work.]
solprovider
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lenya.apache.org
For additional commands, e-mail: dev-help@lenya.apache.org
Re: [1.2] lenya.sh stores PID?
Posted by Richard Frovarp <ri...@sendit.nodak.edu>.
solprovider@apache.org wrote:
> On 2/6/08, Jörn Nettingsmeier <ne...@apache.org> wrote:
>
>> Andreas Hartmann wrote:
>> > I don't know if this behaviour is intended:
>> > When I start my Lenya 1.2.x, the process ID seems to be written to a
>> > file. Ctrl+C doesn't shutdown Jetty, but the next ./lenya.sh kills the
>> > process with the stored ID.
>> >
>> > Is there any special reason for this behaviour? IMO storing PIDs is
>> > dangerous - on the next day, the ID could be assigned to another process
>> > which will be killed when you startup Lenya ... Or is this not possible?
>> only if you own it. if you run lenya as a special user (which you really
>> really should), then there's no danger at all. if you run lenya as root,
>> you get what you deserve :)
>>
>> storing PIDs is standard behaviour for daemons that are singletons.
>> check the /var/run directory of most unix systems for an overview of how
>> many programs do it...
>> Jörn Nettingsmeier
>>
>
> I stored the PID to add the "stop" option. As Jorn states, this is
> standard *nix behavior. The PID being stored is returned from the
> java command, not the process calling lenya.sh. The PID is not stored
> when starting in "cli" mode to prevent interfering with a known user
> process. To allow hijacking through reuse of the PID, a user would
> need to kill the process without using lenya.sh and find a method to
> start a new process on that PID. I thought the OS always assigns the
> PID and has enough PID numbers that I have never seen a rollover.
>
It does assign the PID and they do rollover. There's only 32K of them,
which isn't too many on some systems. My incoming email systems rollover
several times a day, but those are short lived sendmail and various
scanning processes.
> Can an OS assign a new Java/Jetty/Lenya process a PID that is already in use?
>
No, it shouldn't be able to.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lenya.apache.org
For additional commands, e-mail: dev-help@lenya.apache.org
Re: [1.2] lenya.sh stores PID?
Posted by so...@apache.org.
On 2/6/08, Jörn Nettingsmeier <ne...@apache.org> wrote:
> Andreas Hartmann wrote:
> > I don't know if this behaviour is intended:
> > When I start my Lenya 1.2.x, the process ID seems to be written to a
> > file. Ctrl+C doesn't shutdown Jetty, but the next ./lenya.sh kills the
> > process with the stored ID.
> >
> > Is there any special reason for this behaviour? IMO storing PIDs is
> > dangerous - on the next day, the ID could be assigned to another process
> > which will be killed when you startup Lenya ... Or is this not possible?
> only if you own it. if you run lenya as a special user (which you really
> really should), then there's no danger at all. if you run lenya as root,
> you get what you deserve :)
>
> storing PIDs is standard behaviour for daemons that are singletons.
> check the /var/run directory of most unix systems for an overview of how
> many programs do it...
> Jörn Nettingsmeier
I stored the PID to add the "stop" option. As Jorn states, this is
standard *nix behavior. The PID being stored is returned from the
java command, not the process calling lenya.sh. The PID is not stored
when starting in "cli" mode to prevent interfering with a known user
process. To allow hijacking through reuse of the PID, a user would
need to kill the process without using lenya.sh and find a method to
start a new process on that PID. I thought the OS always assigns the
PID and has enough PID numbers that I have never seen a rollover.
Can an OS assign a new Java/Jetty/Lenya process a PID that is already in use?
The stop command is programmed so any call to lenya.sh stops the
running server before starting a new server (or just killing the old
server with the "stop" option.) Without that, a new server would
start, the old server would be orphaned, the new server is
inaccessible because the port was already used by the first server,
and many processes would continue running without purpose.
Remember this was added with code that locks lenya.sh to its
directory. Multiple instances of Lenya (on different ports) can run
from different directories. The stored PID only affects the server
from the directory containing that copy of lenya.sh (which is why the
PID is stored in that directory rather than the typical global
location. Differentiating between multiple instances of Lenya storing
PIDs in one global directory would have been more complicated.)
solprovider
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lenya.apache.org
For additional commands, e-mail: dev-help@lenya.apache.org
Re: [1.2] lenya.sh stores PID?
Posted by Jörn Nettingsmeier <ne...@apache.org>.
Andreas Hartmann wrote:
> Hi Lenya devs,
>
> I don't know if this behaviour is intended:
> When I start my Lenya 1.2.x, the process ID seems to be written to a
> file. Ctrl+C doesn't shutdown Jetty, but the next ./lenya.sh kills the
> process with the stored ID.
>
> Is there any special reason for this behaviour? IMO storing PIDs is
> dangerous - on the next day, the ID could be assigned to another process
> which will be killed when you startup Lenya ... Or is this not possible?
only if you own it. if you run lenya as a special user (which you really
really should), then there's no danger at all. if you run lenya as root,
you get what you deserve :)
storing PIDs is standard behaviour for daemons that are singletons.
check the /var/run directory of most unix systems for an overview of how
many programs do it...
--
Jörn Nettingsmeier
"One of my most productive days was throwing away 1000 lines of code."
- Ken Thompson.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lenya.apache.org
For additional commands, e-mail: dev-help@lenya.apache.org