You are viewing a plain text version of this content. The canonical link for it is here.
Posted to embperl@perl.apache.org by Neil Gunton <ne...@nilspace.com> on 2001/03/09 18:38:29 UTC
Using Lingua::Ispell in Embperl
I am trying to integrate a spell checker into my website, and the
Lingua::Ispell CPAN module looks like it will do what I need. I can
successfully test the module offline, in a perl script, but when I use
the same code in an Embperl page it fails. I see from the module
documentation that it calls Open2::open2 in order to call the ispell
application. I also notice that there is a buildup of defunct aspell
processes when I try to use this module in Embperl.
I am just doing a basic test of the module at this point, nothing
special. The code is as follows:
use Lingua::Ispell;
$Lingua::Ispell::path = "/usr/bin/ispell";
Lingua::Ispell::spellcheck ("hello neil") or print OUT "oops";
And when I run this code, each time I get another zombie process:
(output from ps -aux)
nobody 2296 0.0 0.0 0 0 ? Z 11:50 0:00 [aspell <defunct>]
nobody 2351 1.8 0.0 0 0 ? Z 12:32 0:00 [aspell <defunct>]
I thought that perhaps there was some problem with starting the process
as user nobody, but I have tried running the same code in a simple perl
script on the command line as nobody, and it does work. So what is
different about doing this from within Embperl?
Any ideas?
TIA
-Neil
---------------------------------------------------------------------
To unsubscribe, e-mail: embperl-unsubscribe@perl.apache.org
For additional commands, e-mail: embperl-help@perl.apache.org
Re: Using Lingua::Ispell in Embperl
Posted by Neil Gunton <ne...@nilspace.com>.
Gerald Richter wrote:
>
> > I
> > wonder what else is different when we are running under Apache/mod_perl?
>
> PATH and other environment variables are different
I am getting the impression that this is not so much an Embperl problem
(or even a mod_perl problem), but perhaps it is more of an issue
specifically with Lingua::Ispell. I will try to contact the author
directly and see if he has any ideas.
Meantime, I wonder if anyone else has successfully used modules from
Apache/mod_perl which spawn other processes using the same method (i.e.
calling Open2::open2()). In other words, is this a general issue, or is
it something specific to this Lingua::Ispell module (or, I guess,
perhaps even something specific to ispell itself).
Apart from that, I'll stop bugging you guys about this - but if anyone
reads this stuff later and has an idea then I am all ears!
Thanks again
-Neil
---------------------------------------------------------------------
To unsubscribe, e-mail: embperl-unsubscribe@perl.apache.org
For additional commands, e-mail: embperl-help@perl.apache.org
Re: Using Lingua::Ispell in Embperl
Posted by Neil Gunton <ne...@nilspace.com>.
Gerald Richter wrote:
>
> > I
> > wonder what else is different when we are running under Apache/mod_perl?
>
> PATH and other environment variables are different
The path to ispell is set explicitly in the code:
$SIG{CHLD} = 'IGNORE';
use Lingua::Ispell;
$Lingua::Ispell::path = "/usr/bin/ispell";
Lingua::Ispell::allow_compounds(1);
Lingua::Ispell::spellcheck ("hello neil") or print OUT "oops";
The documentation for the Lingua::Ispell module doesn't mention any
other variables which need to be set.
-Neil
---------------------------------------------------------------------
To unsubscribe, e-mail: embperl-unsubscribe@perl.apache.org
For additional commands, e-mail: embperl-help@perl.apache.org
Re: Using Lingua::Ispell in Embperl
Posted by Gerald Richter <ri...@ecos.de>.
> I
> wonder what else is different when we are running under Apache/mod_perl?
PATH and other environment variables are different
Gerald
---------------------------------------------------------------------
To unsubscribe, e-mail: embperl-unsubscribe@perl.apache.org
For additional commands, e-mail: embperl-help@perl.apache.org
Re: Using Lingua::Ispell in Embperl
Posted by Neil Gunton <ne...@nilspace.com>.
I thought you might be interested in a follow-up to this thread.
I ended up implementing a spellchecking daemon using Paul Kulchenko's
wonderful SOAP::Lite module (http://www.soaplite.com and CPAN). You can
take a look at a demo and download the client & server scripts from my
website:
http://www.neilgunton.com/
I also wrote it up as an article which was published in the current
(September 2001) issue of Dr Dobb's Journal. The server has been up
constantly for a couple of months now, and it seems pretty stable...
It's a very simple script. Let me know if you find it useful...
-Neil
Neil Gunton wrote:
>
> Angus Lees wrote:
> >
> > On Tue, Mar 13, 2001 at 09:42:48AM -0500, Neil Gunton wrote:
> > > But it doesn't seem to affect the main problem of the spawn
> > > failing. This is a real corker. I wonder what else is different when
> > > we are running under Apache/mod_perl? I guess it could still be a
> > > stdin problem, you mentioned that the < /dev/null redirection is not
> > > exactly the same. I am stumped.
> >
> > try "strace -f -p <process id>" on the running apache/mod_perl process.
>
> Thanks Gus. I think I have found the problem - it is a generic one for
> mod_perl scripts spawning other processes. You need to be really careful
> about it; there is a whole section on spawning in the mod_perl
> performance tuning doc, and a mention in the mod_perl traps doc. I
> contacted the author of Lingua::Ispell and he doesn't have any
> experience using mod_perl, though he did confirm someone else having the
> same kind of problem that I am having (but no resolution). Since he is
> apparently unaware of mod_perl spawning issues, I doubt that
> Lingua::Ispell is "mod_perl safe" as it stands.
>
> At this point I am thinking about writing a separate daemon wrapper for
> Lingua::Ispell which I will communicate with from Embperl using sockets.
> This may be the best solution, given the apparent issues with spawning
> processes from mod_perl. If this works then I will pass the code back to
> the author of Lingua::Ispell, because I imagine that I am not the first
> person to want to do trivial spell checking from mod_perl/Embperl
> scripts... hopefully what I can end up with is a simple method call,
> passing in a string and getting back a list of bad words (or, perhaps,
> the same string with bad words marked up somehow, e.g. in HTML). And if
> I make the daemon pre-forking (i.e. able to efficiently handle multiple
> requests in parallel, like apache) then it could be quite useful in a
> web server environment. Plus, for busy websites you could put the
> spellcheck server on its own machine, load balance etc.
>
> Thanks again
>
> -Neil
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: embperl-unsubscribe@perl.apache.org
> For additional commands, e-mail: embperl-help@perl.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: embperl-unsubscribe@perl.apache.org
For additional commands, e-mail: embperl-help@perl.apache.org
Re: Using Lingua::Ispell in Embperl
Posted by Neil Gunton <ne...@nilspace.com>.
Gerald Richter wrote:
>
> > And if
> > I make the daemon pre-forking (i.e. able to efficiently handle multiple
> > requests in parallel, like apache) then it could be quite useful in a
> > web server environment. Plus, for busy websites you could put the
> > spellcheck server on its own machine, load balance etc.
> >
>
> You may checkout the modules Net::Deamon and PlRPC on CPAN, which may help
> you to build such a daemon. (Both are used in DBIProxyServer). I have also
> made a patch to Net::Daemon to do preforking, I am not sure if this patch
> has already gone into the version on CPAN. If not, I could send it to you
Thanks Gerald, I will take a look at these modules. I was looking at the
Perl Cookbook (O'Reilly), which seems to have working code ready-to-use.
However, if these modules already do it...
I'll keep you posted on anything interesting, and thanks again
-Neil
---------------------------------------------------------------------
To unsubscribe, e-mail: embperl-unsubscribe@perl.apache.org
For additional commands, e-mail: embperl-help@perl.apache.org
Re: Using Lingua::Ispell in Embperl
Posted by Gerald Richter <ri...@ecos.de>.
> And if
> I make the daemon pre-forking (i.e. able to efficiently handle multiple
> requests in parallel, like apache) then it could be quite useful in a
> web server environment. Plus, for busy websites you could put the
> spellcheck server on its own machine, load balance etc.
>
You may checkout the modules Net::Deamon and PlRPC on CPAN, which may help
you to build such a daemon. (Both are used in DBIProxyServer). I have also
made a patch to Net::Daemon to do preforking, I am not sure if this patch
has already gone into the version on CPAN. If not, I could send it to you
Gerald
-------------------------------------------------------------
Gerald Richter ecos electronic communication services gmbh
Internetconnect * Webserver/-design/-datenbanken * Consulting
Post: Tulpenstrasse 5 D-55276 Dienheim b. Mainz
E-Mail: richter@ecos.de Voice: +49 6133 925131
WWW: http://www.ecos.de Fax: +49 6133 925152
-------------------------------------------------------------
---------------------------------------------------------------------
To unsubscribe, e-mail: embperl-unsubscribe@perl.apache.org
For additional commands, e-mail: embperl-help@perl.apache.org
Re: Using Lingua::Ispell in Embperl
Posted by Neil Gunton <ne...@nilspace.com>.
Angus Lees wrote:
>
> On Tue, Mar 13, 2001 at 09:42:48AM -0500, Neil Gunton wrote:
> > But it doesn't seem to affect the main problem of the spawn
> > failing. This is a real corker. I wonder what else is different when
> > we are running under Apache/mod_perl? I guess it could still be a
> > stdin problem, you mentioned that the < /dev/null redirection is not
> > exactly the same. I am stumped.
>
> try "strace -f -p <process id>" on the running apache/mod_perl process.
Thanks Gus. I think I have found the problem - it is a generic one for
mod_perl scripts spawning other processes. You need to be really careful
about it; there is a whole section on spawning in the mod_perl
performance tuning doc, and a mention in the mod_perl traps doc. I
contacted the author of Lingua::Ispell and he doesn't have any
experience using mod_perl, though he did confirm someone else having the
same kind of problem that I am having (but no resolution). Since he is
apparently unaware of mod_perl spawning issues, I doubt that
Lingua::Ispell is "mod_perl safe" as it stands.
At this point I am thinking about writing a separate daemon wrapper for
Lingua::Ispell which I will communicate with from Embperl using sockets.
This may be the best solution, given the apparent issues with spawning
processes from mod_perl. If this works then I will pass the code back to
the author of Lingua::Ispell, because I imagine that I am not the first
person to want to do trivial spell checking from mod_perl/Embperl
scripts... hopefully what I can end up with is a simple method call,
passing in a string and getting back a list of bad words (or, perhaps,
the same string with bad words marked up somehow, e.g. in HTML). And if
I make the daemon pre-forking (i.e. able to efficiently handle multiple
requests in parallel, like apache) then it could be quite useful in a
web server environment. Plus, for busy websites you could put the
spellcheck server on its own machine, load balance etc.
Thanks again
-Neil
---------------------------------------------------------------------
To unsubscribe, e-mail: embperl-unsubscribe@perl.apache.org
For additional commands, e-mail: embperl-help@perl.apache.org
Re: Using Lingua::Ispell in Embperl
Posted by Angus Lees <gu...@switchonline.com.au>.
On Tue, Mar 13, 2001 at 09:42:48AM -0500, Neil Gunton wrote:
> But it doesn't seem to affect the main problem of the spawn
> failing. This is a real corker. I wonder what else is different when
> we are running under Apache/mod_perl? I guess it could still be a
> stdin problem, you mentioned that the < /dev/null redirection is not
> exactly the same. I am stumped.
try "strace -f -p <process id>" on the running apache/mod_perl process.
--
- Gus
---------------------------------------------------------------------
To unsubscribe, e-mail: embperl-unsubscribe@perl.apache.org
For additional commands, e-mail: embperl-help@perl.apache.org
Re: Using Lingua::Ispell in Embperl
Posted by Neil Gunton <ne...@nilspace.com>.
Gerald Richter wrote:
> try to run it offline with < /dev/null appended. That simulates what Apache
> does (not exactly the same)
>
> Does it work then ?
Yes - in other words, redirecting stdin to /dev/null doesn't seem to
make the script fail when being run in offline mode. Just as a sanity
check, here is the command line:
bash>embpexec.pl test.html < /dev/null
> Maybe the module Lingua::Ispell simply does not correctly clean up it
> childs. The childs are killed by the OS when your offline scripts
> terminates, but under mod_perl your process never terminates. Somebody wrote
> about setting %SIG, did you tried that ?
Yeah, somehow I misplaced Marcus' email but I finally tried it and it
does seem to get rid of the defunct processes. But it doesn't seem to
affect the main problem of the spawn failing. This is a real corker. I
wonder what else is different when we are running under Apache/mod_perl?
I guess it could still be a stdin problem, you mentioned that the <
/dev/null redirection is not exactly the same. I am stumped.
Hmmmm.
Any wild ideas most welcome...
TIA
-Neil
---------------------------------------------------------------------
To unsubscribe, e-mail: embperl-unsubscribe@perl.apache.org
For additional commands, e-mail: embperl-help@perl.apache.org
Re: Using Lingua::Ispell in Embperl
Posted by Gerald Richter <ri...@ecos.de>.
>
> > The other thing that is different when running under Apache is, that you
> > don't have a stdin. Some programms fail if stdin is /dev/null
>
try to run it offline with < /dev/null appended. That simulates what Apache
does (not exactly the same)
Does it work then ?
Maybe the module Lingua::Ispell simply does not correctly clean up it
childs. The childs are killed by the OS when your offline scripts
terminates, but under mod_perl your process never terminates. Somebody wrote
about setting %SIG, did you tried that ?
Gerald
---------------------------------------------------------------------
To unsubscribe, e-mail: embperl-unsubscribe@perl.apache.org
For additional commands, e-mail: embperl-help@perl.apache.org
Re: Using Lingua::Ispell in Embperl
Posted by Neil Gunton <ne...@nilspace.com>.
Gerald Richter wrote:
>
> > I am trying to integrate a spell checker into my website, and the
> > Lingua::Ispell CPAN module looks like it will do what I need. I can
> > successfully test the module offline, in a perl script, but when I use
> > the same code in an Embperl page it fails. I see from the module
> > documentation that it calls Open2::open2 in order to call the ispell
> > application. I also notice that there is a buildup of defunct aspell
> > processes when I try to use this module in Embperl.
> >
>
> The only thing that comes to my mind, is that there is a permission problem.
> Everything else should be quite the same as offline.
>
> When you run your Embperl page offline, does it work then ?
Yes it does work in offline mode. The same minimal file just fails when
running under Apache/mod_perl. I have also tried running the script
offline as user 'nobody', which is the same as how Apache runs. Is
anyone aware of any other permissions differences, apart from the user
we are running as?
> The other thing that is different when running under Apache is, that you
> don't have a stdin. Some programms fail if stdin is /dev/null
Ok, I don't know much about this, but it sounds reasonable. Since we are
apparently running another program (ispell), I guess there is some kind
of mechanism for piping stuff between the processes, and I could imagine
that using stdin somehow.
If this is the case, then is anyone aware of a workaround for doing this
under Apache/mod_perl? (apart from running as a CGI, which is not really
an option for performance reasons).
TIA
-Neil
> > I am just doing a basic test of the module at this point, nothing
> > special. The code is as follows:
> >
> > use Lingua::Ispell;
> > $Lingua::Ispell::path = "/usr/bin/ispell";
> > Lingua::Ispell::spellcheck ("hello neil") or print OUT "oops";
> >
> > And when I run this code, each time I get another zombie process:
> > (output from ps -aux)
> >
> > nobody 2296 0.0 0.0 0 0 ? Z 11:50 0:00 [aspell <defunct>]
> > nobody 2351 1.8 0.0 0 0 ? Z 12:32 0:00 [aspell <defunct>]
> >
---------------------------------------------------------------------
To unsubscribe, e-mail: embperl-unsubscribe@perl.apache.org
For additional commands, e-mail: embperl-help@perl.apache.org
Re: Using Lingua::Ispell in Embperl
Posted by Gerald Richter <ri...@ecos.de>.
> I am trying to integrate a spell checker into my website, and the
> Lingua::Ispell CPAN module looks like it will do what I need. I can
> successfully test the module offline, in a perl script, but when I use
> the same code in an Embperl page it fails. I see from the module
> documentation that it calls Open2::open2 in order to call the ispell
> application. I also notice that there is a buildup of defunct aspell
> processes when I try to use this module in Embperl.
>
The only thing that comes to my mind, is that there is a permission problem.
Everythingelse should be quite the same as offline.
When you run your Embperl page offline, does it work then ?
The other thing that is different when running under Apache is, that you
don't have a stdin. Some programms fail if stdin is /dev/null
Gerald
> I am just doing a basic test of the module at this point, nothing
> special. The code is as follows:
>
> use Lingua::Ispell;
> $Lingua::Ispell::path = "/usr/bin/ispell";
> Lingua::Ispell::spellcheck ("hello neil") or print OUT "oops";
>
> And when I run this code, each time I get another zombie process:
> (output from ps -aux)
>
> nobody 2296 0.0 0.0 0 0 ? Z 11:50 0:00 [aspell <defunct>]
> nobody 2351 1.8 0.0 0 0 ? Z 12:32 0:00 [aspell <defunct>]
>
> I thought that perhaps there was some problem with starting the process
> as user nobody, but I have tried running the same code in a simple perl
> script on the command line as nobody, and it does work. So what is
> different about doing this from within Embperl?
>
> Any ideas?
>
> TIA
>
> -Neil
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: embperl-unsubscribe@perl.apache.org
> For additional commands, e-mail: embperl-help@perl.apache.org
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: embperl-unsubscribe@perl.apache.org
For additional commands, e-mail: embperl-help@perl.apache.org
Re: Using Lingua::Ispell in Embperl
Posted by Neil Gunton <ne...@nilspace.com>.
"Marcus R. Popetz" wrote:
>
> I don't know about the code failing, but for the zombies, I couldn't find a
> sigchild handler in my brief grep of the code. You might try:
>
> #reap zombies.
> $SIG{CHLD} = 'IGNORE';
>
> To clean up the zombie processes.
>
> or
> local $SIG{CHLD} = 'IGNORE';
> if you like playing with local.
Hi Marcus, sorry about not replying sooner. Your idea works (though I
don't know how exactly) for getting rid of the zombie processes. Thanks
for the suggestion. (It doesn't seem to fix the problem of the process
failing though - that's another email).
Thanks again!
-Neil
---------------------------------------------------------------------
To unsubscribe, e-mail: embperl-unsubscribe@perl.apache.org
For additional commands, e-mail: embperl-help@perl.apache.org
Re: Using Lingua::Ispell in Embperl
Posted by "Marcus R. Popetz" <mp...@netility.com>.
At 09:38 AM 3/9/01, Neil Gunton wrote:
>And when I run this code, each time I get another zombie process:
>(output from ps -aux)
>
>nobody 2296 0.0 0.0 0 0 ? Z 11:50 0:00 [aspell <defunct>]
>nobody 2351 1.8 0.0 0 0 ? Z 12:32 0:00 [aspell <defunct>]
I don't know about the code failing, but for the zombies, I couldn't find a
sigchild handler in my brief grep of the code. You might try:
#reap zombies.
$SIG{CHLD} = 'IGNORE';
To clean up the zombie processes.
or
local $SIG{CHLD} = 'IGNORE';
if you like playing with local.
-mp
---------------------------------------------------------------------
To unsubscribe, e-mail: embperl-unsubscribe@perl.apache.org
For additional commands, e-mail: embperl-help@perl.apache.org