too many files open

[John Michael <jo...@acadiacom.net>]

Hi
I use the perl sub below to create thumbs with image magick on my mod perl server and
never really had any problems with it.  I put it on another server and after
a short while it froze up the server with this error.
too many files open.

It is running under a mod-perl environment because in doing so I am also able to rename images at the same time.
I know that because it is running under a mod perl environment, the lifetime of the script is until roboot of server or
restart of apache.


The freeze happend on one of the lines in this sub so I'm thinking I'm not doing
something right and leaving files open.
I don't think this is a result of the problem with the server.

I would appreciate if someone would look at it and tell me if it is ok or not or give me some suggestions to prevent this problem in the future so that I can continue to run it in mod-perl.

Perlmagick claims that it handles all of the file handling.


sub is called like so:
# Call to build the thumbs here
if ($build_thumbs){&build_thumbs($folder,$working_dir,$thumb_pattern);}


#####################################
sub build_thumbs {
my ($folder,$base_dir,$thumb_pattern) = @_;
my $current_dir = cwd;
my $working_dir = join("/",$base_dir, $folder);

print qq~<p>Building Thumbs in Folder: $folder</p>\n~;
chdir ($working_dir) || die "cannot cd $working_dir ($!)";
my @image_array = glob("*.jpg");
my @images = ();
foreach (@image_array) {
    unless ( $_ =~ /$thumb_pattern/i ) {
    push(@images, $_);
}  }
my $pic;
my $num_thumbs = @images;

foreach $pic (@images){
    my $image = Image::Magick->new(magick=>'JPEG');
    my $file = join("/",$working_dir, $pic);
    open(DATA, "$file") || die "$file $!";
        $image->Read(file=>DATA);
    close(DATA);

## MAKE THUMB HERE ##
my ($width, $height, $size, $format) = split(/,/,$image->Ping($file));
my $rotated = 0;
if ($width > $height){
    $image->Rotate(degrees=>'90');
    $rotated = 1;
}
$image->Resize(geometry=>'55',filter=>'Gaussian', blur=>'.5');
if ($rotated){
    $image->Rotate(degrees=>'-90');
}
## END MAKE THUMB HERE ##

my $out_file = join("/",$working_dir, "$thumb_pattern$pic");

open(DATA, ">$out_file");
    $image->Write(file=>DATA,filename=>$out_file);
close(DATA);

undef @$image;
}

undef $image;
chdir ($current_dir);

print qq~<p>Built <b>$num_thumbs</b> Thumbs in Folder: $folder</p>\n~;
}
############################################################################
###############

THanks in advance
John Michael

Re: too many files open

[___cliff rayman___ <cl...@genwax.com>]

are you sure the new system's OS has a large enough
"open files" parameter?

on linux this could be modified by the parameters
/proc/sys/fs/file-max
/proc/sys/fs/inode-max

it am sure it is different on bsd, solaris etc..

John Michael wrote:

> Hi
> I use the perl sub below to create thumbs with image magick on my mod perl server and
> never really had any problems with it.  I put it on another server and after
> a short while it froze up the server with this error.
> too many files open.

--
___cliff rayman___cliff@genwax.com___http://www.genwax.com/

Re: too many files open

[Marius Feraru <al...@deathsdoor.com>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


Today at 10:14, 'John Michael' wrote:
|JM|Perlmagick claims that it handles all of the file handling.
And it really does it. So why don't you let him do its job!!?!
... meaning, why do you use things like:
|JM|    my $image = Image::Magick->new(magick=>'JPEG');
|JM|    my $file = join("/",$working_dir, $pic);
|JM|    open(DATA, "$file") || die "$file $!";
|JM|        $image->Read(file=>DATA);
|JM|    close(DATA);
and
|JM|open(DATA, ">$out_file");
|JM|    $image->Write(file=>DATA,filename=>$out_file);
|JM|close(DATA);
instead of i.e.:
	my $image = Image::Magick->new;
	my $ret = $image->Read($file);
	return $ret if $ret;
and
	$ret = $image->Write($out_file);
	return $ret if $ret;
?!

Yet, there is no obvious error in your code that should drive you system
crazy. So, there are some suggestions (in addition to the questioned one
above :) that could make your code behave nicer:
- - do check error codes returned by PerlMagick!
- - globing? use it just at home ;-] in fact enhance your program using some 
MIME module to detect properly the file type + check out for files like 
'foo.jpg' that in fact is an animated GIF file so you will get an entire 
group of output files instead of one etc...
- - you will surely gain lots of cpu time if avoiding that rotation stuff***
- - use File::Spec for constructing file paths in a portable way
- - you may wanna use some thumbmakeronly module :)) : Image::GD::Thumbnail
if that gaussian blur it's not so important ;-]


*** as I see it you need some thumbnails size normalization so: how to avoid
image rotation and re-rotation?
example:

my ($w,$h) = calcGeometry($image->Get('width','height'), 50, 50);
$ret = $image->Scale( width => $w, height => $h);
warn $ret if $ret;

sub calcGeometry {
	my ($w,$h,$stdW,$stdH) = @_;
	if($w>$h) {
		$h = $h/($w/$stdW);
		$w = $stdW;
	} else {
		$w = $w/($h/$stdH);
		$h = $stdH;
	}
	return int($w), int($h);
}
                                                    

- -- 
Marius Feraru                                 http://altblue.n0i.net/
"its easy to stop using Perl: I do it after every project"





-----BEGIN PGP SIGNATURE-----

iD8DBQE8DAVdn0ZKufYp8iURAkCWAJ918+VPfGN6ciLx+FnZeZATds4TcwCeI2bx
tzjzTSoz4J/DSiez3rl5xSc=
=KfuI
-----END PGP SIGNATURE-----