You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@lucy.apache.org by goran kent <go...@gmail.com> on 2011/09/27 14:24:04 UTC
[lucy-user] Perl modules used on different machines (very slow search on one, but
not other)
Hi,
I've noticed a difference (and a massive difference in performance) in
what appears to be the Perl modules being used on two different
machines:
Actual search time (on the same small test index) is nice and
respectable on both machines: 0.05s:
But, total run time time is different and it seems to be related to
Perl modules:
machineA (slow)
~% dprofpp
Total Elapsed Time = 1.751774 Seconds
User+System Time = 1.671774 Seconds
Exclusive Times
%Time ExclSec CumulS #Calls sec/call Csec/c Name
18.3 0.306 0.512 5162 0.0001 0.0001 Text::Balanced::_match_variable
13.9 0.233 0.275 4280 0.0001 0.0001 Text::Balanced::_match_quotelike
11.4 0.191 0.962 20 0.0095 0.0481 Switch::filter_blocks
6.52 0.109 0.109 10596 0.0000 0.0000 Text::Balanced::_failmsg
6.40 0.107 0.307 2408 0.0000 0.0001 Text::Balanced::_match_codeblock
4.19 0.070 1.668 16 0.0044 0.1043 main::BEGIN
....
machineB (fast)
~% dprofpp
Total Elapsed Time = 0.369657 Seconds
User+System Time = 0.299657 Seconds
Exclusive Times
%Time ExclSec CumulS #Calls sec/call Csec/c Name
13.3 0.040 0.096 19 0.0021 0.0050 MyModdule0::BEGIN
6.67 0.020 0.020 1 0.0200 0.0200 utf8::AUTOLOAD
6.67 0.020 0.020 9 0.0022 0.0022 MyModule::BEGIN
6.67 0.020 0.088 17 0.0012 0.0052 MyModule2::BEGIN
6.67 0.020 0.213 16 0.0012 0.0133 main::BEGIN
3.34 0.010 0.010 1 0.0100 0.0100 Lucy::Index::Snapshot::read_file
3.34 0.010 0.010 1 0.0100 0.0100 Lucy::Search::PolySearcher::new
3.34 0.010 0.010 3 0.0033 0.0033 utf8::SWASHNEW
...
It looks like it's got something to do with Lucy deciding to use the
Perl module Text::* at configure time, but I may be wrong.
What should I look for to get an idea why this is happening?
More information:
perl Build test on fast machine: all pass.
On slow machine, the following fails:
t/core/051-fs_file_handle.t .......... Failed 42/46 subtests
t/core/103-fs_folder.t ............... 1/108 Can't clean up directory _fstest
S_tear_down at
/home/gk/projects/lucy/lucy/perl/../core/Lucy/Test/Store/TestFSFolder.c
line 67
at t/core/103-fs_folder.t line 20
t/core/103-fs_folder.t ............... Dubious, test returned 39
(wstat 9984, 0x2700)
Failed 106/108 subtests
Lucy: svn trunk (Revision: 1167124)
perl: 5.8.8
I had to install a few perl modules via CPAN to get the required
versions (I'm using CentOS):
Module::Build
Devel::PPPort
Scalar::Util (this was required instead of the builtin since the XS
version was required, etc)
Thanks
Re: [lucy-user] Perl modules used on different machines (very slow
search on one, but not other)
Posted by goran kent <go...@gmail.com>.
OK, removed the _fstest folder and it then tested cleanly.
Just prior to deleting it though, I noticed it contained another
folder called 'foo'. I haven't looked at the code, but if you're
doing 'rmdir _fstest' then it won't remove it since the folder is not
empty (may need to descend into it and rmdir, etc).
cheers
Re: [lucy-user] Perl modules used on different machines (very slow
search on one, but not other)
Posted by goran kent <go...@gmail.com>.
On Tue, Sep 27, 2011 at 11:43 PM, Marvin Humphrey
<ma...@rectangular.com> wrote:
> Text::Balanced and Switch are not dependencies of Lucy, so the reasons they
> are being loaded conditionally lie elsewhere.
Yikes, you were right - they were being sucked in via
Net::Google::SafeBrowsing2, etc (Devel::TraceUse is your friend for
this kind of thing).
Sorry about the noise.
>> On slow machine, the following fails:
>> t/core/051-fs_file_handle.t .......... Failed 42/46 subtests
>> t/core/103-fs_folder.t ............... 1/108 Can't clean up directory _fstest
>> S_tear_down at
>> /home/gk/projects/lucy/lucy/perl/../core/Lucy/Test/Store/TestFSFolder.c
>> line 67
>> at t/core/103-fs_folder.t line 20
>> t/core/103-fs_folder.t ............... Dubious, test returned 39
>> (wstat 9984, 0x2700)
>> Failed 106/108 subtests
>
> The TestFSFolder test file fails intermittently on certain systems (I've seen
> it most often with MSVC), and I've never figured out why; then when it fails,
> it does not clean up after itself properly. Delete that _fstest dir and the
> test may well pass afterwards. If it continues to fail, that's great news --
> you can help me debug it. :)
Weird - I'll play around with the tests a bit and see what I can report back.
Thanks for the help!
Re: [lucy-user] Perl modules used on different machines (very slow
search on one, but not other)
Posted by Marvin Humphrey <ma...@rectangular.com>.
On Tue, Sep 27, 2011 at 02:24:04PM +0200, goran kent wrote:
> 18.3 0.306 0.512 5162 0.0001 0.0001 Text::Balanced::_match_variable
> 13.9 0.233 0.275 4280 0.0001 0.0001 Text::Balanced::_match_quotelike
> 11.4 0.191 0.962 20 0.0095 0.0481 Switch::filter_blocks
> 6.52 0.109 0.109 10596 0.0000 0.0000 Text::Balanced::_failmsg
> 6.40 0.107 0.307 2408 0.0000 0.0001 Text::Balanced::_match_codeblock
> 4.19 0.070 1.668 16 0.0044 0.1043 main::BEGIN
> It looks like it's got something to do with Lucy deciding to use the
> Perl module Text::* at configure time, but I may be wrong.
Text::Balanced and Switch are not dependencies of Lucy, so the reasons they
are being loaded conditionally lie elsewhere.
> What should I look for to get an idea why this is happening?
Probably just grep for those class names in your code base to start with. If
that doesn't reveal the culprit, I seem to recall that there are techniques
for finding out exactly when a module gets loaded, but I forget what they are.
Maybe try perlmonks.org if you're still stumped.
> On slow machine, the following fails:
> t/core/051-fs_file_handle.t .......... Failed 42/46 subtests
> t/core/103-fs_folder.t ............... 1/108 Can't clean up directory _fstest
> S_tear_down at
> /home/gk/projects/lucy/lucy/perl/../core/Lucy/Test/Store/TestFSFolder.c
> line 67
> at t/core/103-fs_folder.t line 20
> t/core/103-fs_folder.t ............... Dubious, test returned 39
> (wstat 9984, 0x2700)
> Failed 106/108 subtests
The TestFSFolder test file fails intermittently on certain systems (I've seen
it most often with MSVC), and I've never figured out why; then when it fails,
it does not clean up after itself properly. Delete that _fstest dir and the
test may well pass afterwards. If it continues to fail, that's great news --
you can help me debug it. :)
Cheers,
Marvin Humphrey