You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@apr.apache.org by Dennis Clarke <dc...@blastwave.org> on 2017/01/10 12:36:47 UTC
[Bug 45615] "Large Files not supported" with 64-bit build
re: https://bz.apache.org/bugzilla/show_bug.cgi?id=45615
I think it best to follow up here as per suggestions by Yann and Rainer
wherein I can run further tests and experiments to determine what is
happening here in these Niagara class systems.
Firstly, sorry for awaking what seems like a long dead cold bug but it
really isn't a "Large Files not supported" bug as opposed to just a
message that needs to be tweaked. Indeed yes this is a 64 bit build and
so off_t and size_t and going to be 64-bit sized :
int
main(int argc, char *argv[])
{
printf ("\nC\n");
printf ( "sizeof char = %d\n", sizeof (char));
printf ( "sizeof double = %d\n", sizeof (double));
printf ( "sizeof float = %d\n", sizeof (float));
printf ( "sizeof int = %d\n", sizeof (int));
printf ( "sizeof long = %d\n", sizeof (long));
printf ( "sizeof long long = %d\n", sizeof (long long));
printf ( "sizeof short = %d\n", sizeof (short));
printf ( "sizeof void * = %d\n", sizeof (void *));
printf ( "sizeof clock_t = %d\n", sizeof (clock_t));
printf ( "sizeof gid_t = %d\n", sizeof (gid_t));
printf ( "sizeof pid_t = %d\n", sizeof (pid_t));
printf ( "sizeof size_t = %d\n", sizeof (size_t));
printf ( "sizeof ssize_t = %d\n", sizeof (ssize_t));
printf ( "sizeof off_t = %d\n", sizeof (off_t));
printf ( "sizeof time_t = %d\n", sizeof (time_t));
printf ( "sizeof uid_t = %d\n", sizeof (uid_t));
printf ( "sizeof \"\" = %d\n", sizeof (""));
return (EXIT_SUCCESS);
}
C
sizeof char = 1
sizeof double = 8
sizeof float = 4
sizeof int = 4
sizeof long = 8
sizeof long long = 8
sizeof short = 2
sizeof void * = 8
sizeof clock_t = 8
sizeof gid_t = 4
sizeof pid_t = 4
sizeof size_t = 8
sizeof ssize_t = 8
sizeof off_t = 8
sizeof time_t = 8
sizeof uid_t = 4
sizeof "" = 1
So that seems clear.
As for the blocking rand, well hrmmm, not unless I was trying to read a
ton of data from /dev/random wherein :
Devices random(7D)
NAME
random, urandom - Strong random number generator device
SYNOPSIS
/dev/random
/dev/urandom
DESCRIPTION
The /dev/random and /dev/urandom files are special files
that are a source for random bytes generated by the kernel
random number generator device. The /dev/random and
/dev/urandom files are suitable for applications requiring
high quality random numbers for cryptographic purposes.
The generator device produces random numbers from data and
devices available to the kernel and estimates the amount of
randomness (or entropy) collected from these sources. The
entropy level determines the amount of high quality random
numbers that are produced at a given time.
Applications retrieve random bytes by reading /dev/random or
/dev/urandom. The /dev/random interface returns random bytes
only when sufficient amount of entropy has been collected.
If there is no entropy to produce the requested number of
bytes, /dev/random blocks until more entropy can be
obtained. Non-blocking I/O mode can be used to disable the
blocking behavior. The /dev/random interface also supports
poll(2). Note that using poll(2) will not increase the speed
at which random numbers can be read.
etc etc ...
So I would expect that /dev/random may slow down but not by too much
while many other processes are running on the system and in this case
the build system is a single virtual zone inside a machine with a
number of zones. There should be plenty of data. However I have never
run a benchmark. I can certainly run a test for "blocking rand()" as
per http://pubs.opengroup.org/onlinepubs/9699919799/functions/rand.html
However I will say that the Apache 2.4.25 server seems to be running
very well with this new apr and apr-util but I am sure we can sort out
this weird test behavior. I certainly have the Sparc hardware sitting
here and can even provide the Oracle Sparc M7 tests if needed.
Dennis
Re: [Bug 45615] "Large Files not supported" with 64-bit build
Posted by Dennis Clarke <dc...@blastwave.org>.
On 01/10/2017 06:03 PM, Yann Ylavic wrote:
> On Tue, Jan 10, 2017 at 6:50 PM, Dennis Clarke <dc...@blastwave.org> wrote:
>>>
>>> Could you strace (truss?) a simple main() that calls rand() to confirm?
>>
>> Even better, Dtrace.
>>
>> Regardless I am juggling a few things and will get right on this.
>
> Thanks for investigating.
>
> Also, when at this, could you please provide the value of DEV_RANDOM
> in "include/arch/unix/apr_private.h" (i.e. /dev/random or
> /dev/urandom) for your ./configure'd tests?
Well I was the one that specified this in configure and therefore it is
correct but for the sake of a trivial test I may do a quick spin with
this :
$ ./configure --prefix=/usr/local \
> --enable-threads --enable-shared --enable-static \
> --enable-other-child --with-devrandom=/dev/urandom --enable-posix-shm
Merely a curiosity to see what happens here as we may be running in
circles looking for a clearly blocking source of entropy where the
well is dry.
dc
Re: [Bug 45615] "Large Files not supported" with 64-bit build
Posted by Dennis Clarke <dc...@blastwave.org>.
> Thanks Dennis for the follow up.
>
> /dev/urandom is the default (if existing), so if you don't ./configure
> --with-devrandom explicitly it should work too.
Well clearly we found the problem and here is some offensive performance
data to reveal the horrible truth :
node000 $ /usr/bin/time -p dd if=/dev/random bs=8192 \
> count=16384 of=dev_random2.dat
0+16384 records in
0+16384 records out
real 2393.45
user 0.21
sys 2391.88
node000 $ /usr/bin/time -p dd if=/dev/urandom bs=8192 \
> count=16384 of=dev_urandom2.dat
0+16384 records in
0+16384 records out
real 5.67
user 0.09
sys 5.57
So there is the proof right there. That was the whole problem.
I may have to recompile httpd-2.4.25 also depending on how apr and
apt-util really interface with httpd and I am curious where I would see
and performance difference.
I am also somewhat curious about the quality difference in entropy from
the /dev/random data vs /dev/urandom and thus I fetched the sources for
"ent" from http://www.fourmilab.ch/random/ and compiled with c99 here
to get this :
node000 $ ./ent -b -c dev_random.dat
Value Char Occurrences Fraction
0 34079262 0.500008
1 34078178 0.499992
Total: 68157440 1.000000
Entropy = 1.000000 bits per bit.
Optimum compression would reduce the size
of this 68157440 bit file by 0 percent.
Chi square distribution for 68157440 samples is 0.02, and randomly
would exceed this value 89.55 percent of the times.
Arithmetic mean value of data bits is 0.5000 (0.5 = random).
Monte Carlo value for Pi is 3.140871554 (error 0.02 percent).
Serial correlation coefficient is 0.000185 (totally uncorrelated = 0.0).
node000 $
node000 $ ./ent -b -c dev_urandom.dat
Value Char Occurrences Fraction
0 34071954 0.499901
1 34085486 0.500099
Total: 68157440 1.000000
Entropy = 1.000000 bits per bit.
Optimum compression would reduce the size
of this 68157440 bit file by 0 percent.
Chi square distribution for 68157440 samples is 2.69, and randomly
would exceed this value 10.12 percent of the times.
Arithmetic mean value of data bits is 0.5001 (0.5 = random).
Monte Carlo value for Pi is 3.142601198 (error 0.03 percent).
Serial correlation coefficient is -0.000063 (totally uncorrelated = 0.0).
node000 $
Essentially no measurable difference for a large number of bits.
Dennis
Re: [Bug 45615] "Large Files not supported" with 64-bit build
Posted by Yann Ylavic <yl...@gmail.com>.
On Tue, Jan 10, 2017 at 9:45 PM, Dennis Clarke <dc...@blastwave.org> wrote:
> On 01/10/2017 06:03 PM, Yann Ylavic wrote:
>>
>> On Tue, Jan 10, 2017 at 6:50 PM, Dennis Clarke <dc...@blastwave.org>
>> wrote:
>>>>
>>>>
>>>> Could you strace (truss?) a simple main() that calls rand() to confirm?
>>>
>>>
>>> Even better, Dtrace.
>>>
>>> Regardless I am juggling a few things and will get right on this.
>>
>>
>> Thanks for investigating.
>>
>> Also, when at this, could you please provide the value of DEV_RANDOM
>> in "include/arch/unix/apr_private.h" (i.e. /dev/random or
>> /dev/urandom) for your ./configure'd tests?
>
> Well let's just come to a full stop here. :-\
>
> Sure enough if we switch over to /dev/urandom then the tests run very
> neatly and at good speed. See log attached.
>
> So .. blocking source of random data has to be the issue.
Thanks Dennis for the follow up.
/dev/urandom is the default (if existing), so if you don't ./configure
--with-devrandom explicitly it should work too.
Re: [Bug 45615] "Large Files not supported" with 64-bit build
Posted by Dennis Clarke <dc...@blastwave.org>.
On 01/10/2017 06:03 PM, Yann Ylavic wrote:
> On Tue, Jan 10, 2017 at 6:50 PM, Dennis Clarke <dc...@blastwave.org> wrote:
>>>
>>> Could you strace (truss?) a simple main() that calls rand() to confirm?
>>
>> Even better, Dtrace.
>>
>> Regardless I am juggling a few things and will get right on this.
>
> Thanks for investigating.
>
> Also, when at this, could you please provide the value of DEV_RANDOM
> in "include/arch/unix/apr_private.h" (i.e. /dev/random or
> /dev/urandom) for your ./configure'd tests?
Well let's just come to a full stop here. :-\
Sure enough if we switch over to /dev/urandom then the tests run very
neatly and at good speed. See log attached.
So .. blocking source of random data has to be the issue.
Dennis
Re: [Bug 45615] "Large Files not supported" with 64-bit build
Posted by Dennis Clarke <dc...@blastwave.org>.
On 01/10/2017 06:03 PM, Yann Ylavic wrote:
> On Tue, Jan 10, 2017 at 6:50 PM, Dennis Clarke <dc...@blastwave.org> wrote:
>>>
>>> Could you strace (truss?) a simple main() that calls rand() to confirm?
>>
>> Even better, Dtrace.
>>
>> Regardless I am juggling a few things and will get right on this.
>
> Thanks for investigating.
>
> Also, when at this, could you please provide the value of DEV_RANDOM
> in "include/arch/unix/apr_private.h" (i.e. /dev/random or
> /dev/urandom) for your ./configure'd tests?
>
node000 # grep "DEV_RANDOM" include/arch/unix/apr_private.h
#define DEV_RANDOM "/dev/random"
Hrmmmm .. makes me wonder what would happen if I try /dev/urandom here.
Just a thought.
Dennis
Re: [Bug 45615] "Large Files not supported" with 64-bit build
Posted by Yann Ylavic <yl...@gmail.com>.
On Tue, Jan 10, 2017 at 6:50 PM, Dennis Clarke <dc...@blastwave.org> wrote:
>>
>> Could you strace (truss?) a simple main() that calls rand() to confirm?
>
> Even better, Dtrace.
>
> Regardless I am juggling a few things and will get right on this.
Thanks for investigating.
Also, when at this, could you please provide the value of DEV_RANDOM
in "include/arch/unix/apr_private.h" (i.e. /dev/random or
/dev/urandom) for your ./configure'd tests?
Regards,
Yann.
Re: [Bug 45615] "Large Files not supported" with 64-bit build
Posted by Dennis Clarke <dc...@blastwave.org>.
On 01/10/2017 12:57 PM, Yann Ylavic wrote:
> On Tue, Jan 10, 2017 at 1:36 PM, Dennis Clarke <dc...@blastwave.org> wrote:
>>
>> re: https://bz.apache.org/bugzilla/show_bug.cgi?id=45615
>>
>
> re: long running (blocking?) apr_skiplist test.
I think we can close the bug about Large File Support and make a note
that this is a poorly worded message. Better yet, perhaps we can find
where that comes from and do a check for LFS support and then issue a
message that is better. Or no message.
> First, is it this test that takes a huge time to complete or was it
> the same for the previous tests too (and which ones)?
A pile of them are very very slow.
Can not say which ones, I started the process and walked away to other
things. If each test exists in its own little code bits then I can
easily insert calls to hrtime and get precise metrics on them all.
>>
>> As for the blocking rand, well hrmmm, not unless I was trying to read a
>> ton of data from /dev/random wherein :
>>
>> Devices random(7D)
>>
>> NAME
>> random, urandom - Strong random number generator device
>>
>> SYNOPSIS
>> /dev/random
>> /dev/urandom
>
> apr_skiplists use the rand() function for its implementation, but on
> most systems (but I can't tell about Solaris), rand() does not use
> /dev/[u]random, so it should not block.
>
> Could you strace (truss?) a simple main() that calls rand() to confirm?
Even better, Dtrace.
Regardless I am juggling a few things and will get right on this.
Dennis
Re: [Bug 45615] "Large Files not supported" with 64-bit build
Posted by Yann Ylavic <yl...@gmail.com>.
On Tue, Jan 10, 2017 at 1:36 PM, Dennis Clarke <dc...@blastwave.org> wrote:
>
> re: https://bz.apache.org/bugzilla/show_bug.cgi?id=45615
>
re: long running (blocking?) apr_skiplist test.
First, is it this test that takes a huge time to complete or was it
the same for the previous tests too (and which ones)?
>
> As for the blocking rand, well hrmmm, not unless I was trying to read a
> ton of data from /dev/random wherein :
>
> Devices random(7D)
>
> NAME
> random, urandom - Strong random number generator device
>
> SYNOPSIS
> /dev/random
> /dev/urandom
apr_skiplists use the rand() function for its implementation, but on
most systems (but I can't tell about Solaris), rand() does not use
/dev/[u]random, so it should not block.
Could you strace (truss?) a simple main() that calls rand() to confirm?
Regards,
Yann.
Re: [Bug 45615] "Large Files not supported" with 64-bit build
Posted by Dennis Clarke <dc...@blastwave.org>.
On 01/10/2017 01:12 PM, Rainer Jung wrote:
> Hi Dennis,
>
Good day fine Sir.
<snip>
>> Firstly, sorry for awaking what seems like a long dead cold bug but it
>> really isn't a "Large Files not supported" bug as opposed to just a
>> message that needs to be tweaked. Indeed yes this is a 64 bit build and
>> so off_t and size_t and going to be 64-bit sized :
>
> I will have a look at the message "Line 345: Large Files not supported"
> coming from "make check" and see whether I can tweak it ("not supported
> or not needed" or similar). Due to the API used by APR I think we can't
> be more precise (effectively distinguish between the two cases) easily.
Hrmmm .. in that case perhaps a more harmless message can be
constructed. Perhaps even "possible Large File support issue" but
even that is misleading. So the API hides the truth? Is that the real
issue here? Or maybe I should say protects the user from the hardware.
>
> ...
>
>> As for the blocking rand, well hrmmm, not unless I was trying to read a
>> ton of data from /dev/random wherein :
>>
>> Devices random(7D)
<snip>
>>
>> So I would expect that /dev/random may slow down but not by too much
>> while many other processes are running on the system and in this case
>> the build system is a single virtual zone inside a machine with a
>> number of zones. There should be plenty of data. However I have never
>> run a benchmark. I can certainly run a test for "blocking rand()" as
>> per http://pubs.opengroup.org/onlinepubs/9699919799/functions/rand.html
>>
>>
>> However I will say that the Apache 2.4.25 server seems to be running
>> very well with this new apr and apr-util but I am sure we can sort out
>> this weird test behavior. I certainly have the Sparc hardware sitting
>> here and can even provide the Oracle Sparc M7 tests if needed.
>
> /dev/random can block because many server systems quickly run out of
> entropy.
Do we really need a crypto grade source of random data? Because we do
have /dev/urandom also and that will feed random noise endlessly.
> If you could follow my suggestions for further debugging your hanging or
Am working on it .. was interrupted by the whole "work" thing among
others. :-)
> extremely long running apr "make check" (using pstack, prstat, truss as
> written in the ticket), I'm sure we can find the culprit. If it is
> /dev/random, we can find that in this type of debug info, if it is
> something else likely we can also determine it from the three.
Could be a time to roll out Dtrace even.
Dennis
Re: [Bug 45615] "Large Files not supported" with 64-bit build
Posted by Rainer Jung <ra...@kippdata.de>.
Hi Dennis,
Am 10.01.2017 um 13:36 schrieb Dennis Clarke:
>
> re: https://bz.apache.org/bugzilla/show_bug.cgi?id=45615
>
> I think it best to follow up here as per suggestions by Yann and Rainer
> wherein I can run further tests and experiments to determine what is
> happening here in these Niagara class systems.
>
> Firstly, sorry for awaking what seems like a long dead cold bug but it
> really isn't a "Large Files not supported" bug as opposed to just a
> message that needs to be tweaked. Indeed yes this is a 64 bit build and
> so off_t and size_t and going to be 64-bit sized :
I will have a look at the message "Line 345: Large Files not supported"
coming from "make check" and see whether I can tweak it ("not supported
or not needed" or similar). Due to the API used by APR I think we can't
be more precise (effectively distinguish between the two cases) easily.
...
> As for the blocking rand, well hrmmm, not unless I was trying to read a
> ton of data from /dev/random wherein :
>
> Devices random(7D)
>
> NAME
> random, urandom - Strong random number generator device
>
> SYNOPSIS
> /dev/random
> /dev/urandom
>
> DESCRIPTION
> The /dev/random and /dev/urandom files are special files
> that are a source for random bytes generated by the kernel
> random number generator device. The /dev/random and
> /dev/urandom files are suitable for applications requiring
> high quality random numbers for cryptographic purposes.
>
> The generator device produces random numbers from data and
> devices available to the kernel and estimates the amount of
> randomness (or entropy) collected from these sources. The
> entropy level determines the amount of high quality random
> numbers that are produced at a given time.
>
> Applications retrieve random bytes by reading /dev/random or
> /dev/urandom. The /dev/random interface returns random bytes
> only when sufficient amount of entropy has been collected.
> If there is no entropy to produce the requested number of
> bytes, /dev/random blocks until more entropy can be
> obtained. Non-blocking I/O mode can be used to disable the
> blocking behavior. The /dev/random interface also supports
> poll(2). Note that using poll(2) will not increase the speed
> at which random numbers can be read.
>
>
> etc etc ...
>
> So I would expect that /dev/random may slow down but not by too much
> while many other processes are running on the system and in this case
> the build system is a single virtual zone inside a machine with a
> number of zones. There should be plenty of data. However I have never
> run a benchmark. I can certainly run a test for "blocking rand()" as
> per http://pubs.opengroup.org/onlinepubs/9699919799/functions/rand.html
>
>
> However I will say that the Apache 2.4.25 server seems to be running
> very well with this new apr and apr-util but I am sure we can sort out
> this weird test behavior. I certainly have the Sparc hardware sitting
> here and can even provide the Oracle Sparc M7 tests if needed.
/dev/random can block because many server systems quickly run out of
entropy.
If you could follow my suggestions for further debugging your hanging or
extremely long running apr "make check" (using pstack, prstat, truss as
written in the ticket), I'm sure we can find the culprit. If it is
/dev/random, we can find that in this type of debug info, if it is
something else likely we can also determine it from the three.
Regards,
Rainer