[OT]: mmap (Was: Templating System (preview Embperl 2.0))

["Paul J. Lucas" <pj...@barefooters.org>]

On Fri, 28 Jul 2000, Kenneth Lee wrote:

> it would be good for the user to choose between mmap or normal i/o at 
> compile time. i'll try HTML::Tree anyway in the meantime.

	It's not that simple.  Using mmap(2) greatly affects how one
	writes code: it's not a drop-in replacement for standard I/O.
	An mmap'd file *becomes* memory in the time it takes the OS to
	handle a page-fault.  You then further get speed by accessing
	the file *as* memory via ordinary pointers rather than function
	calls and I/O buffers.  Inside the HTML Tree code is a generic
	C++ STL-style container class wrapped around mmap...quite nice
	if I do say so myself.

	By definition, file I/O off disk can't be faster than mmap(2).

	- Paul

Re: [OT]: mmap (Was: Templating System (preview Embperl 2.0))

[Malcolm Beattie <mb...@sable.ox.ac.uk>]

Paul J. Lucas writes:
> On Fri, 28 Jul 2000, Malcolm Beattie wrote:
> 
> > Assuming the kernel only keeps track of the last fault position in the file,
> > it won't recognise that it's being read linearly (twice :-) and may well not
> > do the async read ahead and drop behind in page cache that it would do
> > otherwise. Once again, you'll lose performance with mmap.
> 
> 	And reading in the file twice into two separate I/O buffers
> 	is better?

I've forgotten exactly what the original situation was so I'm not
sure whether it's better or not. With mmap you also have the
potential disadvantage that you're taking a page fault into the
kernel for every single page (say 4k or 8k) whereas with read(),
a single system call will fill up a much larger buffer with data.
It's also possible that the kernel can play tricks to get you the
data from a read better (on some architectures it could avoid
copying through the cache or it may be able to get the data
transferred from the source directly into your buffer without
passing through the CPU or suchlike).

Basically, mmap may well be an advantage from the ease or
elegance-of-coding point of view and in some cases it may win out
over read() for performance/scalability or whatever but it's not a
foregone conclusion and there are kernel/architecture-specific
difference which may swing the balance one way or the other. Now
we're getting offtopic though.

I really just wanted to make the points that saying "mmap is
always better than read" is not true and that there are things
going on behind the scenes.

--Malcolm

-- 
Malcolm Beattie <mb...@sable.ox.ac.uk>  I am looking for a Linux (and
Unix Systems Programmer                  maybe Apache/mod_perl) job/contract
Oxford University Computing Services   http://users.ox.ac.uk/~mbeattie/cv.html

Re: [OT]: mmap (Was: Templating System (preview Embperl 2.0))

["Paul J. Lucas" <pj...@barefooters.org>]

On Fri, 28 Jul 2000, Malcolm Beattie wrote:

> Assuming the kernel only keeps track of the last fault position in the file,
> it won't recognise that it's being read linearly (twice :-) and may well not
> do the async read ahead and drop behind in page cache that it would do
> otherwise. Once again, you'll lose performance with mmap.

	And reading in the file twice into two separate I/O buffers
	is better?

	- Paul

Re: [OT]: mmap (Was: Templating System (preview Embperl 2.0))

[Malcolm Beattie <mb...@sable.ox.ac.uk>]

Paul J. Lucas writes:
> > This is the quote from "Philip and Alex..." that I was talking about
> > earlier. I don't know if its relevant or not:
> > 
> > They replied "NaviServer uses memory-mapped file I/O on single-processor
> > machines but automatically configures itself to use operating system read on
> > multi-CPU machines so as to reduce shared memory contention."
> 
> 	mmap(2) is *supposed* to share memory when read-only which is
> 	what's beign done for HTML file reads, so I don't understand
> 	what "contention" there could possibly be.

That's where some of the subtlety comes in. The first that springs
to mind is that with multiple CPUs of an architecture with a
virtually indexed cache, if the mmap maps the file at different
virtual addresses (as it very well may without slow global
communication between the reading processes) then the data will end
up being duplicated in processor cache. That slows things down.

Another point is that it can be harder for the kernel to notice
when a file is being read linearly when all it sees is page faults
through an address space. That means it may not do read-ahead on
that file as well. Most decent Unices will probably try to keep
track these days (certainly Linux does page-ahead and in fact
page-around to try to predict when to read-ahead or drop-behind
or whatever). On the other hand, if you have two concurrent
processes doing the reading then you'll get page faults at, for
example, byte 8k, 16k, 8k(second process starting), 24k, 16k and
so on. Assuming the kernel only keeps track of the last fault
position in the file, it won't recognise that it's being read
linearly (twice :-) and may well not do the async read ahead and
drop behind in page cache that it would do otherwise. Once again,
you'll lose performance with mmap.

--Malcolm

-- 
Malcolm Beattie <mb...@sable.ox.ac.uk>
Unix Systems Programmer
Oxford University Computing Services

Re: [OT]: mmap (Was: Templating System (preview Embperl 2.0))

[Matt Sergeant <ma...@sergeant.org>]

On Fri, 28 Jul 2000, Paul J. Lucas wrote:

> > This is the quote from "Philip and Alex..." that I was talking about
> > earlier. I don't know if its relevant or not:
> > 
> > They replied "NaviServer uses memory-mapped file I/O on single-processor
> > machines but automatically configures itself to use operating system read on
> > multi-CPU machines so as to reduce shared memory contention."
> 
> 	mmap(2) is *supposed* to share memory when read-only which is
> 	what's beign done for HTML file reads, so I don't understand
> 	what "contention" there could possibly be.
> 
> 	Additionally, any OS that doesn't get mmap right is severly
> 	broken and shouldn't be used.

Well Philip does recommend HPUX ;-)

-- 
<Matt/>

Fastnet Software Ltd. High Performance Web Specialists
Providing mod_perl, XML, Sybase and Oracle solutions
Email for training and consultancy availability.
http://sergeant.org | AxKit: http://axkit.org

Re: [OT]: mmap (Was: Templating System (preview Embperl 2.0))

["Paul J. Lucas" <pj...@barefooters.org>]

> This is the quote from "Philip and Alex..." that I was talking about
> earlier. I don't know if its relevant or not:
> 
> They replied "NaviServer uses memory-mapped file I/O on single-processor
> machines but automatically configures itself to use operating system read on
> multi-CPU machines so as to reduce shared memory contention."

	mmap(2) is *supposed* to share memory when read-only which is
	what's beign done for HTML file reads, so I don't understand
	what "contention" there could possibly be.

	Additionally, any OS that doesn't get mmap right is severly
	broken and shouldn't be used.

	- Paul

Re: [OT]: mmap (Was: Templating System (preview Embperl 2.0))

[Matt Sergeant <ma...@sergeant.org>]

On Fri, 28 Jul 2000, Paul J. Lucas wrote:

> On Fri, 28 Jul 2000, Kenneth Lee wrote:
> 
> > it would be good for the user to choose between mmap or normal i/o at 
> > compile time. i'll try HTML::Tree anyway in the meantime.
> 
> 	It's not that simple.  Using mmap(2) greatly affects how one
> 	writes code: it's not a drop-in replacement for standard I/O.
> 	An mmap'd file *becomes* memory in the time it takes the OS to
> 	handle a page-fault.  You then further get speed by accessing
> 	the file *as* memory via ordinary pointers rather than function
> 	calls and I/O buffers.  Inside the HTML Tree code is a generic
> 	C++ STL-style container class wrapped around mmap...quite nice
> 	if I do say so myself.
> 
> 	By definition, file I/O off disk can't be faster than mmap(2).

This is the quote from "Philip and Alex..." that I was talking about
earlier. I don't know if its relevant or not:

>>
My favorite AOLserver story starts in the seventh-floor lounge of the MIT
Artificial Intelligence Laboratory. I was asking Robert Thau, primary
author of the Apache server program, why the Netscape 1.1 server was so
slow. He said "Oh that's because those guys don't understand Unix.
They're actually using the read system call to read files." Everyone in
the room laughed except me. What? What's wrong with that? I asked. He
replied, "Everyone knows you can't use read; you have to use memory-mapped
I/O." 
  I knew that the NaviSoft guys were about to release a new version so I
thought I'd give the naifs in Santa Barbara the benefit of some hard-core
MIT engineering knowledge. I told them the story and asked them what
NaviServer, as it was then called, did. They replied "NaviServer uses
memory-mapped file I/O on single-processor machines but automatically
configures itself to use operating system read on multi-CPU machines so
as to reduce shared memory contention."
<<

-- 
<Matt/>

Fastnet Software Ltd. High Performance Web Specialists
Providing mod_perl, XML, Sybase and Oracle solutions
Email for training and consultancy availability.
http://sergeant.org | AxKit: http://axkit.org