hinting system for poll (was Re: /dev/poll vs. aio_)

[Niels Provos <pr...@citi.umich.edu>]

In message <Pi...@funky.monkey.org>, Chuck L
ever writes:
>Niels Provos is working on a "hinting" poll() for Linux that uses a system
>very similar to this where network devices register themselves with
>poll(), and provide hints when they are ready for I/O.  Niels, maybe you
>can describe your work?
As part of the Linux Scalability Project I have implemented a hinting
system for poll to speed up network intensive applications, e.g. web
servers.

The hinting system works as follows:
The file descriptors are passed along as argument in the device
specific poll() call and the device driver keeps them in a backmapping
list.  When an an event occurs that changes the poll state, the device
driver sets a hint for all file descriptors in the backmapping list.
At process wake up, the process only needs to call the device specific
poll() function for file descriptors that have a hint.

I have run a benchmark with thttpd-2.04 and httperf-0.6 between 2.2.3-ac4
and 2.2.3-ac4 + poll hinting. I obtained the following results for

readprofile -r
httperf --server localhost --port 8080 --num-conns 100000 --rate 650
readprofile | egrep "(poll|select|wait)" | sort -k3 -nr

2.2.3-ac4:
  4017 do_select                                  6.9740
  3024 sock_poll                                 75.6000
  2521 inet_poll                                 57.2955
  2344 tcp_poll                                   9.7667
   375 sys_select                                 0.3189
   187 __pollwait                                 0.9350
   169 free_wait                                  1.0833
   129 max_select_fd                              0.7866
    22 __tcp_select_window                        0.1222

2.2.3-ac4-hint:
  1460 do_select                                  1.5336
   752 sys_select                                 0.6065
   305 max_select_fd                              1.3616
    45 tcp_poll                                   0.1114
    35 __tcp_select_window                        0.1944
    19 poll_free_wait                             0.0651
    15 sock_poll                                  0.3125
    12 tcp_timewait_state_process                 0.0435
    10 expand_poll_hints                          0.0128

They seem to indicate that the hinting system is able to avoid many
calls to {sock,inet,tcp}_poll. Perhaps somebody can explain to me how
comparable the output of readprofile is.

I am still working on this to make sure that it is bug free, but
at the moment I am fairly confident to make patches available
within about a week.

Greetings,
 Niels.

--
Niels Provos <pr...@citi.umich.edu> finger provos@umich.edu for pgp info