You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@river.apache.org by Mark Brouwer <ma...@cheiron.org> on 2007/12/19 16:24:22 UTC
Re: [jira] Created: (RIVER-280) make JERI MUX protocol impl maxFragmentSize
configurable
Bob Scheifler (JIRA) wrote:
> make JERI MUX protocol impl maxFragmentSize configurable
> --------------------------------------------------------
>
> Key: RIVER-280
> URL: https://issues.apache.org/jira/browse/RIVER-280
> Project: River
> Issue Type: Improvement
> Components: net_jini_jeri
> Affects Versions: jtsk_2.1
> Reporter: Bob Scheifler
> Priority: Minor
>
>
> In the current JERI MUX implementation, maxFragmentSize is hardcoded
> to 1024 in the MuxClient and MuxServer constructors. It would be
> useful to make it configurable via system properties, similar to
> initialInboundRation. Applications doing large data transfers can
> get better performance with larger fragments.
>
Hi Bob,
I noticed this issue. Today I found some time to perform some tests
myself with the code as listed in
https://issues.apache.org/jira/browse/RIVER-281 with NIO enabled.
Test are conducted on Windows XP SP2 with Pentium Mobile 1.7GHz and
during the tests I saw no excessive full GC's (max around 0.03 sec) and
I used Sun JDK 1.4.2_16.
Below the matrix of some settings for the ration (client and server) and
the maxFragmentSize and the measured throughput. Settings are in KBytes
while the average throughput is rounded in MBytes/sec.
| ration
fragment |---------------------------------
size | 4 16 32 124
---------|---------------------------------
1 | 9 13 -13- 13
2 | 12 16 18 18
4 | 14 19 21 21
16 | 13 22 25 26
32 | 12 22 25 26
I also ran the test with the Http(Server)Endpoint and the throughput was
only 13 MBytes/sec which means it equals the throughput with the default
settings for the Tcp(Server)Endpoint. You indicates it should perform
more in line with JRMP but I'm not experiencing that.
I also ran the test with JRMP and there I got a throughout of 38
MBytes/sec which is still much better than the 26 MBytes I was able to
get from the Tcp(Server)Endpoint.
I also ran some of the tests with JDK 1.5.0 and there JRMP jumped from
38 to 44 MBytes/sec. The Http(Server)Endpoint test went from 13 to 31
MBytes/sec while the tests with the Tcp(Server)Endpoint seemed to make
no improvements at all. So here indeed the Http(Server)Endpoint performs
more in line with JRMP.
Based on the above it looks like the settings for the ration above 32
KBytes have not that much effect, most effect can be obtained with
increasing the fragment size.
As I have no clue about the internals of the Mux code what would be the
(negative) consequence in case we change the default maxFragmentSize
from 1024 to 4 or more KBytes? I realize that per session more direct
memory will be consumed, but will it also have consequences for other
calls (with less data) relative to calls with lots of data?
--
Mark
Re: [jira] Created: (RIVER-280) make JERI MUX protocol impl maxFragmentSize
configurable
Posted by Mark Brouwer <ma...@cheiron.org>.
Bob Scheifler wrote:
> Mark Brouwer wrote:
>> What was the throughput you got for the 1Gb network and the 100Mb one,
>> also did you test with an mtu of 1500 for gigabit ethernet?
>
> I didn't test gigabit with mtu=1500, not worth it for me.
>
> Here are some numbers, measured using 1000 calls rather than 100,
> to minimize the skew of including VM warmup. To Martin's point,
> it's clear that JERI is using significantly more CPU time than JRMP.
Thanks for the numbers, this also explains why JRMP outperformed Jini
ERI for the various endpoints as I had always 100% CPU load.
Unfortunately I can't test with a proper network because in a moment of
insanity I decided to make my network wireless a few years ago and I
think I have neighbors who test EM bombs, but at least on the positive
site I'm confronted with some of the fallacies of distributed computing.
--
Mark
Re: [jira] Created: (RIVER-280) make JERI MUX protocol impl
maxFragmentSize configurable
Posted by Bob Scheifler <Bo...@Sun.COM>.
Mark Brouwer wrote:
> What was the throughput you got for the 1Gb network and the 100Mb one,
> also did you test with an mtu of 1500 for gigabit ethernet?
I didn't test gigabit with mtu=1500, not worth it for me.
Here are some numbers, measured using 1000 calls rather than 100,
to minimize the skew of including VM warmup. To Martin's point,
it's clear that JERI is using significantly more CPU time than JRMP.
JRMP, 1000Mbps mtu=9000
Finished after 89367 msecs, 111898 KByte/sec
server CPU:
user 21.1
sys 10.4
client CPU:
user 9.4
sys 8.6
JERI TCP, ration=0, frag=8192, 1000Mbps mtu=9000
Finished after 85699 msecs, 116687 KByte/sec
server CPU:
user 48.5
sys 23.5
client CPU:
user 39.5
sys 36.6
JERI TCP NIO, ration=0, frag=8192, 1000Mbps mtu=9000
Finished after 86910 msecs, 115061 KByte/sec
server CPU:
user 55.3
sys 27.1
client CPU:
user 30.9
sys 15.5
JRMP, 100Mbps mtu=1500
Finished after 859347 msecs, 11636 KByte/sec
server CPU:
user 22.8
sys 18.8
client CPU:
user 10.7
sys 10.4
JERI TCP NIO, ration=0, frag=8192, 100Mbps mtu=1500
Finished after 860562 msecs, 11620 KByte/sec
server CPU:
user 58.0
sys 30.9
client CPU:
user 30.4
sys 16.9
JERI HTTP, 100Mbps mtu=1500
Finished after 873766 msecs, 11444 KByte/sec
server CPU:
user 47.8
sys 14.3
client CPU:
user 32.2
sys 14.4
JERI HTTP, 1000Mbps mtu=9000
Finished after 1275516 msecs, 783 KByte/sec [100 calls only]
server CPU:
user 8.8
sys 1.4
client CPU:
user 4.9
sys 0.9
As you can see there's an odd and nasty performance problem with
JERI HTTP on the faster network to be diagnosed.
- Bob
Re: [jira] Created: (RIVER-280) make JERI MUX protocol impl maxFragmentSize
configurable
Posted by Mark Brouwer <ma...@cheiron.org>.
Bob Scheifler wrote:
> Bob Scheifler wrote:
>> I've just recently been testing on Solaris 10 Update 4 and JDK1.6
>> between two machines (2 socket x 2 core Opteron 2.2GHz) connected
>> via a gigabit network with jumbo frames (mtu 9000). By setting
>> the rations to 0 (infinite) and the fragment sizes to 8k,
>> JERI TCP performance without NIO slightly exceeds JRMP performance.
>
> With the same systems, but going across a 100Mb network with mtu 1500,
> the default JERI TCP settings perform only slightly slower than JRMP,
> and with NIO enabled it's nearly the same as JRMP. Setting the rations
> to 0 and increasing the fragment sizes help very little, as the
> network link is already pretty much saturated.
>
> - Bob
I'm pleased to hear you get Jini ERI at the same level as JRMP,
unfortunately I'm not able to near it on my configuration.
What was the throughput you got for the 1Gb network and the 100Mb one,
also did you test with an mtu of 1500 for gigabit ethernet? The numbers
I get for running everything on the host itself and skipping the
datalink layer are beyond the theoretical throughput one should get on a
100Mb network so I can understand that for such a configuration it has
not much use to fiddle with these parameters.
I also conducted some tests with varying ration and fragment size (NIO
enabled) when the payload is only 512 bytes in that case I see with
larger fragment sizes a very small degradation, although due to
making 100000 calls for a test there is a bit more variation in the
results. Still I was pleasantly surprised to get ~0.34 ms/call and a
throughput of ~1.5 MB/sec. For the tests it didn't really matter which
version of the JVM I was running.
Then I decided to do some tests with JRMP and it turned out it is 3
times as fast (~0.11 ms/call) for that type of call, but this is what I
expected due to the large amount of (expensive) security checks and
privileged operations made at various places in the invocation and
dispatch layer.
--
Mark
Re: [jira] Created: (RIVER-280) make JERI MUX protocol impl maxFragmentSize configurable
Posted by "Cornelius, Martin (DWBI)" <Ma...@smiths-heimann.com>.
Hi Bob,
i'd like to note, that the initial goal when i built my little test was,
to find out how JERI and JRMP (and CORBA as well) compare with regard to
*CPU* usage when large data chunks are transferred. Hence, i started
server and client on the same, otherwise unloaded machine. I became a
little alarmed when i noticed, that in this test the throughput with
JERI was below the theoretic capacity of a 100Mbit network, although the
2Ghz CPU was 100% utilized by just shipping the data. In fact, this
would state a knockout against JERI as 'general purpose' transport in
our applications. Thus, i was really happy to find out this is just a
matter of adding 1 line of code to make the fragment size configurable.
BTW, with a fragment size of 8K and unlimited ration, i also get roughly
the same througput (at 100% CPU) with JERI and JRMP when server and
client are on the same machine.
Another BTW: Do you think that unlimited ration has the consequence that
the deadlock-avoidance feature of JERI Multiplexing is compromised ?
Kind regards, Martin
************************************************
The information contained in, or attached to, this e-mail, may contain confidential information and is intended solely for the use of the individual or entity to whom they are addressed and may be subject to legal privilege. If you have received this e-mail in error you should notify the sender immediately by reply e-mail, delete the message from your system and notify your system manager. Please do not copy it for any purpose, or disclose its contents to any other person. The views or opinions presented in this e-mail are solely those of the author and do not necessarily represent those of the company. The recipient should check this e-mail and any attachments for the presence of viruses. The company accepts no liability for any damage caused, directly or indirectly, by any virus transmitted in this email.
************************************************
Re: [jira] Created: (RIVER-280) make JERI MUX protocol impl
maxFragmentSize configurable
Posted by Bob Scheifler <Bo...@Sun.COM>.
Bob Scheifler wrote:
> I've just recently been testing on Solaris 10 Update 4 and JDK1.6
> between two machines (2 socket x 2 core Opteron 2.2GHz) connected
> via a gigabit network with jumbo frames (mtu 9000). By setting
> the rations to 0 (infinite) and the fragment sizes to 8k,
> JERI TCP performance without NIO slightly exceeds JRMP performance.
With the same systems, but going across a 100Mb network with mtu 1500,
the default JERI TCP settings perform only slightly slower than JRMP,
and with NIO enabled it's nearly the same as JRMP. Setting the rations
to 0 and increasing the fragment sizes help very little, as the
network link is already pretty much saturated.
- Bob
Re: [jira] Created: (RIVER-280) make JERI MUX protocol impl
maxFragmentSize configurable
Posted by Bob Scheifler <Bo...@Sun.COM>.
Bob Scheifler wrote:
> I have
> noticed in my tests that enabling NIO on the server side makes
> performance go down in many cases.
Sorry, correction, I have noticed it in a few cases so far, not many.
- Bob
Re: [jira] Created: (RIVER-280) make JERI MUX protocol impl
maxFragmentSize configurable
Posted by Bob Scheifler <Bo...@Sun.COM>.
Mark Brouwer wrote:
> I noticed this issue. Today I found some time to perform some tests
> myself with the code as listed in
> https://issues.apache.org/jira/browse/RIVER-281 with NIO enabled.
Just FYI, I don't understand all of the interactions, but I have
noticed in my tests that enabling NIO on the server side makes
performance go down in many cases.
I've just recently been testing on Solaris 10 Update 4 and JDK1.6
between two machines (2 socket x 2 core Opteron 2.2GHz) connected
via a gigabit network with jumbo frames (mtu 9000). By setting
the rations to 0 (infinite) and the fragment sizes to 8k,
JERI TCP performance without NIO slightly exceeds JRMP performance.
> Based on the above it looks like the settings for the ration above 32
> KBytes have not that much effect, most effect can be obtained with
> increasing the fragment size.
I'd like to see data across a few OSes, without NIO, and across
real networks before coming to firm conclusions.
- Bob