You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Ala' Alkhaldi (JIRA)" <ji...@apache.org> on 2014/07/07 21:08:35 UTC
[jira] [Commented] (CASSANDRA-7316) Windows feature parity - lock
JVM in memory to prevent swapping
[ https://issues.apache.org/jira/browse/CASSANDRA-7316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14054039#comment-14054039 ]
Ala' Alkhaldi commented on CASSANDRA-7316:
------------------------------------------
I compared C* performance using stress test while using large memory pages vs small pages. After running several tests the results were similar (within test error in my opinion). Here is a sample of the results
The tests were run on AWS m1.large Windows server 2008 R2 instance. The stress test client is run from a separate instance.
{code:title=Small Pages}
cassandra-stress mixed ratio(read=1,write=1) -node 10.168.93.204 -mode native cql3
id, ops , op/s, key/s, mean, med, .95, .99, .999, max, time, stderr
4 threadCount, 3800 , 119, 119, 33.5, 31.2, 50.2, 77.7, 112.1, 122.1, 32.0, 0.00970
8 threadCount, 8300 , 253, 253, 31.1, 31.1, 47.6, 67.2, 83.6, 94.1, 32.8, 0.02082
16 threadCount, 233800 , 1123, 1123, 14.2, 14.8, 32.5, 64.6, 142.2, 426.5, 208.2, 0.02105
24 threadCount, 101400 , 1736, 1736, 13.7, 14.3, 29.4, 73.2, 144.7, 632.4, 58.4, 0.02155
36 threadCount, 81700 , 2512, 2512, 14.2, 9.7, 37.6, 94.8, 228.7, 405.6, 32.5, 0.01771
54 threadCount, 99550 , 2944, 2944, 18.1, 12.9, 57.8, 120.5, 274.0, 823.6, 33.8, 0.01549
81 threadCount, 135500 , 2564, 2564, 31.3, 27.3, 77.8, 141.8, 373.9, 796.8, 52.9, 0.02762
121 threadCount, 98200 , 2611, 2611, 45.7, 31.3, 115.5, 236.8, 2093.5, 2422.0, 37.6, 0.01753
181 threadCount, 123800 , 3029, 3029, 58.6, 37.2, 165.9, 355.1, 643.0, 1141.0, 40.9, 0.02268
271 threadCount, 339650 , 3035, 3035, 88.6, 44.3, 225.0, 640.5, 14452.1, 15317.0, 111.9, 0.02556
406 threadCount, 362950 , 4012, 4012, 99.8, 56.6, 263.8, 859.2, 2570.8, 5657.2, 90.5, 0.03540
609 threadCount, 368000 , 4076, 4076, 146.1, 73.2, 500.7, 1268.5, 2478.4, 8818.8, 90.3, 0.03571
913 threadCount, 798700 , 4079, 4079, 219.0, 78.2, 922.8, 2056.4, 10413.4, 17735.8, 195.8, 0.03285
{code}
{code:title=Large Pages}
cassandra-stress mixed ratio(read=1,write=1) -node 10.168.93.204 -mode native cql3
id, ops , op/s, key/s, mean, med, .95, .99, .999, max, time, stderr
4 threadCount, 4200 , 132, 132, 30.1, 31.1, 32.7, 53.4, 78.8, 84.0, 31.8, 0.00832
8 threadCount, 62700 , 320, 320, 25.0, 30.9, 33.1, 54.2, 79.7, 867.6, 196.2, 0.02037
16 threadCount, 161850 , 1210, 1210, 13.1, 8.8, 33.0, 68.6, 126.8, 398.2, 133.8, 0.02185
24 threadCount, 67000 , 1704, 1704, 13.9, 15.0, 26.2, 64.7, 137.6, 377.4, 39.3, 0.02431
36 threadCount, 81000 , 2118, 2118, 16.9, 11.6, 38.1, 87.1, 296.8, 5627.4, 38.2, 0.01901
54 threadCount, 92350 , 2835, 2835, 18.6, 15.7, 32.5, 76.1, 193.1, 360.8, 32.6, 0.03706
81 threadCount, 97700 , 2795, 2795, 28.6, 25.9, 68.4, 1380, 414.9, 677.2, 35.0, 0.01987
121 threadCount, 125950 , 2647, 2647, 45.0, 30.8, 129.4, 227.2, 508.0, 1110.2, 47.6, 0.02548
181 threadCount, 139700 , 3042, 3042, 58.5, 37.5, 161.0, 280.4, 698.6, 1622.6, 45.9, 0.02177
271 threadCount, 203350 , 3512, 3512, 76.0, 41.6, 189.7, 580.3, 1807.0, 3253.5, 57.9, 0.03517
406 threadCount, 683650 , 3701, 3701, 108.9, 59.0, 326.8, 784.8, 1762.7, 10646.9, 184.7, 0.02278
609 threadCount, 560800 , 4319, 4319, 138.7, 69.1, 496.5, 1356.9, 3223.8, 16892.8, 129.8, 0.03182
913 threadCount, 800200 , 4503, 4503, 199.9, 88.1, 843.7, 2151.5, 4249.6, 10486.0, 177.7, 0.02249
{code}
The question of whether the large pages feature is actually used or not follows from this results. How can I verify that windows is assigning large pages to the JVM. Unfortunately, I can't find away to achieve that. Below is a description of the setup and trials I have done:
*Environment Setup*
I am using 64-bit Windows 7 professional edition for my tests. "Lock pages in memory" Windows security policy is enabled as described in [Java Support for Large Memory Pages|http://www.oracle.com/technetwork/java/javase/tech/largememory-jsp-137182.html]. I also verified that Large Pages feature is enabled by running java version command as the following:
{code}
java -XX:+UseLargePages -version
{code}
Where I get the following results, which dedicate that large pages feature is enabled:
{code}
java version "1.7.0_60"
Java(TM) SE Runtime Environment (build 1.7.0_60-b19)
Java HotSpot(TM) 64-Bit Server VM (build 24.60-b09, mixed mode)
{code}
To test large pages usage I consumed all the memory available for the heap using this sample Java program from [this video|https://www.youtube.com/watch?v=6dLAo1LGHrk] in all my trials to consume all the memory available for the java heap:
{code}
public class InfiniteStringHashmap {
public static void main(String[] args) throws Exception{
Map<Long, String> map = new HashMap<Long, String>();
for(long i=1; true; i++){
StringBuilder sb = new StringBuilder();
for(long j=0;j<i;j++) sb.append(" ");
map.put(i, sb.toString());
if(i % 1000 == 0){
System.out.print(".");
Thread.sleep(1000);
}
}
}
}
{code}
I ran this sample program using the following command (with a fixed heap size of 512m for example):
{code}
java -Xms512m -Xmx512m -XX:+UseLargePages InfiniteStringHashmap
{code}
I also tried other heap sizes as small as 12MB and as large as 10GB. Please note that I run my test application immediately after restarting the machine to make sure my free RAM memory is not fragmented.
*Failed trials to monitor the memory*
In order to verify whether Large Memory Pages are used or not, I tried:
# [RamMap tool|http://technet.microsoft.com/en-us/sysinternals/ff700229.aspx] from Windows SystInternals which has a specific field for Large Pages. No matter how I change the heap size, it does not show usage of Large Memory pages. To verify it, I tried [Creating a File Mapping Using Large Pages|http://msdn.microsoft.com/en-us/library/windows/desktop/aa366543%28v=vs.85%29.aspx] Windows API example from the MSDN to narrow down the possibilities. It worked perfectly fine (it print a 2MB page size from inside the code). RamMap tool does not show anything.
# Printing the page size used by the Java process using _sun.misc.Unsafe_ [pageSize()|http://www.docjar.com/docs/api/sun/misc/Unsafe.html#pageSize] as suggested [here|http://stackoverflow.com/questions/19047584/getting-virtual-memory-page-size-by-java-code]. It prints 4096 (the default page size on Windows) all the time. To verify it, I tried Large Pages on Linux version of the Hotspot JVM as described in [this video|https://www.youtube.com/watch?v=6dLAo1LGHrk], and it worked. However, printing the page size didn't work (it prints 4096 all the time).
# [Vadump|http://www.softpedia.com/get/System/System-Miscellaneous/Vadump.shtml] tool that shows virtual memory statistics about a specific process. The "-o" option is supposed to show the types of VM used by a process, the number of used pages, and the total size of occupied by each type which can be used to deduce whether large pages are used. Unfortunately, this command fails with error code 24 when I set the JVM UseLargePages option.
# [VmMap|http://technet.microsoft.com/en-us/sysinternals/dd535533.aspx] tool does not update the looked memory column.
# Simple monitoring tools like TaskManager and Perfmon do not provide detailed information about Large pages.
# Java Mission Controller: Doesn't provide page size info.
# Process Explorer: same as above
Do you know of a way to measure the page size of a process or monitor the usage of large memory pages in Windows?
> Windows feature parity - lock JVM in memory to prevent swapping
> ---------------------------------------------------------------
>
> Key: CASSANDRA-7316
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7316
> Project: Cassandra
> Issue Type: Improvement
> Reporter: Joshua McKenzie
> Assignee: Ala' Alkhaldi
> Priority: Minor
> Labels: Windows, perfomance
> Fix For: 3.0
>
>
> Similar to mlockall() in CLibrary.java for linux, it would be nice to lock the virtual address space on Windows to prevent page faults.
> One option: Reference API: http://msdn.microsoft.com/en-us/library/windows/desktop/aa366895(v=vs.85).aspx
--
This message was sent by Atlassian JIRA
(v6.2#6252)