You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@harmony.apache.org by Xiao-Feng Li <xi...@gmail.com> on 2006/11/01 12:12:44 UTC
[DRLVM][PORT] correct API to retrieve processor number
Hi, I am using port_CPUs_number() for GCv5 to get the processor
number, but my desktop returns one processor with it on Windows
although my processor is dual-core. (port_CPUs_number is defined in
port_sysinfo.h).
I think we need more general form of processor number retrieval API
that can return processor information including that of core and
hyperthreading.
How do you think?
Thanks,
xiaofeng
Re: [DRLVM][PORT] correct API to retrieve processor number
Posted by Xiao-Feng Li <xi...@gmail.com>.
On 11/1/06, Geir Magnusson Jr. <ge...@pobox.com> wrote:
>
>
> Xiao-Feng Li wrote:
> > Hi, I am using port_CPUs_number() for GCv5 to get the processor
> > number, but my desktop returns one processor with it on Windows
> > although my processor is dual-core. (port_CPUs_number is defined in
> > port_sysinfo.h).
> >
> > I think we need more general form of processor number retrieval API
> > that can return processor information including that of core and
> > hyperthreading.
> >
> > How do you think?
>
> I chuckled when I first read this, as if people would disagree - "no, I
> don't think we want to have an API to return accurate information about
> hardware capability".
>
> One thing to keep in mind is to ensure that it's extensible so that
> non-intel features can be detected as well. Easy portability is key.
Of course, since this is only an API issue. It will cover the
vendor-specific information underneath. Along with more and more
multi-processor/multi-core/multi-thread platforms available from
almost all major processor vendors, it is time to have a proper API
for the runtime, unless we give the burden to users with command line
options. Well even that, programming-wise we want an API to keep the
command line options.
Thanks,
xiaofeng
>
> geir
>
Re: [DRLVM][PORT] correct API to retrieve processor number
Posted by "Geir Magnusson Jr." <ge...@pobox.com>.
Xiao-Feng Li wrote:
> Hi, I am using port_CPUs_number() for GCv5 to get the processor
> number, but my desktop returns one processor with it on Windows
> although my processor is dual-core. (port_CPUs_number is defined in
> port_sysinfo.h).
>
> I think we need more general form of processor number retrieval API
> that can return processor information including that of core and
> hyperthreading.
>
> How do you think?
I chuckled when I first read this, as if people would disagree - "no, I
don't think we want to have an API to return accurate information about
hardware capability".
One thing to keep in mind is to ensure that it's extensible so that
non-intel features can be detected as well. Easy portability is key.
geir
Re: [DRLVM][PORT] correct API to retrieve processor number
Posted by Xiao-Feng Li <xi...@gmail.com>.
I will try it, thanks! -xiaofeng
On 11/12/06, Rana Dasgupta <rd...@gmail.com> wrote:
> I added a patch 2154 to look in the Windows kernel32.dll for
> GetNativeSystemInfo first and use it and use GetSystemInfo otherwise.
> Because of the organization of 64 bit and 32 bit kernel dlls as
> kernel32.dll and WOW redirection of kernel32.dll in emulation mode, this
> should ensure that we get back the correct info.
>
> I also added an api port_Cores_number() for Core counting arounf
> GetLogicalProcessorInfo(). We can expand this api as we need to also return
> cache and NUMA information as we discover further requirements. On the Linux
> mailing lists it looks like sysconf() will expand to return cores info for
> multicore systems and the NUMA api's on Linux are a different bunch
> altogether.
>
> Thanks,
> Rana
>
>
> On 11/1/06, Xiao-Feng Li <xi...@gmail.com> wrote:
> >
> > Very informative! Thanks, Rana.
> >
> > -xiaofeng
> >
> > On 11/2/06, Rana Dasgupta <rd...@gmail.com> wrote:
> > > On Windows, there is a bunch of bugs.
> > >
> > > - On W2003 SP1, Vista, XP-64 GetLogicalProcessorInformation() is the
> > > recommended api. It returns # of cores on AMD and # of logical procs
> > on
> > > Intel.
> > > - GetSystemInfo() is on XP-32 and Windows Server. It does not work in
> > > wow mode. I think GetNativeSystemInfo is needed.
> > > - For x64, the VC/VC++ the __cpuid() intrinsic returns
> > > wrong information
> > >
> > > The root cause is that the incorrect versions use the cpuid
> > > instruction incorrectly. __cpuid() uses old style cpuid and takes input
> > from
> > > eax only instead of eax, ecx . GetSystemInfo() uses the registers in
> > short
> > > mode when doing cpuid, so I think it fails for wow. It is amazing how
> > > Windows works at all!
> > >
> > >
> > >
> > >
> > > On 11/1/06, Xiao-Feng Li <xi...@gmail.com> wrote:
> > > >
> > > > On 11/1/06, Alexey Varlamov <al...@gmail.com> wrote:
> > > > > Just a wild guess: this may be caused by x86 emulation on em64t
> > > > > (x86_64). SDK docs advise to use GetNativeSystemInfo() in such case,
> > > > > instead of currently used GetSystemInfo(). (See
> > > > > vm\port\src\misc\win\sysinfo.c).
> > > > >
> > > >
> > > > huh, I guess you are right, since my machine is X86-64bit. :-) I will
> > > > try the API you pointed.
> > > >
> > > > Thanks,
> > > > xiaofeng
> > > >
> > > >
> > >
> > >
> >
>
>
Re: [DRLVM][PORT] correct API to retrieve processor number
Posted by Rana Dasgupta <rd...@gmail.com>.
I added a patch 2154 to look in the Windows kernel32.dll for
GetNativeSystemInfo first and use it and use GetSystemInfo otherwise.
Because of the organization of 64 bit and 32 bit kernel dlls as
kernel32.dll and WOW redirection of kernel32.dll in emulation mode, this
should ensure that we get back the correct info.
I also added an api port_Cores_number() for Core counting arounf
GetLogicalProcessorInfo(). We can expand this api as we need to also return
cache and NUMA information as we discover further requirements. On the Linux
mailing lists it looks like sysconf() will expand to return cores info for
multicore systems and the NUMA api's on Linux are a different bunch
altogether.
Thanks,
Rana
On 11/1/06, Xiao-Feng Li <xi...@gmail.com> wrote:
>
> Very informative! Thanks, Rana.
>
> -xiaofeng
>
> On 11/2/06, Rana Dasgupta <rd...@gmail.com> wrote:
> > On Windows, there is a bunch of bugs.
> >
> > - On W2003 SP1, Vista, XP-64 GetLogicalProcessorInformation() is the
> > recommended api. It returns # of cores on AMD and # of logical procs
> on
> > Intel.
> > - GetSystemInfo() is on XP-32 and Windows Server. It does not work in
> > wow mode. I think GetNativeSystemInfo is needed.
> > - For x64, the VC/VC++ the __cpuid() intrinsic returns
> > wrong information
> >
> > The root cause is that the incorrect versions use the cpuid
> > instruction incorrectly. __cpuid() uses old style cpuid and takes input
> from
> > eax only instead of eax, ecx . GetSystemInfo() uses the registers in
> short
> > mode when doing cpuid, so I think it fails for wow. It is amazing how
> > Windows works at all!
> >
> >
> >
> >
> > On 11/1/06, Xiao-Feng Li <xi...@gmail.com> wrote:
> > >
> > > On 11/1/06, Alexey Varlamov <al...@gmail.com> wrote:
> > > > Just a wild guess: this may be caused by x86 emulation on em64t
> > > > (x86_64). SDK docs advise to use GetNativeSystemInfo() in such case,
> > > > instead of currently used GetSystemInfo(). (See
> > > > vm\port\src\misc\win\sysinfo.c).
> > > >
> > >
> > > huh, I guess you are right, since my machine is X86-64bit. :-) I will
> > > try the API you pointed.
> > >
> > > Thanks,
> > > xiaofeng
> > >
> > >
> >
> >
>
Re: [DRLVM][PORT] correct API to retrieve processor number
Posted by Rana Dasgupta <rd...@gmail.com>.
Xiao Feng,
Did GetNativeSystemInfo() work? I can write our own api if we need it,
but the best way would be to do it around cpuid, since the chip knows best.
I know the cpuid documented behaviour on Intel. On other platforms, I am not
100% sure, though AMD and Intel have somewhat similar feedback from cpuid
(except for brandname string).
One advantage of using the OS api's is that they are already supposed to
have done this work with the platform vendors and should be uptodate.
Rana
On 11/1/06, Xiao-Feng Li <xi...@gmail.com> wrote:
>
> Very informative! Thanks, Rana.
>
> -xiaofeng
>
> On 11/2/06, Rana Dasgupta <rd...@gmail.com> wrote:
> > On Windows, there is a bunch of bugs.
> >
> > - On W2003 SP1, Vista, XP-64 GetLogicalProcessorInformation() is the
> > recommended api. It returns # of cores on AMD and # of logical procs
> on
> > Intel.
> > - GetSystemInfo() is on XP-32 and Windows Server. It does not work in
> > wow mode. I think GetNativeSystemInfo is needed.
> > - For x64, the VC/VC++ the __cpuid() intrinsic returns
> > wrong information
> >
> > The root cause is that the incorrect versions use the cpuid
> > instruction incorrectly. __cpuid() uses old style cpuid and takes input
> from
> > eax only instead of eax, ecx . GetSystemInfo() uses the registers in
> short
> > mode when doing cpuid, so I think it fails for wow. It is amazing how
> > Windows works at all!
> >
> >
> >
> >
> > On 11/1/06, Xiao-Feng Li <xi...@gmail.com> wrote:
> > >
> > > On 11/1/06, Alexey Varlamov <al...@gmail.com> wrote:
> > > > Just a wild guess: this may be caused by x86 emulation on em64t
> > > > (x86_64). SDK docs advise to use GetNativeSystemInfo() in such case,
> > > > instead of currently used GetSystemInfo(). (See
> > > > vm\port\src\misc\win\sysinfo.c).
> > > >
> > >
> > > huh, I guess you are right, since my machine is X86-64bit. :-) I will
> > > try the API you pointed.
> > >
> > > Thanks,
> > > xiaofeng
> > >
> > >
> >
> >
>
Re: [DRLVM][PORT] correct API to retrieve processor number
Posted by Xiao-Feng Li <xi...@gmail.com>.
Very informative! Thanks, Rana.
-xiaofeng
On 11/2/06, Rana Dasgupta <rd...@gmail.com> wrote:
> On Windows, there is a bunch of bugs.
>
> - On W2003 SP1, Vista, XP-64 GetLogicalProcessorInformation() is the
> recommended api. It returns # of cores on AMD and # of logical procs on
> Intel.
> - GetSystemInfo() is on XP-32 and Windows Server. It does not work in
> wow mode. I think GetNativeSystemInfo is needed.
> - For x64, the VC/VC++ the __cpuid() intrinsic returns
> wrong information
>
> The root cause is that the incorrect versions use the cpuid
> instruction incorrectly. __cpuid() uses old style cpuid and takes input from
> eax only instead of eax, ecx . GetSystemInfo() uses the registers in short
> mode when doing cpuid, so I think it fails for wow. It is amazing how
> Windows works at all!
>
>
>
>
> On 11/1/06, Xiao-Feng Li <xi...@gmail.com> wrote:
> >
> > On 11/1/06, Alexey Varlamov <al...@gmail.com> wrote:
> > > Just a wild guess: this may be caused by x86 emulation on em64t
> > > (x86_64). SDK docs advise to use GetNativeSystemInfo() in such case,
> > > instead of currently used GetSystemInfo(). (See
> > > vm\port\src\misc\win\sysinfo.c).
> > >
> >
> > huh, I guess you are right, since my machine is X86-64bit. :-) I will
> > try the API you pointed.
> >
> > Thanks,
> > xiaofeng
> >
> >
>
>
Re: [DRLVM][PORT] correct API to retrieve processor number
Posted by Rana Dasgupta <rd...@gmail.com>.
On Windows, there is a bunch of bugs.
- On W2003 SP1, Vista, XP-64 GetLogicalProcessorInformation() is the
recommended api. It returns # of cores on AMD and # of logical procs on
Intel.
- GetSystemInfo() is on XP-32 and Windows Server. It does not work in
wow mode. I think GetNativeSystemInfo is needed.
- For x64, the VC/VC++ the __cpuid() intrinsic returns
wrong information
The root cause is that the incorrect versions use the cpuid
instruction incorrectly. __cpuid() uses old style cpuid and takes input from
eax only instead of eax, ecx . GetSystemInfo() uses the registers in short
mode when doing cpuid, so I think it fails for wow. It is amazing how
Windows works at all!
On 11/1/06, Xiao-Feng Li <xi...@gmail.com> wrote:
>
> On 11/1/06, Alexey Varlamov <al...@gmail.com> wrote:
> > Just a wild guess: this may be caused by x86 emulation on em64t
> > (x86_64). SDK docs advise to use GetNativeSystemInfo() in such case,
> > instead of currently used GetSystemInfo(). (See
> > vm\port\src\misc\win\sysinfo.c).
> >
>
> huh, I guess you are right, since my machine is X86-64bit. :-) I will
> try the API you pointed.
>
> Thanks,
> xiaofeng
>
>
Re: [DRLVM][PORT] correct API to retrieve processor number
Posted by Xiao-Feng Li <xi...@gmail.com>.
On 11/1/06, Alexey Varlamov <al...@gmail.com> wrote:
> Just a wild guess: this may be caused by x86 emulation on em64t
> (x86_64). SDK docs advise to use GetNativeSystemInfo() in such case,
> instead of currently used GetSystemInfo(). (See
> vm\port\src\misc\win\sysinfo.c).
>
huh, I guess you are right, since my machine is X86-64bit. :-) I will
try the API you pointed.
Thanks,
xiaofeng
> 2006/11/1, Xiao-Feng Li <xi...@gmail.com>:
> > Yes, both SUN JRE and DRLVM returns 1 for me. Java API has the same
> > problem. :-) Probably it should introduce an
> > availableCoresPerProcessor() or something more comprehensive.
> >
> > Thanks,
> > xiaofeng
> >
> > On 11/1/06, Mikhail Fursov <mi...@gmail.com> wrote:
> > > On 11/1/06, Xiao-Feng Li <xi...@gmail.com> wrote:
> > > >
> > > > Are you using Linux? Don't know why it doesn't work for my Pentium D.
> > > > Actually my Windows seems not show two processors at first, while the
> > > > API may depend on OS. My Linux has no problem with this.
> > > >
> > > > On the other hand, even your case is undesirable for Hyperthreading
> > > > since we probably want more detailed info about processor(s) since
> > > > hyperthreading sometimes wants to be treated differently than real SMP
> > > > (or dual-core). I believe there is such kind of API available
> > > > somewhere, at least NUMA support of Linux from SGI has it.
> > > >
> > >
> > > I use WindowsXP and here is more detailed info about CPU:
> > > Number of processors 1
> > > Number of cores 1 per processor
> > > Number of threads 2 (max 2) per processor
> > > Name Intel Pentium 4 660
> > > Code Name Prescott
> > > Specification Intel(R) Pentium(R) 4 CPU 3.60GHz
> > > Package Socket 775 LGA
> > >
> > > And I see 2 CPUs in Windows Task Manager.
> > >
> > > Did you tried Runtime.getRuntime().availableProcessors() ?
> > >
> > >
> > >
> > > --
> > > Mikhail Fursov
> > >
> > >
> >
>
Re: [DRLVM][PORT] correct API to retrieve processor number
Posted by Alexey Varlamov <al...@gmail.com>.
Just a wild guess: this may be caused by x86 emulation on em64t
(x86_64). SDK docs advise to use GetNativeSystemInfo() in such case,
instead of currently used GetSystemInfo(). (See
vm\port\src\misc\win\sysinfo.c).
2006/11/1, Xiao-Feng Li <xi...@gmail.com>:
> Yes, both SUN JRE and DRLVM returns 1 for me. Java API has the same
> problem. :-) Probably it should introduce an
> availableCoresPerProcessor() or something more comprehensive.
>
> Thanks,
> xiaofeng
>
> On 11/1/06, Mikhail Fursov <mi...@gmail.com> wrote:
> > On 11/1/06, Xiao-Feng Li <xi...@gmail.com> wrote:
> > >
> > > Are you using Linux? Don't know why it doesn't work for my Pentium D.
> > > Actually my Windows seems not show two processors at first, while the
> > > API may depend on OS. My Linux has no problem with this.
> > >
> > > On the other hand, even your case is undesirable for Hyperthreading
> > > since we probably want more detailed info about processor(s) since
> > > hyperthreading sometimes wants to be treated differently than real SMP
> > > (or dual-core). I believe there is such kind of API available
> > > somewhere, at least NUMA support of Linux from SGI has it.
> > >
> >
> > I use WindowsXP and here is more detailed info about CPU:
> > Number of processors 1
> > Number of cores 1 per processor
> > Number of threads 2 (max 2) per processor
> > Name Intel Pentium 4 660
> > Code Name Prescott
> > Specification Intel(R) Pentium(R) 4 CPU 3.60GHz
> > Package Socket 775 LGA
> >
> > And I see 2 CPUs in Windows Task Manager.
> >
> > Did you tried Runtime.getRuntime().availableProcessors() ?
> >
> >
> >
> > --
> > Mikhail Fursov
> >
> >
>
Re: [DRLVM][PORT] correct API to retrieve processor number
Posted by Xiao-Feng Li <xi...@gmail.com>.
Yes, both SUN JRE and DRLVM returns 1 for me. Java API has the same
problem. :-) Probably it should introduce an
availableCoresPerProcessor() or something more comprehensive.
Thanks,
xiaofeng
On 11/1/06, Mikhail Fursov <mi...@gmail.com> wrote:
> On 11/1/06, Xiao-Feng Li <xi...@gmail.com> wrote:
> >
> > Are you using Linux? Don't know why it doesn't work for my Pentium D.
> > Actually my Windows seems not show two processors at first, while the
> > API may depend on OS. My Linux has no problem with this.
> >
> > On the other hand, even your case is undesirable for Hyperthreading
> > since we probably want more detailed info about processor(s) since
> > hyperthreading sometimes wants to be treated differently than real SMP
> > (or dual-core). I believe there is such kind of API available
> > somewhere, at least NUMA support of Linux from SGI has it.
> >
>
> I use WindowsXP and here is more detailed info about CPU:
> Number of processors 1
> Number of cores 1 per processor
> Number of threads 2 (max 2) per processor
> Name Intel Pentium 4 660
> Code Name Prescott
> Specification Intel(R) Pentium(R) 4 CPU 3.60GHz
> Package Socket 775 LGA
>
> And I see 2 CPUs in Windows Task Manager.
>
> Did you tried Runtime.getRuntime().availableProcessors() ?
>
>
>
> --
> Mikhail Fursov
>
>
Re: [DRLVM][PORT] correct API to retrieve processor number
Posted by Mikhail Fursov <mi...@gmail.com>.
On 11/1/06, Xiao-Feng Li <xi...@gmail.com> wrote:
>
> Are you using Linux? Don't know why it doesn't work for my Pentium D.
> Actually my Windows seems not show two processors at first, while the
> API may depend on OS. My Linux has no problem with this.
>
> On the other hand, even your case is undesirable for Hyperthreading
> since we probably want more detailed info about processor(s) since
> hyperthreading sometimes wants to be treated differently than real SMP
> (or dual-core). I believe there is such kind of API available
> somewhere, at least NUMA support of Linux from SGI has it.
>
I use WindowsXP and here is more detailed info about CPU:
Number of processors 1
Number of cores 1 per processor
Number of threads 2 (max 2) per processor
Name Intel Pentium 4 660
Code Name Prescott
Specification Intel(R) Pentium(R) 4 CPU 3.60GHz
Package Socket 775 LGA
And I see 2 CPUs in Windows Task Manager.
Did you tried Runtime.getRuntime().availableProcessors() ?
--
Mikhail Fursov
Re: [DRLVM][PORT] correct API to retrieve processor number
Posted by Xiao-Feng Li <xi...@gmail.com>.
Are you using Linux? Don't know why it doesn't work for my Pentium D.
Actually my Windows seems not show two processors at first, while the
API may depend on OS. My Linux has no problem with this.
On the other hand, even your case is undesirable for Hyperthreading
since we probably want more detailed info about processor(s) since
hyperthreading sometimes wants to be treated differently than real SMP
(or dual-core). I believe there is such kind of API available
somewhere, at least NUMA support of Linux from SGI has it.
Anyway before a more comprehensive API available, I really hope PORT
can have my Windows + PentiumD give me two processors. Maybe this
question should go to APR mailing list?
Thanks,
xiaofeng
On 11/1/06, Mikhail Fursov <mi...@gmail.com> wrote:
> I've tried Runtime.getRuntime().availableProcessors() right now and it works
> OK to me and reports "2". I have Prescott, 1CPU, 2 hyperthreads, WindowsXP.
>
> JNIEXPORT jint JNICALL
> Java_java_lang_VMExecutionEngine_getAvailableProcessors
> (JNIEnv *, jclass)
> {
> return port_CPUs_number();
> }
>
> So if it's broken for you it's broken in Java too.
>
> On 11/1/06, Xiao-Feng Li <xi...@gmail.com> wrote:
> >
> > Hi, I am using port_CPUs_number() for GCv5 to get the processor
> > number, but my desktop returns one processor with it on Windows
> > although my processor is dual-core. (port_CPUs_number is defined in
> > port_sysinfo.h).
> >
> > I think we need more general form of processor number retrieval API
> > that can return processor information including that of core and
> > hyperthreading.
> >
> > How do you think?
> >
> > Thanks,
> > xiaofeng
> >
>
>
>
> --
> Mikhail Fursov
>
>
Re: [DRLVM][PORT] correct API to retrieve processor number
Posted by Mikhail Fursov <mi...@gmail.com>.
I've tried Runtime.getRuntime().availableProcessors() right now and it works
OK to me and reports "2". I have Prescott, 1CPU, 2 hyperthreads, WindowsXP.
JNIEXPORT jint JNICALL
Java_java_lang_VMExecutionEngine_getAvailableProcessors
(JNIEnv *, jclass)
{
return port_CPUs_number();
}
So if it's broken for you it's broken in Java too.
On 11/1/06, Xiao-Feng Li <xi...@gmail.com> wrote:
>
> Hi, I am using port_CPUs_number() for GCv5 to get the processor
> number, but my desktop returns one processor with it on Windows
> although my processor is dual-core. (port_CPUs_number is defined in
> port_sysinfo.h).
>
> I think we need more general form of processor number retrieval API
> that can return processor information including that of core and
> hyperthreading.
>
> How do you think?
>
> Thanks,
> xiaofeng
>
--
Mikhail Fursov