You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@mesos.apache.org by be...@apache.org on 2011/06/05 07:02:57 UTC
svn commit: r1131732 - in
/incubator/mesos/trunk/frameworks/torque/nexus-hpl: ./ 24node/
24node/HPL.dat 48node/ 48node/HPL.dat 8node-small/ 8node-small/HPL.dat
8node/ 8node/HPL.dat Make.Linux_PII_CBLAS README
Author: benh
Date: Sun Jun 5 05:02:56 2011
New Revision: 1131732
URL: http://svn.apache.org/viewvc?rev=1131732&view=rev
Log:
Adding nexus-hpl directory with instructions, and parts of the
experiment-framework for running hpl as a workload for the Torque Mesos Framework.
Added:
incubator/mesos/trunk/frameworks/torque/nexus-hpl/
incubator/mesos/trunk/frameworks/torque/nexus-hpl/24node/
incubator/mesos/trunk/frameworks/torque/nexus-hpl/24node/HPL.dat
incubator/mesos/trunk/frameworks/torque/nexus-hpl/48node/
incubator/mesos/trunk/frameworks/torque/nexus-hpl/48node/HPL.dat
incubator/mesos/trunk/frameworks/torque/nexus-hpl/8node/
incubator/mesos/trunk/frameworks/torque/nexus-hpl/8node-small/
incubator/mesos/trunk/frameworks/torque/nexus-hpl/8node-small/HPL.dat
incubator/mesos/trunk/frameworks/torque/nexus-hpl/8node/HPL.dat
incubator/mesos/trunk/frameworks/torque/nexus-hpl/Make.Linux_PII_CBLAS
incubator/mesos/trunk/frameworks/torque/nexus-hpl/README
Added: incubator/mesos/trunk/frameworks/torque/nexus-hpl/24node/HPL.dat
URL: http://svn.apache.org/viewvc/incubator/mesos/trunk/frameworks/torque/nexus-hpl/24node/HPL.dat?rev=1131732&view=auto
==============================================================================
--- incubator/mesos/trunk/frameworks/torque/nexus-hpl/24node/HPL.dat (added)
+++ incubator/mesos/trunk/frameworks/torque/nexus-hpl/24node/HPL.dat Sun Jun 5 05:02:56 2011
@@ -0,0 +1,31 @@
+HPLinpack benchmark input file
+Innovative Computing Laboratory, University of Tennessee
+HPL.out output file name (if any)
+6 device out (6=stdout,7=stderr,file)
+2 # of problems sizes (N)
+500 500 Ns
+2 # of NBs
+50 100 NBs
+0 PMAP process mapping (0=Row-,1=Column-major)
+3 # of process grids (P x Q)
+24 12 6 Ps
+1 2 4 Qs
+16.0 threshold
+1 # of panel fact
+0 PFACTs (0=left, 1=Crout, 2=Right)
+1 # of recursive stopping criterium
+4 NBMINs (>= 1)
+1 # of panels in recursion
+2 NDIVs
+1 # of recursive panel fact.
+0 RFACTs (0=left, 1=Crout, 2=Right)
+1 # of broadcast
+0 BCASTs (0=1rg,1=1rM,2=2rg,3=2rM,4=Lng,5=LnM)
+1 # of lookahead depth
+0 DEPTHs (>=0)
+2 SWAP (0=bin-exch,1=long,2=mix)
+64 swapping threshold
+0 L1 in (0=transposed,1=no-transposed) form
+0 U in (0=transposed,1=no-transposed) form
+1 Equilibration (0=no,1=yes)
+8 memory alignment in double (> 0)
Added: incubator/mesos/trunk/frameworks/torque/nexus-hpl/48node/HPL.dat
URL: http://svn.apache.org/viewvc/incubator/mesos/trunk/frameworks/torque/nexus-hpl/48node/HPL.dat?rev=1131732&view=auto
==============================================================================
--- incubator/mesos/trunk/frameworks/torque/nexus-hpl/48node/HPL.dat (added)
+++ incubator/mesos/trunk/frameworks/torque/nexus-hpl/48node/HPL.dat Sun Jun 5 05:02:56 2011
@@ -0,0 +1,31 @@
+HPLinpack benchmark input file
+Innovative Computing Laboratory, University of Tennessee
+HPL.out output file name (if any)
+6 device out (6=stdout,7=stderr,file)
+1 # of problems sizes (N)
+10000 Ns
+2 # of NBs
+50 100 NBs
+0 PMAP process mapping (0=Row-,1=Column-major)
+1 # of process grids (P x Q)
+6 Ps
+8 Qs
+16.0 threshold
+1 # of panel fact
+0 PFACTs (0=left, 1=Crout, 2=Right)
+1 # of recursive stopping criterium
+4 NBMINs (>= 1)
+1 # of panels in recursion
+2 NDIVs
+1 # of recursive panel fact.
+0 RFACTs (0=left, 1=Crout, 2=Right)
+1 # of broadcast
+0 BCASTs (0=1rg,1=1rM,2=2rg,3=2rM,4=Lng,5=LnM)
+1 # of lookahead depth
+0 DEPTHs (>=0)
+2 SWAP (0=bin-exch,1=long,2=mix)
+64 swapping threshold
+0 L1 in (0=transposed,1=no-transposed) form
+0 U in (0=transposed,1=no-transposed) form
+1 Equilibration (0=no,1=yes)
+8 memory alignment in double (> 0)
Added: incubator/mesos/trunk/frameworks/torque/nexus-hpl/8node-small/HPL.dat
URL: http://svn.apache.org/viewvc/incubator/mesos/trunk/frameworks/torque/nexus-hpl/8node-small/HPL.dat?rev=1131732&view=auto
==============================================================================
--- incubator/mesos/trunk/frameworks/torque/nexus-hpl/8node-small/HPL.dat (added)
+++ incubator/mesos/trunk/frameworks/torque/nexus-hpl/8node-small/HPL.dat Sun Jun 5 05:02:56 2011
@@ -0,0 +1,31 @@
+HPLinpack benchmark input file
+Innovative Computing Laboratory, University of Tennessee
+HPL.out output file name (if any)
+6 device out (6=stdout,7=stderr,file)
+1 # of problems sizes (N)
+200 Ns
+2 # of NBs
+10 20 NBs
+0 PMAP process mapping (0=Row-,1=Column-major)
+2 # of process grids (P x Q)
+1 2 Ps
+8 4 Qs
+16.0 threshold
+1 # of panel fact
+0 PFACTs (0=left, 1=Crout, 2=Right)
+1 # of recursive stopping criterium
+4 NBMINs (>= 1)
+1 # of panels in recursion
+2 NDIVs
+1 # of recursive panel fact.
+0 RFACTs (0=left, 1=Crout, 2=Right)
+1 # of broadcast
+0 BCASTs (0=1rg,1=1rM,2=2rg,3=2rM,4=Lng,5=LnM)
+1 # of lookahead depth
+0 DEPTHs (>=0)
+2 SWAP (0=bin-exch,1=long,2=mix)
+64 swapping threshold
+0 L1 in (0=transposed,1=no-transposed) form
+0 U in (0=transposed,1=no-transposed) form
+1 Equilibration (0=no,1=yes)
+8 memory alignment in double (> 0)
Added: incubator/mesos/trunk/frameworks/torque/nexus-hpl/8node/HPL.dat
URL: http://svn.apache.org/viewvc/incubator/mesos/trunk/frameworks/torque/nexus-hpl/8node/HPL.dat?rev=1131732&view=auto
==============================================================================
--- incubator/mesos/trunk/frameworks/torque/nexus-hpl/8node/HPL.dat (added)
+++ incubator/mesos/trunk/frameworks/torque/nexus-hpl/8node/HPL.dat Sun Jun 5 05:02:56 2011
@@ -0,0 +1,31 @@
+HPLinpack benchmark input file
+Innovative Computing Laboratory, University of Tennessee
+HPL.out output file name (if any)
+6 device out (6=stdout,7=stderr,file)
+2 # of problems sizes (N)
+4000 4000 Ns
+2 # of NBs
+10 20 NBs
+0 PMAP process mapping (0=Row-,1=Column-major)
+2 # of process grids (P x Q)
+1 2 Ps
+8 4 Qs
+16.0 threshold
+1 # of panel fact
+0 PFACTs (0=left, 1=Crout, 2=Right)
+1 # of recursive stopping criterium
+4 NBMINs (>= 1)
+1 # of panels in recursion
+2 NDIVs
+1 # of recursive panel fact.
+0 RFACTs (0=left, 1=Crout, 2=Right)
+1 # of broadcast
+0 BCASTs (0=1rg,1=1rM,2=2rg,3=2rM,4=Lng,5=LnM)
+1 # of lookahead depth
+0 DEPTHs (>=0)
+2 SWAP (0=bin-exch,1=long,2=mix)
+64 swapping threshold
+0 L1 in (0=transposed,1=no-transposed) form
+0 U in (0=transposed,1=no-transposed) form
+1 Equilibration (0=no,1=yes)
+8 memory alignment in double (> 0)
Added: incubator/mesos/trunk/frameworks/torque/nexus-hpl/Make.Linux_PII_CBLAS
URL: http://svn.apache.org/viewvc/incubator/mesos/trunk/frameworks/torque/nexus-hpl/Make.Linux_PII_CBLAS?rev=1131732&view=auto
==============================================================================
--- incubator/mesos/trunk/frameworks/torque/nexus-hpl/Make.Linux_PII_CBLAS (added)
+++ incubator/mesos/trunk/frameworks/torque/nexus-hpl/Make.Linux_PII_CBLAS Sun Jun 5 05:02:56 2011
@@ -0,0 +1,183 @@
+#
+# -- High Performance Computing Linpack Benchmark (HPL)
+# HPL - 1.0a - January 20, 2004
+# Antoine P. Petitet
+# University of Tennessee, Knoxville
+# Innovative Computing Laboratories
+# (C) Copyright 2000-2004 All Rights Reserved
+#
+# -- Copyright notice and Licensing terms:
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+#
+# 1. Redistributions of source code must retain the above copyright
+# notice, this list of conditions and the following disclaimer.
+#
+# 2. Redistributions in binary form must reproduce the above copyright
+# notice, this list of conditions, and the following disclaimer in the
+# documentation and/or other materials provided with the distribution.
+#
+# 3. All advertising materials mentioning features or use of this
+# software must display the following acknowledgement:
+# This product includes software developed at the University of
+# Tennessee, Knoxville, Innovative Computing Laboratories.
+#
+# 4. The name of the University, the name of the Laboratory, or the
+# names of its contributors may not be used to endorse or promote
+# products derived from this software without specific written
+# permission.
+#
+# -- Disclaimer:
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+# ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE UNIVERSITY
+# OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+# ######################################################################
+#
+# ----------------------------------------------------------------------
+# - shell --------------------------------------------------------------
+# ----------------------------------------------------------------------
+#
+SHELL = /bin/sh
+#
+CD = cd
+CP = cp
+LN_S = ln -s
+MKDIR = mkdir
+RM = /bin/rm -f
+TOUCH = touch
+#
+# ----------------------------------------------------------------------
+# - Platform identifier ------------------------------------------------
+# ----------------------------------------------------------------------
+#
+ARCH = Linux_PII_CBLAS
+#
+# ----------------------------------------------------------------------
+# - HPL Directory Structure / HPL library ------------------------------
+# ----------------------------------------------------------------------
+#
+TOPdir = $(HOME)/hpl
+INCdir = $(TOPdir)/include
+BINdir = $(TOPdir)/bin/$(ARCH)
+LIBdir = $(TOPdir)/lib/$(ARCH)
+#
+HPLlib = $(LIBdir)/libhpl.a
+#
+# ----------------------------------------------------------------------
+# - Message Passing library (MPI) --------------------------------------
+# ----------------------------------------------------------------------
+# MPinc tells the C compiler where to find the Message Passing library
+# header files, MPlib is defined to be the name of the library to be
+# used. The variable MPdir is only used for defining MPinc and MPlib.
+#
+MPdir = /usr
+MPinc = -I$(MPdir)/include/mpich2
+MPlib = $(MPdir)/lib/libmpich.a
+#
+# ----------------------------------------------------------------------
+# - Linear Algebra library (BLAS or VSIPL) -----------------------------
+# ----------------------------------------------------------------------
+# LAinc tells the C compiler where to find the Linear Algebra library
+# header files, LAlib is defined to be the name of the library to be
+# used. The variable LAdir is only used for defining LAinc and LAlib.
+#
+LAdir = /root/CBLAS/lib/LINUX
+LAinc =
+LAlib = /usr/lib/libcblas.a /usr/lib/libatlas.a
+#/CBLAS/lib/LINUX/cblas_LINUX.a
+# ----------------------------------------------------------------------
+# - F77 / C interface --------------------------------------------------
+# ----------------------------------------------------------------------
+# You can skip this section if and only if you are not planning to use
+# a BLAS library featuring a Fortran 77 interface. Otherwise, it is
+# necessary to fill out the F2CDEFS variable with the appropriate
+# options. **One and only one** option should be chosen in **each** of
+# the 3 following categories:
+#
+# 1) name space (How C calls a Fortran 77 routine)
+#
+# -DAdd_ : all lower case and a suffixed underscore (Suns,
+# Intel, ...), [default]
+# -DNoChange : all lower case (IBM RS6000),
+# -DUpCase : all upper case (Cray),
+# -DAdd__ : the FORTRAN compiler in use is f2c.
+#
+# 2) C and Fortran 77 integer mapping
+#
+# -DF77_INTEGER=int : Fortran 77 INTEGER is a C int, [default]
+# -DF77_INTEGER=long : Fortran 77 INTEGER is a C long,
+# -DF77_INTEGER=short : Fortran 77 INTEGER is a C short.
+#
+# 3) Fortran 77 string handling
+#
+# -DStringSunStyle : The string address is passed at the string loca-
+# tion on the stack, and the string length is then
+# passed as an F77_INTEGER after all explicit
+# stack arguments, [default]
+# -DStringStructPtr : The address of a structure is passed by a
+# Fortran 77 string, and the structure is of the
+# form: struct {char *cp; F77_INTEGER len;},
+# -DStringStructVal : A structure is passed by value for each Fortran
+# 77 string, and the structure is of the form:
+# struct {char *cp; F77_INTEGER len;},
+# -DStringCrayStyle : Special option for Cray machines, which uses
+# Cray fcd (fortran character descriptor) for
+# interoperation.
+#
+F2CDEFS =
+#
+# ----------------------------------------------------------------------
+# - HPL includes / libraries / specifics -------------------------------
+# ----------------------------------------------------------------------
+#
+HPL_INCLUDES = -I$(INCdir) -I$(INCdir)/$(ARCH) $(LAinc) $(MPinc)
+HPL_LIBS = $(HPLlib) $(LAlib) $(MPlib)
+#
+# - Compile time options -----------------------------------------------
+#
+# -DHPL_COPY_L force the copy of the panel L before bcast;
+# -DHPL_CALL_CBLAS call the cblas interface;
+# -DHPL_CALL_VSIPL call the vsip library;
+# -DHPL_DETAILED_TIMING enable detailed timers;
+#
+# By default HPL will:
+# *) not copy L before broadcast,
+# *) call the BLAS Fortran 77 interface,
+# *) not display detailed timing information.
+#
+HPL_OPTS = -DHPL_CALL_CBLAS
+#
+# ----------------------------------------------------------------------
+#
+HPL_DEFS = $(F2CDEFS) $(HPL_OPTS) $(HPL_INCLUDES)
+#
+# ----------------------------------------------------------------------
+# - Compilers / linkers - Optimization flags ---------------------------
+# ----------------------------------------------------------------------
+#
+CC = /usr/bin/gcc
+CCNOOPT = $(HPL_DEFS)
+CCFLAGS = $(HPL_DEFS) -fomit-frame-pointer -O3 -funroll-loops -lpthread
+#
+# On some platforms, it is necessary to use the Fortran linker to find
+# the Fortran internals used in the BLAS library.
+#
+LINKER = /usr/bin/gfortran
+LINKFLAGS = $(CCFLAGS) -lpthread
+#
+ARCHIVER = ar
+ARFLAGS = r
+RANLIB = echo
+#
+# ----------------------------------------------------------------------
Added: incubator/mesos/trunk/frameworks/torque/nexus-hpl/README
URL: http://svn.apache.org/viewvc/incubator/mesos/trunk/frameworks/torque/nexus-hpl/README?rev=1131732&view=auto
==============================================================================
--- incubator/mesos/trunk/frameworks/torque/nexus-hpl/README (added)
+++ incubator/mesos/trunk/frameworks/torque/nexus-hpl/README Sun Jun 5 05:02:56 2011
@@ -0,0 +1,18 @@
+This installation of HPL was used for the OSDI submission of Mesos (named Nexus at the time). It was compiled for large EC2 instances running Ubuntu (Lucid).
+
+The various "X-node" directories contain just the configuration file HPL.dat which adjusts the size of the problem (e.g. the matrix) in order to provide a mixed workload for experiments. Jobs using these conf files can be launched with the corresponding qsub scripts in the /root/nexus/frameworks/torque/ directory. However, first the xhpl executable must be built and then copied into these directories. The xhpl executable can be built using the Make.Linux_PII_CBLAS, which was constructed with to work on this version of ubuntu.
+
+Building for ubuntu (lucid, EC2 ami-id ami-19a14870):
+1) download and untar http://nexus.berkeley.edu/hpl-nexus.tar.gz
+ NOTE: this package should contain the Makefile we created to build on
+ lucid, which also exists in the same dir as this README
+2) in hpl directory, to build hpl binary, run:
+ make arch=Linux_PII_CBLAS
+
+Then copy bin/xhpl to the "X-node" directories, and you should be able to launch xhpl jobs of various sizes using qsub commands in root/nexus/frameworks/torque/
+
+TODO:
+Dependencies for running hpl on slaves (that should be added to the Mesos cluster deploy scripts):
+- libatlas-base-dev
+- afortran
+