HPL-AMD Configuration

1. Basic

wget http://code.compeng.uni-frankfurt.de/attachments/download/55/hpl-gpu-1.1.0.tar.bz2//HPL

wget http://code.compeng.uni-frankfurt.de/attachments/download/54/caldgemm-1.1.0.tar.bz2 //CALDGEMM

bzip2 -d *.tar.bz2

tar -xvf *.tar

http://code.compeng.uni-frankfurt.de/projects/hpl/wiki

make NO_MEMPOLICY=1 -j      incur errors

gotoblas error undefined symbol prefetchsize ' in operation

gmake clean

gmake TARGET=NEHALEM NO_MEMPOLICY=1 -j

http://code.compeng.uni-frankfurt.de/projects/caldgemm/files
tar xzf *.tgz

-lpthread -ldl -L/usr/X11R6/lib -laticalrt -laticalcl -lgfortran ../GotoBLAS2/
/home/shir/download/AMD-APP-SDK-v2.7-RC-lnx64/lib/x86_64

grep HPL_CUSTOM_CALDGEMM_OPTIONS ./ -r
MV2_ENABLE_AFFINITY=0 MV2_RNDV_PROTOCOL=R3 MV2_USE_RDMA_ONE_SIDED=0

http://code.compeng.uni-frankfurt.de/projects/caldgemm/repository/entry/README
-------------------------------------------
-z (default: disabled)
Enable Multithreading. You definitely want to activate this

-c (default: disabled)
Use CPU for DGEMM. You can supply -g as well to use both CPU and GPU. Supplying neither of them will use GPU only.

-j <dbl> (default: -1)
Ratio of GPU performance to total CPU+GPU performance. Set to -1 for autodetection.


-s (default: disabled)
Dynamic CPU GPU scheduling. Do not use only the fixed ratio specified by -j but use a dynamic CPU/GPU workload scheduling. This includes work-stealing, etc. The value provided by -j is the basis for the scheduling.

-p (default: disabled)
Interleaving Memory Policy. Gotoblas usually activates memory interleaving. This leads to a problem with the CAL library. Interleaving should be activated after memory for the CAL library is allocated. Thus it is recommended to disable interleaving in GotoBLAS (apply the patch provided with caldgemm and set NO_MEMINTERLEAVE in GotoBLAS Make.rule) and use -p.

-A (default: disabled)
Do the DMA transfer to GPU asynchronously. If you are not debugging, always enable this.

-O (default: enabled)
Define backend to use. Available options are:
disabled: CAL
enabled: OpenCL

tried: ./dgemm_bench -c -z -p -m 40960 -n 40960
Error 'Unable to load aticaldd' while initializing CAL
No GPU available, falling back to CPU
Initializing Matrix C
Running Benchmark
Starting DGEMM Run m=40960 k=1024 n=40960 Alpha=-1.000000 Beta=1.000000 LDA=0x408 LDB=0xa008 LDC=0xa008 At=0 Bt=0 ColMajor=0 (A=0x7f5a7553f010, B=0x7f5a6152e010, C=0x7f57412ad010, (C-A=2305843007493692416, (C-B)/w=2251799812046527))
Program: caldgemm Sizes - A: 40960x1024 B: 1024x40960 C:40960x40960 (Host: head.cluster)
System Time 44.493 System Gflops 77.300


HPL.make
-DHPL_NO_MPI_THREAD_CHECK
-DHPL_NO_HACKED_LIB
-DHPL_MPI_FUNNELED_THREADING

root / testing / util / UTIL_cal.cpp
#ifndef HPL_CALDGEMM_BACKEND
------------------------------------------------------------------
cpu1*1  16384  1024 1 1           41.31 314.18   7.099e+01
Avg. matri size per node: 2.00 GiB


Q: http://superuser.com/questions/425084/error-while-compiling-cuda-accelerated-linpack-hpl-2-0-fermi

mpirun_rsh -np 2 -hostfile hosts MV2_CPU_BINDING_LEVEL=socket ./run_linpack
~/cmvapich2/bin/mpirun_rsh -np 2 -hostfile hosts MV2_ENABLE_AFFINITY=0 ./run_linpack
MV2_RNDV_PROTOCOL=R3 MV2_USE_RDMA_ONE_SIDED=0

continued ...