
MPI
文章平均质量分 69
amaowolf
这个作者很懒,什么都没留下…
展开
-
常用MPI 的进程绑定方法
1. 介绍我们常通过CPU 进程绑定(binding or affinity)的方法来提高MPI 程序的性能。通过CPU 进程绑定,可以避免进程在CPU 核之间切换带来的开销,可以减轻cache 争抢现象。特别是当进程数为CPU 总核数一半左右时,有时会发现测试结果不稳定,时好时坏,很可能是因为进程切换造成的,这时不妨尝试进行进程CPU 绑定。MPI 程序的进程CPU 绑定可以转载 2012-11-07 09:58:28 · 16787 阅读 · 2 评论 -
用MPI实现Hadoop Map/Reduce的TeraSort
http://emonkey.blog.sohu.com/166546157.html参与排序的MPI进程首先对输入文件进行划分,以确定每个进程计算的文件偏移量和长度,然后通过MPI-IO读取本进程需要排序的部分每个进程对要处理的数据用qsort进行本地排序,然后收集本进程的最小键值和最大键值,min_key, max_key,通过MPI_Allreduce()调用,每个进程获得全局的转载 2012-10-23 11:22:50 · 2422 阅读 · 0 评论 -
关于MPI-IO
http://emonkey.blog.sohu.com/166467283.html1.基本概念进程私有文件指针:每个参与IO的进程都有一个私有的文件指针,进程间对该指针完全独立进程共享文件指针:所有参与IO的进程共享一个文件指针,任何进程对该指针的操作,都会同步更新其他进程中该指针文件视图:每个参与IO的进程都可以定义自有的文件视图,以表示本进程需读写文件哪些部分,以及这些转载 2012-10-23 11:14:58 · 3075 阅读 · 0 评论 -
安装MPI configure 配置
I was always confused with the parameters used in ./configure command. I did not know how to find the exact meaning of these parameters. Yesterday, I finally found the secrete: --help. After typin转载 2012-10-23 11:07:53 · 5487 阅读 · 0 评论 -
MPI-IO:Open/close/delete
MPI_FILE_OPEN(comm, filename, amode, info, fh)IN comm 组内通信域IN filename 将打开的文件名IN amode 打开方式IN info 传递给运行时的信息OUT fh 返回的文件句柄int MPI_File_open(MPI_Comm comm, char * filename转载 2012-10-23 11:05:05 · 2594 阅读 · 0 评论 -
(1) MPI-IO: 基本函数 open, close, write_shared
/** * int MPI_File_open(MPI_Comm comm, char *filename, int amode, MPI_Info info, MPI_File *fh); * int MPI_File_close(MPI_File *fh); * * int MPI_File_write_shared(MPI_File fh, void *buf, int count,原创 2012-10-23 14:03:09 · 2978 阅读 · 0 评论 -
MPI提供分布式数据服务
借助MPI功能可以提供一种分布式数据服务。多个服务节点通过MPI_COMM形式组成一个逻辑服务器,维护一个“逻辑”上的数据文件,数据文件可能会分布存储在每个服务节点本地盘上,客户端可以从服务节点群获取数据!好处:每个服务节点可以利用本地磁盘和内存,提高响应时间适合:数据只读操作,e.g.,按关键字浏览相关函数:MPI_Comm_s转载 2012-10-23 11:24:20 · 905 阅读 · 0 评论 -
(14) MPI-IO_shared_order: seek/get_position/write/read/iread/iwrite read_ordered/write_ordered
#include#include#include#include"mpi.h"#include #include /** * fh is shared by all processes. * In case of reading, after one process finishes, and the fh moves to the new position, * t原创 2012-10-28 21:50:37 · 719 阅读 · 0 评论 -
(2) MPI-IO: MPI_File_read_at MPI_File_write_at
#include#include#include"mpi.h"/** * int MPI_File_read_at(MPI_File fh, MPI_Offset offset, void * buf, int count, MPI_Datatype datatype, MPI_Statye * status); * int MPI_File_write_at(MPI_File fh,原创 2012-10-23 14:25:37 · 3247 阅读 · 0 评论 -
(3) MPI-IO: MPI_File_read_at_all MPI_File_read_at_all
#include#include#include#include"mpi.h"/** * all process read/write at the same offset * int MPI_File_read_at_all(MPI_File fh, MPI_Offset offset, void *buf, int count, MPI_Datatype datatype, MP原创 2012-10-23 14:39:35 · 1491 阅读 · 0 评论 -
(4) MPI-IO: MPI_File_iread_at MPI_File_iwrite_at MPI_Wait
#include#include#include#include"mpi.h"/** * all process read/write at the same offset * int MPI_File_iread_at(MPI_File fh, MPI_Offset offset, void * buf, int count, MPI_Datatype datatype, MPI_原创 2012-10-23 15:27:57 · 976 阅读 · 0 评论 -
(12)MPI-IO: fileview read_all_begin/end write_all_begin/end
#include#include#include#include"mpi.h"/** * non-blocking read/write in the context of fileview in two stage * because *_end equals to MPI_Wait(&request, &status), MPI_Wait is not necessary *原创 2012-10-28 21:07:39 · 677 阅读 · 0 评论 -
(11) MPI-IO: fileview MPI_File_iread MPI_File_iwrite
#include#include#include#include"mpi.h"/** * non-blocking read/write in the context of fileview * int MPI_File_iread(MPI_File fh, void * buf, int count,Datatype datatype, MPI_Request * request)原创 2012-10-27 20:11:03 · 744 阅读 · 0 评论 -
(10) MPI-IO -- fileview: MPI_File_read_all MPI_File_write_all
#include#include#include#include"mpi.h"/** * all process exe blocking read/write in the context of fileview * int MPI_File_read_all(MPI_file fh, void * buf, int count, MPI_Datatype datatype, MPI原创 2012-10-24 07:32:25 · 1489 阅读 · 0 评论 -
(9) MPI-IO: fileview MPI_File_read MPI_File_write
#include#include#include#include"mpi.h"/** * blocking read/write in the context of a specific process's fileview * * int MPI_File_read(MPI_file fh, void * buf, int count, MPI_Datatype datatype,原创 2012-10-24 07:30:40 · 1575 阅读 · 0 评论 -
(8)MPI-IO: fileview MPI_File_seek MPI_File_get_position MPI_File_get_byte_offset
#include#include#include#include"mpi.h"/** * int MPI_File_seek(MPI_File fh, MPI_Offset offset, int whence) * here whence can be: * MPI_SEEK_SET, point to offset * MPI_SEEK_CUR, point to c原创 2012-10-24 07:19:48 · 839 阅读 · 0 评论 -
(7) MPI-IO: MPI_File_seek
fileview + seek 能组合出非常强大的功能,首先由fileview来抽象进程自己的文件视图,然后再在新的fileview逻辑视图中seek想要达到的位置。而不关心物理位置。#include#include#include#include"mpi.h"/** * int MPI_File_seek(MPI_File fh, MPI_Offset offset,原创 2012-10-23 17:43:55 · 2205 阅读 · 0 评论 -
(6) MPI-IO: fileview MPI_File_set_view MPI_File_get_view
#include#include#include#include"mpi.h"/** * int MPI_File_set_view(MPI_File fh, MPI_Offset disp, MPI_Datatype etype, MPI_Datatype filetype, char * datarep, MPI_Info info); * etype = basic uni原创 2012-10-23 17:26:09 · 1916 阅读 · 0 评论 -
(5) MPI-IO: MPI_File_read_at_all_begin MPI_File_read_at_all_end
#include#include#include#include"mpi.h"/** * two phase non-blocking read and write * * int MPI_File_read_at_all_begin(MPI_File fh, MPI_Offset offset, void * buf, int count, MPI_Datatype dataty原创 2012-10-23 15:45:48 · 1142 阅读 · 0 评论 -
(13)MPI-IO: seek/get_position/write/read all processes share one fh
#include#include#include#include"mpi.h"#include #include /** * fh is shared by all processes. * In case of reading, after one process finishes, and the fh moves to the new position, * t原创 2012-10-28 21:47:52 · 630 阅读 · 0 评论 -
同步操作 MPI_Barrier
todo原创 2011-12-29 12:11:35 · 5323 阅读 · 1 评论 -
N:N
todo原创 2011-12-29 12:09:50 · 811 阅读 · 1 评论 -
Prog1: helloworld.c 最简单的MPI程序
1. 程序源代码#include#include"mpi.h"int main(int argc, char *argv[]){ int totalTaskNum, rankID; int rt = MPI_Init(&argc, &argv); if(rt != MPI_SUCCESS){ pri原创 2011-12-21 10:54:53 · 6230 阅读 · 0 评论 -
Prog3: helloword3.c 显示每个进程执行在哪个机器节点上
1. 根据Prog1改写 #include #include"mpi.h" int main(int argc, char *argv[]){ int totalTaskNum, rankID; int rt = MPI_Init(&argc, &argv);原创 2011-12-21 16:01:55 · 805 阅读 · 0 评论 -
Prog2: hellowrold2.c 由进程0打印执行时间和命令行参数
1. 源码#include#include"mpi.h"int main(int argc, char *argv[]){ int totalTaskNum, rankID; int rt = MPI_Init(&argc, &argv); if(rt != MPI_SUCCESS){原创 2011-12-21 11:30:12 · 858 阅读 · 0 评论 -
P2P non-blocking
1. 非阻塞(non-blocking)通信(1)非阻塞的sender(a)几乎可以立刻返回去干别的事情。不管数据是否从application buffer到了system buffer, 或者数据到了receiver端的application buffer或者system buffer.(b)发送操作由MPI lib选择合适的时间去完成。(c)sender最好不要去立刻修原创 2011-12-29 11:37:58 · 851 阅读 · 0 评论 -
Collective Communication
All or None:Collective communication must involve all processes in the scope of a communicator. All processes are by default, members in the communicator MPI_COMM_WORLD.It is the programmer'原创 2011-12-29 11:53:26 · 1253 阅读 · 0 评论 -
安装openmpi
1.将openmpi-1.5.tar.gz 文件拷贝到一个临时的目录里面(如tem)2.解压文件:$tar -zxvf openmpi-1.5.tar.gz3.进入解压后的目录:$cd openmpi-1.54.$ ./configure --prefix=/home/bjwang/software/openmpi-1.5 CC=icc CXX=icpc F77=ifort FC转载 2012-09-03 15:51:15 · 31053 阅读 · 1 评论 -
Redhat5.4安装mpich2-1.4.1p1全过程
1.配置环境(1) 在两台机器上分别配置/etc/hosts (root用户)192.168.100.37 amao991 (主节点)192.168.100.50 amao992 (从节点)(2) 在两台机器上分别创建amao用户(3) 配置NFS从amao992上mount主节点amao991上/home目录,设为原创 2011-12-21 10:23:48 · 2611 阅读 · 0 评论 -
n->1 MPI_Reduce MPI_Gather
1. MPI_Reduce函数int MPI_Reduce(void *sendBuf, void *receiveBuf, int count, MPI_Datatype dataType, MPI_Op operator, int root, MPI_Comm comm)每个进程从sendBuf向root进程的rec原创 2011-12-29 12:09:22 · 5157 阅读 · 1 评论 -
1->n MPI_Scatter MPI_Bcast
1. 函数 MPI_ScatterMPI_Scatter (&sendbuf,sendcnt,sendtype,&recvbuf,recvcnt,recvtype,root,comm)Distributes distinct messages from a single source task to each task in the group.2. 举例#in原创 2011-12-29 12:06:47 · 5809 阅读 · 1 评论 -
Contiguous Derived Data
1. 连续的构造数据类型(1)把同一内置数据类型/连续的多个元素组合成一个构造数据类型2. 举例#include#include"mpi.h"#define SIZE 4int main(int argc, char *argv[]){ int totalNumTasks, rankID; float sendBuf[SIZE][S原创 2011-12-29 12:23:22 · 625 阅读 · 0 评论 -
P2P Blocking
1. Blocking send / Blocking receive(1)阻塞发送(blocking send)是指sender把消息放入application buffer后,不能返回继续其他操作。直到这个application buffer可修改后。这个时刻有两种可能:(a)异步(asynchronous)阻塞发送: 消息从sender的application bu原创 2011-12-29 10:27:16 · 951 阅读 · 0 评论 -
Derived Data Type
todo原创 2011-12-29 12:13:42 · 898 阅读 · 1 评论 -
Point to Point Communication
General ConceptsTypes of Point-to-Point Operations:MPI point-to-point operations typically involve message passing between two, and only two, different MPI tasks. One task is performing a send o原创 2011-12-29 09:54:23 · 1149 阅读 · 0 评论