rhel5 MOSIX集群

最新推荐文章于 2021-07-05 12:51:59 发布

Fangrn

最新推荐文章于 2021-07-05 12:51:59 发布

阅读量96

点赞数

分类专栏： linux 文章标签： Linux J# UP Vmware 多线程

linux 专栏收录该内容

126 篇文章

订阅专栏

本文详细介绍了如何使用MOSIX在Red Hat Enterprise Linux 5系统上搭建一个由两台虚拟机组成的集群，并通过一个简单的多进程负载测试验证了集群内进程自动迁移的功能。

MOSIX集群(一)–安装

目的: 集群节点内进程能根据负载情况自动迁移

用vmware安装一台rhel5(192.168.100.5)

# 下载MOSIX和kernel代码,准备编译

# 解压到指定目录

[root@rhel5 ~]# tar xjvf MOSIX-2.24.2.2.tbz -C /usr/src/

[root@rhel5 ~]# tar xzvf linux-2.6.26.tar.gz -C /usr/src/

#进入源代码所在目录

[root@rhel5 ~]# cd /usr/src/

#由于other/patch-2.6.26的目标路径是linux-2.6.26.1,做个连接吧(可能是mosix没有为2.6.26单独写patch…,不过还是支持的)

[root@rhel5 src]# ln -s linux-2.6.26/ ./linux-2.6.26.1

#给kernel打上mosix补丁

[root@rhel5 src]# patch -p0 < /usr/src/mosix-2.24.2.2/other/patch-2.6.26

#进入源代码目录,开始编译

[root@rhel5 src]# cd linux-2.6.26

#生成配置文件

[root@rhel5 linux-2.6.26]# make menuconfig

#生成依赖关系

[root@rhel5 linux-2.6.26]# make dep

#编译内核

[root@rhel5 linux-2.6.26]# make bzImage

#编译内核模块

[root@rhel5 linux-2.6.26]# make modules

#安装内核模块

[root@rhel5 linux-2.6.26]# make modules_install

#安装内核

[root@rhel5 linux-2.6.26]# make install

#进入mosix目录

[root@rhel5 mosix-2.24.2.2]# cd ../mosix-2.24.2.2

#安装mosix,一路回车,只用安装,记得把你常用级别的mosix服务打开就可以了.配置以后再说

[root@rhel5 mosix-2.24.2.2]# ./mosix.install

关机以后,用rhel5(192.168.100.5)克隆出slave(192.168.100.6)

安装完成

MOSIX-2.24.2.2/linux-2.6.26集群(二)–配置

将rhel5和slave开启,开机的时候,在grub界面按回车,然后选择2.6.26内核启动

slave启动以后,把ip地址,机器名改好(应为是由rhel5克隆得到的嘛)

[reel5]

#配置mosix

[root@rhel5 ~]# mosconf

MOSIX CONFIGURATION

===================

If this is your cluster's file-server and you want to configure MOSIX

for a set of nodes with a common root, please type their common root

directory. Otherwise, if you want to configure the node that you are

running on, just press <ENTER> :-

What would you like to configure?

=================================

1. Which nodes are in this cluster (ESSENTIAL)

2. Authentication (ESSENTIAL)

3. Logical node numbering (recommended)

4. Queueing policies (recommended)

5. Freezing policies

6. Miscellaneous policies

7. Become part of a multi-cluster organizational Grid

Configure what :- 1

There are no nodes in your cluster yet:

=======================================

To add a new set of nodes to your cluster, type 'n'.

To turn on advanced options, type '+'.

For help, type 'h'.

To save and exit, type 'q'. (to abandon all changes and exit, type 'Q')

Option :- n <==添加节点

Adding new node(s) to the cluster:

First host-name or IP address :- 192.168.100.5 <==节点ip

Number of nodes :- 1 <==节点数

Nodes in your cluster:

======================

1. 192.168.100.5

To add a new set of nodes to your cluster, type 'n'.

To modify an entry, type its number.

To delete an entry, type 'd' followed by that entry-number (eg. d1).

To turn on advanced options, type '+'.

For help, type 'h'.

To save and exit, type 'q'. (to abandon all changes and exit, type 'Q')

Option :- n <==添加节点

Adding new node(s) to the cluster:

First host-name or IP address :- 192.168.100.6 <==节点ip

Number of nodes :- 1 <==节点数

Nodes in your cluster:

======================

1. 192.168.100.5

2. 192.168.100.6

To add a new set of nodes to your cluster, type 'n'.

To modify an entry, type its number.

To delete an entry, type 'd' followed by that entry-number (eg. d2).

To turn on advanced options, type '+'.

For help, type 'h'.

To save and exit, type 'q'. (to abandon all changes and exit, type 'Q')

Option :- q <==保存退出

Cluster configuration was saved.

OK to also update the logical node numbers [Y/n]? y

Suggesting to assign '192.168.100.5'

as the central queue manager for the cluster

(but be cautious if you mix 32-bit and 64-bit nodes in the same cluster)

OK to update it now [Y/n]?

What would you like to configure next?

======================================

1. Which nodes are in this cluster

2. Authentication (ESSENTIAL)

3. Logical node numbering

4. Queueing policies

5. Freezing policies

6. Miscellaneous policies

7. Become part of a multi-cluster organizational Grid

q. Exit

Configure what :- 2 <==设置密码

MOSIX Authentication:

=====================

To protect your MOSIX cluster from abuse, preventing unauthorized

persons from gaining control over your computers, you need to set

up a secret cluster-protection key. This key can include any

characters, but must be identical throughout your cluster.

Your secret cluster-protection key: xxxx <==输入密码

Your key is 5 characters long.

(in the future, please consider a longer one)

To allow your users to send batch-jobs to other nodes in the cluster,

you must set up a secret batch-client key. This key can include any

characters, but must match the 'batch-server' key on the node(s) that

can receive batch-jobs from this node.

Your secret batch-client key: xxxx <==输入密码

Your key is 5 characters long.

(in the future, please consider a longer one)

For this node to accept batch jobs,

you must set up a secret batch-server key. This key can include any

characters, but must match the 'batch-client' key on the sending nodes.

To make your batch-server key the same as your batch-client key, type '+'.

Your secret batch-server key: xxxx <==输入密码

Your key is 5 characters long.

(in the future, please consider a longer one)

#保持退出

[root@rhel5 ~]# service mosix restart

[root@slave ~]# mosconf

....

#操作同rhel5一样

#重启服务

[root@slave ~]# service mosix restart

#看看状态吧

[root@slave ~]# service mosix status

This MOSIX node is: 192.168.100.6 (no features)

Nodes in cluster:

=================

192.168.100.5: proximate

192.168.100.6: proximate

Status: Running Normally (32-bits)

Load: 0.01 (equivalent to about 0.0066 CPU processes)

Speed: 6650 units

CPUS: 1

Frozen: 0

Util: 100%

Avail: YES

Procs: Running 0 MOSIX processes

Accept: Yes, will welcome processes from here

Memory: Available 461MB/503MB

Swap: Available 0.9GB/0.9GB

Daemons:

Master Daemon: Up

MOSIX Daemon : Up

Queue Manager: Up

Remote Daemon: Up

Postal Daemon: Up

Guest processes from other clusters in the grid: 0/8

#我比较喜欢看看端口是不是起来了

#TCP/IP ports 249-253 and UDP/IP ports 249-250 must be available for MOSIX

[root@slave ~]# netstat -antu | grep -E "24|25"

tcp 0 0 0.0.0.0:2401 0.0.0.0:* LISTEN

tcp 0 0 0.0.0.0:249 0.0.0.0:* LISTEN

tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN

tcp 0 0 0.0.0.0:250 0.0.0.0:* LISTEN

tcp 0 0 0.0.0.0:251 0.0.0.0:* LISTEN

tcp 0 0 0.0.0.0:252 0.0.0.0:* LISTEN

udp 0 0 0.0.0.0:249 0.0.0.0:*

udp 0 0 0.0.0.0:250 0.0.0.0:*

#好了,装完了

MOSIX-2.24.2.2/linux-2.6.26集群(三)–应用测试

#先在rehl5和slave上各开启一个终端,运行mon命令,检查

[root@rhel5 ~]# mon

#2个节点上应该都是闲置的吧

#为了能出些效果,做点费cpu的脚本,还必须是多线程的,

#mosix能够迁移的最小单位是进程,而不是指令或者函数,

#所以单进程负载再高也没意义

[root@rhel5 ~]# cat a.sh << EOF

awk 'BEGIN {for(i=0;i<100000;i++)for(j=0;j<100000;j++);}' &

EOF

[root@rhel5 ~]# chmod +x a.sh

#在rhel5上运行a.sh,也就是产生6个进程了

[root@rhel5 ~]# mosrun -e ./a.sh

#开始观察2个节点上的mon画面,刚开始rhel负载很高,然后slave的负载也起来了,能够看到

#能够看到在rhel5上,awk的6个进程还在,但是只有3个在运行,还有3个的状态是T(stop),哈哈,应该是迁移了

[root@rhel5 ~]# ps -aux | grep awk

Warning: bad syntax, perhaps a bogus '-'? See /usr/share/doc/procps-3.2.7/FAQ

root 25648 0.6 0.0 0 0 pts/0 T 16:16 0:00 [awk]

root 25650 0.4 0.0 0 0 pts/0 T 16:16 0:00 [awk]

root 25652 32.0 0.7 4168 3812 pts/0 R 16:16 0:37 awk BEGIN {for(i=0;i<100000;i++)for(j=0;j<100000;j++);}

root 25654 32.0 0.7 4168 3816 pts/0 R 16:16 0:37 awk BEGIN {for(i=0;i<100000;i++)for(j=0;j<100000;j++);}

root 25656 32.0 0.7 4168 3816 pts/0 R 16:16 0:37 awk BEGIN {for(i=0;i<100000;i++)for(j=0;j<100000;j++);}

root 25658 1.4 0.0 0 0 pts/0 T 16:16 0:01 [awk]

root 25665 0.0 0.1 3860 624 pts/0 R+ 16:18 0:00 grep awk

#到slave上top看看吧,明显看到有3个叫remoted的进程占用了cpu,这个就是迁移过来的状态吧

top - 16:19:19 up 3:10, 3 users, load average: 2.78, 1.18, 0.44

Tasks: 99 total, 5 running, 94 sleeping, 0 stopped, 0 zombie

Cpu(s): 99.3%us, 0.3%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.3%si

Mem: 515376k total, 423576k used, 91800k free, 107980k buff

Swap: 1048568k total, 0k used, 1048568k free, 234028k cach

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND

16929 root 20 0 4168 3936 0 R 33.2 0.8 0:48.13 remoted

16925 root 20 0 4168 3932 0 R 32.9 0.8 0:50.57 remoted

16927 root 20 0 4168 3932 0 R 32.9 0.8 0:50.13 remoted

1 root 20 0 2036 664 572 S 0.0 0.1 0:01.36 init

2 root 15 -5 0 0 0 S 0.0 0.0 0:00.00 kthreadd

3 root RT -5 0 0 0 S 0.0 0.0 0:00.00 migratio

4 root 15 -5 0 0 0 S 0.0 0.0 0:02.00 ksoftirq

##############全文测试结束############