Mellanox切换IB为ETH

sw0807

已于 2024-06-22 16:30:29 修改

阅读量3.2k

点赞数 4

CC 4.0 BY-SA版权

分类专栏： Linux 文章标签： linux 运维服务器

于 2024-06-22 15:21:08 首次发布

本文链接：https://blog.youkuaiyun.com/sw0807/article/details/139882616

Linux 专栏收录该内容

4 篇文章

订阅专栏

在网上找了一个很简单的切换方式，就随手将他记录下来，方便自己后续查阅，也给大家提供参考，本章内容以Centos 7.9系统、MT27500-X3网卡为例

1. Mellanox网卡刚插上，使用ip a命令查看，发现第3、4网口名称为ib开头，这种默认就是IB模式了，我们需要将它切换到ETH模式使用（部分有IB需求的无需切换）

[root@localhost ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: enp3s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 00:08:60:00:37:2c brd ff:ff:ff:ff:ff:ff
    inet 192.168.2.16/24 brd 192.168.2.255 scope global noprefixroute dynamic enp3s0
       valid_lft 5999174sec preferred_lft 5999174sec
    inet6 fe80::1c49:f476:24e4:d82c/64 scope link noprefixroute
       valid_lft forever preferred_lft forever
3: ib0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 4092 qdisc pfifo_fast state DOWN group default qlen 256
    link/infiniband 80:00:02:08:fe:80:00:00:00:00:00:00:00:02:ca:03:00:44:0e:bf brd 00:ff:ff:ff:ff:12:40:1b:ff:ff:00:00:00:00:00:00:ff:ff:ff:ff
4: ib1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 4092 qdisc pfifo_fast state DOWN group default qlen 256
    link/infiniband 80:00:02:09:fe:80:00:00:00:00:00:00:00:02:ca:03:00:44:0e:c0 brd 00:ff:ff:ff:ff:12:40:1b:ff:ff:00:00:00:00:00:00:ff:ff:ff:ff
5: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default qlen 1000
    link/ether 52:54:00:89:3b:74 brd ff:ff:ff:ff:ff:ff
    inet 192.168.122.1/24 brd 192.168.122.255 scope global virbr0
       valid_lft forever preferred_lft forever
6: virbr0-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc pfifo_fast master virbr0 state DOWN group default qlen 1000
    link/ether 52:54:00:89:3b:74 brd ff:ff:ff:ff:ff:ff

2. 安装工具，从yum源直接进行安装

[root@localhost ~]# yum install mstflint

3. 查看网卡PCIe总线号

[root@localhost ~]# lspci |grep Mellanox
02:00.0 Network controller: Mellanox Technologies MT27500 Family [ConnectX-3]

4. 查询网卡配置信息

[root@localhost ~]# mstconfig -d 02:00.0 q

Device #1:
----------

Device type:    ConnectX3
Device:         02:00.0

Configurations:                              Next Boot
         SRIOV_EN                            False(0)
         NUM_OF_VFS                          8
         LINK_TYPE_P1                        VPI(3)
         LINK_TYPE_P2                        VPI(3)
         LOG_BAR_SIZE                        3
         BOOT_PKEY_P1                        0
         BOOT_PKEY_P2                        0
         BOOT_OPTION_ROM_EN_P1               True(1)
         BOOT_VLAN_EN_P1                     False(0)
         BOOT_RETRY_CNT_P1                   0
         LEGACY_BOOT_PROTOCOL_P1             PXE(1)
         BOOT_VLAN_P1                        1
         BOOT_OPTION_ROM_EN_P2               True(1)
         BOOT_VLAN_EN_P2                     False(0)
         BOOT_RETRY_CNT_P2                   0
         LEGACY_BOOT_PROTOCOL_P2             PXE(1)
         BOOT_VLAN_P2                        1

5. 修改LINK_TYPE_P1和LINK_TYPE_P2的值（1为IB，2为ETH，3为VPI），我们设置为2，切换到以太网模式就可以

[root@localhost ~]# mstconfig -d 02:00.0 set LINK_TYPE_P1=2 LINK_TYPE_P2=2

Device #1:
----------

Device type:    ConnectX3
Device:         02:00.0

Configurations:                              Next Boot       New
         LINK_TYPE_P1                        VPI(3)          ETH(2)
         LINK_TYPE_P2                        VPI(3)          ETH(2)

 Apply new Configuration? (y/n) [n] : y
Applying... Done!
-I- Please reboot machine to load new configurations.

6. 至此修改完成，重启生效查看如下