需要先安装 mlnx_ofed 驱动。

https://network.nvidia.com/products/infiniband-drivers/linux/mlnx_ofed/

查询网卡信息

# 启动 Mellanox 软件工具
[root@servera ~]# mst start
Starting MST (Mellanox Software Tools) driver set
Loading MST PCI module - Success
Loading MST PCI configuration module - Success
Create devices
Unloading MST PCI module (unused) - Success

# 查看当前的 Mellanox 网卡
[root@servera ~]# mst status
MST modules:
------------
    MST PCI module is not loaded
    MST PCI configuration module loaded

MST devices:
------------
/dev/mst/mt4117_pciconf0         - PCI configuration cycles access.
                                   domain:bus:dev.fn=0000:41:00.0 addr.reg=88 data.reg=92 cr_bar.gw_offset=-1
                                   Chip revision is: 00
/dev/mst/mt4129_pciconf0         - PCI configuration cycles access.
                                   domain:bus:dev.fn=0000:a1:00.0 addr.reg=88 data.reg=92 cr_bar.gw_offset=-1
                                   Chip revision is: 00

# 通过 lspci 查看 Mellanox 的 pci 总线编号
[root@servera ~]# lspci | grep -i Mellanox
41:00.0 Ethernet controller: Mellanox Technologies MT27710 Family [ConnectX-4 Lx]
41:00.1 Ethernet controller: Mellanox Technologies MT27710 Family [ConnectX-4 Lx]
a1:00.0 Ethernet controller: Mellanox Technologies MT2910 Family [ConnectX-7]
a1:00.1 Ethernet controller: Mellanox Technologies MT2910 Family [ConnectX-7]

可以看到需要修改的 Mellanox 网卡的总线信息为 a1:00。

修改 Mellanox 网卡模式

# 查看网卡当前模式
[root@servera ~]# mlxconfig -d /dev/mst/mt4129_pciconf0 q | grep LINK_TYPE_P1
        LINK_TYPE_P1                                ETH(2)

# 设置网卡模式为 IB
[root@servera ~]# mlxconfig -d /dev/mst/mt4129_pciconf0 s LINK_TYPE_P1=1

Device #1:
----------

Device type:        ConnectX7
Name:               MCX755106AS-HEA_Ax
Description:        NVIDIA ConnectX-7 HHHL Adapter Card; 200GbE (default mode) / NDR200 IB; Dual-port QSFP112; PCIe 5.0 x16 with x16 PCIe extension option; Crypto Disabled; Secure Boot Enabled
Device:             /dev/mst/mt4129_pciconf0

Configurations:                                          Next Boot       New
        LINK_TYPE_P1                                ETH(2)               IB(1)

 Apply new Configuration? (y/n) [n] : y
Applying... Done!
-I- Please reboot machine to load new configurations.

# 重启系统
[root@servera ~]# reboot

执行 mlxconfig -d /dev/mst/mt4129_pciconf0 s LINK_TYPE_P1=1 命令修改模式时，LINK_TYPE_P1=1 是设置为 Infiniband 模式，LINK_TYPE_P1=2 是设置为 Ethernet 模式。