**************************************** README **************************************** Chelsio T7 ARM Unified Wire solution for Linux Version : 1.0.0.6 Date : 15/07/2026 Overview ================================================================================ Chelsio T7 ARM Unified Wire solution for Linux is an easy-to-use utility developed to provide installation of ARM Linux–based drivers for Chelsio’s adapters in RC mode. Chelsio T7 ARM Unified Wire solution consists of the following components: - Network (NIC) - LIO iSCSI PDU Offload (Target) - NVMe-oF iWARP (Target) - NVMe-oF RoCEv2 (Target) - NVMe-oF/TCP PDU Offload (Target) ================================================================================ CONTENTS ================================================================================ - 1. Supported Hardware - 2. How To Use - 3. Support Documentation - 4. Customer Support 1. Supported Hardware ================================================================================ Chelsio supported adapter ------------------------- - T72200-FH-DPU 2. How to Use ================================================================================ ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Flashing on the Chelsio T7 adapter ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Connecting the UART cable ========================= Perform the following steps to connect UART cable to the host machine: 1. Check the board type and connect the UART pin to the T7 Adapter. 2. Connect the UART cable USB to the host machine. 3. On the host system, verify the USB device. For the detailed installation steps and Test Bed Architecture, refer to the Chelsio T7 ARM Unified Wire Solution for Linux User Guide. Flash bootstrap firmware, init/vpd, u-boot, and firmware ======================================================== Note: These steps must be performed only once. To operate the ARM device in Root Complex (RC) mode, install the T7 adapter into a Linux host to flash the bootstrap firmware, init/VPD, U-Boot, and firmware. Next, power off the host, remove the T7 adapter, identify Pin J202 and connect a jumper pin to it, and then install the adapter into the target server chassis (or PCIe expansion enclosure) using an RC PCIe riser. 1. Run the following command to flash the bootstrap firmware: [root@host~]# ChelsioARM-1.0.x.x/tools/cudbg_app --loadfw "/sys/bus/pci/devices/" --bootstrap Example: [root@host~]# ChelsioARM-1.0.x.x/tools/cudbg_app --loadfw "/sys/bus/pci/devices/0000\:c7\:00.4" ChelsioARM-1.0.x.x/firmware/t7fwbootstrap-2.0.4.0.bin –bootstrap 2. Run the following command to flash the init VPD: [root@host~]# cd ChelsioARM-1.0.x.x/vpds [root@host~]# ChelsioARM-1.0.x.x/tools/t7seeprom -flash -b write -f: Example: [root@host~]# cd ChelsioARM-1.0.x.x/vpds [root@host~]# ChelsioARM-1.0.x.x/tools/t7seeprom -flash -b c7:00.0 write -f:t72200_fh_init_rc_gen5x16_dpu_vpd_mfg.bin 3. Run the following command to flash the u-boot.bin: [root@host~]# ChelsioARM-1.0.x.x/tools/cudbg_app -load_uboot "/sys/bus/pci/devices/" Example: [root@host~]# ChelsioARM-1.0.x.x/tools/cudbg_app --load_uboot "/sys/bus/pci/devices/0000:c7:00.4" ChelsioARM-1.0.x.x/images/u-boot.bin 4. Run the following command to flash firmware: [root@host~]# ChelsioARM-1.0.x.x/tools/cudbg_app --loadfw "/sys/bus/pci/devices/" Example: [root@host~]# ChelsioARM-1.0.x.x/tools/cudbg_app --loadfw "/sys/bus/pci/devices/0000\:c7\:00.4" ChelsioARM-1.0.x.x/firmware/t7fw-2.1.25.0.bin 5. Run the following command to clear firmware config: [root@host~]# ChelsioARM-1.0.x.x/tools/cudbg_app –loadcfg "/sys/bus/pci/devices/" --clear Example: [root@host~]# ChelsioARM-1.0.x.x/tools/cudbg_app --loadcfg "/sys/bus/pci/devices/0000\:c7\:00.4" --clear 6. Run the following command to flash firmware config: [root@host~]# ChelsioARM-1.0.x.x/tools/cudbg_app –loadcfg "/sys/bus/pci/devices/" Example: [root@host~]# ChelsioARM-1.0.x.x/tools/cudbg_app --loadcfg "/sys/bus/pci/devices/0000\:c7\:00.4" ChelsioARM-1.0.x.x/firmware/t7-config-iep.txt 7. Poweroff the linux machine. 8. Remove the T7 adapter. 9. Identify the Pin J202 and connect a jumper pin to it. NOTE: In the User Guide, pin J202 is labeled as "I" in Figure 1. 10. Install it into a target server chassis using an RC PCIe riser. ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Set up TFTP server on the peer machine ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Note: These steps must be performed only once. Follow these steps to set up a TFTP server on the peer machine: 1. Run the following command to install and configure TFTP Server: [root@host~]# yum install tftp-server.x86_64 2. Run the following command to start and enable the service: [root@host~]# systemctl start tftp.service [root@host~]# systemctl enable tftp.service [root@host~]# systemctl status tftp.service 3. Run the following command to set Permissions on TFTP Directory: [root@host~]# chmod 0777 /var/lib/tftpboot 4. Copy the “.itb file” to “/var/lib/tftpboot/” using the following command: [root@host~]# cp ChelsioARM-1.0.x.x/images/chdpu-t7.itb /var/lib/tftpboot/ 5. Run the following command to assign Peer IP Address: Assign IP to match the expected ‘serverip’ from DUT U-Boot environment. [root@host~]# ip addr add dev Example: [root@host~]# ip addr add 102.11.11.106/24 dev eth0 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Configuring U-Boot and UART access ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Follow these steps to configure U-Boot and UART access on Chelsio T7 ARM whenever the host is rebooted/powercycled. 1. Access UART. Note: Either picocom or minicom can be used for UART access. Using plink =========== Run the following plink commands from host machine: Using picocom ------------- picocom -b 115200 -q /dev/ttyUSB0 On success, you should see a U-Boot prompt: => Using minicom ------------- Configure minicom with below settings: +--------------------------------------------------------------------+ | A - Serial Device : /dev/ttyUSB0 | | | | C - Callin Program : | | D - Callout Program : | | E - Bps/Par/Bits : 115200 8N1 | | F - Hardware Flow Control : No | | G - Software Flow Control : No | On success, one must see a U-Boot prompt: => 2. Print U-Boot environment variables => printenv Sample output: baudrate=115200 bootargs=console=ttyCHD0,115200n8 earlycon rootwait root=/dev/ram0 bootcmd=bdinfo; env print; tftp 0x102020000 chdpu-t7.itb; bootm 0x10202 bootdelay=0 eth1addr=00:07:43:83:b6:38 ethaddr=00:07:43:83:b6:30 fdtcontroladdr=1f4941310 gatewayip= 192.168.1.250 ipaddr= 192.168.1.125 loadaddr=0x102020000 netmask=255.255.255.0 serverip= 192.168.1.250 stderr=serial@47130 stdin=serial@47130 stdout=serial@47130 Environment size: 427/126972 bytes 3. Change the ip for local and tftpserver. => setenv ipaddr 102.11.11.105 => setenv serverip 102.11.11.106 => setenv gatewayip 102.11.11.1 => printenv baudrate=115200 bootargs=console=ttyCHD0,115200n8 earlycon rootwait root=/dev/ram0 bootcmd=bdinfo; env print; tftp 0x102020000 chdpu-t7.itb; bootm 0x102020000 bootdelay=1 fdtcontroladdr=1f4941310 gatewayip=102.11.11.1 ipaddr=102.11.11.105 loadaddr=0x102020000 netmask=255.255.255.0 serverip=102.11.11.106 stderr=serial@47130 stdin=serial@47130 stdout=serial@47130 Environment size: 429/126972 bytes => CTRL-A Z for help | 115200 8N1 | NOR | Minico 4. Boot from TFTP. => tftp chdpu-t7.itb eth_chdpu eth-chdpu-0: link up eth_chdpu eth-chdpu-1: link up Using eth-chdpu-0 device TFTP from server 102.11.11.105; our IP address is 102.11.11.106 Filename 'chdpu-t7.itb'. Load address: 0x102020000 Loading: ######################################################## ######################################################## ######################################################## ######################################################## ######################################################## ######################################################## ######################################################## ######################################################## ######################################################## : : : ######################################################## ######################################################## ######################################################## ######################################################## 50 MiB/s done Bytes transferred = 286020627 (110c5413 hex) => mw 47178 0x1 => bootm 5. Once boot is successful, login prompt appears as below. Type root. Welcome to Buildroot buildroot login: root # # uname -a Linux buildroot 6.12.51 #1 SMP PREEMPT Wed Nov 5 13:35:47 IST 2025 aarch64 GNU/Linux 6. Load the Chelsio NIC driver. # modprobe chdpu_pcie # modprobe cxgb4 7. Configure ARM interfaces. # ifconfig -a dummy0 Link encap:Ethernet HWaddr BA:E1:0E:8B:43:C4 BROADCAST NOARP MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) eth0 Link encap:Ethernet HWaddr 00:07:43:83:B6:30 BROADCAST MULTICAST MTU:1500 Metric:1 RX packets:194850 errors:0 dropped:0 overruns:0 frame:0 TX packets:194883 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:295764414 (282.0 MiB) TX bytes:12473081 (11.8 MiB) eth1 Link encap:Ethernet HWaddr 00:07:43:83:B6:38 BROADCAST MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:27 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:0 (0.0 B) TX bytes:2268 (2.2 KiB) lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:65536 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) sit0 Link encap:IPv6-in-IPv4 NOARP MTU:1480 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) # ifconfig eth0 104.11.11.106 netmask 255.255.255.0 up # ifconfig eth1 103.11.11.178 netmask 255.255.255.0 up # ifconfig -a dummy0 Link encap:Ethernet HWaddr FA:EE:C4:CD:81:F6 BROADCAST NOARP MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) eth0 Link encap:Ethernet HWaddr 00:07:43:83:B6:30 inet addr:104.11.11.106 Bcast:104.11.11.255 Mask:255.255.255.0 inet6 addr: fe80::207:43ff:fe83:b630/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:198032 errors:0 dropped:0 overruns:0 frame:0 TX packets:198156 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:300578741 (286.6 MiB) TX bytes:12684646 (12.0 MiB) eth1 Link encap:Ethernet HWaddr 00:07:43:83:B6:38 inet addr:103.11.11.178 Bcast:103.11.11.255 Mask:255.255.255.0 inet6 addr: fe80::207:43ff:fe83:b638/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:105 errors:0 dropped:0 overruns:0 frame:0 TX packets:236 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:7358 (7.1 KiB) TX bytes:17708 (17.2 KiB) lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:65536 Metric:1 RX packets:2 errors:0 dropped:0 overruns:0 frame:0 TX packets:2 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:168 (168.0 B) TX bytes:168 (168.0 B) sit0 Link encap:IPv6-in-IPv4 NOARP MTU:1480 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) 8. Ping peer over ARM interface. # ping 104.11.11.105 -c 1 PING 104.11.11.105 (104.11.11.105): 56 data bytes 64 bytes from 104.11.11.105: seq=0 ttl=64 time=0.222 ms --- 104.11.11.105 ping statistics --- 1 packets transmitted, 1 packets received, 0% packet loss round-trip min/avg/max = 0.222/0.222/0.222 ms ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Network (NIC) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Driver Loading ============== Perform the following steps to load the driver: 1. Load NIC driver (cxgb4), [root@host~]# modprobe chdpu_pcie [root@host~]# modprobe cxgb4 Driver Unloading ================ To unload the NIC driver, run the following command: [root@host~]# rmmod cxgb4 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ LIO iSCSI PDU Offload (Target) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Driver Loading ============== 1. Load the following drivers: [root@host~]# modprobe chdpu_pcie [root@host~]# modprobe cxgb4 [root@host~]# modprobe cxgbit 2. Bring up the Chelsio interface(s). [root@host~]# ifconfig ethX x.x.x.x up Example: [root@host~]# ifconfig ethX 104.11.11.106 up Configuration ============= Target ------ 1. The following commands configure target using nvmetcli with a LUN: [root@host~]# targetcli /backstores/block/ create name=LUN0 dev= [root@host~]# targetcli /iscsi create wwn=iqn.2017-01.org.lun0.target [root@host~]# targetcli /iscsi/iqn.2017-01.org.lun0.target/tpg1/portals/ delete 0.0.0.0 3260 [root@host~]# targetcli /iscsi/iqn.2017-01.org.lun0.target/tpg1/portals/ create 104.11.11.106 ip_port=3260 [root@host~]# targetcli /iscsi/iqn.2017-01.org.lun0.target/tpg1/luns create /backstores/block/LUN0 [root@host~]# targetcli /iscsi/iqn.2017-01.org.lun0.target/tpg1/portals/104.11.11.106:3260 enable_offload boolean=True [root@host~]# targetcli /iscsi/iqn.2017-01.org.lun0.target/tpg1 set attribute authentication=0 demo_mode_write_protect=0 generate_node_acls=1 cache_dynamic_acls=1 2. Save the target configuration to a file. [root@host~]# targetcli saveconfig /root/iscsi_pdu_offload_target.json 3. To clear the targets, [root@host~]# targetcli clearconfig confirm=TRUE Peer ---- 1. Discover the target. [root@host~]# iscsiadm -m discovery -t st -p : Example: [root@host~]# iscsiadm -m discovery -t st -p 104.11.11.106:3260 2. Connect to target. [root@host~]# iscsiadm -m node -T -p : -l Example: [root@host~]# iscsiadm -m node -T iqn.2017-01.org.lun0.target -p 104.11.11.106:3260 -l 3. List the connected targets. [root@host~]# iscsiadm -m session -P 3 4. Format and mount the iSCSI disks shown with the above command. 5. Logout from the target. [root@host~]# iscsiadm -m node -T -p : -u Example: [root@host~]# iscsiadm -m node -T iqn.2017-01.org.lun0.target -p 104.11.11.106:3260 -u Driver Unloading ================ Follow the steps mentioned below to unload the drivers: 1. Log out from the initiator. 2. Run the following command: [root@host~]# targetcli /iscsi//tpg1/portals/\:3260 enable_offload boolean=False Execute the above command for every portal address listening on the Chelsio interface. Example: [root@host~]# targetcli /iscsi/iqn.2017-01.org.lun0.target/tpg1/portals/104.11.11.106:3260 enable_offload boolean=False 3. Unload the driver. [root@host~]# rmmod cxgbit [root@host~]# rmmod cxgb4 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ NVMe-oF iWARP (Target) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Driver Loading ============== 1. Load the following drivers: [root@host~]# modprobe chdpu_pcie [root@host~]# modprobe cxgb4 [root@host~]# modprobe iw_cxgb4 [root@host~]# modprobe rdma_cm 2. Bring up the Chelsio interface(s). [root@host~]# ifconfig ethX x.x.x.x up Example: [root@host~]# ifconfig ethX 104.11.11.106 up 3. Mount configfs. [root@host~]# mount -t configfs none /sys/kernel/config 4. On target, load the following drivers: [root@host~]# modprobe nvmet [root@host~]# modprobe nvmet-rdma Peer ==== On peer, load the following drivers: [root@host~]# modprobe nvme [root@host~]# modprobe nvme-rdma Configuration ============= Target ------ 1. The following commands configure target using nvmetcli with a LUN: [root@host~]# nvmetcli /> cd subsystems /subsystems> create nvme-ram0 /subsystems> cd nvme-ram0/namespaces /subsystems/n...m0/namespaces> create nsid=1 /subsystems/n...m0/namespaces> cd 1 /subsystems/n.../namespaces/1> set device path= /subsystems/n.../namespaces/1> cd ../.. /subsystems/nvme-ram0> set attr allow_any_host=1 /subsystems/nvme-ram0> cd namespaces/1 /subsystems/n.../namespaces/1> enable /subsystems/n.../namespaces/1> cd ../../../.. /> cd ports /ports> create 1 /ports> cd 1/ /ports/1> set addr adrfam=ipv4. /ports/1> set addr trtype=rdma /ports/1> set addr trsvcid=4420 /ports/1> set addr traddr=104.11.11.106 /ports/1> cd subsystems /ports/1/subsystems> create nvme-ram0 2. Save the target configuration to a file. /ports/1/subsystems> saveconfig /root/nvme-target_setup /ports/1/subsystems> exit 3. To clear the targets, [root@host~]# nvmetcli clear Peer ---- 1. Discover the target. [root@host~]# nvme discover -t rdma -a -s 4420 Example: [root@host~]# nvme discover -t rdma -a 104.11.11.106 -s 4420 2. Connect to target. Connecting to a specific target. [root@host~]# nvme connect -t rdma -a -s 4420 -n Example: [root@host~]# nvme connect -t rdma -a 104.11.11.106 -s 4420 -n nvme-ram0 Connecting to all targets configured on a portal. [root@host~]# nvme connect-all -t rdma -a -s 4420 Example: [root@host~]# nvme connect-all -t rdma -a 104.11.11.106 -s 4420 3. List the connected targets. [root@host~]# nvme list 4. Format and mount the NVMe disks shown with the above command. 5. Disconnect from the target and unmount the disk. [root@host~]# nvme disconnect -d NOTE: nvme_disk_name is the name of the device (Ex:nvme0n1) and not the device path. Driver Unloading ================ Follow the steps mentioned below to unload the drivers: On target, run the following commands: [root@host~]# rmmod nvmet-rdma [root@host~]# rmmod nvmet [root@host~]# rmmod iw_cxgb4 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ NVMe-oF RoCEv2 (Target) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Driver Loading ============== 1. Load the following drivers: [root@host~]# modprobe chdpu_pcie [root@host~]# modprobe cxgb4 [root@host~]# modprobe iw_cxgb4 roce_mode=1 [root@host~]# modprobe rdma_cm 2. Bring up the Chelsio interface(s). [root@host~]# ifconfig ethX x.x.x.x up Example: [root@host~]# ifconfig ethX 104.11.11.106 up 3. Mount configfs. [root@host~]# mount -t configfs none /sys/kernel/config 4. On target, load the following drivers: [root@host~]# modprobe nvmet [root@host~]# modprobe nvmet-rdma Peer ==== Note: The peer must be either an NVIDIA adapter or a SoftRoCE instance. On peer, load the following drivers: [root@host~]# modprobe nvme [root@host~]# modprobe nvme-rdma Configuration ============= Target ------ 1. The following commands configure target using nvmetcli with a LUN: [root@host~]# nvmetcli /> cd subsystems /subsystems> create nvme-ram0 /subsystems> cd nvme-ram0/namespaces /subsystems/n...m0/namespaces> create nsid=1 /subsystems/n...m0/namespaces> cd 1 /subsystems/n.../namespaces/1> set device path= /subsystems/n.../namespaces/1> cd ../.. /subsystems/nvme-ram0> set attr allow_any_host=1 /subsystems/nvme-ram0> cd namespaces/1 /subsystems/n.../namespaces/1> enable /subsystems/n.../namespaces/1> cd ../../../.. /> cd ports /ports> create 1 /ports> cd 1/ /ports/1> set addr adrfam=ipv4. /ports/1> set addr trtype=rdma /ports/1> set addr trsvcid=4420 /ports/1> set addr traddr=104.11.11.106 /ports/1> cd subsystems /ports/1/subsystems> create nvme-ram0 2. Save the target configuration to a file. /ports/1/subsystems> saveconfig /root/nvme-target_setup /ports/1/subsystems> exit 3. To clear the targets, [root@host~]# nvmetcli clear Peer ---- 1. Discover the target. [root@host~]# nvme discover -t rdma -a -s 4420 Example: [root@host~]# nvme discover -t rdma -a 104.11.11.106 -s 4420 2. Connect to target. Connecting to a specific target. [root@host~]# nvme connect -t rdma -a -s 4420 -n Example: [root@host~]# nvme connect -t rdma -a 104.11.11.106 -s 4420 -n nvme-ram0 Connecting to all targets configured on a portal. [root@host~]# nvme connect-all -t rdma -a -s 4420 Example: [root@host~]# nvme connect-all -t rdma -a 104.11.11.106 -s 4420 3. List the connected targets. [root@host~]# nvme list 4. Format and mount the NVMe disks shown with the above command. 5. Disconnect from the target and unmount the disk. [root@host~]# nvme disconnect -d NOTE: nvme_disk_name is the name of the device (Ex:nvme0n1) and not the device path. Driver Unloading ================ Follow the steps mentioned below to unload the drivers: On target, run the following commands: [root@host~]# rmmod nvmet-rdma [root@host~]# rmmod nvmet [root@host~]# rmmod iw_cxgb4 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ NVMe-oF/TCP PDU Offload (Target) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Driver Loading ============== 1. Unload the existing drivers. [root@host~]# rmmod csiostor cxgb4i cxgbit iw_cxgb4 chcr cxgb4vf cxgb4 libcxgbi libcxgb 2. To load NIC driver (cxgb4): [root@host~]# modprobe chdpu_pcie [root@host~]# modprobe cxgb4 3. To load NVMe-oF/TCP PDU Offload Target driver (cnvmet-tcp): [root@host~]# rmmod nvmet-tcp nvmet [root@host~]# modprobe cnvmet-tcp On Peer ------- Load NVMe-oF/TCP kernel driver : [root@host~]# modprobe nvme-tcp nvme-fabrics nvme-core Configuration ============= Target ------ 1. The following commands configure target using nvmetcli with a LUN: [root@host~]# nvmetcli /> cd subsystems /subsystems> create nvme-ram0 /subsystems> cd nvme-ram0/namespaces /subsystems/n...m0/namespaces> create nsid=1 /subsystems/n...m0/namespaces> cd 1 /subsystems/n.../namespaces/1> set device path= /subsystems/n.../namespaces/1> cd ../.. /subsystems/nvme-ram0> set attr allow_any_host=1 /subsystems/nvme-ram0> cd namespaces/1 /subsystems/n.../namespaces/1> enable /subsystems/n.../namespaces/1> cd ../../../.. /> cd ports /ports> create 1 /ports> cd 1/ /ports/1> set addr adrfam=ipv4 /ports/1> set addr trtype=tcp /ports/1> set addr trsvcid=4420 /ports/1> set addr traddr=104.11.11.106 /ports/1> cd subsystems /ports/1/subsystems> create nvme-ram0 2. Save the target configuration to a file. /ports/1/subsystems> saveconfig /root/nvme-target_setup /ports/1/subsystems> exit 3. To clear the targets. [root@host~]# nvmetcli clear Peer ---- 1. Discover the target. [root@host~]# nvme discover -t tcp -a -s 4420 Example: [root@host~]# nvme discover -t tcp -a 104.11.11.106 -s 4420 2. Connect to target. Connecting to a specific target. [root@host~]# nvme connect -t tcp -a -s 4420 -n Example: [root@host~]# nvme connect -t tcp -a 104.11.11.106 -s 4420 -n nvme-ram0 Connecting to all targets configured on a portal. [root@host~]# nvme connect-all -t tcp -a -s 4420 Example: [root@host~]# nvme connect-all -t tcp -a 104.11.11.106 -s 4420 3. List the connected targets. [root@host~]# nvme list 4. Format and mount the NVMe disks shown with the above command. 5. Disconnect from the target and unmount the disk. [root@host~]# nvme disconnect -d NOTE: nvme_disk_name is the name of the device (Ex:nvme0n1) and not the device path. Driver Unloading ================ Follow the steps mentioned below to unload the drivers: On target, run the following command: [root@host~]# rmmod cnvmet-tcp 3. Support Documentation ================================================================================ The documentation for this release can be found inside the ChelsioARM-x.x.x.x/docs folder. It contains: - README (this document) - User Guide - Release Notes ******************************************************************************** Copyright (C) 2026 Chelsio Communications. All Rights Reserved. The information in this document is furnished for informational use only, is subject to change without notice, and should not be construed as a commitment by Chelsio Communications. Chelsio Communications assumes no responsibility or liability for any errors or inaccuracies that may appear in this document or any software that may be provided in association with this document. Except as permitted by such license, no part of this document may be reproduced, stored in a retrieval system, or transmitted in any form or by any means without the express written consent of Chelsio Communications.