**************************************** README **************************************** Chelsio T5/T4 Family Unified Wire for Linux Version : 2.6.0.5 Date : 09/27/2013 Overview ================================================================================ Chelsio Unified Wire software for Linux is an easy to use utility developed to provide installation of 64-bit Linux based drivers and tools for Chelsio's T5 and T4 Unified Wire Adapters. The Chelsio Unified Wire Package provides an interactive installer to install various drivers and utilities. It consists of the following components: a. Network (NIC/TOE) b. Virtual Function Network (vNIC) c. iWARP (RDMA) d. WD-UDP e. WD-TOE f. iSCSI PDU Offload Target g. iSCSI PDU Offload Initiator h. FCoE Full Offload Initiator i. Offload Bonding driver j. UDP Segmentation Offload and Pacing k. Offload IPv6 driver l. Bypass driver m. Classification and Filtering feature n. Traffic Management feature (TM) o. Unified Wire Manager (UM) p. Unified Boot Software q. Utility Tools (cop,cxgbtool,t4_perftune,benchmark tools,sniffer & tracer) r. libs (iWARP and WD-UDP libraries) ================================================================================ CONTENTS ================================================================================ - 1. Requirements - 2. Supported Operating System - 3. Supported Cards - 4. How To Use - 5. Support Documentation - 6. Customer Support 1. Requirements ================================================================================ 1.1. Network (NIC/TOE) ====================== - Chelsio T5 40/10Gb or T4 10/1Gb adapter and system with supported platforms mentioned in section 2. 1.2. Virtual Function Network (vNIC) ==================================== - Chelsio T5 40/10Gb or T4 10/1Gb adapter and an SR-IOV enabled system with supported platforms mentioned in section 2. 1.3. iWARP (RDMA) ================= - Chelsio T5 40/10Gb or T4 10/1Gb adapter and system with supported platforms mentioned in section 2. - If you are planning to upgrade OFED on one member of the cluster, the upgrade needs to be installed on all the members. - If you want to install OFED with NFS-RDMA support, please see "Setting up NFS-RDMA" in iWARP (RDMA) chapter in the User's Guide. 1.4. WD-UDP ============ - Chelsio T5 40/10Gb or T4 10/1Gb adapter and system with supported platforms mentioned in section 2. 1.5. WD-TOE ============ - Chelsio T5 40/10Gb or T4 10/1Gb adapter and system with supported platforms mentioned in section 2. 1.6. iSCSI PDU Offload Target ============================== - Chelsio T5 40/10Gb or T4 10/1Gb adapter and system with supported platforms mentioned in section 2. - iSCSI PDU offload target driver (chiscsi_t4.ko) requires NIC(cxgb4), TOE(t4_tom & toecore,) and iSCSI non-offload(chiscsi_base.ko) modules to work. Whereas the iSCSI non-offload target driver requires only NIC module. 1.7. iSCSI PDU Offload Initiator ================================= - Chelsio T5 40/10Gb or T4 10/1Gb adapter and system with supported platforms mentioned in section 2. - The iSCSI PDU Offload Initiator driver (cxgb4i) runs on top of NIC module(cxgb4) and open-iscsi-2.0.872 only, on a T5/T4 card. - If you're planning to install iSCSI PDU Offload Initiator, please install openssl-devel package. 1.8. FCoE full offload Initiator ================================= - Chelsio T5 40/10Gb or T4 10/1Gb adapter and system with supported platforms mentioned in section 2. 1.9. Offload Bonding driver =========================== - Chelsio T5 40/10Gb or T4 10/1Gb adapter and system with supported platforms mentioned in section 2. 1.10. UDP Segmentation Offload and Pacing driver ================================================= - Chelsio T5 40/10Gb or T4 10/1Gb adapter and system with supported platforms mentioned in section 2. 1.11. Offload IPv6 driver ============================ - Chelsio T5 40/10Gb or T4 10/1Gb adapter and system with supported platforms mentioned in section 2. 1.12. Bypass driver ============================ - Chelsio T4 10/1Gb and system with supported platforms mentioned in section 2. 1.13. Classification and Filtering ======================================= - Chelsio T5 40/10Gb or T4 10/1Gb and system with supported platforms mentioned in section 2. 1.14. Traffic Management ========================== - Chelsio T5 40/10Gb or T4 10/1Gb adapter and system with supported platforms mentioned in section 2. 1.15. Unified Wire Manager ========================== - Chelsio T5 40/10Gb or T4 10/1Gb adapter and system with supported platforms mentioned in section 2. 1.16. Unified Boot Software ================================ - Chelsio T5 40/10Gb or T4 10/1Gb adapter and system with supported platforms mentioned in section 2. - DOS bootable USB flash drive or Floppy Disk. 2. Supported Operating System ================================================================================ The Chelsio Unified Wire software has been developed to run on 64-bit Linux based platforms. Following is the list of Drivers/Software and supported Linux distributions. ########################|######################################################| Linux Distribution | Driver/Software | ########################|######################################################| RHEL5.7,2.6.18-274.el5 |NIC/TOE,vNIC,iWARP,WD-UDP,WD-TOE*,iSCSI Target*, | |iSCSI Initiator*,Bonding,IPv6,Bypass*, | |Sniffer & Tracer,UDP-SO,Filtering,TM | ------------------------|------------------------------------------------------| RHEL5.8,2.6.18-308.el5 |NIC/TOE,vNIC,iWARP,WD-UDP*,WD-TOE*,iSCSI Target*, | |iSCSI Initiator,Bonding,IPv6,Bypass*,Sniffer & Tracer | |UM(Agent,Client),UDP-SO,Filtering,TM | ------------------------|------------------------------------------------------| RHEL6.3, |NIC/TOE,vNIC,iWARP,WD-UDP,WD-TOE*,iSCSI Target*, | 2.6.32-279.el6 |iSCSI Initiator*,FCoE Initiator*, | |Bonding,IPv6,Bypass*,Sniffer & Tracer,UDP-SO, | |UM(Agent,Client),Filtering,TM | ------------------------|------------------------------------------------------| RHEL6.4, |NIC/TOE,vNIC,iWARP,WD-UDP,WD-TOE,iSCSI Target, | 2.6.32-358.el6 |iSCSI Initiator,FCoE Initiator,Bonding,IPv6,Bypass, | |Sniffer & Tracer,UDP-SO,UM(Agent,Client),Filtering,TM | |uBoot(DUD) | ------------------------|------------------------------------------------------| SLES11SP1,2.6.32.12-0.7 |NIC/TOE,vNIC,iWARP,WD-UDP,WD-TOE*,iSCSI Target, | |iSCSI Initiator,FCoE Initiator, | |Bonding,IPv6,Bypass*,Sniffer & Tracer,UDP-SO, | |Filtering,TM | ------------------------|------------------------------------------------------| SLES11SP2,3.0.13-0.27 |NIC/TOE,vNIC,iWARP*,WD-UDP,WD-TOE,iSCSI Target*, | |iSCSI Initiator*,FCoE Initiator*,Bonding, | |IPv6,Bypass*,Sniffer & Tracer*,UDP-SO | |UM(Agent,Client),Filtering,TM,uBoot(DUD) | ------------------------|------------------------------------------------------| SLES11SP3,3.0.76-0.11 |NIC/TOE,vNIC,iWARP,WD-UDP,WD-TOE,iSCSI Target, | |iSCSI Initiator,FCoE Initiator,Bonding,IPv6,Bypass, | |Sniffer & Tracer,UDP-SO,Filtering,TM,uBoot(DUD) | ------------------------|------------------------------------------------------| Ubuntu 12.04, 3.2.0-23 |NIC/TOE,vNIC,iWARP,WD-UDP,iSCSI Initiator, | |Bonding,Bypass,IPv6,UDP-SO,Filtering,TM | ------------------------|------------------------------------------------------| Kernel.org linux-2.6.35 |NIC/TOE*,vNIC,iWARP,WD-UDP,iSCSI Target*, | |iSCSI Initiator*,FCoE Initiator*,IPv6,Bypass*, | |Sniffer & Tracer,UDP-SO,Filtering,TM | ------------------------|------------------------------------------------------| Kernel.org linux-3.4 |NIC/TOE,vNIC,iWARP,WD-UDP,WD-TOE,iSCSI Target, | |iSCSI Initiator,FCoE Initiator,Bonding,IPv6, | |Bypass,Sniffer & Tracer,UDP-SO,Filtering,TM | ------------------------|------------------------------------------------------| Kernel.org linux-3.6 |NIC/TOE*,vNIC*,iWARP*,WD-UDP*,WD-TOE,iSCSI Target*, | |iSCSI Initiator,FCoE Initiator*,Bonding*,IPv6*, | |Bypass*,Sniffer & Tracer*,UDP-SO,Filtering,TM | ------------------------|------------------------------------------------------| Kernel.org linux-3.7 |NIC/TOE,vNIC,iWARP,WD-UDP,WD-TOE,iSCSI Target*, | |iSCSI Initiator*,FCoE Initiator,Bonding,IPv6,Bypass, | |Sniffer & Tracer,UDP-SO,Filtering,TM | -------------------------------------------------------------------------------| NOTE:Other kernel versions have not been tested and are not guaranteed to work. *Limited QA performed 3. Supported Hardware ================================================================================ 3.1. Following is the list of Drivers/Software and supported Chelsio T5/T4 adapters. ------------------------------------------------------------------------- ########################|######################################################| Chelsio T5/T4 Adapter | Driver/Software | ########################|######################################################| T520-LL-CR |NIC,vNIC,iWARP,WD-UDP,WD-TOE,iSCSI Target, | |iSCSI Initiator,FCoE Initiator,Bonding,IPv6,UDP-SO, | |Filtering,TM,UM,uBoot | ------------------------|------------------------------------------------------| T520-SO-CR |NIC,Filtering,uBoot | ------------------------|------------------------------------------------------| T520-CR |NIC,TOE,vNIC,iWARP,WD-UDP,WD-TOE,iSCSI Target, | |iSCSI Initiator,FCoE Initiator,Bonding,IPv6,UDP-SO, | |Filtering,TM,UM,uBoot | ------------------------|------------------------------------------------------| T540-CR |NIC/TOE,iWARP,WD-UDP,iSCSI Target,iSCSI Initiator, | |Bonding,IPv6,Filtering,uBoot | ------------------------|------------------------------------------------------| T580-LP-CR |NIC/TOE,vNIC,iWARP,WD-UDP,WD-TOE,iSCSI Target, | |iSCSI Initiator,Bonding,IPv6,UDP-SO,Filtering,TM,UM, | |uBoot | ------------------------|------------------------------------------------------| T420-CR |NIC/TOE,vNIC,iWARP,WD-UDP,WD-TOE,iSCSI Target, | |iSCSI Initiator,FCoE Initiator,Bonding, | |IPv6,Sniffer & Tracer,UDP-SO,Filtering,TM,UM,uBoot | ------------------------|------------------------------------------------------| T440-CR |NIC/TOE,vNIC,iWARP,WD-UDP,WD-TOE,iSCSI Target, | |iSCSI Initiator,FCoE Initiator,Bonding, | |IPv6,Sniffer & Tracer, UDP-SO,Filtering,TM,UM,uBoot | ------------------------|------------------------------------------------------| T422-CR |NIC/TOE,vNIC,iWARP,WD-UDP,WD-TOE,iSCSI Target, | |iSCSI Initiator,FCoE Initiator,Bonding, | |IPv6,Sniffer & Tracer,UDP-SO,Filtering,TM,UM,uBoot | ------------------------|------------------------------------------------------| T420-SO-CR |NIC/TOE,vNIC,WD-TOE,Bonding,IPv6,Filtering,UM,uBoot | ------------------------|------------------------------------------------------| T404-BT |NIC/TOE,vNIC,iWARP,WD-UDP,WD-TOE,iSCSI Target, | |iSCSI Initiator,FCoE Initiator,Bonding, | |IPv6,Sniffer & Tracer,UDP-SO,Filtering,TM,UM,uBoot | ------------------------|------------------------------------------------------| T420-BCH |NIC/TOE,WD-TOE,iSCSI Target,iSCSI Initiator, | |Bonding,IPv6,Sniffer & Tracer,UDP-SO,Filtering,TM,UM, | |uBoot | ------------------------|------------------------------------------------------| T440-LP-CR |NIC/TOE,vNIC,iWARP,WD-UDP,WD-TOE,iSCSI Target, | |iSCSI Initiator,FCoE Initiator,Bonding,IPv6, | |Sniffer & Tracer, UDP-SO,Filtering,TM,UM,uBoot | ------------------------|------------------------------------------------------| T420-BT |NIC/TOE,vNIC,iSCSI Target,iSCSI Initiator, | |WD-TOE,Bonding,IPv6,Sniffer & Tracer,UDP-SO,Filtering,| |TM,UM,uBoot | ------------------------|------------------------------------------------------| T420-LL-CR |NIC/TOE,vNIC,iWARP,WD-UDP,WD-TOE,iSCSI Target, | |iSCSI Initiator,FCoE Initiator,Bonding,IPv6, | |Sniffer & Tracer,UDP-SO,Filtering,TM,UM,uBoot | ------------------------|------------------------------------------------------| T420-CX |NIC/TOE,vNIC,iWARP,WD-UDP,WD-TOE,iSCSI Target, | |iSCSI Initiator,Bonding,IPv6,Sniffer & Tracer,UDP-SO, | |Filtering,TM,UM | ------------------------|------------------------------------------------------| B420-SR |Bypass | ------------------------|------------------------------------------------------| B404-BT |Bypass | -------------------------------------------------------------------------------| 3.2. T3 adapters supported by Unified Wire Manager: ---------------------------------------------------- - S302E - S302E-C - S310E-CR - S310E-CR-C - S310E-CXA - S310E-SR+ - S310E-SR - S310E-BT - S320E-CR - S320E-LP-CR - S320E-CXA - S320EM-BS - S320EM-BCH - N320E-G2-CR - N320E - N320E-CXA - N320E-BT - N310E - N310E-CXA 3.3. Unified Boot Software ---------------------------- 3.3.1. Supported hardware platforms ----------------------------------------- Following is the list of supported hardware platforms: - DELL PowerEdge T710 - DELL PowerEdge 2950 - DELL PowerEdge T110 - IBM X3650 M2 - IBM X3650 M4 - HP Z210 - HP proliant DL385G2 - Supermicro X7DWE - Supermicro X8DTE-F - Supermicro X8STE - Supermicro X8DT6 - INTEL Q57 TunnelMount - ASUS P5KC - HP DL160 3.3.2. Supported Switch ------------------------ Cisco Nexus 5010 with 5.1(3)N1(1a) firmware. NOTE:Other platforms/switches have not been tested and are not guaranteed to work. 4. How to Use ================================================================================ ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 4.1. Chelsio Unified Wire ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 4.1.1. Installing Chelsio Unified Wire ====================================== There are two main methods to install the Chelsio Unified Wire package: from source and from RPM. If you decide to use source, you can install the package using CLI or GUI mode. If you decide to use RPM, you can install the package using Menu or CLI mode. The following table describes the various "configuration tuning options" available during installation and drivers/software installed with each option by default: ########################################|######################################| T5/T4 Configuration Tuning Option | Driver/Software installed | ########################################|######################################| Unified Wire |NIC/TOE,vNIC,iWARP,WD-UDP, | |iSCSI Target,iSCSI Initiator,Bonding, | |IPv6,Sniffer & Tracer,FCoE Initiator, | |UM(Agent,Client),Filtering,TM | ----------------------------------------|--------------------------------------| Low latency Networking |NIC/TOE,vNIC,iWARP,WD-UDP,WD-TOE,IPv6,| |Sniffer & Tracer,UM(Agent,Client), | |Filtering,TM | ----------------------------------------|--------------------------------------| High capacity RDMA |NIC/TOE,vNIC,iWARP,WD-UDP,Bonding, | |IPv6,Sniffer & Tracer, | |UM(Agent,Client),Filtering,TM | ----------------------------------------|--------------------------------------| High capacity TOE |NIC/TOE,vNIC,Bonding,IPv6, | |UM(Agent,Client),Filtering,TM | ----------------------------------------|--------------------------------------| UDP Seg.Offload & Pacing |NIC/TOE,IPv6,USO,UM(Agent,Client), | |Filtering,TM | ----------------------------------------|--------------------------------------| T5 Wire Direct Latency* |NIC/TOE,iWARP,WD-UDP,WD-TOE, | |UM(Agent,Client) | -------------------------------------------------------------------------------| *Supported only on T5 adapters. NOTE: "High capacity TOE" and "High capacity RDMA" options are currently not supported with T5 Adapters. Follow the steps mentioned below for installation using CLI. For GUI or Menu based installation, refer the User's Guide. a. From source -------------- i. Download the tarball ChelsioUwire-x.x.x.x.tar.gz ii. Untar the tarball [root@host]# tar zxvfm ChelsioUwire-x.x.x.x.tar.gz iii. Change your current working directory to Chelsio Unified Wire package directory. Build the source: [root@host]# make iv. Install the drivers, tools and libraries: [root@host]# make install v. The default configuration tuning option is Unified Wire (includes FCoE Initiator). The configuration tuning can be selected using the following commands: [root@host]# make CONF= [root@host]# make CONF= install NOTE: To view the different configuration tuning options, view help by typing [root@host]# make help NOTE: "High capacity TOE" and "High capacity RDMA" options are currently not supported with T5 Adapters. vi. Reboot your machine for changes to take effect. IMPORTANT:Steps (iv) and (v) mentioned above will NOT install Bypass driver. It will have to be installed manually. Please refer to section "4.1.3. Installing individual drivers" for instructions on installing it. Installation on updated kernels ---------------------------------------- If the kernel version on your Linux distribution is updated, please execute the following command to install the Unified Wire package: [root@host]# make UNAME_R= Where kernel_version is the nearest supported kernel version. For example, if you want to install the package on a RHEL 6 distribution updated to 2.6.32-279.2.1.el6 kernel, run the following commands: [root@host]# make UNAME_R=2.6.32-279.el6 [root@host]# make UNAME_R=2.6.32-279.el6 install To view the list of the supported kernel versions, run the following command: [root@host]# make list_kernels Reboot your machine for changes to take effect. iWARP driver installation on Cluster nodes ------------------------------------------ IMPORTANT: Please make sure that you have enabled password less authentication with ssh on the peer nodes for this feature to work. Chelsio's Unified Wire package allows installing iWARP drivers on multiple Cluster nodes with a single command. Follow the procedure mentioned below: i. Create a file (machinefilename) containing the IP addresses or hostnames of the nodes in the cluster. You can view the sample file, sample_machinefile, provided in the package to view the format in which the nodes have to be listed. ii. Now, execute the following command: [root@host] # ./install.py -C -m iii. Press "1" to read the Chelsio End User License Agreement; Press "q" to go back to installation menu. Press "2" to Accept the EULA and start the installation or "3" to Disagree and and exit from the installer. iv. Select the required T5/T4 configuration tuning option. The tuning options may vary depending on the Linux distribution. v. Select the required Cluster Configuration. vi. If you already have the required version of OFED software installed, you can skip this step. To install OFED-3.5 choose the "Install-OFED option". To install a different version, select "Choose-OFED-Version" and then select the appropriate version. To skip this step, "select Skip-OFED". NOTE: OFED is currently not supported on RHEL 6.4 and SLES11SP3. vii. The selected components will now be installed. b. From RPM (tarball) ---------------------- NOTE: IPv6 should be enabled in the machine to use the RPM Packages. i. Download the tarball specific to your operating system and architecture. ii. Untar the tarball E.g. For RHEL 6.3, untar using the following command: [root@host]# tar zxvfm ChelsioUwire-x.x.x.x-RHEL6.3_x86_64.tar.gz iii. Change your current working directory to Chelsio Unified Wire package directory. Run the following command: [root@host]# ./install.py -i nic_toe :NIC and TOE drivers only all :all Chelsio drivers (including FCoE Initiator) built against inbox OFED bypass :bypass drivers and tools udpso :UDP segmentation offload capable NIC and TOE drivers only wd :Wire Direct drivers and libraries only NOTE: The Installation options may vary depending on the Linux distribution. iv. The default configuration tuning option is Unified Wire (includes FCoE Initiator). The configuration tuning can be selected using the following command: [root@host]# ./install.py -i -c NOTE: To view the different configuration tuning options, view the help by typing [root@host]#./install.py -h NOTE: "High capacity TOE" and "High capacity RDMA" options are currently not supported with T5 Adapters. v. To install OFED and Chelsio Drivers built against OFED, run the above command with -o option. [root@host]# ./install.py -i -c -o vi. Reboot your machine for changes to take effect. NOTE:If the installation aborts with the message "Resolve the errors/dependencies manually and restart the installation", please go through the install.log to resolve errors/dependencies and then start the installation again. Driver installation on cluster nodes --------------------------------------- IMPORTANT:Please make sure that you have enabled password less authentication with ssh on the peer nodes for this feature to work. i. Create a file (machinefilename) containing the IP addresses or hostnames of the nodes in the cluster. You can view the sample file, sample_machinefile, provided in the package to view the format in which the nodes have to be listed. ii. Navigate to ChelsioUwire directory and execute the following command: [root@host] # ./install.py -C -m -i -c -o Here, -o parameter will install OFED and Chelsio drivers built against OFED iii. Reboot your machine for changes to take effect. 4.1.2. Installing individual drivers ====================================== You can also choose to install drivers individually. Provided here are steps to build and install NIC,TOE,iWARP,Bypass,WD-TOE and UDP Segmentation Offload drivers. To know about other drivers, view help by running "make help". i. To build and install NIC driver without offload support : [root@host]# make nic [root@host]# make nic_install ii. To build and install NIC driver with offload support and Offload drivers: [root@host]# make toe [root@host]# make toe_install iii. To build and install Offload drivers without IPv6 support: [root@host]# make toe_ipv4 [root@host]# make toe_ipv4_install iv. To build and install iWARP driver against outbox OFED: [root@host]# make iwarp [root@host]# make iwarp_install v. To build and install all drivers without IPv6 support: [root@host]# make ipv6_disable=1 [root@host]# make ipv6_disable=1 install vi. The above step will not install Bypass and WD-TOE drivers. a. To build and install Bypass driver: [root@host]# make bypass [root@host]# make bypass_install b. To build and install WD-TOE driver: [root@host]# make wdtoe [root@host]# make wdtoe_install vii. To build and install WD-TOE and WD-UDP drivers together: [root@host]# make wdtoe_wdudp [root@host]# make wdtoe_wdudp_install viii. To build and install all drivers with DCBX support (only for T4 adapters) [root@host]# make dcbx=1 [root@host]# make dcbx=1 install ix. The offload drivers support UDP Segmentation Offload with limited number of connections (1024 connections).To build and install UDP Offload drivers which support large number of offload connections (approx 10K), [root@host]# make udp_offload [root@host]# make udp_offload_install x. The default T5/T4 configuration tuning option is Unified Wire (includes FCoE Initiator). The configuration tuning can be selected using the following commands: [root@host]# make CONF= [root@host]# make CONF= NOTE:To view the different configuration tuning options, view the help by typing [root@host]#make help NOTE:If IPv6 is disabled in the machine, the drivers will be built and installed without IPv6 Offload support by default. 4.1.3. Firmware Update ======================================== The T5 (1.8.27.0) and T4 (1.8.24.0) firmwares are installed on the system, typically in /lib/firmware/cxgb4, and the driver will auto-load the firmwares if an update is required. The kernel must be configured to enable userspace firmware loading support: Device Drivers -> Generic Driver Options -> Userspace firmware loading support The firmware version can be verified using ethtool: [root@host]# ethtool -i 4.1.4. Uninstalling Chelsio Unified Wire ======================================== There are two methods to uninstall the Chelsio Unified Wire package: from source and from RPM. If you decide to use source, you can uninstall the package using CLI or GUI mode. Follow the steps mentioned below for uninstallation using CLI. For GUI based uninstallation, refer the User's Guide. a. From source -------------- Navigate to the ChelsioUwire-x.x.x.x directory. Uninstall the source using the following command: [root@host]# make uninstall NOTE:Uninstalling Unified Wire package will not uninstall Unified Wire Manager. Refer to the next section, "Uninstalling individual drivers/software" to remove the software manually. iWARP driver uninstallation on Cluster nodes ---------------------------------------------- To uninstal iWARP drivers on multiple Cluster nodes with a single command, run the following command: [root@host] # ./install.py -C -m -u all The above command will remove Chelsio iWARP (iw_cxgb4) and TOE (t4_tom) drivers from all the nodes listed in the machinefilename file. b. From RPM (tar-ball) ---------------------- Go to the ChelsioUwire-x.x.x.x directory. Run the following command: [root@host]# ./uninstall.py Here, inbox : for removing all Chelsio drivers. ofed : for removing OFED and Chelsio drivers. NOTE:The uninstallation options may vary depending on Linux distribution. View help by running the following command for more information: [root@host]# ./uninstall.py -h NOTE:Uninstalling Unified Wire package will not uninstall Unified Wire Manager. Refer the "Unified Wire Manager (UM)" section to remove the software manually. iWARP driver uninstallation on Cluster nodes ----------------------------------------------- To uninstal iWARP drivers on multiple Cluster nodes with a single command, run the following: [root@host] # ./install.py -C -m -u The above command will remove Chelsio iWARP (iw_cxgb4) and TOE (t4_tom) drivers from all the nodes listed in the machinefilename file. 4.1.5. Uninstalling individual drivers/software ================================================ You can also choose to uninstall drivers/software individually. Provided here are steps to uninstall NIC, TOE, iWARP,Bypass,WD-TOE, UDP Segmentation Offload drivers and Unified Wire Manager (UM). To know about other drivers, view help by running "make help". i. To uninstall NIC driver: [root@host]# make nic_uninstall ii. To uninstall NIC driver with offload support: [root@host]# make toe_uninstall iii. To uninstall iWARP driver: [root@host]# make iwarp_uninstall iv. To uninstall Bypass driver: [root@host]# make bypass_uninstall v. To uninstall UDP Segmentation Offload driver: [root@host]# make udp_offload_uninstall vi. To uninstall WD-TOE driver: [root@host]# make wdtoe_uninstall vii. To uninstall WD-TOE and WD-UPD drivers together [root@host]# make wdtoe_wdudp_uninstall viii. To uninstall Unified Wire Manager (UM) [root@host]# make uninstall UM_UNINST=1 OR [root@host]# make tools_uninstall UM_UNINST=1 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 4.2. Network (NIC/TOE) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 4.2.1. Loading/Unloading the driver =================================== a. Loading the driver ---------------------- The driver must be loaded by the root user. Any attempt to load the driver as a regular user will fail. i. To load the driver in NIC mode(without offload support) [root@host]# modprobe cxgb4 ii. To load driver in TOE mode(with offload support) [root@host]# modprobe t4_tom NOTE:Offload support needs to be enabled upon each reboot of the system. This can be done manually as shown above. b. Unloading the driver ----------------------- i. To unload the NIC driver. [root@host]# rmmod cxgb4 ii. To unload the TOE driver. Please reboot the system to unload the TOE driver. c. Instantiate Virtual Functions ------------------------------ To instantiate the Virtual functions, load the cxgb4 driver with 'num_vf' parameter with a non-zero value. For example: [root@host]# modprobe cxgb4 num_vf=1,0,0,0 Each number instantiates the specified number of Virtual Functions for the corresponding Physical Function. The Virtual Functions can be assigned to Virtual Machines(Guest OS). A maximum of 64 Virtual Functions can be instantiated with 16 Virtual Functions per Physical Function. Loading the cxgb4 driver with "num_vf" parameter loads the cxgb4vf driver by default. Hence unload the cxgb4vf driver (on the host) before assigning Virtual Functions to the Virtual Machines(Guest OS), using the following command: [root@host]# rmmod cxgb4vf 4.2.2. Performance tuning =================================== In order to auto tune the system and TOE devices for best performance, Chelsio recommends installing the tools which will copy t4_perftune.sh script to /sbin directory. Run the script by using the following command: [root@host]# t4_perftune.sh This script will configure RSS and enables Interrupt Coalescing. 4.2.3. Network Device Configuration =================================== Please refer to the operating system documentation for administration and configuration of network devices. NOTE: Some operating systems may attempt to auto-configure the detected hardware and some may not detect all ports on a multi-port adapter. If this happens, please refer to the operating system documentation for manually configuring the network device. ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 4.3. Virtual Function Network (vNIC) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 4.3.1. Instantiate Virtual Functions =================================== To instantiate Chelsio Virtual Functions, please refer to the Network (NIC/TOE) section [section 4.2.1.(c)]. 4.3.2. Loading/Unloading the Driver ================================== a. Loading the driver --------------------- The vNIC (cxgb4vf) driver must be loaded on the Guest OS by the root user. Any attempt to load the driver as a regular user will fail. To load the driver execute the following command: [root@host]# modprobe cxgb4vf b. Unloading the Driver ----------------------- To unload the driver execute the following command: [root@host]# rmmod cxgb4vf ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 4.4. iWARP (RDMA) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 4.4.1. Installing iWARP with OFED package ========================================= i. Download and untar the tarball ChelsioUwire-x.x.x.x.tar.gz ii. Go to the ChelsioUwire-x.x.x.x directory. Build the source using : [root@host]# make iii. Install the drivers, tools and libraries: [root@host]# make install iv. Reboot your machine for changes to take effect. NOTE:WD-UDP libraries are pre-compiled binaries against OFED-1.5.4 package. 4.4.2. Loading/Unloading the driver ================================== a. Loading the driver --------------------- The driver must be loaded by the root user. Any attempt to load the driver as a regular user will fail. To load the iWARP driver we need to load the NIC driver & core RDMA drivers first: [root@host]# modprobe cxgb4 [root@host]# modprobe iw_cxgb4 [root@host]# modprobe rdma_ucm b. Unloading the driver ----------------------- To unload the iWARP driver, run the following command: [root@host]# rmmod iw_cxgb4 NOTE: Set Chelsio driver option for MPI connection changes. Give the below command on all systems: [root@host]# echo 1 > /sys/module/iw_cxgb4/parameters/peer2peer For more information on iWARP and MPI usage please refer to the User Guide. IMPORTANT:On RHEL6.2,SLES11sp2 and later distributions, openmpi-1.4.3 can cause IMB benchmark stalls due to a shared memory BTL issue. This issue is fixed in openmpi-1.4.5 and later releases. Hence, it is recommended that that you download and install the latest stable release from Open MPI's official website, http://www.open-mpi.org ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 4.5. WD-UDP ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 4.5.1. Compiling & Loading Chelsio WD-UDP Drivers ------------------------------------------------- InfiniBand modules from the OFED package should be loaded before proceeding. Load the cxgb4, iw_cxgb4 and rdma_ucm drivers: [root@host]# modprobe cxgb4 [root@host]# modprobe iw_cxgb4 [root@host]# modprobe rdma_ucm 4.5.2. Unloading the driver ---------------------------------------- a. To unload the iWARP driver. [root@host]# rmmod iw_cxgb4 4.5.3. Configuring WD-UDP ---------------------------------------- a. The libcxgb4_sock interface requires a configuration file on each node that defines which UDP port numbers will be accelerated and which T5/T4 interface it will use. The configuration file to use is /etc/libcxgb4_sock.conf, and here is an example: [root@host]# cat /etc/libcxgb4_sock.conf Syntax: endpoint { attributes } ... where attributes include: interface = interface-name port = udp-port-number E.g: endpoint {interface=eth2 port=8888} endpoint {interface=eth3 port=9999} The above file defines 2 accelerated endpoints, port 8888 which will use eth2, and port 9999 which will use eth3. b. Currently the only way to enable WD-UDP sockets is to preload libcxgb4_sock.so when running the desired udp application. E.g. for running netperf: server: [root@host]# LD_PRELOAD=libcxgb4_sock.so netserver -p OR [root@host]# wdload netserver -p NOTE:If you encounter error while using wdload on kernels built on RHEL 5.x distribution, then run the above command as : [root@host]# NUMA=0 wdload netserver -p client: [root@host]# LD_PRELOAD=libcxgb4_sock.so netperf -H -p -t UDP_RR -- -P, OR [root@host]# wdload netperf -H -p -t UDP_RR -- -P, The WD-UDP endpoint is the port number registered for the respective T5/T4 interface in the /etc/libcxgb4_sock.conf file. NOTE: i. In WD-UDP only one application can be run per T5/T4 device per udp port number. For running 2 concurrent netperf UDP_RR tests, each must use a unique UDP port number or T5/T4 device. ii. In order to offload IPv6 UDP sockets, please select "low latency networking" as T5/T4 configuration tuning option during installation. iii. Jumbo frames of 9000B are supported only on kernel 2.6.32 and above. E.g.: endpoint {interface=eth2 port=8888} endpoint {interface=eth2 port=9000} The above file defines 2 accelerated endpoints, port 8888 and port 9000 both will be used by eth2 only. i. install.py script will automatically uninstall the previous versions of OFED and Chelsio WD-UDP. ii. To install/uninstall you will require root privileges. iii. For more information on WD-UDP configuration please refer to the User Guide. ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 4.6. WD-TOE ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 4.6.1. Loading the module ========================================== The drivers must be loaded by the root user. Any attempt to load the drivers as a regular user will fail. Run the following commands to load the drivers: [root@host]# modprobe cxgb4 [root@host]# modprobe t4_tom 4.6.2. Unloading the module ========================================== Reboot the system to unload the driver. 4.6.3. Example ========================================== To run Netperf application with WD-TOE library. i. Start netserver at the PEER, using the following command: [root@host]# PROT=TCP wdload netserver -D -4 ii. On the Test Machine, run the following command to run netperf application. [root@host]# PROT=TCP wdload netperf -H -t TCP_RR -l 10 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 4.7. iSCSI PDU Offload Target ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 4.7.1. Loading the module ========================================== To load the module for RHEL distributions, run modprobe as follows: [root@host]# modprobe chiscsi_t4 For SLES11 SP1: [root@host]# modprobe chiscsi_t4 --allow-unsupported 4.7.2. Obtaining the iSCSI software license ============================================= The iSCSI PDU offload software provided in the Unified Wire package is an evaluation copy fully functional for a period of 30-days. After the trial period, a license key file is required to continue using the software. The license is tied to a selected NIC interface in the machine.The license file will be generated depending on your requirement for a Chelsio iSCSI Target. a. Prepare License information file ----------------------------------- A license information file is required for each license. Run the following command to obtain the license information file: [root@host]# chinfotool The chinfotool scans and lists all the NICs in the system, and prompts the user to select a NIC to which the keyfile will be tied. At the end, it prints out a summary of the license file. For example: ++++++++++++++++++++++++++++++++++++ [root@host]# chinfotool Scanning System for network devices.... License key will be tied to any of the following interfaces. Please select the interface 1. Interface eth1 with INTEL Adapter Linkspeed is 1000 Mbps/s MAC is: 00:30:48:00:00:10. 2. Interface eth2 with CHELSIO Adapter Linkspeed is 10000 Mbps/s MAC is: 00:07:43:00:00:10. ENTER NUMBER:2 SUMMARY: Interface with MAC Address 00:07:43:00:00:10 (10000 Mbps) is selected Please send info file "/etc/xyz_chelsio_infofile" to Chelsio Communications at support@chelsio.com Chelsio Support will send back the keyfile. The keyfile unlocks the software as long as the above interface is present ++++++++++++++++++++++++++++++++++++ For the selected NIC, the following information is gathered: -PCI vendor ID -MAC address -Link speed The above information is used strictly by Chelsio Communications for the iSCSI license management. Chelsio Communications Inc. does not share, sell, rent, or lease the information to any third party. The generated file hostname_chelsio_infofile needs to be sent back to Chelsio at support@chelsio.com as instructed by the chinfotool. NOTE:Please make sure that the selected NIC card is present in the system all the time. If it is removed the license will be invalid and you will need to use chinfotool to rescan the system and get a new keyfile from our support team. b. Install the license file received from Chelsio ------------------------------------------------- Once the information file (generated by chinfotool in the previous step) is processed by Chelsio, the user will receive a license file tailored to the system. Copy the file to /etc/chelsio-iscsi/chiscsi.key [root@host]# cp -f chiscsi.key /etc/chelsio-iscsi/chiscsi.key Now the iSCSI software is ready for use. 4.7.3. Unloading the module ============================ Use the following command to unload the module: [root@host]# rmmod chiscsi_t4 NOTE:i. While using rpm-tar-ball for installation a. Uninstallation will result into chiscsi.conf file renamed into chiscsi.conf.rpmsave. b. It is advised to take a backup of chiscsi.conf file before you do an uninstallation and installation of new/same unified wire package. As re-installing/upgrading unified-wire package may lead to loss of chiscsi.conf file. ii. Installation/uninstallation using source-tar-ball will neither remove the conf file nor rename it. It will always be intact. However it's recommended to always take a backup of your configuration file for both methods of installation. ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 4.8. iSCSI PDU Offload Initiator ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 4.8.1. Loading/Unloading the cxgb4i driver ======================================== a. Loading the driver ---------------------- The driver must be loaded by the root user. Any attempt to loading the driver as a regular user will fail. Load cxgb4i driver using the following command: [root@host]#modprobe cxgb4i The cxgb4i module registers a new transport class "cxgb4i". On SLES11sp1, load the driver with the '--allow' option: [root@host]#modprobe cxgb4i --allow If loading of cxgb4i displays "unknown symbols found" error in dmesg, then kill iscsid, unload all open iSCSI modules and related/dependent modules (like multipath, bnx2i etc) and then load cxgb4i driver. NOTE:i. Sometimes open-iSCSI fails to compile because of dependencies on libopenssl-devel (for SLES11SP1) or openssl-devel(for RHEL 6) packages, so install these packages first and then install the Chelsio Unified Wire package (this openssl-devel dependency will not be seen while using rpm.tar.gz to install package). ii. On SLES, if source package is used and make && make install doesn't install cxgb4i package then please follow these steps to configure your source tree and then re-run make && make install:- [root@host]# cd /usr/src/linux [root@host]# make clean [root@host]# make oldconfig [root@host]# make scripts If loading the driver fails, then follow the steps mentioned below and try again. i. Run the following command to view all the loaded iSCSI modules [root@host]# lsmod | grep iscsi ii. Now, unload them using the following command: [root@host]# rmmod b. Unloading the driver -------------------------- [root@host]#rmmod cxgb4i [root@host]#rmmod libcxgbi ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 4.9. FCoE Full Offload Initiator ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 4.9.1. Loading/Unloading Driver ======================================== a. Loading the driver ------------------------------ The driver must be loaded by the root user. Any attempt to load the driver as a regular user will fail. To load the driver, execute the following command: [root@host]# modprobe csiostor NOTE:To load the driver on SLES11sp1 and SLES11sp2, execute the following command: [root@host]# modprobe csiostor --allow b. Unloading the driver ------------------------------ To unload the driver, execute the following command: [root@host]# modprobe -r csiostor NOTE: If multipath services are running, unload of FCoE driver is not possible. Stop the multipath service and then unload the driver. 4.9.2. Configuring the switch and Troubleshooting =============================================== Please refer the User's Guide. ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 4.10. Offload Bonding driver ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 4.10.1. Loading/Unloading Driver ======================================== The driver must be loaded by the root user. Any attempt to loading the driver as a regular user will fail. a. Loading the driver ------------------------------ To load the Bonding driver (with offload support) [root@host]# modprobe bonding b. Unloading the driver ------------------------------ To unload the bonding driver. [root@host]# rmmod bonding c. Offloading TCP traffic over a bonded interface ------------------------------------------------- To offload TCP traffic over a bonded interface, use the following method: i. Load the network driver with TOE support. [root@host]# modprobe t4_tom ii. Create a bonded interface [root@host]# modprobe bonding mode=1 miimon=100 iii. Bring up the bonded interface and enslave the interfaces to the bond [root@host]# ifconfig bond0 up [root@host]# ifenslave bond0 ethX ethY iv. Assign IPv4/IPv6 address to the bonded interface [root@host]# ifconfig bond0 X.X.X.X/Y All TCP traffic will be offloaded over the bonded interface now. 4.10.2. Network Device Configuration ======================================== Please refer to the operating system documentation for administration and configuration of network devices. NOTE:Some operating systems may attempt to auto-configure the detected hardware and some may not detect all ports on a multi-port adapter. If this happens, please refer to the operating system documentation for manually configuring the network device. ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 4.11. UDP Segmentation Offload and Pacing ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 4.11.1. Loading/Unloading Driver ======================================== The driver must be loaded by the root user. Any attempt to load the driver as a regular user will fail. a. Loading the driver ------------------------------ Run the following commands to load the driver: [root@host]# modprobe cxgb4 [root@host]# modprobe t4_tom Though normally associated with the Chelsio TCP Offload engine, the t4_tom module is required in order to allow for the proper redirection of UDP socket calls. b. Unloading the driver ------------------------------ Reboot the system to unload the driver. +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 4.12. Offload IPv6 driver ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 4.12.1. Loading/Unloading Driver ======================================== IPv6 must be enabled in your system (enabled by default) to use the Offload IPv6 feature.Also, Unified Wire package must be installed with IPv6 support (see Chapter I. Chalsio Unfied Wire in User's Guide for more information). After installing Unified Wire package and rebooting the host, load the NIC (cxgb4) and TOE (t4_tom) drivers. The drivers must be loaded by the root user. Any attempt to load the drivers as a regular user will fail. a. Loading NIC & TOE drivers ------------------------------ [root@host]# modprobe cxgb4 [root@host]# modprobe t4_tom All the IPv6 traffic will be offloaded now. b. Unloading the driver ------------------------------ i. To unload the NIC driver. [root@host]# rmmod cxgb4 ii. To unload the TOE driver. Please reboot the system to unload the TOE driver. ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 4.13. Bypass Driver ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 4.13.1. Loading/Unloading Driver ======================================== The driver must be loaded by the root user. Any attempt to load the driver as a regular user will fail. a. Loading the driver ------------------------------ Run the following command to load the Bypass driver: [root@host]# modprobe cxgb4 b. Unloading the driver ------------------------------ Run the following command to unload the Bypass driver: [root@host]# rmmod cxgb4 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 4.14. WD Sniffing and Tracing ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ The objective of these utilities (wd_sniffer and wd_tcpdump_trace) is to provide sniffing and tracing capabilities by making use of T4's hardware features. Sniffer- Involves targeting specific multicast traffic and sending it directly to user space. Tracer - All tapped traffic is forwarded to user space and also pushed back on the wire via the internal loop back mechanism In either mode the targeted traffic bypasses the kernel TCP/IP stack and is delivered directly to user space by means of a RX queue which is defined by the register MPS_TRC_RSS_CONTROL. For more information,please refer to the User Guide. ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 4.15. Classification and Filtering ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Classification and Filtering feature enhances network security by controlling incoming traffic as they pass through network interface based on source and destination addresses, protocol, source and receiving ports, or the value of some status bits in the packet. 4.15.1. Usage =============== i. Creating Filter Rules ------------------------- Network driver (cxgb4) must be installed before setting the filter rule. If you haven't done already, run the Unified Wire Installer with the appropriate T4 configuration tuning option to install the Network Driver. Next, run the following command to load the network driver: [root@host]# modprobe cxgb4 Now, create filter rules using cxgbtool: [root@host]# cxgbtool ethx filter action [pass/drop/switch] Where, ethX: Chelsio interface index: positive integer set as filter id action: Ingress packet disposition pass: Ingress packets will be passed through set ingress queues switch: Ingress packets will be routed to an output port with optional header rewrite. drop: Ingress packets will be dropped. ii. Listing Filter Rules -------------------------- To list previously set filters, run the following command: [root@host]# cxgbtool ethX filter show iii. Removing Filter Rules ---------------------------- To remove a filter, run the following command with the corresponding filter rule index [root@host]# cxgbtool ethX filter index NOTE:For more information on additional parameters, refer to cxgbtool manual by running the "man cxgbtool" command. ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 4.16. Traffic Management ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Traffic Management capabilities built-in to Chelsio T5/T4 CNAs can shape transmit data traffic through the use of sophisticated queuing and scheduling algorithms built-in to the T5/T4 ASIC hardware which provides fine-grained software control over latency and bandwidth parameters such as packet rate and byte rate. 4.16.1. Usage =============== i. Traffic Management of Non-Offloaded Connections --------------------------------------------------- The following example demonstrates the method to rate limit all TCP connections on class 0 to a rate of 300 Mbps for Non-offload connections: a. Load the Network driver in NIC mode. [root@host]# modprobe cxgb4 b. Bind connections with destination IP address 192.168.5.3 to NIC TX queue 3 [root@host]# tc qdisc add dev eth0 root handle 1: multiq [root@host]# tc filter add dev eth0 parent 1: protocol ip prio 1 u32 match ip dst 192.168.5.3 action skbedit queue_mapping 3 c. Bind the NIC TX queue to class 0 [root@host]# cxgbtool eth0 sched-queue 3 0 d. Set the appropriate rule for class 0 [root@host]# cxgbtool eth0 sched-class params type packet level cl-rl mode class rate-unit bits rate-mode absolute channel 0 class 0 max-rate 300000 pkt-size 1460 ii. Traffic Management of Offloaded Connections ------------------------------------------------- The following example demonstrates the method to rate limit all TCP connections on class 0 to a rate of 300 Mbps for offloaded connections: a. Load the Network driver in TOE mode [root@host]# modprobe t4_tom b. Create a new policy file (say new_policy_file) and add the following line to associate connections with the given scheduling class: src host 102.1.1.1 => offload class 0 c. Compile the policy file using COP [root@host]# cop -d -o d. Apply the COP policy: [root@host]# cxgbtool eth0 policy e. Set the appropriate rule for class 0 [root@host]# cxgbtool ethX sched-class params type packet level cl-rl mode class rate-unit bits rate-mode absolute channel 0 class 0 max-rate 300000 pkt-size 1460 iii. Traffic Management of Offloaded Connections with Modified Application ---------------------------------------------------------------------------- The following example demonstrates the method to rate limit all TCP connections on class 0 to a rate of 300 Mbps for for offloaded connections with modified application. a. Load the Network driver in TOE mode. [root@host]# modprobe t4_tom b. Modify the application as mentioned in the Configuring Traffic Management section in the User's Guide. c. Set the appropriate rule for class 0 [root@host]# cxgbtool ethX sched-class params type packet level cl-rl mode class rate-unit bits rate-mode absolute channel 0 class 0 max-rate 300000 pkt-size 1460 NOTE:For more information on additional parameters, refer cop manual by running the "man cxgbtool" command. ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 4.17. Unified Wire Manager ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Note:This release of Chelsio Unified Wire doesn't support the Web-based management station (Web GUI) component. Please use the CLI client to manage and configure agents. 4.17.1. Installation: ===================== Chelsio Unified Wire has been designed to install Unified Wire Manager (UM) by default. All the three UM components, i.e. Management Agent, Client and Station, will be installed on selecting any of the Terminator 4/Terminator 5 configuration tuning options during installation. Hence, no separate installation is required. 4.17.2. Management Station Configuration : ========================================== i. Running Management Station on RHEL 6.x ----------------------------------------- a. Start/Restart Apache httpd daemon: [root@host]# service httpd start/restart b. Start/Restart the Management Station: [root@host]# /etc/init.d/chelsio-mgmtstd start/restart ii. Running Management Station on SLES11SP2 --------------------------------------------- a. On SLES11SP2, Management Station needs to be configured before running. Hence, execute the following commands and provide valid inputs. [root@host]# cd /etc/apache2 [root@host]# openssl genrsa -des3 -out server.key 1024 [root@host]# openssl req -new -key server.key -out server.csr [root@host]# cp server.key server.key.org [root@host]# openssl rsa -in server.key.org -out server.key [root@host]# openssl x509 -req -days 365 -in server.csr -signkey server.key -out server.crt [root@host]# cp server.crt ./ssl.crt [root@host]# cp server.key ./ssl.key b. Start/Restart Apache services [root@host]# rcapache2 start/restart c. Start/Restart the Management Station: [root@host]# /etc/init.d/chelsio-mgmtstd start/restart 4.17.3. Un-Installation: ========================= Use the following query command to determine the name of the agent/client RPM [root@host]# rpm -qa | grep uwire Now,Uninstall the RPM using the following syntax: [root@host]# rpm -e 4.17.4. Verifying Agent/Client status: ======================================= i. Use following command to check whether Agent is running. [root@host]# /etc/init.d/Chelsio-uwire_mgmtd status ii. Use the following query command to determine if Management Client is installed: [root@host]# chelsio_uwcli -V The above query should confirm that Management Client is installed by displaying a similar result: Unified Manager client CLI version : 2.x.yy 4.17.5. Agent/Station Start/Stop/restart: ========================================== - Use following command to start, stop or restart Agent: [root@host]# /etc/init.d/Chelsio-uwire_mgmtd - Use following command to start, stop or restart Management station: [root@host]# /etc/init.d/chelsio-mgmtstd 4.17.6. Client Usage: ====================== a. Use the following commands to view Management client Help file. [root@host]#chelsio_uwcli /? NOTE: For a detailed explanation regarding usage and configuration of various UM components, please refer the User's Guide. ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 4.18. Unified Boot Software ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 4.18.1. Unified Boot Option ROM ================================== IMPORTANT: - ONLY uEFI 2.1 supported. - Any other uEFI version is NOT SUPPORTED and may render your system unusable. - Both Legacy and uEFI environments are supported on T4 adapters. Whereas, only Legacy PXE is currently supported on T5 adapters. i. Flashing configuration file, firmware & option ROM ========================================================= Chelsio legacy flash utility (cfut4.exe) is used to program the PXE Option ROM image onto the Chelsio CNAs. Example # 1: This example assumes you are using a USB flash drive as a storage media for the necessary files. Follow the steps below: a) Create a DOS bootable USB flash drive. b) Create CHELSIO folder on USB flash drive. c) If you haven't done already, download ChelsioUwire-x.x.x.x.tar.gz from Chelsio Download Center, service.chelsio.com d) Untar the downloaded package. e) Change your working directory depending on the adapter type. For example, for T5 adapters, change your working directory to ChelsioUwire-x.x.x.x/Uboot/OptionROM/T5 f) Copy all the files and place them in the CHELSIO folder created on the USB flash drive. g) Plug-in the USB flash drive in the system on which the Chelsio CNA is installed. h) Reboot the system and enter the system BIOS setup. i) Make the USB flash drive as the primary boot device. Save the changes and exit. j) Once the system boots from the USB flash drive, navigate to the CHELSIO directory: C:\>cd CHELSIO k) Run the following command to list all Chelsio CNAs present in the system. The list displays a unique index for each CNA found. C:\CHELSIO>cfut4 -l l) Delete any previous version of Option ROM flashed on the CNA: C:\CHELSIO>cfut4 -d -xb -xc Here, idx is the CNA index found in step (k) m) Delete any previous firmware and configuration file using the following command: C:\CHELSIO>cfut4 -d -xh -xf n) Run the following command to flash the appropriate configuration file (t4-config.txt for T4 adapters;t5-config.txt for T5 adapters) C:\CHELSIO>cfut4 -d -uh .txt o) Run the following command to flash the appropriate firmware (t4fw-x.x.xx.x.bin for T4 adapters;t5fw-x.x.xx.x.bin for T5 adapters). C:\CHELSIO>cfut4 -d -uf .bin p) Use the following command to flash the appropriate option ROM onto the CNA (cubt4.bin for T4 adapters; cuwlbt4.bin for T5 adapters) C:\CHELSIO>cfut4 -d -ub .bin NOTE: On HP DL160 systems with T4 adapters, please use cubt40.bin instead of cubt4.bin q) Reboot the system for changes to take effect. NOTE: VLAN currently not supported in Legacy and uEFI environments. For more information, please refer the User's Guide. 4.18.2. Driver Update Disk (DUD) ================================== i. Creating Driver Disk for RedHat Enterprise Linux ------------------------------------------------------- a)If you haven't done already, download ChelsioUwire-x.x.x.x.tar.gz from Chelsio Download Center, service.chelsio.com b)Untar the package. c)Change your working directory to the LinuxDUD directory. d)Insert a blank, formatted USB flash drive. e)Depending on the distribution to be installed, copy the corresponding image file to the USB drive.For example, execute the following command for RHEL 6.3 [root@host]# cp Chelsio-DriverUpdateDisk-RHEL6.3-x86_64-x.x.x.x-y.img ii. Creating Driver Disk for Suse Enterprise Linux ----------------------------------------------------- a)If you haven't done already, download ChelsioUwire-x.x.x.x.tar.gz from Chelsio Download Center, service.chelsio.com b)Untar the package c)Insert a blank USB drive. d)Format the USB drive e)Depending on the distribution to be installed, copy the corresponding image file to the USB drive.For example, execute the following command for SLES11sp1: [root@host]# dd if=/root/ChelsioUwire-x.x.x.x/Uboot/LinuxDUD/Chelsio-DriverUpdateDisk-SLES11sp1-x86_64-x.x.x.x-y.img of=/dev/sda NOTE: For more information, please refer User's Guide. IMPORTANT:Please make sure the switch is upgraded to the latest available firmware, before proceeding with the operating system installation. For more information, please refer User's Guide. 5. Support Documentation ================================================================================ The documentation for this release can be found inside the ChelsioUwire-x.x.x.x/docs folder. It contains: - README - Release Notes - User's Guide 6. Customer Support ================================================================================ Please contact Chelsio support at support@chelsio.com for any issues regarding the product. ******************************************************************************** Copyright (C) 2013 Chelsio Communications. All Rights Reserved The information in this document is furnished for informational use only, is subject to change without notice, and should not be construed as a commitment by Chelsio Communications. Chelsio Communications assumes no responsibility or liability for any errors or inaccuracies that may appear in this document or any software that may be provided in association with this document. Except as permitted by such license, no part of this document may be reproduced, stored in a retrieval system,or transmitted in any form or by any means without the express written consent of Chelsio Communications.