**************************************** README **************************************** Chelsio Unified Wire for Windows Version : 6.10.2.0_WIN_006.0.29 Date : 08/13/2018 Overview ================================================================================ Chelsio Unified Wire for Windows provides installation of 64-bit Windows based drivers and tools for Chelsio's Unified Wire adapters. It consists of the following components: - NDIS Function driver (NIC) v6.10.2.0 - NVGRE Offload - VXLAN Task Offload - PacketDirect (Kernel Mode and ARM Moderation) - iWARP (ND) - SMB Direct - Ring Backbone Utility - RDMA/NVGRE concurrent (Mode 2) - NDIS SR-IOV - Guest RDMA (Mode 3) - VMMQ/vRSS - iSCSI Storport Miniport v6.10.2.0 - iSER Initiator v6.10.2.0 - Data Center Bridging (DCB) - Storage Replica (SR) NOTE: Drivers are Azure Stack and WHQL certified. ================================================================================ CONTENTS ================================================================================ - 1. Supported Operating Systems - 2. Supported Cards - 3. How to Use - 4. Support Documentation - 5. Customer Support 1. Supported Operating Systems ================================================================================ The Chelsio Unified Wire software has been developed to run on Windows based platforms. Following is the list of Drivers/Software and supported Windows version(s). |########################|#####################################################| | Windows Version | Driver/Software | |########################|#####################################################| |Server 2016 |NIC,ND,NVGRE,VXLAN,PacketDirect,VMMQ/vRSS, | | |SMB Direct*,Ring,SR,SR-IOV,Guest RDMA,iSCSI,iSER,DCB | |------------------------|-----------------------------------------------------| |10 AU Client |NIC,SMB Direct†^,iSCSI,iSER,DCB | |------------------------|-----------------------------------------------------| |Server 2012 R2 |NIC,ND,NVGRE,SMB Direct†,SR-IOV,iSCSI,DCB | |------------------------|-----------------------------------------------------| |7 Client |NIC | |------------------------|-----------------------------------------------------| |Server 2008 R2 |NIC | |------------------------------------------------------------------------------| * SMB Direct and RDMA/NVGRE concurrent (Mode 2). † SMB Direct Mode 2 not supported. ^ Enterprise and "Pro for Workstations" editions. Other versions have not been tested and are not guaranteed to work. 2. Supported Cards ================================================================================ Following is the list of drivers/software and supported Chelsio adapters: |########################|#####################################################| | Chelsio Adapter | Driver/Software | |########################|#####################################################| |T62100-CR |NIC,NVGRE,VXLAN,PacketDirect,VMMQ/vRSS,SMB Direct*, | | |Ring,SR,SR-IOV,iSCSI,iSER | |------------------------|-----------------------------------------------------| |T62100-LP-CR |NIC,ND,NVGRE,VXLAN,PacketDirect,VMMQ/vRSS,DCB, | | |SMB Direct*,Ring,SR,SR-IOV,Guest RDMA,iSCSI,iSER | |------------------------|-----------------------------------------------------| |T62100-SO-CR |NIC,Ring(NIC only),SR-IOV | |------------------------|-----------------------------------------------------| |T6425-CR** |NIC,NVGRE,VXLAN,SMB Direct*,SR,VMMQ/vRSS,SR-IOV | |------------------------|-----------------------------------------------------| |T6225-CR |NIC,ND,NVGRE,VXLAN,PacketDirect,VMMQ/vRSS,DCB, | | |SMB Direct*,Ring,SR,SR-IOV,Guest RDMA,iSCSI,iSER | |------------------------|-----------------------------------------------------| |T6225-LL-CR |NIC,NVGRE,VXLAN,VMMQ/vRSS,SMB Direct*,Ring,SR,SR-IOV,| | |Guest RDMA,iSCSI,iSER | |------------------------|-----------------------------------------------------| |T580-CR |NIC,ND,NVGRE,VXLAN,PacketDirect,VMMQ/vRSS,DCB, | | |SMB Direct*,Ring,SR,SR-IOV,Guest RDMA,iSCSI,iSER,DCB | |------------------------|-----------------------------------------------------| |T580-LP-CR |NIC,ND,NVGRE,VXLAN,PacketDirect,SMB Direct*,Ring,SR, | | |SR-IOV,Guest RDMA,VMMQ/vRSS,iSCSI,iSER,DCB | |------------------------|-----------------------------------------------------| |T580-SO-CR |NIC,NVGRE,VXLAN,Ring(NIC only),SR-IOV,VMMQ/vRSS | |------------------------|-----------------------------------------------------| |T580-OCP-SO |NIC,NVGRE,VXLAN | |------------------------|-----------------------------------------------------| |T540-CR |NIC,ND,NVGRE,VXLAN,SMB Direct*,SR,SR-IOV,Guest RDMA, | | |VMMQ/vRSS,iSCSI,iSER | |------------------------|-----------------------------------------------------| |T540-LP-CR |NIC,NVGRE,VXLAN,SMB Direct*,iSCSI | |------------------------|-----------------------------------------------------| |T540-BT |NIC,SR-IOV,Guest RDMA,iSCSI | |------------------------|-----------------------------------------------------| |T520-CR |NIC,ND,NVGRE,VXLAN,PacketDirect,SMB Direct*,Ring,SR, | | |SR-IOV,Guest RDMA,VMMQ/vRSS,iSCSI,iSER,DCB | |------------------------|-----------------------------------------------------| |T520-LL-CR |NIC,ND,NVGRE,VXLAN,SMB Direct*,Ring,SR,SR-IOV, | | |VMMQ/vRSS,iSCSI,iSER,DCB | |------------------------|-----------------------------------------------------| |T520-SO-CR |NIC,NVGRE,Ring(NIC only),VXLAN,SR-IOV,VMMQ/vRSS | |------------------------|-----------------------------------------------------| |T520-OCP-SO |NIC,NVGRE,VXLAN,iSCSI,iSER | |------------------------|-----------------------------------------------------| |T520-BT |NIC,ND,NVGRE,VXLAN,SMB Direct*,Ring,SR,SR-IOV, | | |VMMQ/vRSS,iSCSI,iSER | |------------------------------------------------------------------------------| * SMB Direct and RDMA/NVGRE concurrent (Mode 2) ** All 4-ports of T6425-CR adapter will be functional only if PCIe x8 -> 2x PCIe x4 slot bifurcation is supported by the system and enabled in BIOS. Otherwise, only 2-ports will be functional. NOTE: T4 family of adapters are not supported on Windows Server 2016 & 10 AU Client versions. Memory-free Adapters ==================== |########################|#####################################################| | Chelsio Adapter | Driver/Software | |########################|#####################################################| |T6225-SO-CR |NIC,SR-IOV,SMB Direct*^,iSCSI Initiator^,Ring | |------------------------|-----------------------------------------------------| * SMB Direct and RDMA/NVGRE concurrent (Mode 2) ^ Limited number of offload connections supported. 3. How to Use ================================================================================ ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Chelsio Unified Wire ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Pre-requisites ============== Please make sure that the following requirements are met before installation: - Latest Microsoft Visual C++ Redistributable Packages for Visual Studio (x86 and x64) are required if you are planning to use installer. - Latest Windows OS updates should be installed on Windows 7 Client and 2008 R2 Server. Installation ============ Chelsio Unified Wire can be installed using the Installer or the Zip package. Refer the appropriate section below depending on the method of installation. IMPORTANT: If you are planning to install Unified Wire in a cluster, it should be done on all the members. Using different versions of Unified Wire is not supported. Unified Wire Installer (Windows GUI Machine) -------------------------------------------- Enabling Test Signing ---------------------- In case the drivers in the package are not WHQL certified, follow the steps mentioned below to enable test signing on your system: a) Goto Start->Run command option, enter "cmd" and press OK. This will open the command prompt utility. b) Run the following command: C:\Users\Administrator> bcdedit /set testsigning on c) Reboot the machine for the changes to take effect. GUI Based Installation ---------------------- a) Run the "ChelsioUwire_x.x.x.x_WIN_yyy.y.yy.exe" application. b) Click the Next button for the Chelsio End User License Agreement Window. c) Select the radio button "I accept the terms of the license agreement" and click on "Next" button. d) Select "Complete" for complete package installation or else select "Custom" radio button to customize the installation. NOTE: Complete option will not install iSCSI Storport Miniport and iSER Initiator drivers. Use the Custom option to install them. VBD and NIC drivers are required for iSER Initiator to work, whereas iSCSI Storport Miniport needs VBD to function. e) Now clicking on "Install" button will start installation with the selected options. f) Click on the "Finish" button to exit from the installer. CLI Based (Silent) Installation ------------------------------- To install drivers/software using this feature, open command prompt and execute the following command: C:\Users\Administrator>ChelsioUwire_x.x.x.x_WIN_yyy.y.yy.exe -in E.g.: C:\Users\Administrator>ChelsioUwire_x.x.x.x_WIN_yyy.y.yy.exe -in all The above command will install all drivers, except iSCSI Storport Miniport and iSER Initiator. To install a particular driver (custom installation), execute the following command: C:\Users\Administrator>ChelsioUwire_x.x.x.x_WIN_yyy.y.yy.exe -in NIC The above command will install the Network driver. To add a driver to an existing list of already installed drivers, use the following command: C:\Users\Administrator>ChelsioUwire_x.x.x.x_WIN_yyy.y.yy.exe -add To know more about other parameters and options, execute the following command: C:\Users\Administrator>ChelsioUwire_x.x.x.x_WIN_yyy.y.yy.exe -help The above command will create a help file, ChelsioUwire_x.x.x.x_WIN_yyy.y.yy_help.txt, in the present working directory which contains the complete list of command line syntax required for performing all the necessary CLI operations. NOTE: A log file, ChelsioUwire_x.x.x.x_WIN_yyy.y.yy.log, is created in the same directory which keeps a record of all the commands executed and their results. Zip Package ----------- Enabling Test Signing --------------------- In case of non-WHQLed drivers, please follow the steps mentioned below to enable test signing before installing Unified Wire: a) Open command prompt and run the following command to enable test signing: C:\Users\Administrator>bcdedit /set testsigning on b) Change your working directory to "ChelsioUwire_x.x.x.x_WIN_yyy.y.yy\Selfsign\" and run the following commands: C:\ChelsioUwire_x.x.x.x_WIN_yyy.y.yy\Selfsign> .\certmgr.exe -add .\chelsiocert1.cer -s -r LocalMachine root -all C:\ChelsioUwire_x.x.x.x_WIN_yyy.y.yy\Selfsign> .\certmgr.exe -add .\chelsiocert1.cer -s -r LocalMachine trustedpublisher -all c) Reboot the machine for the changes to take effect. Drivers location ---------------- The driver files should be manually picked from: - ChelsioUwire_x.x.x.x_WIN_yyy.y.yy\drivers\Win10 for Server 2016 and 10 AU Client. - ChelsioUwire_x.x.x.x_WIN_yyy.y.yy\drivers\Win7 for Server 2012 R2, 2008 R2 and 7 Client. Manual Installation ------------------- In Manual method, the driver components will have to be individually installed using the Device Manager. Please refer User's Guide for instructions. The components must be installed strictly in the order specified in the document. Any other order is not supported. Offline Windows Image --------------------- You can use Windows Deployment Image Servicing and Management (DISM) tool to add Chelsio driver components (.inf files) to an offline Windows image. For more information, visit the following links: - http://technet.microsoft.com/en-us/library/hh825070.aspx - http://blogs.technet.com/b/heyscriptingguy/archive/2012/09/27/use-the-powershell-dism-cmdlets-to-manage-windows-8.aspx Firmware Update =============== The driver will auto-load firmware (v1.20.8.0) if an update is required. The firmware version can be verified using cxgbtool: C:\Users\Administrator>cxgbtool nic0 debug version In case of installer, the firmware binaries will be copied to \ChelsioUwire\Firmware\ folder during installation. For the zip package, the binaries will be present in ChelsioUwire_x.x.x.x_WIN_yyy.y.yy\firmware\ folder. Driver Update ============= Chelsio Unified Wire can be updated using Windows update, installer or zip package. Please refer the user's guide for a detailed explanation regarding each method. IMPORTANT: If you are planning to upgrade Unified Wire in a cluster, it should be done on all the members. Using different versions of Unified Wire is not supported. Configuring 40G Adapters ======================== The "chelsio_adapter_config.ps1" configuration script will configure the Chelsio 40G adapters in any of the following three operational modes: - DEFAULT (2X40G): This is the default mode of operation where each port functions as 40Gbps link. The port nearest to the motherboard will appear as the first network interface (Port 0). - SPIDER (4X10G): In this mode, port 0 functions as 4 10Gbps links and port 1 is disabled. - QSA (2X10G): This mode adds support for QSA (QSFP to SFP+) modules, enabling smooth, cost-effective, connections between 40 Gigabit Ethernet adapters and 1 or 10 Gigabit Ethernet networks using existing SFP+ based cabling. The port farthest from the motherboard will appear as the first network interface (Port 0). Windows GUI Machine ------------------- a) In case of Installer, the configuration script and cxgbtool.exe will be copied to "\Windows\System32" during installation. If you are using the zip package, copy cxgbtool.exe from ChelsioUwire_x.x.x.x_WIN_yyy.y.yy folder to "\Windows\System32" and change your working directory to "ChelsioUwire_x.x.x.x_WIN_yyy.y.yy\Adapter Configuration". b) Open PowerShell with administrative privileges. c) Run the adapter configuration script and enter the index of the 40G adapter for which the configuration needs to be updated. Hit [Enter]. Installer: PS C:\Users\Administrator>chelsio_adapter_config.ps1 Zip Package: PS D:\ChelsioUwire_x.x.x.x_WIN_yyy.y.yy\Adapter Configuration> .\chelsio_adapter_config.ps1 d) Select "Port settings" (option 3) as the configuration type. Hit [Enter]. e) Select the operation mode and then enter "y" to confirm. Hit [Enter]. f) Open Device Manager, click on System Devices and right-click on the 40G Chelsio adapter selected in step (c). g) Select Disable and then Enable for changes to take effect. Un-installation =============== Installer --------- GUI --- a) Run the "ChelsioUwire_x.x.x.x_WIN_yyy.y.yy.exe" application. b) Select the radio button "Remove" and click "Next". c) Click on the "Finish" button to exit from the installer. NOTE: This method of uninstallation is possible only if the drivers were installed using Unified Wire Installer. CLI --- To uninstall all the drivers, execute the following command: C:\Users\Administrator>ChelsioUwire_x.x.x.x_WIN_yyy.y.yy.exe -un all To uninstall a particular driver, execute the following command: C:\Users\Administrator>ChelsioUwire_x.x.x.x_WIN_yyy.y.yy.exe -rm NOTE: Uninstalling Unified Wire package using the above method will not uninstall Unified Wire Manager. You will have to manually uninstall it using "Programs and Features" in the Control Panel. Zip Package ----------- If the driver components were installed using the zip package, they will have to be uninstalled manually. Please refer to User's Guide for step-by-step instructions. IMPORTANT: The driver components must be uninstalled manually and strictly in the order specified in the document. Any other order is not supported. ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ NDIS Function ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Chelsio’s Terminator series of Unified Wire adapters provide extensive support for NIC operation, including all stateless offload mechanisms for both IPv4 and IPv6. NVGRE Offload ============= Chelsio’s Terminator based adapters are uniquely capable of offloading the processing of NVGRE encapsulated frames such that all stateless offloads(checksums, LSO, VMQ, RSS and filtering) are preserved, resulting in significant performance benefits. This feature is enabled by default. Configure the relevant customer and provider network settings on the host. VXLAN Task Offload ================== Chelsio’s Terminator based adapters are capable of offloading the processing of VXLAN encapsulated frames such that all stateless offloads (checksums, LSO, VMQ, RSS and filtering) are preserved, resulting in significant performance benefits. This feature is enabled by default. PacketDirect ============= PacketDirect Provider Interface (PDPI), an extension to the existing NDIS miniport driver model, gives applications complete control over buffers, poll processors, and sending and receiving packets over a miniport adapter. Chelsio’s Unified Wire adapters offer extensive support for this feature leading to a higher throughput performance. This feature is disabled by default. NOTE: Please refer User's Guide to configure the PacketDirect feature. VMMQ/vRSS ========= Network throughput of a multi-core virtual machine is limited by the processing power of a single virtual processor, since only one processor handles all the interrupts from a virtual network adapter. Virtual Machine Multi-Queue (VMMQ) or Virtual Switch Receive-side Scaling (vRSS) solves this bottleneck by distributing the network traffic across multiple virtual processors. If NDIS function driver is installed using Unified Wire installer, VMMQ/vRSS will be enabled by default. However, if the driver is installed manually (using zip package), the feature will be disabled. Use NDIS Miniport Driver Parameters (Device manager -> Chelsio Network Adapter Properties -> Advanced Tab -> Virtual Switch RSS) to enable it. Additionally, VMMQ/vRSS must be enabled for each Virtual Machine using the below command on the host: PS C:\Users\Administrator> Set-VMNetworkAdapter -VMName -VrssEnabled:$true -VmmqEnabled:$true -VmmqQueuePairs Example: PS C:\Users\Administrator> Set-VMNetworkAdapter -VMName VM -VrssEnabled:$true -VmmqEnabled:$true -VmmqQueuePairs 8 Dynamic VMMQ/RSSv2 ------------------ NOTE: Available on Windows Server 2016 version 1709 and later. The latest update, Dynamic VMMQ/RSSv2, makes RSS algorithm dynamic by taking CPU measurement and making balancing decision at DISPATCH level. The feature also avoids taking global locks by changing only the subset of indirection table which affects current CPUs only. The feature will be enabled/disabled depending on how NDIS function driver was installed, as described previously. Additionally, the feature must be enabled for each VM on the host using: PS C:\Users\Administrator> Set-VMNetworkAdapter -VMName -VrssEnabled 1 -VmmqEnabled 1 -VmmqQueuePairs -VrssQueueSchedulingMode Dynamic ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ iWARP (ND) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Chelsio Unified Wire Installer registers Chelsio iWARP provider automatically during installation. In case of zip package, run the following command to register: C:\Users\Administrator>chinstallsp.exe -i The iWARP provider should show up using: C:\Users\Administrator>chinstallsp.exe -l You should see a similar output: 0000001019 - Chelsio Network Direct provider ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ SMB Direct ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Enabling SMB Direct =================== Windows Server 2016/Server 2012 R2 ---------------------------------- SMB Direct is installed and enabled by default on Windows Server 2016/Server 2012 R2. RDMA functionality on Chelsio adapters will be enabled after installing the driver from the package. Windows 10 AU Client -------------------- On Windows 10 AU Client systems, SMB Direct feature is installed but disabled by default. To enable it, follow the steps mentioned below: a) Open Control Panel and click Programs. b) Under Programs and Features, click Turn Windows features on or off. c) In the Windows Features window, scroll down and select the SMB Direct checkbox. d) Click OK. RDMA functionality on Chelsio adapters will be enabled after installing the driver from the package. Configuration -------------- You can use the PowerShell command "Get-NetAdapterRdma" to get a list of RDMA capable interfaces in your system. At least one adapter should indicate "RDMA Capable" true to take advantage of SMB RDMA capability. The next step is to configure your network so that SMB server and client machines can communicate with each other. Please configure the Microsoft firewall to allow traffic between the SMB clients and server. RDMA/NVGRE concurrent (Mode 2) ============================== In this mode, you can run RDMA traffic using a virtual switch. This feature is enabled by default in the driver, but must be enabled on the Hyper-V Virtual Ethernet adapter. NOTE: For more information on the features mentioned above, please refer to User's Guide. Storage Replica =============== Currently, Unified Wire supports Server to Server replication mode. In this mode, data is synced between two servers and each server keeps an identical copy of the same volume. For more information on how to setup and configure in this mode, please refer to Microsoft’s official documentation at https://technet.microsoft.com/en-us/windows-server-docs/storage/storage-replica/server-to-server-storage-replication HMA === The Host Memory Access(HMA) implements a bridge between the Memory Arbiter (MA) and the PCIE so that the Host Memory is available to all clients. The translation from the MA memory map to the Host Memory map is performed based on the client number and the address of the transaction, both of which are used as index to look up a 64-bit offset within the Host Memory. Currently 256 IPv4/128 IPv6 offload connections are supported on T6225-SO-CR adapter. You can view the number of connections offloaded by using C:\Users\Administrator> cxgbtool hardware tid_info ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Ring Backbone Utility ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Chelsio Ring Backbone Utility is an easy to use tool developed to configure machines (with Chelsio adapters) in a ring backbone. This replaces the need for a ToR switch to connect a set of servers. Pre-requisites ============== Please ensure that the following requirements are met on each machine in the topology before proceeding with the installation: - There must be an active management network (apart from Chelsio test network) which enables the machines to be accessed to carry out the required configuration. - PowerShell remoting should be enabled using the following command: PS C:Users\Administrator> Enable-PSRemoting -Force - Windows Remote Management(WinRM) should be enabled using the following command: PS C:Users\Administrator> Set-item wsman:localhost\client\trustedhosts -value * - Firewall should be disabled from Control Panel. - All the nodes should have the same user credentials. Installation ============ a) Ensure that Unified Wire is installed on all the machines in the ring backbone (Refer Chelsio Unified Wire Chapter). b) In case of Installer, the "backbone_cfg.ps1" script and cxgbtool.exe will be copied to "\Windows\System32" during installation. If you are using the zip package, copy "backbone_cfg.ps1" script from "\RingBackbone" and "cxgbtool.exe" from "\drivers" to "\Windows\System32" folder. Configuration ============= IMPORTANT: - Port 1 should be not configured for any other purpose in the operating system. - Only Single port (port 0) is available to the user after configuring ring backbone. a) Connect the machines in ring backbone. Connect port 1 of one machine to the port 0 of next machine such that a closed ring is formed. b) Open PowerShell and run the backbone_cfg.ps1 script on any one of the machines to configure ring backbone. PS C:\Users\Administrator> backbone_cfg.ps1 -Servers -Username -Password -BaseIpAddress c) You should be able to run traffic between the machines/nodes now. Re-configuring Ring Backbone ============================ Ring Backbone must be reconfigured if - any of the driver parameters is changed (driver will be reloaded). - vSwitch is created. - a new VM is added. To reconfigure, run the following command on any node: PS C:\Users\Administrator> backbone_cfg.ps1 -Servers -Username -Password -Reconfigure host1,host2: Only affected hosts should be listed here. Clear Configuration =================== To clear previously set configuration: PS C:\Users\Administrator> backbone_cfg.ps1 -Servers -Username -Password -Clear ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ NDIS SR-IOV ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Pre-requisites ============== SR-IOV should be enabled in the machine. Installation ============ Please ensure that Unified Wire is installed on the host before proceeding. Refer Unified Wire's Installation section for step-by-step instructions. Enabling SR-IOV =============== IMPORTANT: The adapter's configuration should be updated to make use of the SR-IOV feature. You can do this by running the "chelsio_adapter_config.ps1" configuration script. To use the adapter in non-SRIOV scenarios in Windows or with other operating systems, the configuration type must be changed to "NON-SRIOV (Default)". Windows GUI Machine ------------------- In case of Installer, the script will be copied to "\Windows\System32" during installation. If you are using the zip package, copy "cxgbtool.exe" from "ChelsioUwire_x.x.x.x_WIN_yyy.y.yy" folder to "\Windows\System32". Also, change your working directory to "ChelsioUwire_x.x.x.x_WIN_yyy.y.yy\Adapter Configuration". a) Open PowerShell with administrative privileges. b) Check if the machine is SR-IOV capable using the following command: PS C:\Users\Administrator> Get-VMHost | fl *iov* The "IovSupport" field should display "True" c) Run the adapter configuration script and enter the index of the adapter for which the configuration needs to be updated. Hit [Enter]. Installer: PS C:\Users\Administrator> chelsio_adapter_config.ps1 Zip: PS D:\ChelsioUwire_x.x.x.x_WIN_yyy.y.yy\Adapter Configuration> .\chelsio_adapter_config.ps1 d) Select "SRIOV" (option 2) as the configuration type and enter "y" to confirm. Hit [Enter]. e) Reboot the system for the changes to take effect. Host Configuration ================== a) Create a vSwitch with SR-IOV enabled using Chelsio adapter. b) Assign a virtual network adapter with SR-IOV enabled to the VM. c) Bring up the VM. NOTE: For more detailed information, refer User's Guide. Guest (VM) Configuration ======================== a) Install Unified Wire on the Guest (VM): - If the guest has internet access, proceed with the installation as described in the "Software/Driver Installation" section of the "Unified Wire" chapter. - If the guest doesn’t have internet access, create an ISO image of the installer or zip package contents on the host. Open "Hyper-V Manager->VM-> Settings->IDE Controller" and attach the image to the virtual CD/DVD drive of the guest. Proceed with the installation as described in the "Software/Driver Installation" section of the "Chelsio Unified Wire" chapter. b) Verify if SR-IOV was enabled successfully on the host: PS C:\Users\Administrator> Get-VMNetworkAdapter | fl The VFDataPathActive parameter must display "True". This, however, is possible only if the physical port has an active link. To uninstall Unified Wire, please refer Unified Wire's "Un-installation" section. Guest RDMA (Mode 3) =================== In this mode, you can run RDMA traffic on Guest (VM) using a virtual switch. This feature is enabled by default in the driver, but must be enabled on the Microsoft Hyper-V Virtual Ethernet adapter. Please refer User's Guide for step-by-step instructions on how to enable this feature. ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ iSCSI Storport Miniport ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Configuring iSCSI Initiator =========================== Configuring IP -------------- There are two ways to assign IP to Chelsio iSCSI interface: DHCP: NIC driver must be installed (see Software/Driver Installation). Upon installing the driver, DHCP will be enabled by default and the NIC interface will be assigned an IP automatically. The same IP can be used by the iSCSI interface as well. Static: To assign a static IP to the iSCSI interface use the Device Manager, as explained below: a. Open Device Manager, right click on Chelsio iSCSI Initiator and click on Properties. b. Click on Advanced tab and select IP option and add IP. Similarly, add subnet mask and default gateway. Configuring Optional Parameters ------------------------------- Configure the below optional parameters based on your requirement. - The Max PDU Length parameter can be used to improve iSCSI performance. The default value of the parameter (indicated by 0) is set based on the MTU: 1024 for MTU 1500, 8192 for MTU 9000. However, these default values can be overridden by entering the value in increments of 512. - Set the maximum iSCSI queue sets. This parameter can be used to improve performance. - Select MTU and assign value as per requirement. - VLAN is disabled by default. To enable it, click on Vlan Insertion and select value as Enabled. - Assign the required VLAN ID and click OK. Repeat the above mentioned steps to assign these driver properties on other ports. iSCSI Target Discovery and Login ================================ Before target discovery, make sure an IP address, either static or using DHCP, is assigned to the respective Chelsio iSCSI node (see previous section). NOTE: - Connecting to more than 64 targets using single port is currently not supported. - In case of Microsoft target, a maximum of 6 targets can be discovered by default. To increase this, change the maximum iSCSI PDU length to 8192 in Device Manager as explained in the "Configuring iSCSI Initiator" section. Following are the set of instructions to discover the target and login to it using the Chelsio iSCSI interface: a. To start the Initiator configuration, go to Control Panel and click on the iSCSI Initiator option. b. Choose the Discovery tab and click on Discover Portal. c. Click on the Advanced button. d. In the Advanced Settings window, select Chelsio iSCSI interface as the local adapter. e. Select the corresponding Chelsio iSCSI Interface IP as the Initiator IP and click OK. f. Enter the IP address (DNS name not supported) of the target machine and the corresponding port number and click OK on the Discover Target Portal Window. Note that the default port number for iSCSI traffic is 3260. g. Once target portal is added, details like target and initiator machine IP, Port number and Chelsio iSCSI interface IP will be displayed. h. Click on the Targets tab to see the list of targets available, choose a particular target and click on Connect. i. A window pops up showing the Target Name, Click OK. j. After logging in to the target, the state of the target will change from Inactive to Connected. k. After successful login, go to Disk Management and make the respective volume online. HMA === The Host Memory Access(HMA) implements a bridge between the Memory Arbiter (MA) and the PCIE so that the Host Memory is available to all clients. The translation from the MA memory map to the Host Memory map is performed based on the client number and the address of the transaction, both of which are used as index to look up a 64-bit offset within the Host Memory. Currently 256 IPv4/128 IPv6 iSCSI offload connections are supported on T6225-SO-CR adapter. You can view the number of connections offloaded by using C:\Users\Administrator> cxgbtool hardware tid_info NOTE: This is a Beta Release. ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ iSER Initiator ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Configuring iSER Target ----------------------- The iSER target must be configured before establishing connection. For more information, please refer Unified Wire for Linux documentation available at the Chelsio Download Center, service.chelsio.com Configuring iSER Initiator -------------------------- The iSER interface uses the same IP as the NIC interface. If not already done, assign an IP to Chelsio interface as described in the "Assigning IP Address" section in the User Guide, before proceeding. iSER Target Discovery and Login -------------------------------- Please refer the User Guide for step-by-step instructions on how to discover and login to iSER target using the Chelsio iSER interface. ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Data Center Bridging (DCB) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Chelsio network adapters work seamlessly with any legacy switches and do not require the complexity of configuring Data Center Bridging (DCB) protocols either on adapter or network switch. In case DCB is already configured on the switch, the adapter will automatically negotiate the PFC settings configured on the switch. NOTE: Only IEEE configuration supported. Network QoS (Optional) ========================= Network Quality of Service (QoS) is an advanced Windows feature that can be used to distribute bandwidth between different kinds of outgoing traffic. This feature ensures efficient usage of resources and minimizes the impact of bandwidth congestion. The bandwidth percentage can be configured either on switch or host. Enabling QoS on Host -------------------- Network QoS is disabled by default. Use NDIS Miniport Driver Parameters (Device manager -> Chelsio Network Adapter Properties -> Advanced Tab -> NDIS QoS) to enable it. NOTE: Chelsio QoS requires a minimum of 8 cores to work. In case of multiple sockets, where the number of cores per socket is less than 8, Chelsio adapters must be configured to utilize all the available cores before enabling QoS. Run the following command on all Chelsio ports: PS C:\Users\Administrator> Set-NetAdapterRss -Name -MaxProcessorGroup 3 Configuring Bandwidth Allocation -------------------------------- Switch ------- Here is an example of setting bandwidth allocation on a Dell Force10 switch: a) Log in to the switch and enter the configuration mode: Login: admin Password: Force10#enable Force10#configure b) Enable DCB and LLDP: Force10(conf)#dcb enable Force10(conf)#protocol lldp Force10(conf-lldp)#no disable c) Set DCBx version to auto: Force10(conf-lldp)#dcbx version auto d) Specify the intervals between hello packets: Force10(conf-lldp)#hello 5 e) Create a DCB map. For example, here we are configuring 2 groups/classes: - Group 0 has BW of 80% with pfc enabled. - Group 1 had BW of 20% with pfc disabled. - Assigning priority groups: Priority 5-7 are under group 0 and 0-4 are under group 1. Force10(conf)#dcb-map win_dcb Force10(conf-dcbmap-win_dcb)# priority-group 0 bandwidth 80 pfc on Force10(conf-dcbmap-win_dcb)# priority-group 1 bandwidth 20 pfc off Force10(conf-dcbmap-win_dcb)# priority-pgid 1 1 1 1 1 0 0 0 f) Enter interface configuration mode and configure protocol LLDP: Force10(conf)#interface tengigabitethernet 0/17 Force10(conf-if-te-0/17)#protocol lldp Force10(conf-if-te-0/17-lldp)#no disable g) Set DCBx version to auto: Force10(conf-if-te-0/17-lldp)#dcbx version auto h) Apply the DCB-MAP created: Force10(conf-if-te-0/17)#dcb-map win_dcb The host adapter will honor the bandwidth allocation settings configured on the switch. Host ---- Here is an example of setting bandwidth allocation on the host: a) Open PowerShell with administrative privileges and run the following command: PS C:\Users\Administrator> Install-WindowsFeature -Name "Data-Center-Bridging" -IncludeAllSubFeature -IncludeManagementTools b) Create new policy: PS C:\Users\Administrator> New-NetQosPolicy -Name rdma -PriorityValue 5 -NetDirectPortMatchCondition 445 c) Create new rule with bandwidth percentage: PS C:\Users\Administrator> New-NetQosTrafficClass -Name rdma -Priority 5 -Algorithm ETS -BandwidthPercentage 80 -Verbose The above command allocates 80% of the bandwidth to the policy named 'rdma'. To change the percentage of an existing ETS rule: PS C:\Users\Administrator> Set-NetQosTrafficClass -Name rdma -Priority 5 -Algorithm ETS -BandwidthPercentage 10 -Verbose 4. Support Documentation ================================================================================ The documentation for this release can be found in the following directory in the system: Windows Unified Wire Installer (after installation): \ChelsioUwire\docs\ Zip Package: ChelsioUwire_x.x.x.x_WIN_yyy.y.yy\docs\ It contains: - README - Release Notes - User's Guide 5. Customer Support ================================================================================ Please contact Chelsio support at support@chelsio.com for any issues regarding the product. ******************************************************************************** Copyright (C) 2018 Chelsio Communications. All Rights Reserved. The information in this document is furnished for informational use only, is subject to change without notice, and should not be construed as a commitment by Chelsio Communications. Chelsio Communications assumes no responsibility or liability for any errors or inaccuracies that may appear in this document or any software that may be provided in association with this document. Except as permitted by such license, no part of this document may be reproduced, stored in a retrieval system,or transmitted in any form or by any means without the express written consent of Chelsio Communications.