**************************************** README **************************************** Chelsio Unified Wire for Windows Version : 5.0.0.88 Date : 06/26/2017 Overview ================================================================================ Chelsio Unified Wire for Windows provides installation of 64-bit Windows based drivers and tools for Chelsio's Unified Wire adapters. It consists of the following components: - NDIS Function driver (NIC) v6.6.4.0 - NVGRE Offload - VXLAN Task Offload - PacketDirect (Kernel Mode and ARM Moderation) - iWARP (ND) - SMB Direct - RDMA/NVGRE concurrent (Mode 2) - NDIS SR-IOV - Guest RDMA (Mode 3) - VMMQ/vRSS - iSCSI Storport Miniport v6.6.4.0 - Data Center Bridging (DCB) - Storage Replica (SR) - Unified Wire Manager (UM) v2.4.62 - UM for Win PE v2.4.62 NOTE: Drivers are WHQL certified. ================================================================================ CONTENTS ================================================================================ - 1. Requirements - 2. Supported Operating Systems - 3. Supported Cards - 4. How to Use - 5. Support Documentation - 6. Customer Support 1. Requirements ================================================================================ Unified Wire Installer ====================== Latest Microsoft Visual C++ Redistributable Packages for Visual Studio (x86 and x64). NDIS SR-IOV =========== - Chelsio adapter and system with supported platform mentioned in section 2. - SR-IOV should be enabled in the machine. Unified Wire Manager (UM) ========================= Management Agent ---------------- Microsoft .Net Framework 3.5 Management Station ------------------ - Install Python 2.6.6 (32-bit) and ensure that the path to python binary (typically "C:\Python26") is added to PATH system variable. - Install Apache HTTP Server 2.2 with SSL - If the Apache Server is running, stop it before starting the installation process. 2. Supported Operating Systems ================================================================================ The Chelsio Unified Wire software has been developed to run on Windows based platforms. Following is the list of Drivers/Software and supported Windows version(s). |########################|#####################################################| | Windows Version | Driver/Software | |########################|#####################################################| |Server 2016 |NIC,ND,NVGRE,VXLAN,PacketDirect,VMMQ/vRSS, | | |SMB Direct*,SR,SR-IOV,Guest RDMA,iSCSI,DCB | |------------------------|-----------------------------------------------------| |Nano Server 2016 |NIC,VMMQ/vRSS,SMB Direct*,SR-IOV,DCB | |------------------------|-----------------------------------------------------| |10 AU Client |NIC,SMB Direct†,iSCSI,DCB | |------------------------|-----------------------------------------------------| |Server 2012 R2 |NIC,ND,NVGRE,SMB Direct†,SR-IOV,iSCSI,DCB,UM^ | |------------------------|-----------------------------------------------------| |Server 2012 |NIC | |------------------------------------------------------------------------------| * SMB Direct and RDMA/NVGRE concurrent (Mode 2) ^ UM and UM for Win PE † Mode 2 not supported Other versions have not been tested and are not guaranteed to work. 3. Supported Cards ================================================================================ Following is the list of drivers/software and supported Chelsio adapters: |########################|#####################################################| | Chelsio Adapter | Driver/Software | |########################|#####################################################| |T62100-CR |NIC,NVGRE,VXLAN,PacketDirect,VMMQ/vRSS,SMB Direct*, | | |SR,SR-IOV,iSCSI | |------------------------|-----------------------------------------------------| |T62100-LP-CR |NIC,ND,NVGRE,VXLAN,PacketDirect,VMMQ/vRSS,DCB, | | |SMB Direct*,SR,SR-IOV,Guest RDMA,iSCSI | |------------------------|-----------------------------------------------------| |T62100-SO-CR |NIC,SR-IOV | |------------------------|-----------------------------------------------------| |T6425-CR |NIC,NVGRE,VXLAN,SMB Direct*,SR,VMMQ/vRSS,SR-IOV | |------------------------|-----------------------------------------------------| |T6225-CR |NIC,ND,NVGRE,VXLAN,PacketDirect,VMMQ/vRSS,DCB, | | |SMB Direct*,SR,SR-IOV,Guest RDMA,iSCSI | |------------------------|-----------------------------------------------------| |T6225-LL-CR |NIC,NVGRE,VXLAN,VMMQ/vRSS,SMB Direct*,SR,SR-IOV, | | |Guest RDMA,iSCSI | |------------------------|-----------------------------------------------------| |T6225-SO-CR |NIC,SR-IOV | |------------------------|-----------------------------------------------------| |T580-CR |NIC,ND,NVGRE,VXLAN,PacketDirect,VMMQ/vRSS,DCB, | | |SMB Direct*,SR,SR-IOV,Guest RDMA,iSCSI,DCB,UM^ | |------------------------|-----------------------------------------------------| |T580-LP-CR |NIC,ND,NVGRE,VXLAN,PacketDirect,SMB Direct*,SR, | | |SR-IOV,Guest RDMA,VMMQ/vRSS,iSCSI,DCB,UM^ | |------------------------|-----------------------------------------------------| |T580-SO-CR |NIC,NVGRE,VXLAN,SR-IOV,VMMQ/vRSS,UM^ | |------------------------|-----------------------------------------------------| |T580-OCP-SO |NIC,NVGRE,VXLAN,iSCSI | |------------------------|-----------------------------------------------------| |T540-CR |NIC,ND,NVGRE,VXLAN,SMB Direct*,SR,SR-IOV,Guest RDMA, | | |VMMQ/vRSS,iSCSI,UM^ | |------------------------|-----------------------------------------------------| |T520-CR |NIC,ND,NVGRE,VXLAN,PacketDirect,SMB Direct*,SR, | | |SR-IOV,Guest RDMA,VMMQ/vRSS,iSCSI,DCB,UM^ | |------------------------|-----------------------------------------------------| |T520-LL-CR |NIC,ND,NVGRE,VXLAN,SMB Direct*,SR,SR-IOV,VMMQ/vRSS, | | |iSCSI,DCB,UM^ | |------------------------|-----------------------------------------------------| |T520-SO-CR |NIC,NVGRE,VXLAN,SR-IOV,VMMQ/vRSS,UM^ | |------------------------|-----------------------------------------------------| |T520-OCP-SO |NIC,NVGRE,VXLAN,iSCSI | |------------------------|-----------------------------------------------------| |T520-BT |NIC,ND,NVGRE,VXLAN,SMB Direct*,SR,SR-IOV,VMMQ/vRSS, | | |iSCSI | |------------------------------------------------------------------------------| * SMB Direct and RDMA/NVGRE concurrent (Mode 2) ^ UM and UM for Win PE NOTE: T4 family of adapters are not supported on Windows Server 2016 & 10 AU Client versions. 4. How to Use ================================================================================ ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Chelsio Unified Wire ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1. Installation =============== Chelsio Unified Wire can be installed using the Installer or the Zip package. Refer the appropriate section below depending on the method of installation. 1.1. Unified Wire Installer (Windows GUI Machine) ------------------------------------------------- Enabling test signing ---------------------- In case the drivers in the package are not WHQL certified, follow the steps mentioned below to enable test signing on your system: a) Goto Start->Run command option, enter "cmd" and press OK. This will open the command prompt utility. b) Run the following command: C:\Users\Administrator> bcdedit /set testsigning on c) Reboot the machine for the changes to take effect. GUI Based Installation ---------------------- a) Run the "ChelsioUwire-x.x.x.xx.exe" application. b) Click the Next button for the Chelsio End User License Agreement Window. c) Select the radio button "I accept the terms of the license agreement" and click on "Next" button. d) Select "Complete" for complete package installation or else select "Custom" radio button to customize the installation. e) Now clicking on "Install" button will start installation with the selected options. f) After successful installation of the drivers, the Unified Wire Manager installer will be invoked. g) Follow on-screen instructions. h) Click on the "Finish" button to exit from the installer. CLI Based (Silent) Installation ------------------------------- To install drivers/software using this feature, open command prompt and execute the following command: C:\Users\Administrator>ChelsioUwire-x.x.x.xx.exe -in E.g.: C:\Users\Administrator>ChelsioUwire-x.x.x.xx.exe -in all The above command will install all the drivers and UM Agent. To install a particular driver (custom installation), execute the following command: C:\Users\Administrator>ChelsioUwire-x.x.x.x.exe -in NIC The above command will install the Network driver. To add a driver to an existing list of already installed drivers, use the following command: C:\Users\Administrator>ChelsioUwire-x.x.x.x.exe -add To know more about other parameters and options, execute the following command: C:\Users\Administrator>ChelsioUwire-x.x.x.xx.exe -help The above command will create a help file,ChelsioUwire-x.x.x.xx_help.txt, in the present working directory which contains the complete list of command line syntax required for performing all the necessary CLI operations. NOTE: A log file,ChelsioUwire-x.x.x.xx.log, is created in the same directory which keeps a record of all the commands executed and their results. 1.2. Nano Server Installer ========================== a) Download and unzip the driver package Chelsio-NANO-installer-x.x.x.xx.zip b) Connect to Nano Server Machine and copy the package contents. c) Open PowerShell with administrative privileges and change your working directory to Chelsio-NANO-installer-x.x.x.xx d) Run the installer [nanomachine-ip]: PS D:\Chelsio-NANO-installer-x.x.x.xx> .\install.ps1 -in all A log file, "Chelsio-installer.log", containing installation summary will be created in the same folder. NOTE: In case of non-WHQLed drivers, if Nano Server is freshly installed, the installer will enable Test Signing. Reboot the machine and run the installer again. To know more about Installer parameters and options, view the help: [nanomachine-ip]: PS D:\Chelsio-NANO-installer-x.x.x.xx> .\install.ps1 -h 1.3. Zip Package ---------------- Enabling Test Signing --------------------- In case of non-WHQLed drivers, please follow the steps mentioned below to enable test signing before installing Unified Wire: a) Open command prompt and run the following command to enable test signing: C:\Users\Administrator>bcdedit /set testsigning on b) Change your working directory to "ChelsioUwire-x.x.x.xx\Selfsign\" and run the following commands: C:\ChelsioUwire-x.x.x.xx\Selfsign> .\certmgr.exe -add .\chelsiocert1.cer -s -r LocalMachine root -all C:\ChelsioUwire-x.x.x.xx\Selfsign> .\certmgr.exe -add .\chelsiocert1.cer -s -r LocalMachine trustedpublisher -all c) Reboot the machine for the changes to take effect. Manual Installation ------------------- In Manual method, the driver components will have to be individually installed using the Device Manager. Please refer User's Guide for instructions. The components must be installed strictly in the order specified in the document. Any other order is not supported. Offline Windows Image --------------------- You can use Windows Deployment Image Servicing and Management (DISM) tool to add Chelsio driver components (.inf files) to an offline Windows image. For more information, visit the following links: - http://technet.microsoft.com/en-us/library/hh825070.aspx - http://blogs.technet.com/b/heyscriptingguy/archive/2012/09/27/use-the-powershell-dism-cmdlets-to-manage-windows-8.aspx Nano Sever: Driver Installation ------------------------------- Before installing Nano Server, the Chelsio driver components need to be added to the boot image. Please refer to the User's Guide for step-by-step instructions. 2. Firmware Update ================== The driver will auto-load firmware (v1.16.45.0) if an update is required. The firmware version can be verified using cxgbtool: C:\Users\Administrator>cxgbtool nic0 debug version In case of installer, the firmware binaries will be copied to \ChelsioUwire\Firmware\ folder during installation. For the zip package, the binaries will be present in ChelsioUwire-x.x.x.xx\firmware\ folder. For Nano Server Installer, the binaries will be present in Chelsio-NANO-installer-x.x.x.xx\firmware\ folder. 3. Driver Update ================ Chelsio Unified Wire can be updated using Windows update, installer or zip package. Please refer the user's guide for a detailed explanation regarding each method. 4. Configuring 40G Adapters =========================== The "chelsio_adapter_config.ps1" configuration script will configure the Chelsio 40G adapters in any of the following three operational modes: - DEFAULT (2X40G): This is the default mode of operation where each port functions as 40Gbps link. The port nearest to the motherboard will appear as the first network interface (Port 0). - SPIDER (4X10G): In this mode, port 0 functions as 4 10Gbps links and port 1 is disabled. - QSA (2X10G): This mode adds support for QSA (QSFP to SFP+) modules, enabling smooth, cost-effective, connections between 40 Gigabit Ethernet adapters and 1 or 10 Gigabit Ethernet networks using existing SFP+ based cabling. The port farthest from the motherboard will appear as the first network interface (Port 0). Windows GUI Machine ------------------- a) In case of Installer, the configuration script and cxgbtool.exe will be copied to "\Windows\System32" during installation. If you are using the zip package, copy cxgbtool.exe from ChelsioUwire-x.x.x.xx folder to "\Windows\System32" and change your working directory to "ChelsioUwire-x.x.x.xx\Adapter Configuration". b) Open PowerShell with administrative privileges. c) Run the adapter configuration script and select "Windows GUI" (option 1) as the Windows version. Hit [Enter]. Installer: PS C:\Users\Administrator> chelsio_adapter_config.ps1 Zip Package: PS D:\ChelsioUwire-x.x.x.xx\Adapter Configuration> .\chelsio_adapter_config.ps1 d) Enter the index of the 40G adapter for which the configuration needs to be updated. Hit [Enter]. e) Select "Port settings" (option 3) as the configuration type. Hit [Enter]. f) Select the operation mode and then enter "y" to confirm. Hit [Enter]. g) Open Device Manager, click on System Devices and right-click on the 40G Chelsio adapter selected in step (c). h) Select Disable and then Enable for changes to take effect. Nano Server Machine (Nano Server Installer & Zip Package) --------------------------------------------------------- a) Connect to Nano Server Machine. b) In case of Nano Server Installer, the configuration script and cxgbnano.exe will be copied to "\Windows\System32" during installation. If you are using the zip package, copy cxgbnano.exe from ChelsioUwire-x.x.x.xx folder to \Windows\System32 and change your working directory to "ChelsioUwire-x.x.x.xx\Adapter Configuration". c) Run the adapter configuration script and select "Windows Nano Server" (option 2) as the Windows version. Hit [Enter]. Nano Server Installer: [nanomachine-ip]: PS D:\> chelsio_adapter_config.ps1 Zip Package: [nanomachine-ip]: PS D:\ChelsioUwire-x.x.x.xx\Adapter Configuration> .\chelsio_adapter_config.ps1 d) Enter the index of the 40G adapter for which the configuration needs to be updated. Hit [Enter]. e) Select "Port settings" (option 3) as the configuration type. Hit [Enter]. f) Select the operation mode and then enter "y" to confirm. Hit [Enter]. g) Determine the Instance ID of the adapter using the "Get-PnpDevice" command. h) Use the Instance ID obtained from the previous step to Disable and Enable the Adapter for changes to take effect. 5. Mass Deployment (Installer) ============================== Using Unified Wire Installer's Mass Deployment feature, you can flash or erase Option ROM on multiple systems (nodes) simultaneously. The Installer utilizes Unified Wire Manager's Agent and Client components to implement this feature. The Client component has to be installed on the host machine and the Agent component on the remote nodes. This release of Unified Wire Installer contains option ROM image v1.0.0.68. A detailed explanation regarding this feature is included in the User's Guide. Please refer to the "Mass Deployment" section in "Chelsio Unified Wire" chapter. 6. Un-installation ================== 6.1. Installer -------------- GUI --- a) Run the "ChelsioUwire-x.x.x.xx.exe" application. b) Select the radio button "Remove" and click "Next". c) Click on the "Finish" button to exit from the installer. NOTE: This method of uninstallation is possible only if the drivers were installed using Unified Wire Installer. CLI --- To uninstall all the drivers, execute the following command: C:\Users\Administrator>ChelsioUwire-x.x.x.xx.exe -un all To uninstall a particular driver, execute the following command: C:\Users\Administrator>ChelsioUwire-x.x.x.x.exe -rm NOTE: Uninstalling Unified Wire package using the above method will not uninstall Unified Wire Manager. You will have to manually uninstall it using "Programs and Features" in the Control Panel. 6.2. Zip Package ---------------- If the driver components were installed using the zip package, they will have to be uninstalled manually. Please refer to User's Guide for step-by-step instructions. IMPORTANT: The driver components must be uninstalled manually and strictly in the order specified in the document. Any other order is not supported. ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ NDIS Function ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Chelsio’s Terminator series of Unified Wire adapters provide extensive support for NIC operation, including all stateless offload mechanisms for both IPv4 and IPv6. NVGRE Offload ============= Chelsio’s Terminator based adapters are uniquely capable of offloading the processing of NVGRE encapsulated frames such that all stateless offloads(checksums, LSO, VMQ, RSS and filtering) are preserved, resulting in significant performance benefits. This feature is enabled by default. Configure the relevant customer and provider network settings on the host. VXLAN Task Offload ================== Chelsio’s Terminator based adapters are capable of offloading the processing of VXLAN encapsulated frames such that all stateless offloads (checksums, LSO, VMQ, RSS and filtering) are preserved, resulting in significant performance benefits. This feature is enabled by default. PacketDirect ============= PacketDirect Provider Interface (PDPI), an extension to the existing NDIS miniport driver model, gives applications complete control over buffers, poll processors, and sending and receiving packets over a miniport adapter. Chelsio’s Unified Wire adapters offer extensive support for this feature leading to a higher throughput performance. This feature is disabled by default. NOTE: Please refer User's Guide to configure the PacketDirect feature. VMMQ/vRSS ========= Network throughput of a multi-core virtual machine is limited by the processing power of a single virtual processor, since only one processor handles all the interrupts from a virtual network adapter. Virtual Machine Multi-Queue (VMMQ) or Virtual Switch Receive-side Scaling (vRSS) solves this bottleneck by distributing the network traffic across multiple virtual processors. If NDIS function driver is installed using Unified Wire installer, VMMQ/vRSS will be enabled by default. However, if the driver is installed manually (using zip package), the feature will be disabled. Use NDIS Miniport Driver Parameters (Device manager -> Chelsio Network Adapter Properties -> Advanced Tab -> Virtual Switch RSS) to enable it. Additionally, VMMQ/vRSS needs to be enabled for each Virtual Machine using the below command on the host: PS C:\Users\Administrator> Set-VMNetworkAdapter -VMName -VrssEnabled:$true -VmmqEnabled:$true -VmmqQueuePairs Example: PS C:\Users\Administrator> Set-VMNetworkAdapter -VMName VM -VrssEnabled:$true -VmmqEnabled:$true -VmmqQueuePairs 8 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ iWARP (ND) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Chelsio Unified Wire Installer registers Chelsio iWARP provider automatically during installation. In case of zip package, run the following command to register: C:\Users\Administrator>chinstallsp.exe -i The iWARP provider should show up using: C:\Users\Administrator>chinstallsp.exe -l You should see a similar output: 0000001019 - Chelsio Network Direct provider ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ SMB Direct ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1. Enabling SMB Direct ====================== Windows Server 2016/Server 2012 R2 ---------------------------------- SMB Direct is installed and enabled by default on Windows Server 2016/Server 2012 R2. RDMA functionality on Chelsio adapters will be enabled after installing the driver from the package. Windows 10 AU Client -------------------- On Windows 10 AU Client systems, SMB Direct feature is installed but disabled by default. To enable it, follow the steps mentioned below: a) Open Control Panel and click Programs. b) Under Programs and Features, click Turn Windows features on or off. c) In the Windows Features window, scroll down and select the SMB Direct checkbox. d) Click OK. RDMA functionality on Chelsio adapters will be enabled after installing the driver from the package. Configuration -------------- You can use the PowerShell command "Get-NetAdapterRdma" to get a list of RDMA capable interfaces in your system. At least one adapter should indicate "RDMA Capable" true to take advantage of SMB RDMA capability. The next step is to configure your network so that SMB server and client machines can communicate with each other. Please configure the Microsoft firewall to allow traffic between the SMB clients and server. 2. RDMA/NVGRE concurrent (Mode 2) ================================= In this mode, you can run RDMA traffic using a virtual switch. This feature is enabled by default in the driver, but must be enabled on the Hyper-V Virtual Ethernet adapter. NOTE: For more information on the features mentioned above, please refer to User's Guide. 3. Storage Replica ===================== Currently, Unified Wire supports Server to Server replication mode. In this mode, data is synced between two servers and each server keeps an identical copy of the same volume. For more information on how to setup and configure in this mode, please refer to Microsoft’s official documentation at https://technet.microsoft.com/en-us/windows-server-docs/storage/storage-replica/server-to-server-storage-replication ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ NDIS SR-IOV ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1. Installation =============== Please ensure that Unified Wire is installed on the host before proceeding. Refer Unified Wire's Installation section for step-by-step instructions. 2. Enabling SR-IOV ================== IMPORTANT: The adapter's configuration should be updated to make use of the SR-IOV feature. You can do this by running the "chelsio_adapter_config.ps1" configuration script. To use the adapter in non-SRIOV scenarios in Windows or with other operating systems, the configuration type must be changed to "NON-SRIOV (Default)". Windows GUI Machine ------------------- In case of Installer, the script will be copied to "\Windows\System32" during installation. If you are using the zip package, copy "cxgbtool.exe" from "ChelsioUwire-x.x.x.xx" folder to "\Windows\System32". Also, change your working directory to "ChelsioUwire-x.x.x.xx\Adapter Configuration". a) Open PowerShell with administrative privileges. b) Check if the machine is SR-IOV capable using the following command: PS C:\Users\Administrator> Get-VMHost | fl *iov* The "IovSupport" field should display "True" c) Run the adapter configuration script and select "Windows GUI" (option 1) as the Windows version. Hit [Enter]. Installer: PS C:\Users\Administrator> chelsio_adapter_config.ps1 Zip: PS D:\ChelsioUwire-x.x.x.xx\Adapter Configuration> .\chelsio_adapter_config.ps1 d) Enter the index of the adapter for which the configuration needs to be updated. Hit [Enter]. e) Select "SRIOV" (option 2) as the configuration type and enter "y" to confirm. Hit [Enter]. f) Reboot the system for the changes to take effect. Nano Server (Nano Server Installer & Zip Package) ------------------------------------------------- a) Connect to Nano Server Machine. b) Check if the machine is SR-IOV capable using the following command: [nanomachine-ip]: PS C:\> Get-VMHost | fl *iov* The "IovSupport" field should display "True" c) If Unified wire was installed on the Nano Server machine using the Nano Server installer, skip to step (g). If the driver components were manually installed using the boot image (Zip package), follow all the steps mentioned below. d) Copy "ChelsioUwire-x.x.x.xx" folder to a desired location. e) Copy "cxgbnano.exe" from "ChelsioUwire-x.x.x.xx" folder to "\Windows\System32" f) Change your working directory to "ChelsioUwire-x.x.x.xx\Adapter Configuration" g) Run the adapter configuration script and select "Windows Nano Server" (option 2) as the Windows version. Hit [Enter]. Nano Server Installer: [nanomachine-ip]: PS D:\> chelsio_adapter_config.ps1 Zip Package: [nanomachine-ip]: PS D:\ChelsioUwire-x.x.x.xx\Adapter Configuration> .\chelsio_adapter_config.ps1 h) Enter the index of the Chelsio adapter for which the configuration needs to be updated. Hit [Enter]. i) Select "SR-IOV" (option 2) as the configuration type and enter "y" to confirm. Hit [Enter]. j) Reboot the Nano Server system for the changes to take effect. 3. Host Configuration ===================== a) Create a vSwitch with SR-IOV enabled using Chelsio adapter. b) Assign a virtual network adapter with SR-IOV enabled to the VM. c) Bring up the VM. NOTE: For more detailed information, refer User's Guide. 4. Guest (VM) Configuration =========================== a) Install Unified Wire on the guest by referring Unified Wire's "Installation" section. b) Verify if SR-IOV was enabled successfully on the host: PS C:\Users\Administrator> Get-VMNetworkAdapter | fl The VFDataPathActive parameter must display "True" To uninstall Unified Wire, please refer Unified Wire's "Un-installation" section. 5. Guest RDMA (Mode 3) ====================== In this mode, you can run RDMA traffic on Guest (VM) using a virtual switch. This feature is enabled by default in the driver, but must be enabled on the Microsoft Hyper-V Virtual Ethernet adapter. Please refer User's Guide for step-by-step instructions on how to enable this feature. ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ iSCSI Storport Miniport ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1. Configuring iSCSI Initiator ============================== Follow the steps mentioned below to assign IP, Subnet Mask, Default Gateway and VLAN IDs on port 0. a. Open Device Manager, right click on Chelsio iSCSI Initiator and click on Properties. b. Click on Advanced tab and select IP option and add IP. Similarly add subnet mask and default gateway. c. Select MTU and assign value as per requirement. d. VLAN is disabled by default, so to enable, click on Vlan Insertion and select value as Enabled. e. Assign the required VLAN ID and click OK. Repeat the above mentioned steps to assign these driver properties on other ports. 2. iSCSI Target Discovery and Login =================================== Before target discovery, make sure a static IP address is assigned in the Device Manager to the respective Chelsio iSCSI node. Following are the set of instructions to discover the target and login to it using the Chelsio iSCSI interface: a. To start the Initiator configuration, go to Control Panel and click on the iSCSI Initiator option. b. Choose the Discovery tab and click on Discover Portal. c. Click on the Advanced button. d. In the Advanced Settings window, select Chelsio iSCSI interface as the local adapter. e. Select the corresponding Chelsio iSCSI Interface IP as the Initiator IP and click OK. f. Enter the IP address (DNS name not supported) of the target machine and the corresponding port number and click OK on the Discover Target Portal Window. Note that the default port number for iSCSI traffic is 3260. g. Once target portal is added, details like target and initiator machine IP, Port number and Chelsio iSCSI interface IP will be displayed. h. Click on the Targets tab to see the list of targets available, choose a particular target and click on Connect. i. A window pops up showing the Target Name, Click OK. j. After logging in to the target, the state of the target will change from Inactive to Connected. k. After successful login, go to Disk Management and make the respective volume online. ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Data Center Bridging (DCB) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Chelsio network adapters work seamlessly with any legacy switches and do not require the complexity of configuring Data Center Bridging (DCB) protocols either on adapter or network switch. In case DCB is already configured on the switch, the adapter will automatically negotiate the PFC settings configured on the switch. Network QoS (Optional) ========================= Network Quality of Service (QoS) is an advanced Windows feature that can be used to distribute bandwidth between different kinds of outgoing traffic. This feature ensures efficient usage of resources and minimizes the impact of bandwidth congestion. The bandwidth percentage can be configured either on switch or host. 1. Enabling QoS on Host ----------------------- Network QoS is enabled by default. NOTE: Chelsio QoS requires a minimum of 8 cores to work. In case of multiple sockets, where the number of cores per socket is less than 8, Chelsio adapters must be configured to utilize all the available cores before enabling QoS. Run the following command on all Chelsio ports: PS C:\Users\Administrator> Set-NetAdapterRss -Name -MaxProcessorGroup 3 2. Configuring Bandwidth Allocation ----------------------------------- - Switch --------- Here is an example of setting bandwidth allocation on a Dell Force10 switch: a) Log in to the switch and enter the configuration mode: Login: admin Password: Force10#enable Force10#configure b) Enable DCB and LLDP: Force10(conf)#enable dcb Force10(conf)#protocol lldp Force10(conf-lldp)#no disable c) Set DCBx version to auto: Force10(conf-lldp)#dcbx version auto d) Specify the intervals between hello packets: Force10(conf-lldp)#hello 5 e) Create a DCB map. For example, here we are configuring 2 groups/classes: - Group 0 has BW of 80% with pfc enabled. - Group 1 had BW of 20% with pfc disabled. - Assigning priority groups: Priority 5-7 are under group 0 and 0-4 are under group 1. Force10(conf)#dcb-map win_dcb Force10(conf-dcbmap-win_dcb)# priority-group 0 bandwidth 80 pfc on Force10(conf-dcbmap-win_dcb)# priority-group 1 bandwidth 20 pfc off Force10(conf-dcbmap-win_dcb)# priority-pgid 1 1 1 1 1 0 0 0 f) Enter interface configuration mode and configure protocol LLDP: Force10(conf)#interface tengigabitethernet 0/17 Force10(conf-if-te-0/17)#protocol lldp Force10(conf-if-te-0/17-lldp)#no disable g) Set DCBx version to auto: Force10(conf-if-te-0/17-lldp)#dcbx version auto h) Apply the DCB-MAP created: Force10(conf-if-te-0/17)#dcb-map win_dcb The host adapter will honor the bandwidth allocation settings configured on the switch. - Host ------ Here is an example of setting bandwidth allocation on the host: a) Open PowerShell with administrative privileges and run the following command: PS C:\Users\Administrator> Install-WindowsFeature -Name "Data-Center-Bridging" -IncludeAllSubFeature -IncludeManagementTools b) Create new policy: PS C:\Users\Administrator> New-NetQosPolicy -Name rdma -PriorityValue 5 -NetDirectPortMatchCondition 445 c) Create new rule with bandwidth percentage: PS C:\Users\Administrator> New-NetQosTrafficClass -Name rdma -Priority 5 -Algorithm ETS -BandwidthPercentage 80 -Verbose The above command allocates 80% of the bandwidth to the policy named 'rdma'. To change the percentage of an existing ETS rule: PS C:\Users\Administrator> Set-NetQosTrafficClass -Name rdma -Priority 5 -Algorithm ETS -BandwidthPercentage 10 -Verbose ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Unified Wire Manager ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1. Installation =============== The Unified Wire installer has been designed to install Unified Wire Manager (UM) along with driver components by default, hence no separate installation is required. In case of zip package, you will have to manually run the UM installer located in ChelsioUwire-x.x.x.xx\UM folder (See User's Guide for more information). Based on the Windows version running on the system, the three UM components, i.e. Management Agent, Client and Station will be installed. If an older version of UM exists, the installer will upgrade it to the version provided in the package. 2. Management Station Configuration =================================== If you've already configured Management Station using the Unified Wire Installer during installation, it is already set-up and you can skip this step. If you, however, decide to manually configure or make modifications to the default settings, please refer "Unified Wire Manager" chapter in the User's Guide and make necessary changes. 3. Un-Installation ================== To uninstall Unified Wire Manager, use "Programs and Features" in the Control Panel. 4. Verifying UM components status ================================= Verifying Management Agent -------------------------- a) On the system running Management Agent, launch Services from the Control Panel. You can also type services.msc in the Run command. This will open the Services window. b) Search for Unified Wire Manager in the list of services. You should get the options to stop and restart in the Actions menu on the left side, indicating that Management Agent is already running. Verifying Management Client --------------------------- Execute the following query command to determine if Management Client is installed: C:\Users\Administrator>chelsio_uwcli.exe -V The above query should confirm that Management Client is installed by displaying a similar result: Unified Manager client CLI version : 2.x.yy Verifying Management Station ---------------------------- a) Launch Services from the Control Panel. You can also type services.msc in the Run command. This will open the Services window. b) Search for Unified Wire Management Station in the list of services. You should get the options to stop,start and restart the service in the Actions menu on the left side. 5. Agent/Station Start/Stop/restart =================================== a) Launch Services from the Control Panel. You can also type services.msc in the Run command. This will open the Services window. b) Scroll and highlight the "Unified Wire Manager" service. c) Click on Start/Stop/Restart to carry out the respective tasks in the Description section on the left. 6. Client Usage =============== Use the following commands to view Management client Help file. C:\Users\Administrator>chelsio_uwcli --help ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Chelsio UM for WinPE ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Chelsio "UM for Win PE" is a light-weight management tool developed for Windows PE (Windows Preinstallation Environment) platform and provides features to view and configure different aspects of the system, including Chelsio adapter installed. The tool provides command-line options to view system related information like list of drivers installed, Chelsio adapters present, etc. You can also view and configure Chelsio adapter settings like card and port properties, flash and erase option ROM, etc. To view the complete list of commands available, invoke the help by typing chelsio_uwlite.exe in the command prompt. The tool is copied to \ChelsioUwire\docs\ during Unified Wire installation.In order to use the tool, you will have to add it to boot image when creating bootable Windows PE media. For more information, visit the following link: http://technet.microsoft.com/en-us/library/cc749312%28v=ws.10%29.aspx NOTE: - Only local systems can be managed currently using this tool. - To view syntax and usage examples, please refer User's Guide. 5. Support Documentation ================================================================================ The documentation for this release can be found in the following directory in the system: Windows Unified Wire Installer (after installation): \ChelsioUwire\docs\ Zip Package: ChelsioUwire-x.x.x.xx\docs\ Nano Server Installer: Chelsio-NANO-installer-x.x.x.xx\docs\ It contains: - README - Release Notes - User's Guide 6. Customer Support ================================================================================ Please contact Chelsio support at support@chelsio.com for any issues regarding the product. ******************************************************************************** Copyright (C) 2017 Chelsio Communications. All Rights Reserved. The information in this document is furnished for informational use only, is subject to change without notice, and should not be construed as a commitment by Chelsio Communications. Chelsio Communications assumes no responsibility or liability for any errors or inaccuracies that may appear in this document or any software that may be provided in association with this document. Except as permitted by such license, no part of this document may be reproduced, stored in a retrieval system,or transmitted in any form or by any means without the express written consent of Chelsio Communications.