**************************************** RELEASE NOTES **************************************** Chelsio Unified Wire for VMware ESXi 6.7 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Drivers ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Version : 3.1.0.29 Date : 06/04/2019 ================================================================================ Native Network Driver ********************* 1. CHANGES ---------------------------------------- - Driver VMware certified. 2. ERRATA (Open Issues) ---------------------------------------- - 37629: Need GENEVE support. - 36014: Unloading cxl driver on host with VFs attached is not recommended. - 35033: Need DCB support. - 34022: IPv6 ping fails on NIC interface in VM when VM has both SR-IOV and NIC interfaces. - 33990: Errors observed on 25G link connected to mellanox Switch. NOTE: The errors are seen only with this switch. To workaround it, Enable RS FEC on the Chelsio port using cxgbtool. [root@host:~] /opt/chelsio/bin/cxgbtool -c fec -upd rs - 32617: (DUP!) seen in ping while running vxlan traffic between VMs. iSCSI/iSER Offload Initiator Driver *********************************** 1. CHANGES ---------------------------------------- - Fixed the issue Lun's are not available after machine comes up from reboot. - Driver VMware certified. 2. ERRATA (Open Issues) ----------------------------------------. - 37607: old ima driver not getting cleaned up on driver upgrade. - 37079: Need support for "IPv6 Router Advertisement Configuration" and "IPv6 Linklocal Auto Configuration" - 36816: All the available targets must be displayed under static discovery when number of targets is > 128 ================================================================================ ================================================================================ Version : 3.1.0.22 (Beta) Date : 03/29/2019 ================================================================================ Native Network Driver ********************* 1. CHANGES ---------------------------------------- - Upgraded firmware to 1.23.4.0 - Fixed the incorrect queues displayed in cxgbtool qinfo. - Fixed the issue vxlan cso counters are not incremented on tx end while vxlan traffic is running. 2. ERRATA (Open Issues) ---------------------------------------- - 37629: Need GENEVE support. - 36014: Unloading cxl driver on host with VFs attached is not recommended. - 35033: Need DCB support. - 34022: IPv6 ping fails on NIC interface in VM when VM has both SR-IOV and NIC interfaces. - 33990: Errors observed on 25G link connected to mellanox Switch. NOTE: The errors are seen only with this switch. To workaround it, Enable RS FEC on the Chelsio port using cxgbtool. [root@host:~] /opt/chelsio/bin/cxgbtool -c fec -upd rs - 32617: (DUP!) seen in ping while running vxlan traffic between VMs. iSCSI/iSER Offload Initiator Driver *********************************** 1. CHANGES ---------------------------------------- - Added cxgbtool statistics for iSCSI error types. - Fixed issues seen with iscsi target lun reset. - Fixed the issues with cheiscsi DriverLoadUnload test. - Remove static target added by dynamic target when dynamic target is removed. 2. ERRATA (Open Issues) ----------------------------------------. - 37679: Lun's not available after machine comes up from reboot. Workaround: Rescan the Initiator to see the LUNs. - 37607: old ima driver not getting cleaned up on driver upgrade. - 37079: Need support for support for "IPv6 Router Advertisement Configuration" and "IPv6 Linklocal Auto Configuration" - 36816: All the available targets must be displayed under static discovery when number of targets is > 128 ================================================================================ ================================================================================ Version : 3.1.0.11 (Beta) Date : 12/26/2018 ================================================================================ Native Network Driver ********************* 1. CHANGES ---------------------------------------- - Upgraded firmware to 1.22.8.0 - Fixed failures seen during VMware ESX NIC certification. - Fixed issue with IPv6 Autoconfig IP. - Fixed memory leaks observed in some cases. - Fixed issue with RSS queues. 2. ERRATA (Open Issues) ---------------------------------------- - 37215: Incorrect queues displayed in cxgbtool qinfo. - 37214: vxlan cso counters are not incremented on tx end while vxlan traffic is running. - 36014: Unloading cxl driver on host with VFs attached is not recommended. - 35033: Need DCB support. - 34022: IPv6 ping fails on NIC interface in VM when VM has both SR-IOV and NIC interfaces. - 33990: Errors observed on 25G link connected to mellanox Switch. NOTE: The errors are seen only with this switch. To workaround it, Enable RS FEC on the Chelsio port using cxgbtool. [root@host:~] /opt/chelsio/bin/cxgbtool -c fec -upd rs - 32617: (DUP!) seen in ping while running vxlan traffic between VMs. - 29607: Enabling LRO in the module option shows less performance, when LRO is also enabled in Linux Virtual machine. iSCSI/iSER Offload Initiator Driver *********************************** 1. CHANGES ---------------------------------------- - 100G iSCSI Performance improvements. - Fixed compatibility issue with scst iser target. - Fixed failures seen during VMware ESX iSCSI certification. - Fixed PSOD seen while trying to login to 256 targets in iSER mode. - Fixed PSOD seen while doing dynamic discovery in beta build. - Fixed PSOD seen when executing "cxgbtool -c isess" while trying to discover 256 targets. - Fixed PSOD seen during memory allocation failure. - Fixed issue seen during ESXi installation with Chelsio iSCSI CBFT. - Fixed issue seen with multiple SO card in target login. - Fixed Duplicate target session coming with link toggle. 2. ERRATA (Open Issues) ----------------------------------------. - 37079: Need support for support for "IPv6 Router Advertisement Configuration" and "IPv6 Linklocal Auto Configuration" - 36816: All the available targets must be displayed under static discovery when number of targets is > 128 - 35177: need cxgbtool statistics for iSCSI error types. ================================================================================ ================================================================================ Version : 3.0.0.18 (Beta) Date : 10/08/2018 ================================================================================ Native Network Driver ********************* 1. CHANGES ---------------------------------------- - Upgraded firmware to 1.21.5.0 - Added support for ESXi 6.7 - Added support for T540-SO-CR and T540-LP-CR adapters. - Added options to read FEC settings using cxgbtool. - Fixed the issues observed on doing plug/unplug of vmnic from vSwitch while running traffic. - Fixed the issue "SRIOVNIC" stats do not increment on the Host. - Fixed the issue while collecting MC. - Fixed the issue esxcli is not listing link parameters. - Fixed the continuous flood of messages in dmesg on loading the driver in line interrupt mode. - Fixed the issue while running vxlan traffic along with link toggle and driver unload/load. 2. ERRATA (Open Issues) ---------------------------------------- - 36014: Unloading cxl driver on host with VFs attached is not recommended. - 35033: Need DCB support. - 34022: IPv6 ping fails on NIC interface in VM when VM has both SR-IOV and NIC interfaces. - 33990: Errors observed on 25G link connected to mellanox Switch. NOTE: The errors are seen only with this switch. To workaround it, Enable RS FEC on the Chelsio port using cxgbtool. [root@host:~] /opt/chelsio/bin/cxgbtool -c fec -upd rs - 32617: (DUP!) seen in ping while running vxlan traffic between VMs. - 29607: Enabling LRO in the module option shows less performance, when LRO is also enabled in Linux Virtual machine. iSCSI/iSER Offload Initiator Driver *********************************** 1. CHANGES ---------------------------------------- - Added support for ESXi 6.7 - Added support for iSER offload initiator. - Added support for memory-free adapters. - Added support for T540-LP-CR adapter. - Fixed the issues on unloading and loading the cheiscsi driver. - Fixed the issues observed after loggin in using Ports 2 and 3. - Fixed the issue unable to get iSCSI initiator state to CHAP default settings. - Fixed the issue of initiator discovering 127 instead of 128 targets. - Fixed the issues seen while running network traffic with RSS and iSCSI driver loaded. 2. ERRATA (Open Issues) ---------------------------------------- - 36851: Executing "cxgbtool -c isess" while trying to discover 256 targets is not supported. - 36816: All the available targets must be displayed under static discovery when number of targets is > 128 - 36768: Toggling switch port continuously while running xdd traffic is not supported. - 35177: need cxgbtool statistics for iSCSI error types. ================================================================================ ================================================================================ Version : 2.2.0.14 (Beta) Date : 07/16/2018 ================================================================================ Native Network Driver ********************* 1. CHANGES ---------------------------------------- - Added support for adapters with new flash parts. - Fixed the issue unable to flash esx configuration on few T5 adapters. 2. ERRATA (Open Issues) ---------------------------------------- - 36276: Plug/unplug of vmnic from vSwitch while running traffic is not supported. - 36014: Unloading cxl driver on host with VFs attached is not recommended. - 35033: Need DCB support. - 34022: IPv6 ping fails on NIC interface in VM when VM has both SR-IOV and NIC interfaces. - 34021: Warnings observed on PEER while running VXLAN traffic and rebooting DUT machine. - 32617: (DUP!) seen in ping while running vxlan traffic between VMs. - 30452: "SRIOVNIC" stats do not increment on the host. - 29607: Enabling LRO in the module option shows less performance, when LRO is also enabled in Linux Virtual machine. iSCSI/iSER Offload Initiator Driver *********************************** 1. CHANGES ---------------------------------------- - N/A 2. ERRATA (Open Issues) ---------------------------------------- - 36175: Need support for memory free adapters. - 35177: need cxgbtool statistics for iSCSI error types. ================================================================================ ================================================================================ Version : 2.2.0.12 (Beta) Date : 07/06/2018 ================================================================================ Native Network Driver ********************* 1. CHANGES ---------------------------------------- - Upgraded firmware to 1.20.8.0 2. ERRATA (Open Issues) ---------------------------------------- - 36276: Plug/unplug of vmnic from vSwitch while running traffic is not supported. - 36014: Unloading cxl driver on host with VFs attached is not recommended. - 35033: Need DCB support. - 34022: IPv6 ping fails on NIC interface in VM when VM has both SR-IOV and NIC interfaces. - 34021: Warnings observed on PEER while running VXLAN traffic and rebooting DUT machine. - 32617: (DUP!) seen in ping while running vxlan traffic between VMs. - 30452: "SRIOVNIC" stats do not increment on the host. - 29607: Enabling LRO in the module option shows less performance, when LRO is also enabled in Linux Virtual machine. iSCSI/iSER Offload Initiator Driver *********************************** 1. CHANGES ---------------------------------------- - Added support for ESXi iSCSI boot. - Fixed the PSOD while logging in to the redirection target. 2. ERRATA (Open Issues) ---------------------------------------- - 36175: Need support for memory free adapters. - 35177: need cxgbtool statistics for iSCSI error types. ================================================================================ ================================================================================ Version : 2.2.0.1 (Beta) Date : 03/27/2018 ================================================================================ Native Network Driver ********************* 1. CHANGES ---------------------------------------- - Upgraded firmware to 1.18.9.0 - Added support for T61100-OCP adapter. 2. ERRATA (Open Issues) ---------------------------------------- - 34156: Plug/unplug of vmnic from vSwitch while running traffic is not supported. Workaround: Disable RSS on the VMs. - 34022: IPv6 ping fails on NIC interface in VM when VM has both SR-IOV and NIC interfaces. - 34021: Warnings observed on PEER while running VXLAN traffic and rebooting DUT machine. - 32617: (DUP!) seen in ping while running vxlan traffic between VMs. - 30452: "SRIOVNIC" stats do not increment on the host. - 29607: Enabling LRO in the module option shows less performance, when LRO is also enabled in Linux Virtual machine. iSCSI/iSER Offload Initiator Driver *********************************** 1. CHANGES ---------------------------------------- - Fixed PSOD seen while logging into redirection target. - Made DNS IP field optional while configuring initiator. - Added support to discover iSCSI targets using host name. 2. ERRATA (Open Issues) ---------------------------------------- - 34744: Need ESXi iSCSI boot support. - 35624: PSOD seen while loging out of redirection target. - 35509: PSOD seen while running login logout script in loop. - 35177: need cxgbtool statistics for iSCSI error types. - 34296: Need support to discover iSCSI targets using host name. ================================================================================ ================================================================================ Version : 2.0.0.32 (Beta) Date : 02/28/2018 ================================================================================ Native Network Driver ********************* 1. CHANGES ---------------------------------------- - Upgraded firmware to 1.18.4.0 2. ERRATA (Open Issues) ---------------------------------------- - 34934: Issues observed with T61100-OCP card. - 34156: Plug/unplug of vmnic from vSwitch while running traffic is not supported. Workaround: Disable RSS on the VMs. - 34022: IPv6 ping fails on NIC interface in VM when VM has both SR-IOV and NIC interfaces. - 34021: Warnings observed on PEER while running VXLAN traffic and rebooting DUT machine. - 32617: (DUP!) seen in ping while running vxlan traffic between VMs. - 30452: "SRIOVNIC" stats do not increment on the host. - 29607: Enabling LRO in the module option shows less performance, when LRO is also enabled in Linux Virtual machine. iSCSI/iSER Offload Initiator Driver *********************************** 1. CHANGES ---------------------------------------- - Fixed CHAP authentication issue. - Fixed PSOD issue seen when removing discovered target one by one. - Fixed PSOD issue seen when sending ABORT from SNABLAZE target. - Fixed PSOD issue seen while running iozone traffic with target interface up/down. - Added iSCSI boot support. 2. ERRATA (Open Issues) ---------------------------------------- - 35274: Junk portal address displayed once logged in into target using IPv6 address. - 35075: CHAP settings are not persistent. - 35177: need cxgbtool statistics for iSCSI error types - 34788: "esxcli iscsi session list -A vmhba64" does not list iSCSI session. - 34296: Need support to discover iSCSI targets using host name. - 34602: "esxcli iscsi adapter discovery statictarget list" does not list the discovered target. ================================================================================ ================================================================================ Version : 2.0.0.21 Date : 02/13/2018 ================================================================================ Native Network Driver ********************* 1. CHANGES ---------------------------------------- - Upgraded firmware to 1.17.14.0 - Fixed the issue of dmesg printing failure messages in case of fw_attach=0. - Fixed the "WARNING: cxl: cleanup_module:849: WARN - Adapter links list is not empty!" observed while unloading cxl - Fixed the issue executing "/opt/chelsio/bin/cxgbtool -c version" command after loading driver with fw_attch=0 leads to PSOD. 2. ERRATA (Open Issues) ---------------------------------------- - 34934: Issues observed with T61100-OCP card. - 34905: "WARNING: cxl: cxl_attach:3123: No of ports exceeded the limit" seen with 3 adapters. - 34156: Plug/unplug of vmnic from vSwitch while running traffic is not supported. Workaround: Disable RSS on the VMs. - 34022: IPv6 ping fails on NIC interface in VM when VM has both SR-IOV and NIC interfaces. - 34021: Warnings observed on PEER while running VXLAN traffic and rebooting DUT machine. - 33028: Need an option to erase config files stored in adapter flash. - 32617: (DUP!) seen in ping while running vxlan traffic between VMs. - 30452: "SRIOVNIC" stats do not increment on the host. - 29607: Enabling LRO in the module option shows less performance, when LRO is also enabled in Linux Virtual machine. iSCSI/iSER Offload Initiator Driver *********************************** 1. CHANGES ---------------------------------------- - N/A (first release) 2. ERRATA (Open Issues) ---------------------------------------- - 34756: Need to support all types of CHAP authentication. - 34755: Unable to set CHAP authentication. - 34788: "esxcli iscsi session list -A vmhba64" does not list iSCSI session. - 34799: Ping fails between iSCSI interface and the VM on same host. - 34712: PSOD seen while running diskspd traffic using 4 LUNs. - 34296: Need support to discover iSCSI targets using host name. - 34728: ESXi initiator fails to login when connecting to multiple target machine. - 34602: "esxcli iscsi adapter discovery statictarget list" does not list the discovered target. - 34731: Persistent login fails. ================================================================================ ================================================================================ Version : 1.1.0.64 Date : 10/10/2017 ================================================================================ Native Network Driver ********************* 1. CHANGES ---------------------------------------- - Upgraded Firmware to 1.16.63.0 - Fixed the PSOD on Unloading/loading driver with fw_attach=0 - Removed unallocated tx queue display from cxgbtool - Fixed the warning observed in dmesg when total number of ports=8 2. ERRATA (Open Issues) ---------------------------------------- - 34156: Plug/unplug of vmnic from vSwitch while running traffic is not supported. Workaround: Disable RSS on the VMs. - 34051: Simultaneously powering up VMs with VFs attached is not supported. - 34022: IPv6 ping fails on NIC interface in VM when VM has both SR-IOV and NIC interfaces. - 34021: Warnings observed on PEER while running VXLAN traffic and rebooting DUT machine. - 33028: Need an option to erase config files stored in adapter flash. - 32617: (DUP!) seen in ping while running vxlan traffic between VMs. - 30452: "SRIOVNIC" stats do not increment on the host. - 29607: Enabling LRO in the module option shows less performance, when LRO is also enabled in Linux Virtual machine. ================================================================================ ================================================================================ Version : 1.1.0.61 (Alpha) Date : 08/14/2017 ================================================================================ Native Network Driver ********************* 1. CHANGES ---------------------------------------- - Upgraded firmware to 1.16.59.0 - Fixed the issue of Rx queue stats failing to increment on certain queues after a period of time while VM's traffic is running. - Fixed the issue of cxl_sema_wakeup:1666 error while running vxlan traffic. - Fixed the issue of cxgbtool not displaying stats for port 2 and 3 of a 4 port-adapter in multiple adapters scenario. - 100G performance improvements. - Added support for T540-BT adapter. 2. ERRATA (Open Issues) ---------------------------------------- - 33663: Need to remove unallocated tx queue display from cxgbtool. - 33028: Need an option to erase config files stored in adapter flash. - 32617: (DUP!) seen in ping while running vxlan traffic between VMs. - 31922: T62100-LP-CR shows link speed "34464Mbps" NOTE: This is an OS issue; not a driver issue. It will not be seen on ESXi 6.5. - 30452: "SRIOVNIC" stats do not increment on the host. - 29607: Enabling LRO in the module option shows less performance, when LRO is also enabled in Linux Virtual machine. ================================================================================ ================================================================================ Version : 1.1.0.46 (Beta) Date : 05/18/2017 ================================================================================ Native Network Driver ********************* 1. CHANGES ---------------------------------------- - Upgraded Firmware to 1.16.43.0 - Fixed the PSOD seen while toggling the interface. - Fixed the issue iozone is hung and incomplete on NFS mounted partition through VXLAN. - Fixed the PSOD observed on starting traffic between 16 VMs. - Added support for T6425-CR and T6225-LL-CR adapters. 2. ERRATA (Open Issues) ---------------------------------------- - 33028: Need an option to erase config files stored in adapter flash. - 33026: Adding and removing vmnic from vSwitch with traffic is not supported. - 33014: Continuous unload and load of driver with traffic is not supported. - 33000: "rx_fs_err" counters are incrementing while running traffic. - 32829: Link fails to come up intermittently with SO cards. - 32619: Ping failure observed while running traffic with e1000 as the vNIC. - 32772: cxl: cxl_sema_wakeup:1666: warnings seen while running vxlan traffic. - 32617: (DUP!) seen in ping while running vxlan traffic between VMs. - 32589: "WARNING:cxl:cxl_isr_fl:2853:No Netpoll. Pkt dropped" observed while doing load/unload (or) interface toggle. - 31922: T62100-LP-CR shows link speed "34464Mbps" NOTE: This is an OS issue; not a driver issue. It will not be seen on ESXi 6.5. - 30452: "SRIOVNIC" stats do not increment on the host. - 29607: Enabling LRO in the module option shows less performance, when LRO is also enabled in Linux Virtual machine. ================================================================================ ================================================================================ Version : 1.1.0.33 (Beta) Date : 02/24/2017 ================================================================================ Native Network Driver ********************* 1. CHANGES ---------------------------------------- - Upgraded Firmware to 1.16.33.0 - Added support for ESXi 6.5a - Fixed the issue of PSOD after unloading and loading the cxl driver while vxlan traffic is running. - Added support for T62100-SO-CR and T6225-SO-CR adapters. 2. ERRATA (Open Issues) ---------------------------------------- - 32358: Interface toggle with esxcli while running traffic is not supported. - 32354: vxlan cso counters are not incremented on rx end while vxlan traffic is running. - 32043: iozone is hung and incomplete on nfs mounted partition through vxlan. - 31922: [ESXi 6.0]T62100-LP-CR shows link speed "34464Mbps" NOTE: This is an OS issue; not a driver issue. It will not be seen on ESXi 6.5. - 31396: T6 link fails to come up with few switches. Workaround: Disable autonegotiation on the switches. - 30452: "SRIOVNIC" stats do not increment on the host. - 29607: Enabling LRO in the module option shows less performance, when LRO is also enabled in Linux Virtual machine. - 29102: Errors counters increment while running traffic on standby uplink. - 29072: Changing MTU while running bidirectional traffic is not recommended. ================================================================================ ================================================================================ Version : 1.1.0.31 Date : 01/27/2017 ================================================================================ Native Network Driver ********************* 1. CHANGES ---------------------------------------- - Upgraded firmware to 1.16.29.0. - Fixed the PSOD observed while unloading driver after mailbox timed out. - Fixed the VLAN priority setting related issues. - Added support to control Link FEC. 2. ERRATA (Open Issues) ---------------------------------------- - 32043: iozone is hung and incomplete on nfs mounted partition through vxlan. - 31922: [ESXi 6.0]T62100-LP-CR shows link speed "34464Mbps" NOTE: This is an OS issue; not a driver issue. It will not be seen on ESXi 6.5. - 31396: T6 link fails to come up with few switches. Workaround: Disable autonegotiation on the switches. - 30452: "SRIOVNIC" stats do not increment on the host. - 29607: Enabling LRO in the module option shows less performance, when LRO is also enabled in Linux Virtual machine. ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Firmware ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ T6 Firmware =========== Version : 1.23.4.0 Date : 03/27/2019 ================================================================================ Fixes ----- ETH: - Fixed an issue in UDP-SO (segmentation offload) handling where Tx packets were getting truncated on wire. - Fixed a crash on adding more than 200 vlans. OFLD: - Pass wrid in error CQE. FOiSCSI: - IPV6 extension header support added. - ARP and neighbor solicitation retry timers are tuned to reduce the iscsi login time. - drop fragmented ICMP_TYPE_ECHO request packets. Enhancements ------------ BASE: - New command FW_PARAMS_PARAM_PFVF_LINK_STATE added to control VF link state notification to VF driver. - nvf parameter in firmware config file is deprecated. ETH: - Tx performance improvement in VF NIC path (when mac acl and non-transparent vlan are not used). Non-transparent vlan means when VF is aware of VLAN. - Default NIC WR completion timer changed to 150us which significantly lower completion Interrupt overhead. OFLD: - hashfilter enabled by default along with offload in firmware embedded config file. - Adjust iwarp ird/ord values according to the adapter memory. This enables larger value of ird/ord on T6 25G adapters for long distance links. - New APIs added to modify filterModes and filterMask from driver dynamically. - Fixed an issue where iwarp connection cleanup is not complete. ================================================================================ ================================================================================ Version : 1.22.8.0 Date : 12/26/2018 ================================================================================ Fixes ----- BASE: - Fixed the PTP issue observed on port 1. - Fixed the issue of filter rule based on outer vlan not getting honored. - Fixed the spurious temprature value returned by firmware. ETH: - Fixed a VF traffic hang when it is run along with PF traffic. - Made AN (auto-negotiation) mandatory for 10GBASE-T PHYs. - 100G Link handling with Cisco 3000/9000 series switches fixed. OFLD: - Fixed errors and connection drop on other connections when CRC error occurs on one iwarp connection. - Issue in CQE error handling fixed. - Fixed RDMA connection close hangs under heavy traffic conditions. FOiSCSI: - Fixed a bug in redirect target handling by Option ROM. - Fixed a crash observed while clearing iSNS server/client. Enhancements ------------ BASE: - New field vfvld and vin added to FW_VI_CMD. - smtid field added to FW_VI_MAC_CMD. - New device param command FW_PARAMS_PARAM_DEV_OPAQUE_VIID_SMT_EXTN added to check if firmware supports extended VIID format or not. Note that driver must not look inside(interpret) VIID. Instead use vfvld, vin and smtid field as returned by the firmware. - Per PF parameter FW_PARAMS_PARAM_PFVF_GET_SMT_START and FW_PARAMS_PARAM_PFVF_GET_SMT_SIZE added. - Enable hashfilter along with offload config. - New API ptp_rx_ctrl added to clear MPS_RX_PTP_CTL per port settings. - Enabled 32 traffic classes per port. - Enabled transparent vlan (VST) support for VF. A new API FW_PARAMS_PARAM_DEV_VF_TRVLAN added to indicate that the firmware has transparent vlan support in tx direction. Driver needs to use FW_PORT_CMD[action=FW_PORT_ACTION_L2_CFG] to enable transparent vlan in Rx direction. - ovlan_filt bit added to FW_PORT_CMD to enable/disable ovlan/ivlan based filtering in MPS. ETH: - New APIs (bit F_FW_EQ_ETH_CMD_TIMEREN and FW_PARAMS_CMD) added to enable Doorbell timer facility. This feature enables h/w to return Egress Queue Consumer Index (CIDX) updates to the host. Driver can change the SGE DBQ timer index in an existing ethernet egress context. - Added FW_PARAMS_CMD[DEV, TIMERTICK] Firmware API to allow a Host driver to read/write the SGE Doorbell Queue Timer Tick in units of 1us. - GPIO LED function definition updated for 25G and 100G OCP adapters. - Send (vlan or mac) ACL error message to master PF on asynchronous firmware event queue instead of mailbox. OFLD: - New WR FW_TLS_TUNNEL_OFLD_WR is added to carry NIC_TLS traffic. ================================================================================ ================================================================================ Version : 1.21.5.0 Date : 09/26/2018 ================================================================================ Fixes ----- BASE: - Incorrect mac reference count issue fixed. - VF FLR resetting VLAN issue fixed. - Always enable PFC if ETS is configured. - Incosistent VF rate limiting fixed. ETH: - Restart auto negotiation on seeing sigdet change in fallback mode. - Fixes a bug where pause doesn't get enabled when enabling PFC if switch doesn't support pause negotiation. OFLD: - Fixes a hang in offload loopback traffic on link toggle. - Connection hang due to CQE error after memory read fixed. FOiSCSI: - Ping to iscsi interface failure fixed. - Firmware crash while running IO and start/stop target in a loop fixed. Enhancements ------------ BASE: - Allow sending FW_TP_WR and FW_ULPTX_WR on NIC queues (needed for inline NIC-TLS usecase). - 2 new bits, F_FW_FILTER2_WR_DROP_ENCAP to drop encapsulation header and F_FW_FILTER2_WR_TX_LOOP to loopback nat'ed packets are added to FW_FILTER2_WR. ETH: - New API FW_PARAMS_PARAM_PFVF_MAX_PKTS_PER_ETH_TX_PKTS_WR added to know the max number of packets allwoed in FW_ETH_TX_PKTS_WR and FW_ETH_TX_PKTS_VM_WR. - Performance improvement in handling coalesced packet from VMs. - More than 64K ethofld flows supported. iSCSI: - iSCSI performance improvement on SO (memfree) adpater. ================================================================================ ================================================================================ Version : 1.20.8.0 Date : 07/06/2018 ================================================================================ Fixes ----- ETH: - Allows PF to access VI stats of any of its VF. - Fixes a bug in DCBX where dynamic ETS changes were not obeyed by fw. - Regression due to hpfilter fixed. OFLD: - Enabled 28K offload connection setup with traffic. FOiSCSI: - Ping response with data above 56 bytes fixed. - IPv6 hdr size not substracted from MSS value is fixed. Enhancements ------------ BASE: - FW_VI_MAC_CMD modified to accept loopback port number. Now MPS-TCAM entries can be created for loopback port from driver. - FW_VI_MAC_CMD modified to query/lookup index of SMAC/DMAC address. ETH: - Added support for 40/100G KR. OFLD: - Enabled hashfilter support along with other offloads (toe/iscsi/rdma). iSCSI: - SO (memfree) adapter support enabled for 256 iscsi connections. ================================================================================ ================================================================================ Version : 1.19.1.0 Date : 04/23/2018 ================================================================================ Fixes ----- BASE: - Fixed traffic stall when rate-limit is modified while running traffic. - Fixes a firmware crash in FW_ETH_TX_EO_WR handling. - Fixes host DCB support when FW_PORT_CMD is used. ETH: - Exit Auto-Negotiation if we don't receive base page from peer within 10s. This fixes some cases where in we keep on restarting auto negotiation without ever exiting, resulting in link failure. - Fixes an issue where VF packets counter were not increasing if VF packets coalesced WR is used by driver. OFLD: - Kernel and user mode NVMEoF performance enhancements. FOiSCSI: - Fixes fw crash when trying to connect to non-existence IPv6 iSNS target. ================================================================================ ================================================================================ Version : 1.18.9.0 Date : 03/27/2018 ================================================================================ Fixes ----- BBASE: - For Ethernet frames less than 64B, pad them with zero bytes as per IEEE spec (RFC 894). - Added a new parameter iqtype to FW_IQ_CMD to identify the ingress NIC or offload queues. This fixes an issue where driver was receiving interrupt with no new messages in queue. - FW_PARAMS_CMD processes all the valaid paramaters and returns value 0UL for any unknown parameter. OFLD: - Fixes connection failure during SRQ reuse. - Fixes incorrect cqe in case of WRITE with immediate operation. FOiSCSI: - Fixes a fw crash when wrong node-id is passed to FW_FOISCSI_CTRL_WR. FOFCoE: - Fixes a fw hang while creating NPIV. Enhancements ------------ ETH: - A new WR FW_ETH_TX_PKTS_VM_WR added to support VM packet coalescing. ================================================================================ ================================================================================ Version : 1.18.4.0 Date : 02/28/2018 ================================================================================ Fixes ----- BASE: - Fixed Rate limiting not working for 101Mbps<=rate limit<=163Mbps range. - Fixed starting more than 32 VMs on PF4 causing firmware hang. ETH: - Fixed link failure due to FEC mismatch with optics. - Fixed link failure with link toggle stress tests. - Only BaseR FEC is supported for 50G. - Fixed a bug in next page handling which sometimes causes link down. - Fixed port down due to failre to read eeprom contents of some modules. - Fixed a bug causing adapter to fail with spider configuration. FOiSCSI: - Fixed a bug causing login failure when connecting to multiple targets. Enhancements ------------ BASE: - Added a new firmware API to retrieve the maximum temperaturethreshold for the chip (FW_PARAM_DEV_DIAG_MAXTMPTHRESH). ETH: - Added support for user to contol pause negotiation during auto negotiation. FOiSCSI: - Added a new facility to redirect few fw events to offload rx queue (based on driver's configration) - Driver can ignore providing ipv6 prefix len during ipv6 address configuration. ================================================================================ ================================================================================ Version : 1.17.14.0 Date : 12/27/2017 ================================================================================ FIXES ----- BASE: - Fixed an FLR failure during simulteneous power up of VM. - Fixed an issue in vlan acl which was limiting vlan range to 1024. ETH: - Enabled RS-FEC for 25G active copper cable and 25GBASE-SR. - When auto negotiation is enabled, final pause settings are resolved based on local and peer pause settings. - Handle NACK for an I2C access. OFLD: - Fixed rdma connection cleanup in SO adpater. - Fixed rdma connections during read invalidate. - Fixed the crash when invalid BW rate is passed to fw. - Fixed the traffic hang when BW allocation is changed from switch during traffic. FOFCoE: - Fixed an issue where initiator remains logged-in even after LLDP is disabled on switch. ENHANCEMENTS ------------ BASE: - Added support for 248 VFs. - Added fw driver periodic calibration for MC. ETH: - Added XLAUI port type support. - Added raw mac entry deletion support (FW_VI_MAC_ID_BASED_FREE). OFLD: - Inline IPSec support added (flag F_FW_ULPTX_WR_DATA indicates the inline IPSec WR). - New work request FW_RI_RDMA_WRITE_CMPL_WR (write with completion) added to ================================================================================ ================================================================================ Version : 1.16.63.0 Date : 09/29/2017 ================================================================================ FIXES ----- BASE: - Fixed a fw crash when configured traffic rate limit is less than 10kbps. - Fixed traffic rate limiting for smaller traffic rate value. ETH: - Fixed 40G link failure when interface is toggled. - Fixed adpter crash when interface is toggled during traffic. - Fixed 25G link failure when PEER, only supports consortium mode autoneg for 25G. - Fixed 100G optics link failure when cable is plugged in after bringing up the interface. - Enable RS FEC as default if speed is 100G. - Fixed DCBX configuration refreash failure. OFLD: - Fixed 0B iWARP ingress read failure. - Fixed iWARP SRQ reuse failure. FOiSCSI: - Fixed vlan interface ping failure. - Fixed target discovery failures. - Fixed mutual chap login failure. ================================================================================ ================================================================================ Version : 1.16.59.0 Date : 08/14/2017 ================================================================================ FIXES ----- BASE: - Fixed fw crash caused by MC parity error in SO adapters. - Generate Timer0Int interrupt if fw crashes due to unaligned access error. Host driver must look into PCIE_FW register to see if any fw fatal error has encountered. If PCIE_FW doesn't indicate any error then driver must ignore this interrupt. - Fixed receive buffer threshold settings which was resulting in error frames on receive side. ETH: - Fixed an issue in connection traffic shaping when FLOWC_WR->FW_FLOWC_MNEM_SCHEDCLASS is not received in first WR on the connection. - Fixed link failure when speed is changed from 10G-1G-10G due to incorrect flag check. - Fixed improper LED behaviour for blink test and when traffic is running. - Removed storage of previous fec settings from fw. Driver needs to pass the user settings whenever a new module is plugged in as fw resets these when a module is unplugged. OFLD: - OVS offload: TP cache is flushed periodically to get the accuate filters stats (hit count). ENHANCEMENTS ------------ BASE: - Ring backbone feature added. New FW_PARAMS_PARAM_DEV_RING_BACKBONE param type added to query and enable ring backbone support. - VNI support added for filtering. New entry_type FW_VI_MAC_TYPE_EXACTMAC_VNI added to FW_VI_MAC_CMD. - Added new API FW_PARAM_PARAM_DEV_MPSBGMAP to read the priority to buffer group mapping for the ports. - FW_PARAMS_PARAM_DEV_TPCHMAP API added to read the port to channel mapping. - HMA (Host memory access) support added. New FW_HMA_CMD and FW_PARAMS_PARAM_DEV_HMA_SIZE added to query and configure the HMA. It enables the memfree support (256 connections) for iwarp. - PTP support enabled. ETH: - Added consortium mode 50G support. - Added the ability to allow only selected speeds to be advertised during auto negotiation. - Increased port capability from 16 to 32 bits to support more speeds. FW_PARAMS_PARAM_PFVF_PORT_CAPS32 added to query whether fw supports 16 or 32 bit port capability. OFLD: - RDMA Write with immediate support added (iwarp 2.0 feature) - FW_TLS_KEYCTX_TX_WR removed and security key management moved to driver. - 256 offloaded connections support for iwarp on SO adapters. iSCSI: - New param FW_PARAMS_PARAM_DEV_PPOD_EDRAM added for iscsi ppod configuration in EDRAM (performance improvement). FOiSCSI: - iSCSI Command offload target support added. FOFCoE: - FCoE support enabled. ================================================================================ ================================================================================ Version : 1.16.43.0 Date : 05/05/2017 ================================================================================ FIXES ----- BASE: - Fixed default DCB mode to AUTO. - Fixed DCBX bugs when AUTO mode is configured in config file. - Fixed an issue where even after removing PFC from switch, PFC wasn't getting reset. - Fixed DDR3/DDR4 ECC errors. - Fixed an FLR issue where FLR completion was going to host before FLR processing is finished in fw. ETH: - Fixed bug in writing multi-bytes using i2c interface. - Fixed the link failure when optical cable is inserted into the QSA module after loading the driver. - Fixed false link up when peer interface was brought down. - Enabling RS FEC by default for 100Gbase-SR4 according to 802.3BJ standard. - Fixed bugs related to negotiated fec based local/peer fec ability and request. - Fixed auto-neg failure with few switches. - T6 Performance improvement fixes. OFLD: - Fixed an extra credit issue if FW_RI_TYPE_FINI is delayed in fw due to backpressure. - Added a new queue type FW_IQ_TYPE_VF_CQ to handle the FW_PARAMS_PARAM_DMAQ* commands. queue type will be part of the FW_PARAMS_PARAM_DMAQ_IQ_INTIDX value. Used in guest RDMA (RDMA from VM/VF) usecase. - T6 Crypto Coprocessor mode bug fixes. - T6 Crypto TLS-inline mode bug fixes. ENHANCEMENTS ------------ BASE: - Added new API FW_PARAM_PARAM_DEV_MPSBGMAP to read the priority to buffer group mapping for the ports. ETH: - Added broadcom consortium next page support for 25G CR. This can be enabled using flags=an_brcm option in the t6-config.txt file. - Added spider mode support. - Added support for 10G-BaseT converter sfp+ module. - Added support for additional 25G/100G cables. - Added support to enable/disable auto-neg using ethtool. ================================================================================ ================================================================================ Version : 1.16.33.0 Date : 02/24/2017 ================================================================================ Fixes ----- BASE: - Fixed DDR4 uncorrectable errors. ETH: - Enabled link auto negotiation (AN) by default in config file. - Added AN and FEC control api. Host driver and application can enable/disable AN and FEC. ENHANCEMENTS ------------ BASE: - Enabled High priorty filter. - Added T6425 adapter support. ETH: - Added new workrequest ETH_TX_PKTS2_WR (see fw api document for more details). ================================================================================ ================================================================================ Version : 1.16.29.0 Date : 01/27/2017 ================================================================================ FIXES ----- BASE: - Set multiple fec values only if AN is enabled in config file and when module is connected. - Fixed intermittent DDR3/4 ECC errors. - max number of ethctrl queue in VF set to 2 (reverted the last change because it causes problem in VF drivers). ETH: - Made devlog more verbose by printing cable information in redable form. - Updated AN settings to work with more 25G/100G switches. - Added support for more SFP28/QSFP28 cables. - Fixed an issue of link going down after few hours of idle time. OFLD: - Fixed an issue in TLS which was causing fw crash on running TLS traffic. FOiSCSI: - Fixed the failure of PXE boot OS install on an iscsi lun. ENHANCEMENTS ------------ OFLD: - Added filtering support for NAT. New WR FW_FILTER2_WR and FW_PARAMS_PARAM_DEV_FILTER2_WR added for the same. - Added RDMA guest mode (mode 3 or RDMA from VF) support. T5 Firmware =========== Version : 1.23.4.0 Date : 03/27/2019 ================================================================================ Fixes ----- ETH: - Fixed a crash when ports are configured as 4x10Gbase-KR. - Fixed an issue in UDP-SO (segmentation offload) handling where Tx packets were getting truncated on wire. - Fixed a crash on adding more than 200 vlans. OFLD: - Pass wrid in error CQE. FOiSCSI: - IPV6 extension header support added. - ARP and neighbor solicitation retry timers are tuned to reduce the iscsi login time. - drop fragmented ICMP_TYPE_ECHO request packets. Enhancements ------------ BASE: - New command FW_PARAMS_PARAM_PFVF_LINK_STATE added to control VF link state notification to VF driver. - nvf parameter in firmware config file is deprecated. ETH: - Tx performance improvement in VF NIC path (when mac acl and non-transparent vlan are not used). Non-transparent vlan means when VF is aware of VLAN. - Default NIC WR completion timer changed to 150us which significantly lower completion Interrupt overhead. OFLD: - hashfilter enabled by default along with offload in firmware embedded config file. - Adjust iwarp ird/ord values according to the adapter memory. This enables larger value of ird/ord on T6 25G adapters for long distance links. - New APIs added to modify filterModes and filterMask from driver dynamically. - Fixed an issue where iwarp connection cleanup is not complete. ================================================================================ ================================================================================ Version : 1.22.8.0 Date : 12/26/2018 ================================================================================ Fixes ----- BASE: - Fixed the PTP issue observed on port 1. - Fixed the spurious temprature value returned by firmware. ETH: - Fixed a VF traffic hang when it is run along with PF traffic. - Made AN (auto-negotiation) mandatory for 10GBASE-T PHYs. OFLD: - Fixed errors and connection drop on other connections when CRC error occurs on one iwarp connection. - Issue in CQE error handling fixed. - Fixed RDMA connection close hangs under heavy traffic conditions. FOiSCSI: - Fixed a bug in redirect target handling by Option ROM. Enhancements ------------ BASE: - New field vfvld and vin added to FW_VI_CMD. - smtid field added to FW_VI_MAC_CMD. - New device param command FW_PARAMS_PARAM_DEV_OPAQUE_VIID_SMT_EXTN added to check if firmware supports extended VIID format or not. Note that driver must not look inside(interpret) VIID. Instead use vfvld, vin and smtid field as returned by the firmware. - Per PF parameter FW_PARAMS_PARAM_PFVF_GET_SMT_START and FW_PARAMS_PARAM_PFVF_GET_SMT_SIZE added. - Enable hashfilter along with offload config. - New API ptp_rx_ctrl added to clear MPS_RX_PTP_CTL per port settings. ETH: - Send (vlan or mac) ACL error message to master PF on asynchronous firmware event queue instead of mailbox. ================================================================================ ================================================================================ Version : 1.21.5.0 Date : 09/26/2018 ================================================================================ Fixes ----- BASE: - Incorrect mac reference count issue fixed. - VF FLR resetting VLAN issue fixed. - Always enable PFC if ETS is configured. - Incosistent VF rate limiting fixed. OFLD: - Connection hang due to CQE error after memory read fixed. FOiSCSI: - Ping to iscsi interface failure fixed. - Firmware crash while running IO and start/stop target in a loop fixed. Enhancements ------------ BASE: - 2 new bits, F_FW_FILTER2_WR_DROP_ENCAP to drop encapsulation header and F_FW_FILTER2_WR_TX_LOOP to loopback nat'ed packets are added to FW_FILTER2_WR. ETH: - New API FW_PARAMS_PARAM_PFVF_MAX_PKTS_PER_ETH_TX_PKTS_WR added to know the max number of packets allwoed in FW_ETH_TX_PKTS_WR and FW_ETH_TX_PKTS_VM_WR. - Performance improvement in handling coalesced packet from VMs. ================================================================================ ================================================================================ Version : 1.20.8.0 Date : 07/06/2018 ================================================================================ Fixes ----- ETH: - Allows PF to access VI stats of any of its VF. - Fixes a bug in DCBX where dynamic ETS changes were not obeyed by fw. FOiSCSI: - Ping response with data above 56 bytes fixed. - IPv6 hdr size not substracted from MSS value is fixed. Enhancements ------------ BASE: - FW_VI_MAC_CMD modified to accept loopback port number. Now MPS-TCAM entries can be created for loopback port from driver. - FW_VI_MAC_CMD modified to query/lookup index of SMAC/DMAC address. OFLD: - Enabled MSS segmentation offload support. ================================================================================ ================================================================================ Version : 1.19.1.0 Date : 04/23/2018 ================================================================================ Fixes ----- BASE: - Fixes a firmware crash in FW_ETH_TX_EO_WR handling. - Fixes host DCB support when FW_PORT_CMD is used. ETH: - Fixes an issue where VF packets counter were not increasing if VF packets coalesced WR is used by driver. OFLD: - Fixes an issue where fw hangs if max traffic rate passed is 0. FOiSCSI: - Fixes fw crash when trying to connect to non-existence IPv6 iSNS target. ================================================================================ ================================================================================ Version : 1.18.9.0 Date : 03/27/2018 ================================================================================ Fixes ----- BASE: - For Ethernet frames less than 64B, pad them with zero bytes as per IEEE spec (RFC 894). - Added a new parameter iqtype to FW_IQ_CMD to identify the ingress NIC or offload queues. This fixes an issue where driver was receiving interrupt with no new messages in queue. ETH: - Pad the Ethernet packets of size less than 64B with zeros. This fixes the incorrect checksum generation of packets less then 64B. FOiSCSI: - Fixes a fw crash when wrong node-id is passed to FW_FOISCSI_CTRL_WR. FOFCoE: - Fixes a fw hang while creating NPIV. Enhancements ------------ ETH: - A new WR FW_ETH_TX_PKTS_VM_WR added to support VM packet coalescing. ================================================================================ ================================================================================ Version : 1.18.4.0 Date : 02/28/2018 ================================================================================ Fixes ----- BASE: - Fixed starting more than 32 VMs on PF4 causing firmware hang. FOiSCSI: - Fixed a bug causing login failure when connecting to multiple targets. Enhancements ------------ BASE: - Added a new firmware API to retrieve the maximum temperaturethreshold for the chip (FW_PARAM_DEV_DIAG_MAXTMPTHRESH). ETH: - Added support for user to contol pause negotiation during auto negotiation. FOiSCSI: - Added a new facility to redirect few fw events to offload rx queue (based on driver's configration) - Driver can ignore providing ipv6 prefix len during ipv6 address configuration. ================================================================================ ================================================================================ Version : 1.17.14.0 Date : 12/27/2017 ================================================================================ FIXES ----- BASE: - Fixed an issue in vlan acl which was limiting vlan range to 1024. ETH: - Corrected lane inversion logic. - Fixed improper LED behavior in T580 cards. - When auto negotiation is enabled, final pause settings are resolved based on local and peer pause settings. - Handle NACK for an I2C access. OFLD: - Fixed rdma connections during read invalidate. FOiSCSI: - Fixed a connections hang when link is toggled frequently. FOFCoE: - Fixed an issue where initiator remains logged-in even after LLDP is disabled on switch. ENHANCEMENTS ------------ BASE: - Added support for 124 VFs. ETH: - Added XLAUI port type support. - Added raw mac entry deletion support (FW_VI_MAC_ID_BASED_FREE). OFLD: - New work request FW_RI_RDMA_WRITE_CMPL_WR (write with completion) added to optimize NVMEoF write. ================================================================================ ================================================================================ Version : 1.16.63.0 Date : 09/29/2017 ================================================================================ FIXES ----- BASE: - Fixed offload memory overcommit in case of SO adapter. ETH: - Fixed DCBX configuration refreash failure. OFLD: - Fixed 0B iWARP ingress read failure. FOiSCSI: - Fixed vlan interface ping failure. ================================================================================ ================================================================================ Version : 1.16.59.0 Date : 08/14/2017 ================================================================================ FIXES ----- BASE: - Fixed an FLR issue which was causing error when VF attached VM was powered on. ETH: - Fixed an issue in connection traffic shaping when FLOWC_WR->FW_FLOWC_MNEM_SCHEDCLASS is not received in first WR on the connection. - Fixed link failure when speed is changed from 10G-1G-10G due to incorrect flag check. - Fixed T580 link failure with few switches which take more time for establishing link. ENHANCEMENTS ------------ BASE: - Ring backbone feature added. New FW_PARAMS_PARAM_DEV_RING_BACKBONE param type added to query and enable ring backbone support. - Added new API FW_PARAM_PARAM_DEV_MPSBGMAP to read the priority to buffer group mapping for the ports. - FW_PARAMS_PARAM_DEV_TPCHMAP API added to read the port to channel mapping. FOiSCSI: - iSCSI Command offload target support added. ================================================================================ ================================================================================ Version : 1.16.43.0 Date : 05/05/2017 ================================================================================ FIXES ----- BASE: - Fixed default DCB mode to AUTO. - Fixed DCBX bugs when AUTO mode is configured in config file. - Fixed an issue where even after removing PFC from switch, PFC wasn't getting reset. ETH: - Fixed bug in writing multi-bytes using i2c interface. - Fixed the link failure when optical cable is inserted into the QSA module after loading the driver. OFLD: - Fixed an extra credit issue if FW_RI_TYPE_FINI is delayed in fw due to backpressure. - Added a new queue type FW_IQ_TYPE_VF_CQ to handle the FW_PARAMS_PARAM_DMAQ* commands. queue type will be part of the FW_PARAMS_PARAM_DMAQ_IQ_INTIDX value. Used in guest RDMA (RDMA from VM/VF) usecase. ENHANCEMENTS ------------ BASE: - Added new API FW_PARAM_PARAM_DEV_MPSBGMAP to read the priority to buffer group mapping for the ports. ================================================================================ ================================================================================ Version : 1.16.33.0 Date : 02/24/2017 ================================================================================ ENHANCEMENTS ------------ ETH: - Added new workrequest ETH_TX_PKTS2_WR (see fw api document for more details). ================================================================================ ================================================================================ Version : 1.16.29.0 Date : 01/27/2017 ================================================================================ FIXES ----- BASE: - max number of ethctrl queue in VF set to 2 (reverted the last change because it causes problem in VF drivers). FOiSCSI: - Fixed the failure of PXE boot OS install on an iscsi lun. ENHANCEMENTS ------------ OFLD: - Added filtering support for NAT. New WR FW_FILTER2_WR and FW_PARAMS_PARAM_DEV_FILTER2_WR added for the same. - Added RDMA guest mode (mode 3 or RDMA from VF) support. ******************************************************************************** Copyright (C) 2019 Chelsio Communications. All Rights Reserved. The information in this document is furnished for informational use only, is subject to change without notice, and should not be construed as a commitment by Chelsio Communications. Chelsio Communications assumes no responsibility or liability for any errors or inaccuracies that may appear in this document or any software that may be provided in association with this document. Except as permitted by such license, no part of this document may be reproduced, stored in a retrieval system,or transmitted in any form or by any means without the express written consent of Chelsio Communications.