**************************************** README **************************************** Chelsio T5 NVMeF Driver for Linux Version : 1.0.0.0 (Alpha) Date : 07/12/2016 Overview ================================================================================ Chelsio T5 NVMeF driver package for Linux is for Chelsio's 10/40Gb Unified Wire adapters. The package provides an installer to install the following components: - Kernel 4.7+ (RC2) with NVMe Over Fabrics support - iWARP libraries v1.3.6 - nvme-cli utility (only for initiator) Note: The kernel is taken from git://git.infradead.org/nvme-fabrics.git, nvmf-all.3 branch. ================================================================================ CONTENTS ================================================================================ - 1. Requirements - 2. Supported Operating Systems - 3. Supported Cards - 4. How To Use - 5. Support Documentation - 6. Customer Support 1. Requirements ================================================================================ - Chelsio T5 10/40G adapter and system with supported platforms mentioned in section 2. 2. Supported Operating Systems ================================================================================ - RHEL7.2, 3.10.0-327.el7 - RHEL7.1, 3.10.0-229.el7 Other versions have not been tested and are not guaranteed to work. 3. Supported Hardware ================================================================================ - T580-CR - T580-LP-CR - T540-CR - T520-CR - T520-LL-CR - T520-BT 4. How to Use ================================================================================ 4.1. Installation ================= Follow the steps mentioned below to install NVMeF supported kernel and related components on both target and initiator machines: a) Download the driver package, Chelsio-NVMeF-x.x.x.x.tar.gz b) Untar the tarball and change your working directory to Chelsio-NVMeF-x.x.x.x [root@host~]# tar zxvfm Chelsio-NVMeF-x.x.x.x.tar.gz [root@host~]# cd Chelsio-NVMeF-x.x.x.x c) On target, run the following command: [root@host~]# ./setup.sh -t On initiator, run the following command: [root@host~]# ./setup.sh -i d) Reboot system to installed 4.7+ (RC2) kernel 4.2. Driver Loading =================== Follow the steps mentioned below on both target and initiator machines: a) Load the following modules: [root@host~]# modprobe iw_cxgb4 [root@host~]# modprobe rdma_ucm b) Bring up the Chelsio interface(s): [root@host~]# ifconfig ethX x.x.x.x up c) On target, run the following commands: [root@host~]# modprobe null_blk [root@host~]# modprobe nvmet [root@host~]# modprobe nvmet-rdma On initiator, run the following commands: [root@host~]# modprobe nvme [root@host~]# modprobe nvme-rdma 4.3. Firmware Update ==================== The T5 (v1.15.37.0) firmware is installed on the system, typically in /lib/firmware/cxgb4, and the driver will auto-load the firmware if an update is required. The kernel must be configured to enable userspace firmware loading support: Device Drivers -> Generic Driver Options -> Userspace firmware loading support The firmware version can be verified using ethtool: [root@host~]# ethtool -i 4.4. Configuration =================== 4.4.1. Target ------------- The following commands will configure target with a LUN: [root@host~]# mkdir /sys/kernel/config/nvmet/subsystems/ [root@host~]# mkdir /sys/kernel/config/nvmet/subsystems//namespaces/1 [root@host~]# echo -n >/sys/kernel/config/nvmet/subsystems//namespaces/1/device_path [root@host~]# echo 1 > /sys/kernel/config/nvmet/subsystems//attr_allow_any_host [root@host~]# echo 1 > /sys/kernel/config/nvmet/subsystems//namespaces/1/enable [root@host~]# mkdir /sys/kernel/config/nvmet/ports/1 [root@host~]# echo "ipv4" > /sys/kernel/config/nvmet/ports/1/addr_adrfam [root@host~]# echo "rdma" > /sys/kernel/config/nvmet/ports/1/addr_trtype [root@host~]# echo "4420" > /sys/kernel/config/nvmet/ports/1/addr_trsvcid [root@host~]# echo "" > /sys/kernel/config/nvmet/ports/1/addr_traddr [root@host~]# ln -s /sys/kernel/config/nvmet/subsystems/ /sys/kernel/config/nvmet/ports/1/subsystems/ 4.4.2. Initiator ---------------- a) Discover the target [root@host~]# nvme discover -t rdma -a -s 4420 b) Connect to target Connecting to a specific target: [root@host~]# nvme connect -t rdma -a -s 4420 -n Connecting to all targets configured on a portal [root@host~]# nvme connect-all -t rdma -a -s 4420 c) List the connected targets [root@host~]# nvme list d) Format and mount the NVMe disks shown with the above command. e) Disconnect from the target and unmount the disk: [root@host~]# nvme disconnect -d 5. Support Documentation ================================================================================ The documentation for this release can be found inside the Chelsio-NVMeF-x.x.x.x/docs folder. It contains: - README - Release Notes 6. Customer Support ================================================================================ Please contact Chelsio support at support@chelsio.com for any issues regarding the product. ******************************************************************************** Copyright (C) 2016 Chelsio Communications. All Rights Reserved. The information in this document is furnished for informational use only, is subject to change without notice, and should not be construed as a commitment by Chelsio Communications. Chelsio Communications assumes no responsibility or liability for any errors or inaccuracies that may appear in this document or any software that may be provided in association with this document. Except as permitted by such license, no part of this document may be reproduced, stored in a retrieval system,or transmitted in any form or by any means without the express written consent of Chelsio Communications.