Chelsio T3 Solaris driver
-------------------------
Solaris Operating System Support
----------------------------------------------------------
Solaris Driver is GLDv3 based driver but for backward compatibility we
also support GLDv2 based Solaris OS.
Solaris 10
-------------------
Solaris 10 update 1(GLDv2)
Solaris 10 update 2(GLDv2)
Solaris 10 update 3(GLDv2)
Solaris 10 update 4(GLDv3)
Solaris 10 update 5(GLDv3)
Solaris 10 update 6(GLDv3)
Opensolaris
-------------------
Opensolaris 2008.11
(Note: Opensolaris 2008.11 is based on opensolaris build nv101.
Drive may work for builds upto nv104 but has not been tested
and is not guaranteed to work. Any builds above opensolaris
nv104 will not work.)
Chelsio Card Support
-------------------------------------------------------------
This release of the Chelsio Solaris Ethernet driver supports the following
Chelsio network adapters:
N320E (SFP+)
S320E (SFP+)
N320E-CXA
S320E-CXA
N310E (SFP+)
S310E (SFP+)
N310E-CXA
S310E-CXA
S302E
Driver Installation
-----------------------------------------------------------
The Chelsio T3 adapter driver supports 1G and 10G adapters.
The driver is made up of two components a Nexus driver
which attaches to the card and a function driver which
applies to an Ethernet port provided by the Nexus.
The driver installs 2 files into $PLATFORM/kernel/drv
cxgen - Is the bus nexus.
cxge - Is the Ethernet Driver.
The steps to getting the driver working is as follows.
1. Uncompress/Untar the cxge-solaris-1.0.tar.gz file.
2. In the directory containing the Uncompressed/
untared files run the 'pkgadd' command
as follows.
pkgadd -d .
3. The Installation will present 4 packages.
CHLSt3ei - The 32 bit driver set for x86.
CHLSt3eix - The 64 bit driver set for x86.
CHLSt3es - The 32 bit driver set for SPARC.
CHLSt3esx - The 64 bit driver set for SPARC.
4. Follow the instuctions to complete the
installation.
5. At this point it's a good time to configure usage of
Jumbo frames. This can be done by creating and
editing the cxge.conf file. This file is placed
under /kernel/drv.
The required addition to this file is
accept-jumbo=1;
This will cause all of the 'cxge' instances to be enabled
in Jumbo Frames mode. If a more direct per instance approach
is required the following type of configuration line needs
to be added. The 'unit-address' is the 'cxge' instance
number.
name="cxge" parent="cxgen" unit-address="0" accept-jumbo=1;
6. Once the installation is complete reboot the system.
7. Once the system has rebooted the nexus driver 'cxgen'
should have attached to the card and created
the 'cxge' nodes, use prtconf' to verify this.
Output should contain the following for a successful
install.
cxgen, instance #0
cxge, instance #0
8. Having confirmed the successful attach of the cxgen, cxge
pair you can proceed to brining up the Ethernet Interface
using 'ifconfig'.
ifconfig cxge0 plumb 123.123.123.123 netmask 0xffffff00 Broadcast + up
9. You can check the link state using 'kstat'
kstat cxge:0 | grep link
10. If the link is up and an IP network is established one can
begin to 'ping' and run any type of benchmark.
11. The interface mtu is set by default to 1500 bytes, but can be
adjusted to Jumbo frames by configuring the driver as described
in step 6 above.
To see the current mtu setting as indicated by the driver use
ifconfig as follows....
ifconfig cxge0
cxge0: flags=1000803
inet 102.50.50.27 netmask ffffff00 broadcast 102.50.50.255
In the above example teh MTU ia 9000, which is considered Jumbo frames.
Once set to Jumbo frame the mtu can be adjusted back to regular frame
size simply by using 'ifconfig' as follows
ifconfig cxge0 mtu 1500
To see the update do
ifconfig cxge0
cxge0: flags=1000803
inet 102.50.50.27 netmask ffffff00 broadcast 102.50.50.255
12. For the benefit of maximum performance there are a few tuning parameters
that are useful to increase the tcp/udp performance.
Those parameters can be adjusted using ndd.
/usr/sbin/ndd -set /dev/tcp tcp_max_buf 4194304
/usr/sbin/ndd -set /dev/tcp tcp_recv_hiwat 1048576
/usr/sbin/ndd -set /dev/tcp tcp_xmit_hiwat 1048576
/usr/sbin/ndd -set /dev/udp udp_max_buf 4194304
/usr/sbin/ndd -set /dev/udp udp_xmit_hiwat 1048576
/usr/sbin/ndd -set /dev/udp udp_recv_hiwat 1048576
13. Driver Tuning Parameters
Driver tuning parameters can be set and read using ndd interface.
"#ndd /dev/cxge0 ?" will give you all the parameters which can
be set. This are
1."accpet-jumbo" this can be set to "1" or "0" to enable/disable
jumbo frame support dynamically.
2."tx-lb" this can set set to 1 or 0 to enable/disable tx load
balancing.
3."hw-csum" this can set to 1 or 0 to enable/disable hardware checksum
support
4."rx-pause" this can be set to 1 or 0 to enable/disable receiving pause
frames.
5."tx_pause" this can be set to 1 or 0 to enable/disable transmit pause
frames.
6."coalesce" this is set to 5 by defaults which can be set in range from
1 to 250.
All the configuration using ndd take effect after unplumb/plumb of the
interface. Also all the configuration are lost on a reboot. To set static
parameters across a reboot please set them as variable in /kernen/drv/cxge.conf
and /kernel/drc/cxgen.conf file.
* cxgen.conf can have these tunables:
-sge-engines
-interrupt-types
* cxge.conf can have these tunables:
-accept-jumbo
-tx-lb
-hw-csum
-lso
-rx_pause
-tx_pause