Kernel-2.6.32-573.12.1.el6_s2io

Release notes for Neterion’s (Formerly S2io) Xframe I/II PCI-X 10GbE driver.

Contents

    1. Introduction
    1. Identifying the adapter/interface
    1. Features supported
    1. Command line parameters
    1. Performance suggestions
    1. Available Downloads
  1. Introduction:
    This Linux driver supports Neterion’s Xframe I PCI-X 1.0 and
    Xframe II PCI-X 2.0 adapters. It supports several features
    such as jumbo frames, MSI/MSI-X, checksum offloads, TSO, UFO and so on.
    See below for complete list of features.
    All features are supported for both IPv4 and IPv6.

  2. Identifying the adapter/interface:
    a. Insert the adapter(s) in your system.
    b. Build and load driver

    insmod s2io.ko

    c. View log messages

    dmesg | tail -40

    You will see messages similar to:
    eth3: Neterion Xframe I 10GbE adapter (rev 3), Version 2.0.9.1, Intr type INTA
    eth4: Neterion Xframe II 10GbE adapter (rev 2), Version 2.0.9.1, Intr type INTA
    eth4: Device is on 64 bit 133MHz PCIX(M1) bus

The above messages identify the adapter type(Xframe I/II), adapter revision,
driver version, interface name(eth3, eth4), Interrupt type(INTA, MSI, MSI-X).
In case of Xframe II, the PCI/PCI-X bus width and frequency are displayed
as well.

To associate an interface with a physical adapter use “ethtool -p “.
The corresponding adapter’s LED will blink multiple times.

  1. Features supported:
    a. Jumbo frames. Xframe I/II supports MTU upto 9600 bytes,
    modifiable using ifconfig command.

b. Offloads. Supports checksum offload(TCP/UDP/IP) on transmit
and receive, TSO.

c. Multi-buffer receive mode. Scattering of packet across multiple
buffers. Currently driver supports 2-buffer mode which yields
significant performance improvement on certain platforms(SGI Altix,
IBM xSeries).

d. MSI/MSI-X. Can be enabled on platforms which support this feature
(IA64, Xeon) resulting in noticeable performance improvement(upto 7%
on certain platforms).

e. Statistics. Comprehensive MAC-level and software statistics displayed
using “ethtool -S” option.

f. Multi-FIFO/Ring. Supports up to 8 transmit queues and receive rings,
with multiple steering options.

  1. Command line parameters
    a. tx_fifo_num
    Number of transmit queues
    Valid range: 1-8
    Default: 1

b. rx_ring_num
Number of receive rings
Valid range: 1-8
Default: 1

c. tx_fifo_len
Size of each transmit queue
Valid range: Total length of all queues should not exceed 8192
Default: 4096

d. rx_ring_sz
Size of each receive ring(in 4K blocks)
Valid range: Limited by memory on system
Default: 30

e. intr_type
Specifies interrupt type. Possible values 0(INTA), 2(MSI-X)
Valid values: 0, 2
Default: 2

  1. Performance suggestions
    General:
    a. Set MTU to maximum(9000 for switch setup, 9600 in back-to-back configuration)
    b. Set TCP windows size to optimal value.
    For instance, for MTU=1500 a value of 210K has been observed to result in
    good performance.

    sysctl -w net.ipv4.tcp_rmem=”210000 210000 210000”

    sysctl -w net.ipv4.tcp_wmem=”210000 210000 210000”

    For MTU=9000, TCP window size of 10 MB is recommended.

    sysctl -w net.ipv4.tcp_rmem=”10000000 10000000 10000000”

    sysctl -w net.ipv4.tcp_wmem=”10000000 10000000 10000000”

Transmit performance:
a. By default, the driver respects BIOS settings for PCI bus parameters.
However, you may want to experiment with PCI bus parameters
max-split-transactions(MOST) and MMRBC (use setpci command).
A MOST value of 2 has been found optimal for Opterons and 3 for Itanium.
It could be different for your hardware.
Set MMRBC to 4K**.

For example you can set
For opteron
#setpci -d 17d5:* 62=1d
For Itanium
#setpci -d 17d5:* 62=3d

For detailed description of the PCI registers, please see Xframe User Guide.

b. Ensure Transmit Checksum offload is enabled. Use ethtool to set/verify this
parameter.
c. Turn on TSO(using “ethtool -K”)

ethtool -K tso on

Receive performance:
a. By default, the driver respects BIOS settings for PCI bus parameters.
However, you may want to set PCI latency timer to 248.
#setpci -d 17d5:* LATENCY_TIMER=f8
For detailed description of the PCI registers, please see Xframe User Guide.
b. Use 2-buffer mode. This results in large performance boost on
certain platforms(eg. SGI Altix, IBM xSeries).
c. Ensure Receive Checksum offload is enabled. Use “ethtool -K ethX” command to
set/verify this option.
d. Enable NAPI feature(in kernel configuration Device Drivers —> Network
device support —> Ethernet (10000 Mbit) —> S2IO 10Gbe Xframe NIC) to
bring down CPU utilization.

** For AMD opteron platforms with 8131 chipset, MMRBC=1 and MOST=1 are
recommended as safe parameters.
For more information, please review the AMD8131 errata at
http://www.amd.com/us-en/assets/content_type/white_papers_and_tech_docs/26310.pdf

  1. Available Downloads
    Neterion “s2io” driver in Red Hat and Suse 2.6-based distributions is kept up
    to date, also the latest “s2io” code (including support for 2.4 kernels) is
    available via “Support” link on the Neterion site: http://www.neterion.com.

For Xframe User Guide (Programming manual), visit ftp site ns1.s2io.com,
user: linuxdocs password: HALdocs

  1. Support
    For further support please contact either your 10GbE Xframe NIC vendor (IBM,
    HP, SGI etc.) or click on the “Support” link on the Neterion site:
    http://www.neterion.com.