350 lines
13 KiB
Plaintext
350 lines
13 KiB
Plaintext
Linux* Base Driver for the Intel(R) Ethernet 10 Gigabit PCI Express Family of
|
|
Adapters
|
|
=============================================================================
|
|
|
|
Intel 10 Gigabit Linux driver.
|
|
Copyright(c) 1999 - 2013 Intel Corporation.
|
|
|
|
Contents
|
|
========
|
|
|
|
- Identifying Your Adapter
|
|
- Additional Configurations
|
|
- Performance Tuning
|
|
- Known Issues
|
|
- Support
|
|
|
|
Identifying Your Adapter
|
|
========================
|
|
|
|
The driver in this release is compatible with 82598, 82599 and X540-based
|
|
Intel Network Connections.
|
|
|
|
For more information on how to identify your adapter, go to the Adapter &
|
|
Driver ID Guide at:
|
|
|
|
http://support.intel.com/support/network/sb/CS-012904.htm
|
|
|
|
SFP+ Devices with Pluggable Optics
|
|
----------------------------------
|
|
|
|
82599-BASED ADAPTERS
|
|
|
|
NOTES: If your 82599-based Intel(R) Network Adapter came with Intel optics, or
|
|
is an Intel(R) Ethernet Server Adapter X520-2, then it only supports Intel
|
|
optics and/or the direct attach cables listed below.
|
|
|
|
When 82599-based SFP+ devices are connected back to back, they should be set to
|
|
the same Speed setting via ethtool. Results may vary if you mix speed settings.
|
|
82598-based adapters support all passive direct attach cables that comply
|
|
with SFF-8431 v4.1 and SFF-8472 v10.4 specifications. Active direct attach
|
|
cables are not supported.
|
|
|
|
Supplier Type Part Numbers
|
|
|
|
SR Modules
|
|
Intel DUAL RATE 1G/10G SFP+ SR (bailed) FTLX8571D3BCV-IT
|
|
Intel DUAL RATE 1G/10G SFP+ SR (bailed) AFBR-703SDDZ-IN1
|
|
Intel DUAL RATE 1G/10G SFP+ SR (bailed) AFBR-703SDZ-IN2
|
|
LR Modules
|
|
Intel DUAL RATE 1G/10G SFP+ LR (bailed) FTLX1471D3BCV-IT
|
|
Intel DUAL RATE 1G/10G SFP+ LR (bailed) AFCT-701SDDZ-IN1
|
|
Intel DUAL RATE 1G/10G SFP+ LR (bailed) AFCT-701SDZ-IN2
|
|
|
|
The following is a list of 3rd party SFP+ modules and direct attach cables that
|
|
have received some testing. Not all modules are applicable to all devices.
|
|
|
|
Supplier Type Part Numbers
|
|
|
|
Finisar SFP+ SR bailed, 10g single rate FTLX8571D3BCL
|
|
Avago SFP+ SR bailed, 10g single rate AFBR-700SDZ
|
|
Finisar SFP+ LR bailed, 10g single rate FTLX1471D3BCL
|
|
|
|
Finisar DUAL RATE 1G/10G SFP+ SR (No Bail) FTLX8571D3QCV-IT
|
|
Avago DUAL RATE 1G/10G SFP+ SR (No Bail) AFBR-703SDZ-IN1
|
|
Finisar DUAL RATE 1G/10G SFP+ LR (No Bail) FTLX1471D3QCV-IT
|
|
Avago DUAL RATE 1G/10G SFP+ LR (No Bail) AFCT-701SDZ-IN1
|
|
Finistar 1000BASE-T SFP FCLF8522P2BTL
|
|
Avago 1000BASE-T SFP ABCU-5710RZ
|
|
|
|
82599-based adapters support all passive and active limiting direct attach
|
|
cables that comply with SFF-8431 v4.1 and SFF-8472 v10.4 specifications.
|
|
|
|
Laser turns off for SFP+ when device is down
|
|
-------------------------------------------
|
|
"ip link set down" turns off the laser for 82599-based SFP+ fiber adapters.
|
|
"ip link set up" turns on the laser.
|
|
|
|
|
|
82598-BASED ADAPTERS
|
|
|
|
NOTES for 82598-Based Adapters:
|
|
- Intel(R) Network Adapters that support removable optical modules only support
|
|
their original module type (i.e., the Intel(R) 10 Gigabit SR Dual Port
|
|
Express Module only supports SR optical modules). If you plug in a different
|
|
type of module, the driver will not load.
|
|
- Hot Swapping/hot plugging optical modules is not supported.
|
|
- Only single speed, 10 gigabit modules are supported.
|
|
- LAN on Motherboard (LOMs) may support DA, SR, or LR modules. Other module
|
|
types are not supported. Please see your system documentation for details.
|
|
|
|
The following is a list of 3rd party SFP+ modules and direct attach cables that
|
|
have received some testing. Not all modules are applicable to all devices.
|
|
|
|
Supplier Type Part Numbers
|
|
|
|
Finisar SFP+ SR bailed, 10g single rate FTLX8571D3BCL
|
|
Avago SFP+ SR bailed, 10g single rate AFBR-700SDZ
|
|
Finisar SFP+ LR bailed, 10g single rate FTLX1471D3BCL
|
|
|
|
82598-based adapters support all passive direct attach cables that comply
|
|
with SFF-8431 v4.1 and SFF-8472 v10.4 specifications. Active direct attach
|
|
cables are not supported.
|
|
|
|
|
|
Flow Control
|
|
------------
|
|
Ethernet Flow Control (IEEE 802.3x) can be configured with ethtool to enable
|
|
receiving and transmitting pause frames for ixgbe. When TX is enabled, PAUSE
|
|
frames are generated when the receive packet buffer crosses a predefined
|
|
threshold. When rx is enabled, the transmit unit will halt for the time delay
|
|
specified when a PAUSE frame is received.
|
|
|
|
Flow Control is enabled by default. If you want to disable a flow control
|
|
capable link partner, use ethtool:
|
|
|
|
ethtool -A eth? autoneg off RX off TX off
|
|
|
|
NOTE: For 82598 backplane cards entering 1 gig mode, flow control default
|
|
behavior is changed to off. Flow control in 1 gig mode on these devices can
|
|
lead to Tx hangs.
|
|
|
|
Intel(R) Ethernet Flow Director
|
|
-------------------------------
|
|
Supports advanced filters that direct receive packets by their flows to
|
|
different queues. Enables tight control on routing a flow in the platform.
|
|
Matches flows and CPU cores for flow affinity. Supports multiple parameters
|
|
for flexible flow classification and load balancing.
|
|
|
|
Flow director is enabled only if the kernel is multiple TX queue capable.
|
|
|
|
An included script (set_irq_affinity.sh) automates setting the IRQ to CPU
|
|
affinity.
|
|
|
|
You can verify that the driver is using Flow Director by looking at the counter
|
|
in ethtool: fdir_miss and fdir_match.
|
|
|
|
Other ethtool Commands:
|
|
To enable Flow Director
|
|
ethtool -K ethX ntuple on
|
|
To add a filter
|
|
Use -U switch. e.g., ethtool -U ethX flow-type tcp4 src-ip 10.0.128.23
|
|
action 1
|
|
To see the list of filters currently present:
|
|
ethtool -u ethX
|
|
|
|
Perfect Filter: Perfect filter is an interface to load the filter table that
|
|
funnels all flow into queue_0 unless an alternative queue is specified using
|
|
"action". In that case, any flow that matches the filter criteria will be
|
|
directed to the appropriate queue.
|
|
|
|
If the queue is defined as -1, filter will drop matching packets.
|
|
|
|
To account for filter matches and misses, there are two stats in ethtool:
|
|
fdir_match and fdir_miss. In addition, rx_queue_N_packets shows the number of
|
|
packets processed by the Nth queue.
|
|
|
|
NOTE: Receive Packet Steering (RPS) and Receive Flow Steering (RFS) are not
|
|
compatible with Flow Director. IF Flow Director is enabled, these will be
|
|
disabled.
|
|
|
|
The following three parameters impact Flow Director.
|
|
|
|
FdirMode
|
|
--------
|
|
Valid Range: 0-2 (0=off, 1=ATR, 2=Perfect filter mode)
|
|
Default Value: 1
|
|
|
|
Flow Director filtering modes.
|
|
|
|
FdirPballoc
|
|
-----------
|
|
Valid Range: 0-2 (0=64k, 1=128k, 2=256k)
|
|
Default Value: 0
|
|
|
|
Flow Director allocated packet buffer size.
|
|
|
|
AtrSampleRate
|
|
--------------
|
|
Valid Range: 1-100
|
|
Default Value: 20
|
|
|
|
Software ATR Tx packet sample rate. For example, when set to 20, every 20th
|
|
packet, looks to see if the packet will create a new flow.
|
|
|
|
Node
|
|
----
|
|
Valid Range: 0-n
|
|
Default Value: 1 (off)
|
|
|
|
0 - n: where n is the number of NUMA nodes (i.e. 0 - 3) currently online in
|
|
your system
|
|
1: turns this option off
|
|
|
|
The Node parameter will allow you to pick which NUMA node you want to have
|
|
the adapter allocate memory on.
|
|
|
|
max_vfs
|
|
-------
|
|
Valid Range: 1-63
|
|
Default Value: 0
|
|
|
|
If the value is greater than 0 it will also force the VMDq parameter to be 1
|
|
or more.
|
|
|
|
This parameter adds support for SR-IOV. It causes the driver to spawn up to
|
|
max_vfs worth of virtual function.
|
|
|
|
|
|
Additional Configurations
|
|
=========================
|
|
|
|
Jumbo Frames
|
|
------------
|
|
The driver supports Jumbo Frames for all adapters. Jumbo Frames support is
|
|
enabled by changing the MTU to a value larger than the default of 1500.
|
|
The maximum value for the MTU is 16110. Use the ip command to
|
|
increase the MTU size. For example:
|
|
|
|
ip link set dev ethx mtu 9000
|
|
|
|
The maximum MTU setting for Jumbo Frames is 9710. This value coincides
|
|
with the maximum Jumbo Frames size of 9728.
|
|
|
|
Generic Receive Offload, aka GRO
|
|
--------------------------------
|
|
The driver supports the in-kernel software implementation of GRO. GRO has
|
|
shown that by coalescing Rx traffic into larger chunks of data, CPU
|
|
utilization can be significantly reduced when under large Rx load. GRO is an
|
|
evolution of the previously-used LRO interface. GRO is able to coalesce
|
|
other protocols besides TCP. It's also safe to use with configurations that
|
|
are problematic for LRO, namely bridging and iSCSI.
|
|
|
|
Data Center Bridging, aka DCB
|
|
-----------------------------
|
|
DCB is a configuration Quality of Service implementation in hardware.
|
|
It uses the VLAN priority tag (802.1p) to filter traffic. That means
|
|
that there are 8 different priorities that traffic can be filtered into.
|
|
It also enables priority flow control which can limit or eliminate the
|
|
number of dropped packets during network stress. Bandwidth can be
|
|
allocated to each of these priorities, which is enforced at the hardware
|
|
level.
|
|
|
|
To enable DCB support in ixgbe, you must enable the DCB netlink layer to
|
|
allow the userspace tools (see below) to communicate with the driver.
|
|
This can be found in the kernel configuration here:
|
|
|
|
-> Networking support
|
|
-> Networking options
|
|
-> Data Center Bridging support
|
|
|
|
Once this is selected, DCB support must be selected for ixgbe. This can
|
|
be found here:
|
|
|
|
-> Device Drivers
|
|
-> Network device support (NETDEVICES [=y])
|
|
-> Ethernet (10000 Mbit) (NETDEV_10000 [=y])
|
|
-> Intel(R) 10GbE PCI Express adapters support
|
|
-> Data Center Bridging (DCB) Support
|
|
|
|
After these options are selected, you must rebuild your kernel and your
|
|
modules.
|
|
|
|
In order to use DCB, userspace tools must be downloaded and installed.
|
|
The dcbd tools can be found at:
|
|
|
|
http://e1000.sf.net
|
|
|
|
Ethtool
|
|
-------
|
|
The driver utilizes the ethtool interface for driver configuration and
|
|
diagnostics, as well as displaying statistical information. The latest
|
|
ethtool version is required for this functionality.
|
|
|
|
The latest release of ethtool can be found from
|
|
http://ftp.kernel.org/pub/software/network/ethtool/
|
|
|
|
FCoE
|
|
----
|
|
This release of the ixgbe driver contains new code to enable users to use
|
|
Fiber Channel over Ethernet (FCoE) and Data Center Bridging (DCB)
|
|
functionality that is supported by the 82598-based hardware. This code has
|
|
no default effect on the regular driver operation, and configuring DCB and
|
|
FCoE is outside the scope of this driver README. Refer to
|
|
http://www.open-fcoe.org/ for FCoE project information and contact
|
|
e1000-eedc@lists.sourceforge.net for DCB information.
|
|
|
|
MAC and VLAN anti-spoofing feature
|
|
----------------------------------
|
|
When a malicious driver attempts to send a spoofed packet, it is dropped by
|
|
the hardware and not transmitted. An interrupt is sent to the PF driver
|
|
notifying it of the spoof attempt.
|
|
|
|
When a spoofed packet is detected the PF driver will send the following
|
|
message to the system log (displayed by the "dmesg" command):
|
|
|
|
Spoof event(s) detected on VF (n)
|
|
|
|
Where n=the VF that attempted to do the spoofing.
|
|
|
|
|
|
Performance Tuning
|
|
==================
|
|
|
|
An excellent article on performance tuning can be found at:
|
|
|
|
http://www.redhat.com/promo/summit/2008/downloads/pdf/Thursday/Mark_Wagner.pdf
|
|
|
|
|
|
Known Issues
|
|
============
|
|
|
|
Enabling SR-IOV in a 32-bit or 64-bit Microsoft* Windows* Server 2008/R2
|
|
Guest OS using Intel (R) 82576-based GbE or Intel (R) 82599-based 10GbE
|
|
controller under KVM
|
|
------------------------------------------------------------------------
|
|
KVM Hypervisor/VMM supports direct assignment of a PCIe device to a VM. This
|
|
includes traditional PCIe devices, as well as SR-IOV-capable devices using
|
|
Intel 82576-based and 82599-based controllers.
|
|
|
|
While direct assignment of a PCIe device or an SR-IOV Virtual Function (VF)
|
|
to a Linux-based VM running 2.6.32 or later kernel works fine, there is a
|
|
known issue with Microsoft Windows Server 2008 VM that results in a "yellow
|
|
bang" error. This problem is within the KVM VMM itself, not the Intel driver,
|
|
or the SR-IOV logic of the VMM, but rather that KVM emulates an older CPU
|
|
model for the guests, and this older CPU model does not support MSI-X
|
|
interrupts, which is a requirement for Intel SR-IOV.
|
|
|
|
If you wish to use the Intel 82576 or 82599-based controllers in SR-IOV mode
|
|
with KVM and a Microsoft Windows Server 2008 guest try the following
|
|
workaround. The workaround is to tell KVM to emulate a different model of CPU
|
|
when using qemu to create the KVM guest:
|
|
|
|
"-cpu qemu64,model=13"
|
|
|
|
|
|
Support
|
|
=======
|
|
|
|
For general information, go to the Intel support website at:
|
|
|
|
http://support.intel.com
|
|
|
|
or the Intel Wired Networking project hosted by Sourceforge at:
|
|
|
|
http://e1000.sourceforge.net
|
|
|
|
If an issue is identified with the released source code on the supported
|
|
kernel with a supported adapter, email the specific information related
|
|
to the issue to e1000-devel@lists.sf.net
|