path: root/host/docs/usrp_x3x0_config.dox

/*! \page page_usrp_x3x0_config System Configuration for USRP X3x0 Series

\tableofcontents

\section x3x0cfg_hostpc Configuring your Host PC

The USRP X3x0 is capable of delivering very fast sample rates to the
host PC, and even high-powered desktops can have trouble keeping up at
the higher rates. You can improve the performance of your host by
configuring a number of settings that affect the performance of your
computer.

These are:

-   Kernel Version
-   Network Configuration
-   Power Management Configuration
-   Real-Time & Priority Scheduling
-   Building with ORC & Volk

These items are covered in more detail, below.

\subsection x3x0cfg_hostpc_kernel Kernel Version

Performance issues may be encountered with Linux kernels earlier than
3.11. Ettus Research strongly recommends using kernel version 3.11 or
higher for high sample rates.

\subsection x3x0cfg_hostpc_netcfg Network Configuration

When using Ethernet interfaces to communicate with the device, it is
necessary to configure a number of facets regarding your network
connection.

\subsubsection x3x0cfg_hostpc_netcfg_nwmgr Configuring NetworkManager

Fedora and Ubuntu both use NetworkManager to manage network connections.
Unfortunately, NetworkManager often tries to take control of a
connection and will disconnect the interface.

You should open your NetworkManager configuration and tell it to ignore
the network interface you are using. **This is not the same as simply
setting a static IP address.** You *must* tell NetworkManager to ignore
the interface.

\subsubsection x3x0cfg_hostpc_netcfg_ip Configuring the host's IP address

On a Linux system, you can add a static IP address very easily by using the 'ip' command:

    sudo ip addr add 192.168.10.1/24 dev <interface>

Note that `<interface>` is usually something like `eth0`.  You can discover the
names of the network interfaces in your computer by running:

    ip addr show

\subsubsection x3x0cfg_hostpc_netcfg_sockbuff Configuring the Socket Buffers

It is necessary to increase the maximum size of the socket buffers to
avoid potential overflows and underruns at high sample rates. Add the
following entries into /etc/sysctl.conf (root privileges required):

    net.core.rmem_max=33554432
    net.core.wmem_max=33554432

Either restart the system or issue the following commands:

    sudo sysctl -w net.core.rmem_max=33554432
    sudo sysctl -w net.core.wmem_max=33554432

\subsubsection x3x0cfg_hostpc_netcfg_mtu Configuring the MTU

In order to achieve maximum performance, we recommend setting the MTU
size to 9000 for 10 GigE and 1500 for 1 GigE. It is possible to use
smaller MTUs, but this can affect performance. With some NICs, setting
the MTU too high can also cause issues. To set the MTU to 9000, you can
use the following command:

    sudo ifconfig <interface> mtu 9000 # For 10 GigE
    sudo ifconfig <interface> mtu 1500 # For 1 GigE

Using these MTUs will set the frame sizes for UHD communication to 8000
and 1472, respectively.

In some cases, specifying the frame size manually by adding the argument
`<send/recv>_frame_size=1472` can solve issues. Note that a frame
size of 1472 will limit the available sampling rate, although this is
not a problem on 1 GigE.

\subsubsection x3x0cfg_hostpc_netcfg_firewall Configuring the Firewall

Many Linux distributions come installed with a Firewall, by default. The
Firewall will often interfere with your ability to communicate with your
USRP. You should configure your firewall to "trust" the interface you
are using. Setting this properly depends on your OS and firewall
configuration method.

When using UHD software, if an IP address for the USRP-X Series device is not specified,
the software will use UDP broadcast packets to locate the USRP-X Series device.
On some systems, the firewall will block UDP broadcast packets.
It is recommended that you change or disable your firewall settings.

\subsubsection x3x0cfg_hostpc_netcfg_if Interface Configuration File (Fedora)

On Fedora systems, you can configure the network interface mostly from
one place (with the exception of the socket buffers). Each interface on
your system should have a file in:

    /etc/sysconfig/network-scripts/

As an example, if your 1GigE interface is "em1", your "ifcfg-em1"
configuration file should look something like this, when configured for
use with a USRP X3xx:

    TYPE="Ethernet"
    BOOTPROTO="none"
    IPADDR0="192.168.10.1"
    DEFROUTE="yes"
    IPV4_FAILURE_FATAL="no"
    IPV6INIT="no"
    IPV6_FAILURE_FATAL="no"
    NAME="em1"
    UUID="<specific to your device>"
    ONBOOT="no"
    HWADDR"<specific to your device>"
    PEERDNS="yes"
    PEERROUTES="yes"
    ZONE="trusted"
    MTU="9000"
    NM_MANAGED="no"

The above file was generated and modified on a Fedora 20 system.

\subsection x3x0cfg_hostpc_pcie PCI Express Configuration

\b Note: These instructions are \b only relevant for using a
USRP X3xx device over a cabled PCIe transport (PCIe over MXIe cable).
If you are using your USRP X3xx device over 1 Gigabit Ethernet or
10 Gigabit Ethernet, these instructions do <em>not apply to you.</em>

\subsubsection x3x0cfg_hostpc_pcie_linux Linux Systems

In order to use the USRP X300/X310 with PCI Express on Linux,
NI USRP RIO kernel modules needs to be installed. In order to use
the PCIe kernel module, and thus PCIe-over-cable with the X3xx,
your kernel version must be supported by the kernel module.
Only 64-bit kernels are supported. To check if your kernel is supported
by the drivers, check the NI USRP RIO website.

#### Downloading and Extracting Installer

- Download the installer from here: http://files.ettus.com/binaries/niusrprio/niusrprio-installer.tar.gz
- Extract it with the following command:

    tar zxf niusrprio-installer.tar.gz

The files will be extracted into a directory called `niusrprio-installer`.

#### Installation

To install the NI USRPRIO kernel modules and RPC server, run the following command:

    sudo niusrprio-installer/INSTALL

Select `y` for each prompt, and the script will install all necessary components.
This script will automatically load all necessary kernel modules for
the duration of the session.

#### Enabling/Disabling Usage

Once everything is installed, run the following commands to enable
use of the X300/X310 over PCI Express (the assumption is that `niusrprio_pcie`
was installed into your `$PATH`):

    sudo niusrprio_pcie start

To stop these processes, run the following command:

    sudo niusrprio_pcie stop

To check if the kernel modules are loaded and if the RPC server
is running, run the following command:

    niusrprio_pcie status

\b Note: `niusrprio_pcie` start does not run when the host system is booted.
If you would like the USRP PCIe device to be available automatically after
a system restart, please create an init.d script that runs `niusrprio_pcie start`.

#### Hot-plugging/Power-cycling

The USRP X3x0, NI USRP-294x and NI USRP-295x devices cannot be hot-swapped when
connected over PCI Express. Unplugging the PCI Express connection or powering
the device should be done only after disabling the device using the following command.

    sudo /usr/local/bin/niusrprio_pcie stop

\b Warning: If the device is unplugged without running the above command, the system
can become unstable and crash.

#### Linux kernel upgrades

The NI USRP RIO kernel modules are built for a specific kernel version.
If you upgrade/downgrade the linux kernel on the host to a version different
from the one that the installer was run on, then you may see the following
error message when running `niusrprio start`:

    ERROR: could not insert 'NiRioSrv': Unknown symbol in module, or unknown parameter (see dmesg)
    ERROR: could not insert 'niusrpriok': Unknown symbol in module, or unknown parameter (see dmesg)

To rebuild the kernel modules for the currently running kernel, simple run the following

    sudo /usr/local/bin/niusrprio_pcie stop
    sudo /usr/local/bin/updateNIDrivers --no-prompt
    sudo /usr/local/bin/niusrprio_pcie start

#### Uninstallation

To uninstall the NI USRP RIO kernel modules and RPC server, run the following command:

    sudo niusrprio-installer/UNINSTALL

Select `y` at the prompt, and the script will uninstall all installed components.

\subsubsection x3x0cfg_hostpc_pcie_windows Windows

In order to use the USRP X300/X310 with PCI Express on Windows, the
NI-RIO driver package and the NI-USRP RIO kernel driver needs to be installed.

\b Note: The kernel driver is only supported with certain Windows versions.
Check the NI USRP RIO website to see if your version is supported.

#### Installing NI-USRP

The NI-USRP 1.3 installer can be downloaded from this location: http://www.ni.com/download/ni-usrp-1.3/4711/en/

You will need to create a free NI User Account to download the installer.
Perform the following steps to download and install the NI-USRP exe driver package:

- Choose the "2. Standard Download:NIUSRP130.exe" option to download NIUSRP130.exe to your computer
- Run NIUSRP130.exe as an Administrator and extract the contents to C:\\National Instruments Downloads\\NI-USRP\\1.3
- In the extract location, run setup.exe and follow the prompts.

Reboot the computer after both the NI-USRP package has been installed.

#### Enabling/Disabling Usage

Once everything is installed and the system is rebooted, your X300/X310 PCI Express device should automatically be detected by the Windows Device Manager. The device should be enabled by default.

- To disable the USRPRIO device, navigate to "Device Manager", locate your USRPRIO-X3x0 device, right-click on it and choose "Disable".
- To enable the USRPRIO device, navigate to "Device Manager", locate your USRPRIO-X3x0 device, right-click on it and choose "Enable".

#### Hot-plugging/Power-cycling

The USRP X3x0, NI USRP-294x and NI USRP-295x devices cannot be hot-swapped when connected over PCI Express. Unplugging the PCI Express connection or powering the device should be done only after disabling the device.

\b Warning: If the device is unplugged without running the above command, the system can become unstable and crash

#### Uninstalling NI-USRP

Navigate to the Control Panel and open "Programs". Then select National Instruments
Software and select NI-USRP and NI-RIO from the list. Click on uninstall
to remove the drivers from your system.

\subsection x3x0cfg_hostpc_pwr Power Management

Power management on the host system attempts to save power by reducing
clock frequencies or even powering off devices while not in use. This
can lead to significant performance issues when trying to operate at
high sample rates. Ettus Research strongly recommends disabling all
power management.

\subsubsection x3x0cfg_hostpc_pwr_cpugov Setting the CPU Governors

In Linux, the CPU governors dictate the frequency at which the CPU
operates and attempt to reduce the CPU frequencies at certain times to
save power. When running at high sample rates, reduction of CPU
frequencies can cause significant performance issues. To prevent those
issues, set the governor to "performance".

\b Ubuntu:

1. Install cpufrequtils:

    sudo apt-get install cpufrequtils

2.  Edit `/etc/init.d/cpufrequtils` and set `GOVERNOR="performance"` on the
    appropriate line (run as root):

    sed s/^GOVERNOR=.*$/GOVERNOR=\"performance\"/g /etc/init.d/cpufrequtils > /etc/init.d/cpufrequtils

3.  Restart cpufrequtils:

    sudo /etc/init.d/cpufrequtils restart

\b Fedora:

    sudo cpupower frequency-set -g performance

\subsection x3x0cfg_hostpc_rtprio Real-Time & Priority Scheduling

Enabling real-time and priority scheduling can improve the total
processing throughput of your application. Priority scheduling should be
enabled for UHD, and real-time scheduling can be enabled by your
application.

\subsubsection x3x0cfg_hostpc_rtprio_thread Thread Priority Scheduling with UHD

For information regarding how to enable priority scheduling for UHD on
your system, please see \ref page_general.

\subsubsection x3x0cfg_hostpc_rtprio_app Real-Time Scheduling in your Application

Please note that turning on real-time scheduling in your application
**may lock up your computer** if the processor cannot keep up with the
application. You should generally avoid using real-time scheduling
unless you need to.

Real-time scheduling is enabled via different methods depending on your
application and operating system. In GNU Radio Companion, it can be
turned on in each individual flowgraph.

\subsection x3x0cfg_hostpc_volk Building with ORC & Volk

Especially when running high-performance applications, processing
performance can be dramatically improved by SIMD instructions. UHD uses
ORC to provide SIMD capability, and GNU Radio includes a SIMD library
called "Volk". These should both be used to guarantee optimum
performance.

\subsubsection x3x0cfg_hostpc_volk_orc Compiling UHD with ORC

ORC, the <a href="http://code.entropywave.com/orc/">Oil Runtime Compiler</a>,
is a third-party compiler that UHD uses to create efficient SIMD code for
your particular computer. ORC is generally easily installed from your
OS's package manager.

On Fedora:

    $ sudo yum update; sudo yum install orc-compiler orc-devel

On Ubuntu:

    $ sudo apt-get update; sudo apt-get install liborc-<version> liborc-<version>-dev

After installing ORC, when building UHD from source, you should see
"ORC" as one of the configured UHD components.

    -- ######################################################
    -- # UHD enabled components
    -- ######################################################
    --   * LibUHD
         <cut for brevity>
    --   * ORC

\subsubsection x3x0cfg_hostpc_volk_volk Compiling GNURadio with Volk

If you are using GNURadio to build applications, you should compile
GNURadio with Volk. For instructions on how to do this,
<a href="http://gnuradio.org/redmine/projects/gnuradio/wiki/Volk">refer to the GNURadio wiki</a>.

\section x3x0cfg_hosthw Host PC Hardware Selection

\subsection x3x0cfg_hosthw_mb Motherboard

Testing has shown that some motherboards do not provide enough PCIe bus
bandwidth to support higher sample rates. Motherboards with PCIe 3.0 are
required and the PCIe architecture of the motherboard should be
carefully considered. Slots with dedicated PCIe lanes should be used for
PCIe or 10GbE cards that will be connected to the X3x0 device.

\subsection x3x0cfg_hosthw_10gige 10GbE NIC

Intel or Myricom 10GbE NICs are recommended. Mellanox, SolarFlare, and
Chelsio 10GbE NICs are not currently recommended. The Ethernet card
should be plugged into the slot that has the most direct connection with
the CPU (PCIe lanes are not shared with another slot). Refer to the
motherboard manual for more information on PCIe architecture.

\section x3x0cfg_hosthw_troubleshooting Troubleshooting Performance Issues

The output on the host console provides indicators of performance issues
in the form of single upper-case letters. The following table lists the
letters, their meanings, and possible causes:

  Indicator  |Meaning               | Possible Causes
  -----------|----------------------|---------------------------------------------------------------------------------
  O          |Overflow on RX        |- Data is not being consumed by user's application fast enough.<br>- CPU governor or other power management not configured correctly.
  D          |Dropped packet on RX  |- Network hardware failure. (Check host NIC, cable, switch, etc...)<br>- PCIe bus on host cannot sustain throughput. (Check ethtool -S \<interface\>).<br>- CPU governor or other power management not configured correctly.<br>- Frame size might not work with the current NIC's MTU.
  U          |Underflow on TX       |- Samples are not being produced by user's application fast enough.<br>- CPU governor or other power management not configured correctly.
  L          |Late packet (usually on MIMO TX)|- Samples are not being produced by user's application fast enough.<br>- CPU governor or other power management not configured correctly.<br>- Incorrect/invalid time_spec provided.
  S          |Sequence error on TX  |- Network hardware failure. (Check host NIC, cable, switch, etc...)<br>- Frame size might not work with the current NIC's MTU.

\subsection x3x0cfg_hosthw_troubleshooting_eth Troubleshooting Ethernet Issues

1.  First, check `ifconfig <interface>` to see if there are any errors
    reported on the interface. If there are errors, it is most likely a
    network hardware problem.
2.  Next, check the output of `ethtool -S <interface>`. The output is
    driver-specific, but may give important clues as to what may be
    happening. For example, a high value on rx_missed_errors for an
    Intel NIC indicates that the bus (i.e. PCIe) is not keeping up.
3.  Finally, Wireshark can be used to validate the traffic between the
    host and device and make sure there is no unwanted traffic on the
    interface.

*/
// vim:ft=doxygen: