path: root/host/docs/rd_testing.dox

/*! \page page_rdtesting R&D Testing Procedures

All defined R&D test procedures are listed here. These tests are meant as a tool
for Ettus R&D to enable faster and more reliable development. Note these tests
are no replacement for manufacturing or production tests, and should not be
treated as such. Instead, they are meant to catch common failure modes during
development. As a result, test definitions are fairly light-weight.

\section rdtesting_gpsdo GPSDO Tests

| Test Code        | Device    | Peripherals       | Manual Test Procedure        | Automatic Test Procedure  |
|------------------|-----------|-------------------|------------------------------|---------------------------|
| GPS-X310-TCXO-v1 | USRP X310 | Jackson Labs TCXO | \ref rdtesting_gpsdo_manual  | \ref rdtesting_gpsdo_auto |
| GPS-X310-OCXO-v1 | USRP X310 | Jackson Labs OCXO | \ref rdtesting_gpsdo_manual  | \ref rdtesting_gpsdo_auto |
| GPS-X300-TCXO-v1 | USRP X300 | Jackson Labs TCXO | \ref rdtesting_gpsdo_manual  | \ref rdtesting_gpsdo_auto |
| GPS-X300-OCXO-v1 | USRP X300 | Jackson Labs OCXO | \ref rdtesting_gpsdo_manual  | \ref rdtesting_gpsdo_auto |
| GPS-B200-TCXO-v1 | USRP B200 | Jackson Labs TCXO | \ref rdtesting_gpsdo_manual  | \ref rdtesting_gpsdo_auto |
| GPS-B210-TCXO-v1 | USRP B210 | Jackson Labs TCXO | \ref rdtesting_gpsdo_manual  | \ref rdtesting_gpsdo_auto |


\subsection rdtesting_gpsdo_recommendations Recommendations

For cursory testing, not all tests within a device family are required (e.g.,
only testing the OCXO on any X-Series, and testing the TCXO on any B-Series is
sufficient).

The following tests are recommended for a minimum test (N stands for the latest
version of this test):
- One of GPS-X310-OCXO-vN or GPS-X300-OCXO-vN
- One of GPS-B210-TCXO-vN or GPS-B200-TCXO-vN

\subsection rdtesting_gpsdo_requirements Requirements

All of these tests require a device that is GPSDO capable (e.g., X3x0, B2x0,
N2x0). For those devices that have a separate GPS component (such as the Jackson
Labs GPSDOs), this component is also required (called the "peripheral" in the
following).

\subsection rdtesting_gpsdo_manual GPSDO: Manual Test Procedure

1. Without connecting the peripheral to the device, run `uhd_usrp_probe` on the
   device and verify that the lack of GPSDO is correctly reported under "sensors".
   No warning or error must be printed.
2. This and the following tests are run with the peripheral connected: Run
   `uhd_usrp_probe` and verify that the GPSDO is correctly reported under "sensors".
   Power down the device before connecting the peripheral. The GPSDO must be
   reported found, and no error or warning must be printed.
3. OCXO only: Without connecting the GPS antenna input, run
   `utils/query_gpsdo_sensors`. To pass, it must report the GPSDO as found, lock to
   the external reference, but then report not being locked to GPS. The tool
   will report a valid GPS time, and a string such as "GPS and UHD Device time
   are aligned" in case of success.
4. Connect a GPS antenna to the input and make sure it is in a position to
   receive GPS satellite data. Confirm that GPS lock is reported by running
   `utils/query_gpsdo_sensors` within 20 minutes of connecting the antenna.
   The tool `query_gpsdo_sensors` will print a string such as "GPS Locked" in
   case of success.

All of these tests must pass for a 'pass' validation.

\subsection rdtesting_gpsdo_auto GPSDO: Automatic Test Procedure

tbd


\section rdtesting_devtest Devtests

| Test Code           | Device        | Peripherals | Manual Test Procedure         | Automatic Test Procedure    |
|---------------------|---------------|-------------|-------------------------------|-----------------------------|
| DEVTEST-X310-XG-v1  | USRP X310     | None        | \ref rdtesting_devtest_manual | \ref rdtesting_devtest_auto |
| DEVTEST-X310-HG-v1  | USRP X310     | None        | \ref rdtesting_devtest_manual | \ref rdtesting_devtest_auto |
| DEVTEST-X300-XG-v1  | USRP X300     | None        | \ref rdtesting_devtest_manual | \ref rdtesting_devtest_auto |
| DEVTEST-X300-HG-v1  | USRP X300     | None        | \ref rdtesting_devtest_manual | \ref rdtesting_devtest_auto |
| DEVTEST-E310-SG1-v1 | USRP E310-SG1 | None        | \ref rdtesting_devtest_manual | \ref rdtesting_devtest_auto |
| DEVTEST-E310-SG3-v1 | USRP E310-SG3 | None        | \ref rdtesting_devtest_manual | \ref rdtesting_devtest_auto |
| DEVTEST-B200-v1     | USRP B200     | None        | \ref rdtesting_devtest_manual | \ref rdtesting_devtest_auto |
| DEVTEST-B210-v1     | USRP B210     | None        | \ref rdtesting_devtest_manual | \ref rdtesting_devtest_auto |
| DEVTEST-B200m-v1    | USRP B200mini | None        | \ref rdtesting_devtest_manual | \ref rdtesting_devtest_auto |
| DEVTEST-B205m-v1    | USRP B205mini | None        | \ref rdtesting_devtest_manual | \ref rdtesting_devtest_auto |

The devtests are hardware tests built in to the UHD make system. They can be run
directly from the build directory and require no configuration.
Devtests are designed to always run, regardless of the actual device
configuration. This means, by definition, that devtests cannot require special
cabling, specific daughtercards, etc.

Note: The actual devtests can change, since they're part of the code. This does
not require a version bump on the test code.

\subsection rdtesting_devtest_requirements Requirements

Devtests are only defined for some devices. When running a devtest, all
peripherals must be disconnected (e.g., no daughterboards on the X-Series, no
GPSDOs on the B- and X-Series).

Running these tests requires the yaml package. On Ubuntu, run
`sudo apt-get install python-yaml` to install the Python 2 version of the YAML
library.

\subsection rdtesting_devtest_manual Devtest: Manual Test Procedure

### X3x0 procedure

1. Make sure no peripherals are connected to the device (no daughterboards, no
   GSPDO, front panel GPIO is unconnected).
2. When testing the HG image, run a test once for each connection (1 GigE and
   10 GigE). When testing the XG image, a test on either connection (SFP0 or
   SFP1) is sufficient. In both cases, also test via PCIe.
3. When the device is connected, simply run `make test_x3x0` from the command
   line in the build directory. Multiple devices connected will all get tested,
   there is no requirement to only connect a single device at a time (because
   devtest will run sequentially anyway).
4. Devtest must report no failures for a 'pass' validation.

### B2xx procedure

Note: The test codes with an 'm' suffix refer to B200mini and B205mini,
respectively.

1. Make sure no peripherals are connected to the device (no GPSDO if applicable,
   GPIO pins unconnected)
2. Test once via USB3, once via USB2.
3. Simply run `make test_b2xx`
4. Devtest must report no failures for a 'pass' validation.

### E310 procedure

1. Make sure GPIO pins are unconnected.
2. Tests need to be run natively on the device. If the build environment is
   available on the device, running `make test_e3xx` is sufficient.
3. In general, there is no build environment on the device (e.g. when doing a
   typical sshfs mount of an environment). In this case, copy the contents of
   the devtest directory onto the device, and run the following command (the
   environment variables need to point to the location of the devtest code, the
   location of the UHD build, and where you want to store log files, respectively):

       $DEVTEST_DIR/run_testsuite.py --src-dir $DEVTEST_DIR \
                                     --devtest-pattern e3xx \
                                     --build-type na \
                                     --build-dir $BUILD_DIR \
                                     --device-filter e3x0 \
                                     --log-dir $LOG_DIR

4. Devtest must report no failures for a 'pass' validation.

\subsection rdtesting_devtest_auto Devtest: Automatic Test Procedure

As all these tests can be run unsupervised, they can be run automatically given
the correct device setup. The return code of the tests can be used to check for
pass/fail conditions (return code 0 means 'pass').

\section rdtesting_fpga_testbenches FPGA: Testing through Simulations

Test benches provide a faster way to verify the design through simulations.

| Test Code        | Device    | Peripherals       | Manual Test Procedure                   | Automatic Test Procedure             |
|------------------|-----------|-------------------|-----------------------------------------|--------------------------------------|
| FPGATB-v1        | None      | None              | \ref rdtesting_fpga_testbenches_manual  | \ref rdtesting_fpga_testbenches_auto |

\subsection rdtesting_fpga_testbenches_requirement Requirements

These tests are simulations and do not need any device. Vivado 15.4 should be
installed.

\subsection rdtesting_fpga_testbenches_manual Manual Test Procedure

1. Go to the fpga directory depending on which test needs to be run.

  1. NoC block test benches:
     Most of the NoC blocks have a test bench written in System Verilog that provides stimuli to the noc_block to verify it. The test bench for a block resides in `<fpga-dir>/usrp3/lib/rfnoc/&zwj;*_tb`.

  2. Running unit test benches:
     A few sub-blocks like noc-shell and sine_tone are used within the bigger noc_blocks. They have their own test benches. Their test benches reside in `<fpga-dir>/usrp3/lib/sim/rfnoc/&zwj;*`.

  3. Radio test bench:
     The radio test bench resides in `<fpga-dir>/usrp3/lib/radio/noc_block_radio_core_tb/`.

  4. Device specific test benches:
     IPs specific to a device have test benches that exist in `<fpga-dir>/usrp3/top/x300/sim/*`. e.g. DMA testbench, PCIe, etc.

2. Setup the environment by running `source <fpga-dir>/usrp3/top/<device>/setupenv.sh`.

3. In the test bench directory and run the test bench by 'make xsim' or 'make vsim'.

4. All of these tests must report no failure with a 'PASS' validation. Example testbench output:

\code
========================================================
TESTBENCH STARTED: noc_block_skeleton
========================================================
[TEST CASE   1] (t=000000000) BEGIN: Wait for Reset...
[TEST CASE   1] (t=000001002) DONE... Passed
[TEST CASE   2] (t=000001002) BEGIN: Check NoC ID...
Read Skeleton NOC ID: 1234000000000000
[TEST CASE   2] (t=000001238) DONE... Passed
[TEST CASE   3] (t=000001238) BEGIN: Connect RFNoC blocks...
Connecting noc_block_tb (SID: 1:0) to noc_block_skeleton (SID: 0:0)
Connecting noc_block_skeleton (SID: 0:0) to noc_block_tb (SID: 1:0)
[TEST CASE   3] (t=000005457) DONE... Passed
[TEST CASE   4] (t=000005457) BEGIN: Write / readback user registers...
[TEST CASE   4] (t=000006888) DONE... Passed
[TEST CASE   5] (t=000006888) BEGIN: Test sequence...
[TEST CASE   5] (t=000007403) DONE... Passed
========================================================
TESTBENCH FINISHED: noc_block_skeleton
 - Time elapsed:   7500 ns
 - Tests Expected: 5
 - Tests Run:      5
 - Tests Passed:   5
Result: PASSED
========================================================
\endcode

Failing tests can be debugged by checking the waveform in a Vivado GUI by
running 'make GUI=1 xsim'. More details on
debugging: https://kb.ettus.com/Debugging_FPGA_images

\subsection rdtesting_fpga_testbenches_auto Automatic Test Procedure

 Go to <fpga-dir>/usrp3/ and run 'build.py xsim all'. All tests should report 'PASS'.

\section rdtesting_fpgadspverif FPGA DSP Verification

| Test Code                | Device        | Peripherals | Manual Test Procedure              | Automatic Test Procedure         |
|--------------------------|---------------|-------------|------------------------------------|----------------------------------|
| FPGADSPVERIF-X310-HG-v1  | USRP X310     | 2x UBX      | \ref rdtesting_fpgadspverif_manual | \ref rdtesting_fpgadspverif_auto |
| FPGADSPVERIF-X310-XG-v1  | USRP X310     | 2x UBX      | \ref rdtesting_fpgadspverif_manual | \ref rdtesting_fpgadspverif_auto |
| FPGADSPVERIF-X300-HG-v1  | USRP X300     | 2x UBX      | \ref rdtesting_fpgadspverif_manual | \ref rdtesting_fpgadspverif_auto |
| FPGADSPVERIF-X300-XG-v1  | USRP X300     | 2x UBX      | \ref rdtesting_fpgadspverif_manual | \ref rdtesting_fpgadspverif_auto |
| FPGADSPVERIF-E310-SG1-v1 | USRP E310 SG1 | None        | \ref rdtesting_fpgadspverif_manual | \ref rdtesting_fpgadspverif_auto |
| FPGADSPVERIF-E310-SG3-v1 | USRP E310 SG3 | None        | \ref rdtesting_fpgadspverif_manual | \ref rdtesting_fpgadspverif_auto |

\subsection rdtesting_fpgadspverif_requirements Requirements

- Signal generator and spectrum analyzer
- X300 & X310 with 2x UBX daughterboard
- E310 SG1 & SG3 with SSH access

\subsection rdtesting_fpgadspverif_manual FPGA DSP Verification: Manual Test Procedure

This procedure tests the DDC and DUC signal quality and the block's capability
to change sample rate while streaming.

#### RX testing

1. Run calibration on device, if applicable
2. Using a signal generator, inject a sine tone into RX channel 0 at 915.5 MHz @
   -40 dBm
3. Inspect the received spectrum using `uhd_fft`
  - X3x0: `uhd_fft -f 915e6 -s 10e6 -g 10`
  - E3xx: `uhd_fft -f 915e6 -s 2e6 -g 50`
  - Embedded devices will require either using network mode or using X
    forwarding over ssh to run the app natively
4. In the GUI, inspect the spectrum. There should be a strong tone at the test
   tone frequency. There may be a small tone at the carrier frequency due to DC
   offset and a quadrature image due to IQ imbalance.
5. Check the input tone frequencies outlined below. The tone should shift from
   left to right as the frequency changes and may have some amplitude variation,
   especially at the band edges.
  - X3x0: 910 MHz to 920 MHz in 1 MHz steps
  - E3xx: 914 MHz to 916 MHz in 200 kHz steps
6. Set input tone back to 915.5 MHz. Check the sampling rate as outlined below.
   The spectrum should reflect the change in sample rate.
  - X3x0: 1, 5, 20, 33.333, 50, 66.666, 100, 200 MHz
  - E3x0: 0.1, 0.5, 1, 1.143, 1.684 MHz
7. Repeat on each RX channel of the device.
8. This test fails if:
  - DC offset and IQ imbalance tones are unusually large
  - There are any other strong tones or spectrum distortion
  - The spectrum changes significantly between frequencies or sample rates
    - An initial transient distortion is acceptable
    - Amplitude variation on the order of +/-10 dB is acceptable
  - Console reports any of the following:
    - Overruns 'O' if continuous and not due to host computer's lack of
      processing performance
    - Dropped packets 'D'
    - Sequence number errors 'S'
    - Timeouts

#### TX testing

1. Run calibration on device, if applicable
2. Using `uhd_siggen_gui`, generate a sine tone TX channel 0 at 915.5 MHz:
  - X3x0: `uhd_siggen_gui -f 915e6 -s 10e6 -g 10 -x 500e3 --sine`
  - E3xx: `uhd_siggen_gui -f 915e6 -s 2e6 -g 50 -x 500e3 --sine`
3. Using a spectrum analyzer, inspect the output spectrum. There should be a
   strong tone at the test tone frequency. There may be a small tone at the
   carrier frequency due to DC offset and a quadrature image due to IQ
   imbalance.
4. Using the GUI, test the follow offset frequencies. The tone should shift from
   left to right as the frequency changes and may have some amplitude variation,
   especially at the band edges.
  - X3x0: -5 to +5 MHz in 1 MHz steps
  - E3xx: -1 to +1 MHz in 200 kHz steps
5. Set output tone offset back to 500e3. Change sampling rate as outlined below.
   The spectrum should not significantly differ between sample rates.
  - X3x0: 1, 5, 20, 33.333, 50, 66.666, 100, 200 MHz
  - E3x0: 0.1, 0.5, 1, 1.143, 1.684 MHz
6. Repeat on each TX channel of the device
7. This test fails if:
  - DC offset and IQ imbalance tones are unusually large
  - There are any other strong tones or spectrum distortion
  - The spectrum changes significantly between sample rates
    - An initial transient distortion is acceptable
  - Console reports any of the following:
    - Underruns 'U' if continuous and not due to host computer's lack of
      processing performance
    - Late packets 'L'
    - Sequence number errors 'S'

\subsection rdtesting_fpgadspverif_auto FPGA DSP Verification: Automatic Test Procedure

tbd

\section rdtesting_fpgafuncverif FPGA Functional Verification

| Test Code                 | Device        | Peripherals | Manual Test Procedure               | Automatic Test Procedure          |
|---------------------------|---------------|-------------|-------------------------------------|-----------------------------------|
| FPGAFUNCVERIF-X310-HG-v1  | USRP X310     | 2x UBX      | \ref rdtesting_fpgafuncverif_manual | \ref rdtesting_fpgafuncverif_auto |
| FPGAFUNCVERIF-X310-XG-v1  | USRP X300     | 2x UBX      | \ref rdtesting_fpgafuncverif_manual | \ref rdtesting_fpgafuncverif_auto |
| FPGAFUNCVERIF-X300-HG-v1  | USRP X310     | 2x UBX      | \ref rdtesting_fpgafuncverif_manual | \ref rdtesting_fpgafuncverif_auto |
| FPGAFUNCVERIF-X300-XG-v1  | USRP X300     | 2x UBX      | \ref rdtesting_fpgafuncverif_manual | \ref rdtesting_fpgafuncverif_auto |
| FPGAFUNCVERIF-E310-SG1-v1 | USRP E310 SG1 | None        | \ref rdtesting_fpgafuncverif_manual | \ref rdtesting_fpgafuncverif_auto |
| FPGAFUNCVERIF-E310-SG3-v1 | USRP E310 SG3 | None        | \ref rdtesting_fpgafuncverif_manual | \ref rdtesting_fpgafuncverif_auto |
| FPGAFUNCVERIF-N310-v1     | USRP N310     | None        | \ref rdtesting_fpgafuncverif_manual | \ref rdtesting_fpgafuncverif_auto |
| FPGAFUNCVERIF-N300-v1     | USRP N300     | None        | \ref rdtesting_fpgafuncverif_manual | \ref rdtesting_fpgafuncverif_auto |

The FPGA functional verification tests exercise the Digital Downconverter (DDC),
Digital Upconverter (DUC), and Radio Core RFNoC blocks.

\subsection rdtesting_fpgafuncverif_requirements Requirements

- X300 & X310 with two daughterboards
  - 2x UBX recommended
  - HG tests require a single 10 GigE connection, XG requires two for the 2x RX
    200 MSPS test
  - 1 GigE and PCIe adapters and cabling for optional tests
- E310: SG1 & SG3 with SSH access
- N310: No special requirements

\subsection rdtesting_fpgafuncverif_manual FPGA Functional Verification: Manual Test Procedure

This procedure verifies that the DDC, DUC, and Radio Core can run at various sample
rates and channel configurations without any data flow issues.

1. Run `benchmark_rate` using the parameters outlined in the tables below
2. Unless otherwise noted, to pass each test:
  - Benchmark rate must run without reporting any of the following:
    - Underruns 'U'
    - Overruns 'O'
    - Dropped packets 'D'
    - Sequence number errors 'S'
    - Late commands 'L'
    - Timeouts
  - Appropriate TX/RX LEDs must be illuminated
3. Unless specified in 'Notes' column, use default values for unlisted
   parameters
4. Example commands:
  - X3x0: `benchmark_rate --tx_rate 1e6 --rx_rate 1e6 --channels 0,1 --duration 120`
  - E3xx: `benchmark_rate --args="master_clock_rate=10e6" --tx_rate 1e6 --rx_rate 1e6 --channels 0,1 --duration 120`

#### USRP X3x0: 10 GigE Interface

- Required images to test: X310 HG
- Optional images to test: X310 XG, X300 HG, X300 XG
- Note: On TX tests, initial Us within the first 5 seconds can be ignored and do not fail the test

| Channels      | Sample Rates             | Duration | Notes              |
|---------------|--------------------------|----------|--------------------|
| 1x RX         | 10e6, 50e6, 100e6, 200e6 | 60       | Test both channels |
| 2x RX         | 10e6, 50e6, 100e6        | 60       |                    |
| 2x RX         | 200e6                    | 60       | 2x 10G, XG only    |
| 1x TX         | 10e6, 50e6, 100e6, 200e6 | 60       | Test both channels |
| 2x TX         | 10e6, 50e6, 100e6        | 60       |                    |
| 1x RX & 1x TX | 10e6, 50e6, 100e6        | 60       | Test both channels |
| 1x RX & 1x TX | 200e6                    | 60       | Use channel 0      |
| 2x RX & 2x TX | 10e6, 50e6               | 60       |                    |
| 1x RX & 1x TX | 200e6                    | 3600     | Use channel 1      |
| 2x RX & 2x TX | 100e6                    | 3600     |                    |

#### USRP X3x0: 1 GigE Interface

- Required images to test: None
- Optional images to test: X310 HG, X310 XG, X300 HG, X300 XG
- Note: On TX tests, initial Us within the first 5 seconds can be ignored and do not fail the test

| Channels      | Sample Rates            | Duration |
|---------------|-------------------------|----------|
| 1x RX         | 1e6, 10e6, 25e6         | 60       |
| 2x RX         | 1e6, 10e6               | 60       |
| 1x TX         | 1e6, 10e6, 25e6         | 60       |
| 2x TX         | 1e6, 10e6               | 60       |
| 1x RX & 1x TX | 1e6, 10e6, 25e6         | 60       |
| 2x RX & 2x TX | 1e6, 10e6               | 60       |

#### USRP X3x0: PCIe Interface

- Required images to test: None
- Optional images to test: X310 HG, X310 XG, X300 HG, X300 XG
- Note: On TX tests, initial Us within the first 5 seconds can be ignored and do not fail the test

| Channels      | Sample Rates             | Duration |
|---------------|--------------------------|----------|
| 1x RX         | 10e6, 50e6, 100e6, 200e6 | 60       |
| 2x RX         | 10e6, 50e6, 100e6        | 60       |
| 1x TX         | 10e6, 50e6, 100e6, 200e6 | 60       |
| 2x TX         | 10e6, 50e6, 100e6        | 60       |
| 1x RX & 1x TX | 10e6, 50e6, 100e6        | 60       |
| 1x RX & 1x TX | 200e6                    | 60       |
| 2x RX & 2x TX | 10e6, 50e6               | 60       |

Note: On TX tests, initial Us within the first 5 seconds can be ignored and do not fail the test

#### USRP E3xx (SG3 Required, SG1 Optional)

| Channels      | Master Clock Rate       | Sample Rate | Duration | Notes              |
|---------------|-------------------------|-------------|----------|--------------------|
| 1x RX         | 10e6                    | 1e6         | 60       | Test both channels |
| 1x RX         | 61.44e6                 | 1.024e6     | 60       | Test both channels |
| 1x TX         | 10e6                    | 1e6         | 60       | Test both channels |
| 1x TX         | 61.44e6                 | 1.024e6     | 60       | Test both channels |
| 2x RX         | 10e6                    | 1e6         | 60       |                    |
| 2x RX         | 30.72e6                 | 1.024e6     | 60       |                    |
| 2x TX         | 10e6                    | 1e6         | 60       |                    |
| 2x TX         | 30.72e6                 | 1.024e6     | 60       |                    |
| 1x RX & 1x TX | 10e6                    | 1e6         | 60       | Test both channels |
| 1x RX & 1x TX | 61.44e6                 | 1.024e6     | 60       | Use channel 1      |
| 2x RX & 2x TX | 10e6                    | 1e6         | 60       |                    |
| 2x RX & 2x TX | 30.72e6                 | 1.024e6     | 60       |                    |
| 1x RX & 1x TX | 61.44e6                 | 1e6         | 3600     | Use channel 0      |
| 2x RX & 2x TX | 30.72e6                 | 1e6         | 3600     |                    |

#### USRP N300/N310: 1 GigE Interface

- Required images to test: HG
- Note: On TX tests, initial Us within the first 5 seconds can be ignored and do not fail the test

| Channels      | Master Clock Rate | Sample Rates            | Duration | Notes                             |
|---------------|-------------------|-------------------------|----------|-----------------------------------|
| 1x RX         | 125e6             | 1.25e6                  | 60       | One test each for all 4 channels  |
| 1x RX         | 122.88e6          | 1.2288e6                | 60       | One test each for all 4 channels  |
| 1x RX         | 153.6e6           | 1.536e6                 | 60       | One test each for all 4 channels  |
| 1x TX         | 125e6             | 1.25e6                  | 60       | One test each for all 4 channels  |
| 1x TX         | 122.88e6          | 1.2288e6                | 60       | One test each for all 4 channels  |
| 1x TX         | 153.6e6           | 1.536e6                 | 60       | One test each for all 4 channels  |
| 2/3/4x RX     | 125e6             | 1.25e6                  | 60       | 3 tests total                     |
| 2/3/4x RX     | 122.88e6          | 1.2288e6                | 60       | 3 tests total                     |
| 2/3/4x RX     | 153.6e6           | 1.536e6                 | 60       | 3 tests total                     |
| 2/3/4x TX     | 125e6             | 1.25e6                  | 60       | 3 tests total                     |
| 2/3/4x TX     | 122.88e6          | 1.2288e6                | 60       | 3 tests total                     |
| 2/3/4x TX     | 153.6e6           | 1.536e6                 | 60       | 3 tests total                     |
| 4x RX & 4x TX | 125e6             | 1.25e6                  | 60       | Drop to 2 channels for N300
| 4x RX & 4x TX | 122.88e6          | 1.2288e6                | 60       | Drop to 2 channels for N300
| 4x RX & 4x TX | 153e6             | 1.536e6                 | 60       | Drop to 2 channels for N300

#### USRP N300/N310: 10 GigE Interface

- Required images to test: N310 HG + XG
- Note: On TX tests, initial Us within the first 5 seconds can be ignored and do not fail the test

| Channels      | Master Clock Rate | Sample Rates            | Duration | Notes                             |
|---------------|-------------------|-------------------------|----------|-----------------------------------|
| 1x RX         | 125e6             | 1.25e6, 125e6           | 60       | One test each for all 4 channels  |
| 1x RX         | 122.88e6          | 1.2288e6, 122.88e6      | 60       | One test each for all 4 channels  |
| 1x RX         | 153.6e6           | 1.536e6, 153.6e6        | 60       | One test each for all 4 channels  |
| 2/3/4x RX     | 125e6             | 1.25e6                  | 60       | 3 tests total                     |
| 2/3/4x RX     | 122.88e6          | 1.2288e6                | 60       | 3 tests total                     |
| 2/3/4x RX     | 153.6e6           | 1.536e6                 | 60       | 3 tests total                     |
| 4x RX         | 125e6             | 1.25e6, 62.5e6          | 60       | N310 only
| 4x TX         | 125e6             | 1.25e6, 12.5e6          | 60       | N310 only
| 4x RX & 4x TX | 125e6             | 1.25e6, 62.5e6          | 60       | Drop to 2 channels for N300
| 4x RX & 4x TX | 122.88e6          | 1.2288e6, 61.44e6       | 60       | Drop to 2 channels for N300
| 4x RX & 4x TX | 153e6             | 1.536e6, 76.8e6         | 60       | Drop to 2 channels for N300
| 4x RX & 4x TX | 125e6             | 62.5e6                  | 3600     | Drop to 2 channels for N300
| 4x RX & 4x TX | 122.88e6          | 61.44e6                 | 3600     | Drop to 2 channels for N300
| 4x RX & 4x TX | 153e6             | 76.8e6                  | 3600     | Drop to 2 channels for N300
| 4x RX & 4x TX | 125e6             | 125e6 RX, 62.5e6 TX     | 60       | Use dual 10GigE, N310 only
| 4x RX & 4x TX | 122.88e6          | 122.88e6 RX, 61.44e6 TX | 60       | Use dual 10GigE, N310 only
| 4x RX & 4x TX | 153e6             | 153e6 RX, 76.8e6 TX     | 60       | Use dual 10GigE, N310 only

\subsection rdtesting_fpgafuncverif_auto FPGA Functional Verification: Automatic Test Procedure

tbd

\section rdtesting_phasealignment Phase alignment tests

| Test Code           | Device    | Peripherals        | Manual Test Procedure                | Automatic Test Procedure    |
|---------------------|-----------|--------------------|--------------------------------------|-----------------------------|
| PHASE-Twin-RX-v1    | 2xTwinRX  | 1xX3x0 + LOSharing cables | \ref rdtesting_phase_rx_X3x0_twinrx | \ref rdtesting_phase_rx_auto |
| PHASE-UBX-40-RX-v1  | 2xUBX-40  | 2xX3x0  | \ref rdtesting_phase_rx_X3x0_sbx_ubx | \ref rdtesting_phase_rx_auto |
| PHASE-UBX-160-RX-v1  | 2xUBX-160  | 2xX3x0 | \ref rdtesting_phase_rx_X3x0_sbx_ubx | \ref rdtesting_phase_rx_auto |
| PHASE-SBX-40-RX-v1  | 2xSBX-40  | 2xX3x0 | \ref rdtesting_phase_rx_X3x0_sbx_ubx | \ref rdtesting_phase_rx_auto |
| PHASE-SBX-120-RX-v1  | 2xSBX-120  | 2xX3x0 | \ref rdtesting_phase_rx_X3x0_sbx_ubx | \ref rdtesting_phase_rx_auto |
| PHASE-N2x0-MIMO-v1 | 2x N2x0 + MIMO cable | 2x SBX | \ref rdtesting_phase_rx_N2x0_MIMO | \ref rdtesting_phase_rx_auto |


| Device \anchor phase_band_table        | Frequency Range         | Number of bands |
|---------------|-------------------------|-----------------|
| TwinRX        | 10 - 6000 MHz           |      12         |
| UBX-{160, 40} | 10 - 6000 MHz           |      12         |
| SBX-{120, 40} | 400 - 4400 MHz          |      7          |

Phase alignment testing is necessary to verify device synchronization across multiple daughter- and motherboards is working as expected for CBX, SBX and UBX daughterboards. To enable efficient Phase alignment testing a GNU Radio Out-of-Tree module gr-usrptest exists in tools/gr-usrptest. It is required for testing RX testcases and later may be required to perform TX testcases.

To test phase alignment we measure phase offset between DUTs at an offset of 2 MHz offset from the selected center frequency. The phase difference for a given center frequency has to stay the same across retunes and power cycles of the DUT.

Correct synchronization with PPS and 10 MHz references is required for these tests.

\subsection rdtesting_phase_rx_manual Manual phase alignment testing (Receiver)

Equipment Required
- Octoclock-G
- Signal Generator
- 2-way splitter that covers frequency range for daughterboard (4-way for TwinRX)
- 5+ SMA Cables

Software Required
- UHD
- gnuradio
- gr-usrptest

\subsection rdtesting_phase_rx_X3x0_twinrx X3x0 with TwinRX
1.  Make sure correct FPGA image is loaded on X3x0.
2.  Place first daughterboard in slot A and second daughterboard in slot B.
3.  Connect LO sharing cables between boards.
4.  Connect host to device via 1 GbE, 10 GbE, or PCIe.
5.  Connect 10 MHz and PPS from Octoclock-G to X3x0.
6.  Connect Signal Generator to input of 4-way splitter and outputs of the splitter to the 2 RX ports on each daughterboard.
7.  Set Signal Generator output power at -30 dBm.
8.  From the top of the UHD source, run the command:

  -  `./tools/gr-usrptest/apps/usrp_phasealignment.py --spec "A:0 A:1 B:0 B:1" --channels 0,1,2,3 --sync pps --time-source external --clock-source external -s 5e6 -g 75 -f 10e6 --freq-bands 12 --start-freq 10e6 --stop-freq 6e9 --duration 2.0 --auto --lo-export True,False,False,False --lo-source internal,companion,external,external`

9.  At each frequency step, tune Signal Generator to the displayed frequency + 1 MHz and increase output power by 3 dB.
10.  Analyze terminal output.  The "run avg" across all runs should not deviate more than 1 degree and the "stddev" for any run should not deviate more than 1 degree.

\subsection rdtesting_phase_rx_X3x0_sbx_ubx X3x0 with SBX or UBX
1.  Set different IP addresses on each X3x0 and make sure correct FPGA image is loaded on each.
2.  Place first daughterboard in slot A of first X3x0 and second daughterboard in slot A of second X3x0.
3.  Connect host to both X3x0s.
4.  Connect 10 MHz and PPS from Octoclock-G to both X3x0s.
5.  Connect Signal Generator to input of splitter and outputs of the splitter to the RX2 port on each daughterboard.
6.  Set Signal Generator output power at -30 dBm.
7.  From the top of the UHD source, run the command:

  -  `./tools/gr-usrptest/apps/usrp_phasealignment.py --args "addr0=<first X3x0 IP addr>,addr1=<second X3x0 IP addr>,dboard_clock_rate=20e6" --clock-source external --time-source external --sync pps --spec "A:0" --channels 0,1 -s 10e6 -g 25 -f \<lowest DB freq\> --freq-bands \<# frequency bands\> --start-freq \<lowest freq\> --stop-freq \<highest freq\> --duration 2.0 --auto`

8.  At each frequency step, tune Signal Generator to the displayed frequency + 1 MHz and increase output power by 2dB.
9.  Analyze terminal output.  The "run avg" across all runs should not deviate more than 2 degrees and the "stddev" for any run should not deviate more than 2 degrees.

\subsection rdtesting_phase_rx_N2x0_MIMO N2x0 MIMO with SBX
1.  Set different IP addresses on each N2x0 and make sure correct FPGA image and firmware are loaded.
2.  Connect MIMO cable between devices.
3.  Connect host to master device via 1 GbE.
4.  Connect 10 MHz and PPS from Octoclock-G to master device only.
5.  Connect Signal Generator to input of splitter and outputs of the splitter to the RX2 port on each daughterboard.
6.  Set Signal Generator output power at -36 dBm.
7.  From the top of the UHD source, run the command:

  -  `./tools/gr-usrptest/apps/usrp_phasealignment.py --args "addr0=<IP address of master>,addr1=<IP address of slave>" --clock-source external,mimo --time-source external,mimo --sync pps --channels 0,1 -s 10e6 -f 400e6 -g 31.5 --freq-bands 7 --start-freq 400e6 --stop-freq 4400e6 --duration 2.0 --auto`

8.  At each frequency step, tune Signal Generator to the displayed frequency + 1 MHz and increase output power by 2 dB.
9.  Analyze terminal output.  The "run avg" across all runs should not deviate more than 2 degrees and the "stddev" for any run should not deviate more than 5 degrees.

\subsection rdtesting_phase_rx_auto Automatic phase alignment testing (Receiver)

tbd

\section rdtesting_bist BISTs

| Test Code           | Device    | Peripherals         | Manual Test Procedure           | Automatic Test Procedure      |
|---------------------|-----------|---------------------|---------------------------------|-------------------------------|
| BIST-N310-v1        | 1xN310    | DB-15 GPIO Loopback | \ref rdtesting_bist_n3x0_manual | \ref rdtesting_bist_n3x0_auto |
| BIST-N300-v1        | 1xN300    | DB-15 GPIO Loopback | \ref rdtesting_bist_n3x0_manual | \ref rdtesting_bist_n300_auto |

Some of our devices have built-in self-tests (BISTs).

\subsection rdtesting_bist_n310_manual N300/N310 Manual Procedure

Note: The N300 and N310 have identical BISTs.

1. Connect the front-panel GPIO loopback to the front panel
   (see \ref rdtesting_peripherals_gpiolb)
2. Execute the following commands:

        $ n3xx_bist standard # Note: This will run multiple tests
        $ n3xx_bist gpio

3. Load the AA image from a host computer. Adapt the following command to your
   system:

        $ uhd_image_loader --args type=n3xx,addr=ni-n3xx-$SERIAL --fpga-path=/path/to/usrp_n310_fpga_AA.bit

4. The final BIST is the SFP test, and there are three valid ways of executing
   them. For the purpose of running the test, only one of these needs to be
   run, and can be chosen based on the available peripherals.
  a. Connect an SFP loopback module to both SFP0 and SFP1. Run the command
     `n3xx_bist sfp0_loopback sfp1_loopback`.
  b. If only one SFP loopback module is available, connect the loopback module
     to SFP0 and run `n3xx_bist sfp0_loopback`. Then, connect the loopback
     module to SFP1 and run the command `n3xx_bist sfp1_loopback`.
  c. If no SFP loopback module is available, connect an SFP cable to both SFP0
     and SFP1, and run the command `n3xx_bist sfp_loopback`. Note while this
     option is a legitimate substitute to the previous two options, it is of
     limited use when trying to debug actual SFP issues.

Every test will produce a JSON-serialized dictionary. All tests have passed if
the "status" key is "true", or the return code for `n3xx_bist` is 0.

Note: Keep in mind that after the test, an Aurora image is loaded. If this is
not desired, re-run `uhd_image_loader` to load whatever image is requested.

\subsection rdtesting_bist_n3x0_auto N300/N310 Automatic Procedure

Note: The N300 and N310 have identical BISTs.

Assuming the peripherals described in \ref rdtesting_bist_n310_manual are all
plugged in, the test can trivially be executed automatically by running

        $ n3xx_bist standard
        $ n3xx_bist gpio
	$ n3xx_bist sfp_loopback # Or sfp0_loopback and sfp1_loopback

and making sure that all return values are 0.

\section rdtesting_peripherals Required Peripherals

\section rdtesting_peripherals_gpiolb DB15 GPIO Loopback

This is a cable or breakout board which connects to the DB15 connector and loops
back the following pins:

- 0<->6
- 1<->7
- 2<->8
- 3<->9
- 4<->10
- 5<->11

\section rdtesting_defining Defining R&D Tests

Tests can be added any time to define procedures for pass/fail validation. Any
test must include the following:

- An unambiguous test code. This code consists of three characters that
  identify the test, a short description of the devices required, and a version
  suffix. Example: `GPS-X310-OCXO-v1` is a GPS-related test, requires an X310
  and an OCXO to run, and is version 1 of this test.
- A manual testing procedure. This must unambiguously define a set of tasks,
  and clearly identify whether or not a test has failed or passed. Tests do not
  require any other defined outcome other than 'pass' and 'fail'.
- Optional, but highly recommended: An automatic test procedure. This must
  consist of a command, or a script, or a set of commands that can be
  automatically executed, and that will report a failure condition by means of
  returning a non-zero return value.

Basic understanding of the operation of USRPs by the test operator should be
assumed when authoring test procedures. The descriptions should be as short as
possible to fully describe, unambiguously, how to reach a pass/fail conclusion.

Test procedures may be updated at any time. If this happens, a new test code
must be generated, with the version number increased. Old test codes are
considered deprecated (if there exists a version 2 of a test, version 1 should
not be run any more).

*/
// vim:ft=doxygen: