/*! \page page_rdtesting R&D Testing Procedures All defined R&D test procedures are listed here. These tests are meant as a tool for Ettus R&D to enable faster and more reliable development. Note these tests are no replacement for manufacturing or production tests, and should not be treated as such. Instead, they are meant to catch common failure modes during development. As a result, test definitions are fairly light-weight. \section rdtesting_phase Phase Alignment Tests tbd \section rdtesting_gpsdo GPSDO Tests | Test Code | Device | Peripherals | Manual Test Procedure | Automatic Test Procedure | |------------------|-----------|-------------------|------------------------------|---------------------------| | GPS-X310-TCXO-v1 | USRP X310 | Jackson Labs TCXO | \ref rdtesting_gpsdo_manual | \ref rdtesting_gpsdo_auto | | GPS-X310-OCXO-v1 | USRP X310 | Jackson Labs OCXO | \ref rdtesting_gpsdo_manual | \ref rdtesting_gpsdo_auto | | GPS-X300-TCXO-v1 | USRP X300 | Jackson Labs TCXO | \ref rdtesting_gpsdo_manual | \ref rdtesting_gpsdo_auto | | GPS-X300-OCXO-v1 | USRP X300 | Jackson Labs OCXO | \ref rdtesting_gpsdo_manual | \ref rdtesting_gpsdo_auto | | GPS-B200-TCXO-v1 | USRP B200 | Jackson Labs TCXO | \ref rdtesting_gpsdo_manual | \ref rdtesting_gpsdo_auto | | GPS-B210-TCXO-v1 | USRP B210 | Jackson Labs TCXO | \ref rdtesting_gpsdo_manual | \ref rdtesting_gpsdo_auto | \subsection rdtesting_gpsdo_recommendations Recommendations For cursory testing, not all tests within a device family are required (e.g., only testing the OCXO on any X-Series, and testing the TCXO on any B-Series is sufficient). The following tests are recommended for a minimum test (N stands for the latest version of this test): - One of GPS-X310-OCXO-vN or GPS-X300-OCXO-vN - One of GPS-B210-TCXO-vN or GPS-B200-TCXO-vN \subsection rdtesting_gpsdo_requirements Requirements All of these tests require a device that is GPSDO capable (e.g., X3x0, B2x0, N2x0). For those devices that have a separate GPS component (such as the Jackson Labs GPSDOs), this component is also required (called the "peripheral" in the following). \subsection rdtesting_gpsdo_manual GPSDO: Manual Test Procedure 1. Without connecting the peripheral to the device, run `uhd_usrp_probe` on the device and verify that the lack of GPSDO is correctly reported. No warning or error must be printed. 2. This and the following tests are run with the peripheral connected: Run `uhd_usrp_probe` and verify that the GPSDO is correctly reported. Power down the device before connecting the peripheral. The GPSDO must be reported found, and no error or warning must be printed. 3. Without connecting the GPS antenna input, run `query_gpsdo_sensors`. To pass, it must report the GPSDO as found, lock to the external reference, but then report not being locked to GPS. The tool will report a valid GPS time, and a string such as "GPS and UHD Device time are aligned" in case of success. 4. Connect a GPS antenna to the input and make sure it is in a position to receive GPS satellite data. Confirm that GPS lock is reported using `query_gpsdo_sensors` within 20 minutes of connecting the antenna. The tool `query_gpsdo_sensors` will print a string such as "GPS Locked" in case of success. All of these tests must pass for a 'pass' validation. \subsection rdtesting_gpsdo_auto GPSDO: Automatic Test Procedure tbd \section rdtesting_devtest Devtests | Test Code | Device | Peripherals | Manual Test Procedure | Automatic Test Procedure | |---------------------|---------------|-------------|-------------------------------|-----------------------------| | DEVTEST-X310-XG-v1 | USRP X310 | None | \ref rdtesting_devtest_manual | \ref rdtesting_devtest_auto | | DEVTEST-X310-HG-v1 | USRP X310 | None | \ref rdtesting_devtest_manual | \ref rdtesting_devtest_auto | | DEVTEST-X300-XG-v1 | USRP X300 | None | \ref rdtesting_devtest_manual | \ref rdtesting_devtest_auto | | DEVTEST-X300-HG-v1 | USRP X300 | None | \ref rdtesting_devtest_manual | \ref rdtesting_devtest_auto | | DEVTEST-E310-SG1-v1 | USRP E310-SG1 | None | \ref rdtesting_devtest_manual | \ref rdtesting_devtest_auto | | DEVTEST-E310-SG3-v1 | USRP E310-SG3 | None | \ref rdtesting_devtest_manual | \ref rdtesting_devtest_auto | | DEVTEST-B200-v1 | USRP B200 | None | \ref rdtesting_devtest_manual | \ref rdtesting_devtest_auto | | DEVTEST-B210-v1 | USRP B210 | None | \ref rdtesting_devtest_manual | \ref rdtesting_devtest_auto | | DEVTEST-B200m-v1 | USRP B200mini | None | \ref rdtesting_devtest_manual | \ref rdtesting_devtest_auto | | DEVTEST-B205m-v1 | USRP B205mini | None | \ref rdtesting_devtest_manual | \ref rdtesting_devtest_auto | The devtests are hardware tests built in to the UHD make system. They can be run directly from the build directory and require no configuration. Devtests are designed to always run, regardless of the actual device configuration. This means, by definition, that devtests cannot require special cabling, specific daughtercards, etc. Note: The actual devtests can change, since they're part of the code. This does not require a version bump on the test code. \subsection rdtesting_devtest_requirements Requirements Devtests are only defined for some devices. When running a devtest, all peripherals must be disconnected (e.g., no daughterboards on the X-Series, no GPSDOs on the B- and X-Series). \subsection rdtesting_devtest_manual Devtest: Manual Test Procedure ### X3x0 procedure 1. Make sure no peripherals are connected to the device (no daughterboards, no GSPDO, front panel GPIO is unconnected). 2. When testing the HG image, run a test once for each connection (1 GigE and 10 GigE). When testing the XG image, a test on either connection (SFP0 or SFP1) is sufficient. In both cases, also test via PCIe. 3. When the device is connected, simply run `make test_x3x0` from the command line in the build directory. Multiple devices connected will all get tested, there is no requirement to only connect a single device at a time (because devtest will run sequentially anyway). 4. Devtest must report no failures for a 'pass' validation. ### B2xx procedure Note: The test codes with an 'm' suffix refer to B200mini and B205mini, respectively. 1. Make sure no peripherals are connected to the device (no GPSDO if applicable, GPIO pins unconnected) 2. Test once via USB3, once via USB2. 3. Simply run `make test_b2xx` 4. Devtest must report no failures for a 'pass' validation. ### E310 procedure 1. Make sure GPIO pins are unconnected. 2. Tests need to be run natively on the device. If the build environment is available on the device, running `make test_e3xx` is sufficient. 3. In general, there is no build environment on the device (e.g. when doing a typical sshfs mount of an environment). In this case, copy the contents of the devtest directory onto the device, and run the following command (the environment variables need to point to the location of the devtest code, the location of the UHD examples such as benchmark_rate, and where you want to store log files, respectively): $DEVTEST_DIR/run_testsuite.py --src-dir $DEVTEST_DIR \ --devtest-pattern e3xx \ --build-type na \ --build-dir $EXAMPLES_DIR \ --device-filter e3x0 \ --log-dir $LOG_DIR 4. Devtest must report no failures for a 'pass' validation. \subsection rdtesting_devtest_auto Devtest: Automatic Test Procedure As all these tests can be run unsupervised, they can be run automatically given the correct device setup. The return code of the tests can be used to check for pass/fail conditions (return code 0 means 'pass'). \section rdtesting_fpga_testbenches FPGA: Testing through Simulations Test benches provide a faster way to verify the design through simulations. | Test Code | Device | Peripherals | Manual Test Procedure | Automatic Test Procedure | |------------------|-----------|-------------------|------------------------------|---------------------------| | FPGATB-v1 | None | None | \ref rdtesting_fpga_testbenches_manual | \ref rdtesting_fpga_testbenches_auto | \subsection rdtesting_fpga_testbenches_requirement: Requirements These tests are simulations and do not need any device. Vivado 15.4 should be installed. \subsection rdtesting_fpga_testbenches_manual: Manual Test Procedure 1. Go to the fpga directory depending on which test needs to be run. 1. NoC block test benches: Most of the NoC blocks have a test bench written in System Verilog that provides stimuli to the noc_block to verify it. The test bench for a block resides in /usrp3/lib/rfnoc/*_tb. 2. Running unit test benches: A few sub-blocks like noc-shell and sine_tone are used within the bigger noc_blocks. They have their own test benches. Their test benches reside in /usrp3/lib/sim/rfnoc/*. 3. Radio test bench: The radio test bench resides in /usrp3/lib/radio/noc_block_radio_core_tb/. 4. Device specific test benches: IPs specific to a device have test benches that exist in /usrp3/top/x300/sim/*. e.g. DMA testbench, PCIe, etc. 2. Setup the environment by running 'source /usrp3/top//setupenv.sh'. 3. In the test bench directory and run the test bench by 'make xsim' or 'make vsim'. 4. All of these tests must report no failure with a 'PASS' validation. Example testbench output: ======================================================== TESTBENCH STARTED: noc_block_skeleton ======================================================== [TEST CASE 1] (t=000000000) BEGIN: Wait for Reset... [TEST CASE 1] (t=000001002) DONE... Passed [TEST CASE 2] (t=000001002) BEGIN: Check NoC ID... Read Skeleton NOC ID: 1234000000000000 [TEST CASE 2] (t=000001238) DONE... Passed [TEST CASE 3] (t=000001238) BEGIN: Connect RFNoC blocks... Connecting noc_block_tb (SID: 1:0) to noc_block_skeleton (SID: 0:0) Connecting noc_block_skeleton (SID: 0:0) to noc_block_tb (SID: 1:0) [TEST CASE 3] (t=000005457) DONE... Passed [TEST CASE 4] (t=000005457) BEGIN: Write / readback user registers... [TEST CASE 4] (t=000006888) DONE... Passed [TEST CASE 5] (t=000006888) BEGIN: Test sequence... [TEST CASE 5] (t=000007403) DONE... Passed ======================================================== TESTBENCH FINISHED: noc_block_skeleton - Time elapsed: 7500 ns - Tests Expected: 5 - Tests Run: 5 - Tests Passed: 5 Result: PASSED ======================================================== Failing tests can be debugged by checking the waveform in a Vivado GUI by running 'make GUI=1 xsim'. More details on debugging https://kb.ettus.com/Debugging_FPGA_images \subsection rdtesting_fpga_testbenches_auto: Automatic Test Procedure Go to /usrp3/ and run 'build.py xsim all'. All tests should report 'PASS'. \section rdtesting_defining Defining R&D Tests Tests can be added any time to define procedures for pass/fail validation. Any test must include the following: - An unambiguous test code. This code consists of three characters that identify the test, a short description of the devices required, and a version suffix. Example: `GPS-X310-OCXO-v1` is a GPS-related test, requires an X310 and an OCXO to run, and is version 1 of this test. - A manual testing procedure. This must unambiguously define a set of tasks, and clearly identify whether or not a test has failed or passed. Tests do not require any other defined outcome other than 'pass' and 'fail'. - Optional, but highly recommended: An automatic test procedure. This must consist of a command, or a script, or a set of commands that can be automatically executed, and that will report a failure condition by means of returning a non-zero return value. Basic understanding of the operation of USRPs by the test operator should be assumed when authoring test procedures. The descriptions should be as short as possible to fully describe, unambiguously, how to reach a pass/fail conclusion. Test procedures may be updated at any time. If this happens, a new test code must be generated, with the version number increased. Old test codes are considered deprecated (if there exists a version 2 of a test, version 1 should not be run any more). */ // vim:ft=doxygen: