rfnoc: Add a new builder script for FPGA images based on eRFNoC.

The builder has two major jobs:
 * generate an image core file which reflects the FPGA image
   configuration given by the user
 * invoke Xilinx toolchain to actually build the FPGA image

For this purpose it needs to know where to find the FPGA source tree.
This tree can be give by the -F option.

The code that represents the user configurable part of the image is
written to a file called <device>_rfnoc_sandbox.v. To generate the file
these configuration files are needed:
 * io_signatures.yml: A file describing the known IO signatures. This file
                      is global for all devices and contains the superset
                      of all signatures (not all signatures are used by all
                      devices). It resides in usrp3/top/ of the tree given
                      by -F.
* bsp.yml: A file describing interfaces of a specific device such as AXIS
           transport interfaces or IO ports as well as device specific
           settings. It resides in usrp3/top/<device> of the tree given by -F.
* <image>.yml: a file provided by the user with freely chosen name.
               It describes which elements the image should contain
               (RFNoC blocks, streaming endpoints, IO ports) and how
               to connect them. The file also contains image setting
               such as the CHDR width to be used.

The script uses mako templates to generate the sandbox file. Before the
template engine is invoked sanity checks are executed to ensure the
configuration is synthactic correct. The script also build up structures
to ease Verilog code generation in the template engine. The engine should
not invoke more Python than echoing variables or iterating of lists or
dictionaries. This eases debugging as errors in the template engine are
hard to track and difficult to read for the user.

All Python code is placed in a package called rfnoc. The templates used
by the builder are also part of this package. image_builder.py contains
a method called build_image which is the main entry point for the builder.
It can also be utilized by other Python programs. To align with the
existing uhd_image_builder there is also a wrapper in bin called
rfnoc_image_builder which expects similar commands as the uhd_image_builder.

For debugging purpuse the script can be invoked from host/utils using
$ PYTHONPATH=. python bin/rfnoc_image_builder <options>

When installed using cmake/make/make install the builder installs to
${CMAKE_INSTALL_PREFIX}bin and can be invoked without specifying a
PYTHONPATH.

One can also install the package using pip from host/utils
$ pip install .

Image config generation can also be done from  GNU Radio Companion
files. The required GRC files are merged into gr-ettus.

Example usage:

$ rfnoc_image_builder -F ~/src/fpgadev -d x310 \
    -r path/to/x310_rfnoc_image_core.grc \
    -b path/to/gr-ettus/grc

Co-Authored-By: Alex Williams <alex.williams@ni.com>
Co-Authored-By: Sugandha Gupta <sugandha.gupta@ettus.com>
Co-Authored-By: Martin Braun <martin.braun@ettus.com>

1 files changed, 194 insertions, 0 deletions

diff --git a/host/utils/bin/rfnoc_image_builder b/host/utils/bin/rfnoc_image_builder
new file mode 100755
index 000000000..0df0d5d3c
--- /dev/null
+++ b/host/utils/bin/rfnoc_image_builder
@@ -0,0 +1,194 @@
+#!/usr/bin/env python
+"""
+Copyright 2019 Ettus Research, A National Instrument Brand
+
+This program is free software: you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation, either version 3 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program.  If not, see <http://www.gnu.org/licenses/>.
+"""
+
+from __future__ import print_function
+
+import sys
+import argparse
+import hashlib
+import logging
+import os
+import re
+import yaml
+
+logging.basicConfig(format='[%(levelname).3s] %(message)s')
+# CMAKE will dump final site-package folder into this to allow the script find
+# its package folder
+sys.path.insert(0, "@RFNOC_PACKAGE_DIR@")
+#pylint : disable = wrong - import - position
+import rfnoc.image_builder
+import rfnoc.yaml_utils
+
+
+def setup_parser():
+    """
+    Create argument parser
+    """
+    parser = argparse.ArgumentParser(
+        description="Build UHD image using RFNoC blocks",
+    )
+
+    config_group = parser.add_mutually_exclusive_group(required=True)
+    config_group.add_argument(
+        "-y", "--yaml_config",
+        help="Path to yml configuration file")
+    config_group.add_argument(
+        "-r", "--grc_config",
+        help="Path to grc file to generate config from")
+
+    parser.add_argument(
+        "-F", "--fpga-dir",
+        help="Path directory of the FPGA source tree",
+        required=True,
+        default=None)
+    parser.add_argument(
+        "-o", "--image-core-output",
+        help="Path to where to save the image core Verilog source. "
+             "Defaults to the location of the YAML file.")
+    parser.add_argument(
+        "-x", "--router-hex-output",
+        help="Path to where to save the static router hex file. "
+             "Defaults to the location of the YAML file, filename $device_static_router.hex",
+        default=None)
+
+    parser.add_argument(
+        "-I", "--include-dir",
+        help="Path directory of the RFNoC Out-of-Tree module",
+        nargs='+',
+        default=None)
+
+    parser.add_argument(
+        "-b", "--grc-blocks",
+        help="Path directory of GRC block descriptions (needed for --grc-config only)",
+        default=None)
+    parser.add_argument(
+        "-l", "--log-level",
+        help="Adjust log level",
+        default='info')
+    parser.add_argument(
+        "--generate-only",
+        help="Just generate files without building IP",
+        action="store_true")
+    parser.add_argument(
+        "-d", "--device",
+        help="Device to be programmed [x300, x310, e310, e320, n300, n310, n320]",
+        default="x310")
+    parser.add_argument(
+        "-t", "--target",
+        help="Build target (e.g. X310_HG, N320_XG, ...)",
+        default=None)
+    parser.add_argument(
+        "-g", "--GUI",
+        help="Open Vivado GUI during the FPGA building process",
+        action="store_true")
+    parser.add_argument(
+        "-c", "--clean-all",
+        help="Cleans the IP before a new build",
+        action="store_true")
+
+    return parser
+
+
+def image_config(args):
+    """
+    Load image configuration.
+
+    The configuration can be either passed as RFNoC image configuration or as
+    GNU Radio Companion grc. In latter case the grc files is converted into a
+    RFNoC image configuration on the fly.
+    :param args: arguments passed to the script.
+    :return: image configuration as dictionary
+    """
+    if args.yaml_config:
+        return rfnoc.yaml_utils.load_config(args.yaml_config, get_config_path()), \
+                args.yaml_config
+
+    with open(args.grc_config) as grc_file:
+        config = yaml.load(grc_file)
+        logging.info("Converting GNU Radio Companion file to image builder format")
+        config = rfnoc.image_builder.convert_to_image_config(config, args.grc_blocks)
+        return config, args.grc_config
+
+
+def resolve_path(path, local):
+    """
+    Replaced path by local if path is enclosed with "@" (placeholder markers)
+    :param path: the path to check
+    :param local: new path content if path is placeholder
+    :return: path if path is not a placeholder else local
+    """
+    return re.sub("^@.*@$", local, path)
+
+
+def get_fpga_path(args):
+    """
+    Returns FPGA path. This is the fpga_dir of arguments, If fpga_dir does
+    not exists it is the predefined path of of the script.
+    :param args: arguments passed to the script
+    :return: FPGA root path
+    """
+    result = args.fpga_dir
+    if not os.path.isdir(result):
+        logging.info("%s is not a valid directory.", result)
+        result = resolve_path("@FPGA_PATH@", os.path.join(
+            os.path.dirname(__file__), '..', '..', '..', 'fpga-src'))
+        logging.info("Fall back to %s", result)
+    return result
+
+
+def get_config_path():
+    """
+    Returns path that contains configurations files (yml descriptions for
+    block, IO signatures and device bsp).
+    :return: Configuration path
+    """
+    return os.path.normpath(resolve_path("@CONFIG_PATH@", os.path.join(
+        os.path.dirname(__file__), '..', '..', 'include', 'uhd')))
+
+
+def main():
+    """
+    Wrapper for rfnoc.image_builder.build_image.
+    :return: exit code
+    """
+    args = setup_parser().parse_args()
+    if args.log_level is not None:
+        logging.root.setLevel(args.log_level.upper())
+
+    config, source = image_config(args)
+    source_hash = hashlib.sha256()
+    with open(source, "rb") as source_file:
+        source_hash.update(source_file.read())
+
+    rfnoc.image_builder.build_image(
+        config=config,
+        fpga_path=args.fpga_dir,
+        config_path=get_config_path(),
+        device=args.device,
+        target=args.target,
+        generate_only=args.generate_only,
+        clean_all=args.clean_all,
+        gui=args.GUI,
+        source=source,
+        source_hash=source_hash.hexdigest(),
+        output_path=args.image_core_output,
+        router_hex_path=args.router_hex_output,
+        )
+
+if __name__ == "__main__":
+    exit(main())