Merge FPGA repository back into UHD repository

The FPGA codebase was removed from the UHD repository in 2014 to reduce
the size of the repository. However, over the last half-decade, the
split between the repositories has proven more burdensome than it has
been helpful. By merging the FPGA code back, it will be possible to
create atomic commits that touch both FPGA and UHD codebases. Continuous
integration testing is also simplified by merging the repositories,
because it was previously difficult to automatically derive the correct
UHD branch when testing a feature branch on the FPGA repository.

This commit also updates the license files and paths therein.

We are therefore merging the repositories again. Future development for
FPGA code will happen in the same repository as the UHD host code and
MPM code.

== Original Codebase and Rebasing ==

The original FPGA repository will be hosted for the foreseeable future
at its original local location: https://github.com/EttusResearch/fpga/

It can be used for bisecting, reference, and a more detailed history.

The final commit from said repository to be merged here is
05003794e2da61cabf64dd278c45685a7abad7ec. This commit is tagged as
v4.0.0.0-pre-uhd-merge.

If you have changes in the FPGA repository that you want to rebase onto
the UHD repository, simply run the following commands:

- Create a directory to store patches (this should be an empty
  directory):

    mkdir ~/patches

- Now make sure that your FPGA codebase is based on the same state as
  the code that was merged:

    cd src/fpga # Or wherever your FPGA code is stored
    git rebase v4.0.0.0-pre-uhd-merge

  Note: The rebase command may look slightly different depending on what
  exactly you're trying to rebase.

- Create a patch set for your changes versus v4.0.0.0-pre-uhd-merge:

    git format-patch v4.0.0.0-pre-uhd-merge -o ~/patches

  Note: Make sure that only patches are stored in your output directory.
  It should otherwise be empty. Make sure that you picked the correct
  range of commits, and only commits you wanted to rebase were exported
  as patch files.

- Go to the UHD repository and apply the patches:

    cd src/uhd # Or wherever your UHD repository is stored
    git am --directory fpga ~/patches/*
    rm -rf ~/patches # This is for cleanup

== Contributors ==

The following people have contributed mainly to these files (this list
is not complete):

Co-authored-by: Alex Williams <alex.williams@ni.com>
Co-authored-by: Andrej Rode <andrej.rode@ettus.com>
Co-authored-by: Ashish Chaudhari <ashish@ettus.com>
Co-authored-by: Ben Hilburn <ben.hilburn@ettus.com>
Co-authored-by: Ciro Nishiguchi <ciro.nishiguchi@ni.com>
Co-authored-by: Daniel Jepson <daniel.jepson@ni.com>
Co-authored-by: Derek Kozel <derek.kozel@ettus.com>
Co-authored-by: EJ Kreinar <ej@he360.com>
Co-authored-by: Humberto Jimenez <humberto.jimenez@ni.com>
Co-authored-by: Ian Buckley <ian.buckley@gmail.com>
Co-authored-by: Jörg Hofrichter <joerg.hofrichter@ni.com>
Co-authored-by: Jon Kiser <jon.kiser@ni.com>
Co-authored-by: Josh Blum <josh@joshknows.com>
Co-authored-by: Jonathon Pendlum <jonathan.pendlum@ettus.com>
Co-authored-by: Martin Braun <martin.braun@ettus.com>
Co-authored-by: Matt Ettus <matt@ettus.com>
Co-authored-by: Michael West <michael.west@ettus.com>
Co-authored-by: Moritz Fischer <moritz.fischer@ettus.com>
Co-authored-by: Nick Foster <nick@ettus.com>
Co-authored-by: Nicolas Cuervo <nicolas.cuervo@ettus.com>
Co-authored-by: Paul Butler <paul.butler@ni.com>
Co-authored-by: Paul David <paul.david@ettus.com>
Co-authored-by: Ryan Marlow <ryan.marlow@ettus.com>
Co-authored-by: Sugandha Gupta <sugandha.gupta@ettus.com>
Co-authored-by: Sylvain Munaut <tnt@246tNt.com>
Co-authored-by: Trung Tran <trung.tran@ettus.com>
Co-authored-by: Vidush Vishwanath <vidush.vishwanath@ettus.com>
Co-authored-by: Wade Fife <wade.fife@ettus.com>

25 files changed, 5555 insertions, 0 deletions

diff --git a/fpga/usrp3/lib/axi/Makefile.srcs b/fpga/usrp3/lib/axi/Makefile.srcs
new file mode 100644
index 000000000..28f63104d
--- /dev/null
+++ b/fpga/usrp3/lib/axi/Makefile.srcs
@@ -0,0 +1,36 @@
+#
+# Copyright 2012-2013 Ettus Research LLC
+# Copyright 2014 Ettus Research, a National Instruments Company
+#
+# SPDX-License-Identifier: LGPL-3.0-or-later
+#
+
+##################################################
+# FIFO Sources
+##################################################
+AXI_SRCS = $(abspath $(addprefix $(BASE_DIR)/../lib/axi/, \
+axi_chdr_header_trigger.v \
+axi_chdr_test_pattern.v \
+axi_defs.v \
+axi_dma_fifo.v \
+axi_dma_master.v \
+axi_replay.v \
+axi_embed_tlast.v \
+axi_extract_tlast.v \
+axi_fast_extract_tlast.v \
+axi_embed_tlast_tkeep.v \
+axi_extract_tlast_tkeep.v \
+axi_fast_fifo.v \
+axi_to_strobed.v \
+axis_data_swap.v \
+axi_dummy.v \
+strobed_to_axi.v \
+axi_add_preamble.v \
+axi_strip_preamble.v \
+crc_xnor.v \
+axis_packet_flush.v \
+axis_shift_register.v \
+axis_upsizer.v \
+axis_downsizer.v \
+axis_width_conv.v \
+))
diff --git a/fpga/usrp3/lib/axi/axi_add_preamble.v b/fpga/usrp3/lib/axi/axi_add_preamble.v
new file mode 100644
index 000000000..a66b4229d
--- /dev/null
+++ b/fpga/usrp3/lib/axi/axi_add_preamble.v
@@ -0,0 +1,157 @@
+//
+// Copyright 2016 Ettus Research LLC
+// Copyright 2018 Ettus Research, a National Instruments Company
+//
+// SPDX-License-Identifier: LGPL-3.0-or-later
+//
+// Adds preamble, EOP, and CRC/num_words check
+// <preamble> <packet> <EOP> [control_chksum,word_count,payload_chksum] 
+// <preamble>   = 64'h9E6774129E677412
+// <EOP>        = 64'h2A1D632F2A1D632F
+
+module axi_add_preamble #(
+  parameter WIDTH=64
+) (
+   input clk,
+   input reset,
+   input clear,
+   //
+   input [WIDTH-1:0] i_tdata,
+   input i_tlast,
+   input i_tvalid,
+   output i_tready,
+   //
+   output reg [WIDTH-1:0] o_tdata,
+   output o_tvalid,
+   input o_tready
+);
+
+    function [0:0] cvita_get_has_time;
+        input [63:0] header;
+        cvita_get_has_time = header[61];
+    endfunction
+
+   //States
+   localparam IDLE = 0;
+   localparam PREAMBLE = 1;
+   localparam PASS = 3;
+   localparam EOP = 4;
+   localparam CRC = 5;
+   
+   localparam PAYLOAD_WORDCOUNT_WIDTH = 16;
+   localparam PAYLOAD_CHKSUM_WIDTH = 32;
+   localparam CONTROL_CHKSUM_WIDTH = 16;
+   
+   reg [2:0]  state, next_state;
+   
+   reg [PAYLOAD_WORDCOUNT_WIDTH-1:0] word_count;
+   reg [PAYLOAD_WORDCOUNT_WIDTH-1:0] cntrl_length = 16'd2;
+   wire [PAYLOAD_CHKSUM_WIDTH-1:0] payload_chksum;
+   wire [CONTROL_CHKSUM_WIDTH-1:0] control_chksum;
+
+  // Payload LFSR
+  crc_xnor #(.INPUT_WIDTH(WIDTH), .OUTPUT_WIDTH(PAYLOAD_CHKSUM_WIDTH)) payload_chksum_gen (
+     .clk(clk), .rst(word_count<=cntrl_length), .hold(~(i_tready && i_tvalid)),
+     .input_data(i_tdata), .crc_out(payload_chksum)
+  );
+  
+  // Control LFSR
+  crc_xnor #(.INPUT_WIDTH(WIDTH), .OUTPUT_WIDTH(CONTROL_CHKSUM_WIDTH)) control_chksum_gen (
+     .clk(clk), .rst(word_count=='d0), .hold(~(i_tready && i_tvalid) || word_count>=cntrl_length),
+     .input_data(i_tdata), .crc_out(control_chksum)
+  );
+  
+  //Update control length so control checksum is correct
+  always @(posedge clk) begin
+    if (state == IDLE && i_tvalid)
+        cntrl_length <= cvita_get_has_time(i_tdata) ? 16'd2 : 16'd1;
+  end
+  
+  //Note that word_count includes EOP
+  always @(posedge clk) begin
+     if (state == IDLE) begin
+        word_count <= 0;
+     end else if (i_tready && i_tvalid || (o_tready && state == EOP)) begin
+        word_count <= word_count+1;
+     end
+  end
+      
+   always @(posedge clk) 
+      if (reset | clear) begin
+         state <= IDLE;
+      end else begin
+         state <= next_state;
+      end 
+
+   always @(*) begin
+      case(state)
+         IDLE: begin
+            if (i_tvalid) begin
+               next_state = PREAMBLE;
+            end else begin
+               next_state = IDLE;
+            end
+         end 
+         
+         PREAMBLE: begin
+            if(o_tready) begin
+                next_state = PASS;
+            end else begin
+                next_state = PREAMBLE;
+            end
+         end                
+
+         PASS: begin
+            if(i_tready && i_tvalid && i_tlast) begin
+                 next_state = EOP;
+             end else begin
+                 next_state = PASS;
+             end
+         end
+         
+         EOP: begin
+            if(o_tready) begin
+                next_state = CRC;
+            end else begin
+                next_state = EOP;
+            end
+         end          
+
+         CRC: begin
+            if(o_tready) begin
+                 next_state = IDLE;
+             end else begin
+                 next_state = CRC;
+             end
+         end
+         
+         default: begin
+            next_state = IDLE;
+         end
+        
+      endcase
+   end 
+
+   //
+   // Muxes
+   //
+   always @*
+      begin
+         case(state)
+            IDLE:           o_tdata = 0; 
+            PASS:           o_tdata = i_tdata;
+            PREAMBLE:       o_tdata = 64'h9E6774129E677412;
+            EOP:            o_tdata = 64'h2A1D632F2A1D632F;
+            CRC:            o_tdata = {control_chksum,word_count,payload_chksum};
+            default:        o_tdata = 0;
+            
+         endcase 
+      end
+
+   assign o_tvalid = (state == PASS) ? i_tvalid : (state != IDLE);
+   assign i_tready = (state == PASS) ? o_tready : 1'b0;
+
+endmodule 
+
+
+
diff --git a/fpga/usrp3/lib/axi/axi_chdr_header_trigger.v b/fpga/usrp3/lib/axi/axi_chdr_header_trigger.v
new file mode 100644
index 000000000..452e85052
--- /dev/null
+++ b/fpga/usrp3/lib/axi/axi_chdr_header_trigger.v
@@ -0,0 +1,43 @@
+
+// Copyright 2014 Ettus Research LLC
+// Copyright 2018 Ettus Research, a National Instruments Company
+//
+// SPDX-License-Identifier: LGPL-3.0-or-later
+
+
+module axi_chdr_header_trigger
+  #(
+    parameter WIDTH=64,
+    parameter SID=0
+   )
+    (input clk, input reset, input clear,
+     input [WIDTH-1:0] i_tdata, input i_tlast, input i_tvalid, input i_tready,
+     output trigger
+     );
+
+   
+   reg 	  state;
+   localparam IDLE = 0;
+   localparam RUN  = 1;
+ 
+
+   always @(posedge clk)
+     if(reset | clear)
+       state <= IDLE;
+     else
+       case (state)
+	 IDLE :
+	   if(i_tvalid && i_tready)
+	     state <= RUN;
+
+	 RUN :
+	   if(i_tready && i_tvalid && i_tlast)	    
+	     state <= IDLE;
+
+	 default :
+	   state <= IDLE;
+       endcase // case (state)
+
+   assign     trigger =  i_tvalid && i_tready && (state == IDLE) && (i_tdata[15:0] != SID);
+
+endmodule // axi_chdr_header_trigger
diff --git a/fpga/usrp3/lib/axi/axi_chdr_test_pattern.v b/fpga/usrp3/lib/axi/axi_chdr_test_pattern.v
new file mode 100644
index 000000000..e73eaaa9d
--- /dev/null
+++ b/fpga/usrp3/lib/axi/axi_chdr_test_pattern.v
@@ -0,0 +1,505 @@
+//
+// Copyright 2014 Ettus Research, a National Instruments Company
+//
+// SPDX-License-Identifier: LGPL-3.0-or-later
+//
+//
+// Synthesizable test pattern generator and checker
+// for AXI-Stream that can be used to test transparent blocks
+// (FIFOs, switches, etc)
+//
+
+module axi_chdr_test_pattern #(
+  parameter SR_BASE     = 8'h0,         //Base address for settings in this module
+  parameter DELAY_MODE  = "DYNAMIC",    //Are delays configurable at runtime {STATIC, DYNAMIC}
+  parameter SID_MODE    = "DYNAMIC",    //Is the SID configurable at runtime {STATIC, DYNAMIC}
+  parameter STATIC_SID  = 32'h0,        //SID Value if it is static
+  parameter BW_COUNTER  = 1             //Instantiate counters to measure bandwidth (Cycles of Data Xfer / Total cycles)
+) (
+  input             clk,
+  input             reset,
+
+  // AXI stream to hook up to input of DUT
+  output reg [63:0] i_tdata,
+  output reg        i_tlast,
+  output reg        i_tvalid,
+  input             i_tready,
+
+  // AXI stream to hook up to output of DUT
+  input      [63:0] o_tdata,
+  input             o_tlast,
+  input             o_tvalid,
+  output reg        o_tready,
+
+  //Settings bus interface
+  input             set_stb,
+  input       [7:0] set_addr,
+  input      [31:0] set_data,
+
+  // Test flags
+  output reg        running,    //Test is currently in progress
+  output reg        done,       //(Sticky) Test has finished executing
+  output reg  [1:0] error,      //Error code from last test execution
+
+  output    [127:0] status_vtr, //More information about test failure.
+  output     [95:0] bw_ratio    //Bandwidth counter info
+);
+
+  //
+  // Error Codes
+  //
+  localparam ERR_SUCCESS                  = 0;
+  localparam ERR_DATA_MISMATCH            = 1;
+  localparam ERR_SIZE_MISMATCH_TOO_LONG   = 2;
+  localparam ERR_SIZE_MISMATCH_TOO_SHORT  = 3;
+  
+  localparam ERR_TIMEOUT_LOG2             = 10;
+
+  //
+  // Settings
+  //
+  wire        bist_size_ramp;
+  wire [1:0]  bist_test_patt;
+  wire [12:0] bist_max_pkt_size;
+  wire        bist_go, bist_cont, bist_ctrl_wr;
+  wire [1:0]  bist_ctrl_reserved;
+  wire [17:0] bist_max_pkts;
+  wire [15:0] bist_tx_pkt_delay;
+  wire [7:0]  bist_rx_samp_delay;
+  wire [31:0] bist_cvita_sid;
+
+  localparam TEST_PATT_ZERO_ONE     = 2'd0;
+  localparam TEST_PATT_CHECKERBOARD = 2'd1;
+  localparam TEST_PATT_COUNT        = 2'd2;
+  localparam TEST_PATT_COUNT_INV    = 2'd3;
+
+  // SETTING: Test Control Register
+  // Fields:
+  // - [0]    : (Strobe) Start the test if 1, otherwise stop a running test.
+  //            If no test is running then reset the status. (Reseting a
+  //            continuously running test requires two writes to this reg)
+  // - [1]    : Start the test in continuous mode. (Run until reset or failure)
+  // - [3:2]  : <Unused>
+  // - [5:4]  : Test pattern:
+  //            * 00 = Zeros and Ones (0x0000000000000000 <-> 0xFFFFFFFFFFFFFFFF)
+  //            * 01 = Checkerboard   (0x5555555555555555 <-> 0xAAAAAAAAAAAAAAAA)
+  //            * 10 = Counter        (Each byte will count up)
+  //            * 11 = Invert Counter (Each byte will count up and invert)
+  setting_reg #(
+    .my_addr(SR_BASE + 0), .width(6), .at_reset(3'b0)
+  ) reg_ctrl (
+    .clk(clk), .rst(reset),
+    .strobe(set_stb), .addr(set_addr), .in(set_data),
+    .out({bist_test_patt, bist_ctrl_reserved, bist_cont, bist_go}),.changed(bist_ctrl_wr)
+  );
+  
+  wire bist_start = bist_ctrl_wr & bist_go;
+  wire bist_clear = bist_ctrl_wr & ~bist_go;
+
+  // SETTING: Test Packet Configuration Register
+  // Fields:
+  // - [17:0]  : Number of packets to transfer per BIST execution
+  // - [30:18] : Max number of bytes of payload per packet
+  // - [31]    : Send variable (ramping) sized packets
+  setting_reg #(
+    .my_addr(SR_BASE + 1), .width(32), .at_reset(32'b0)
+  ) reg_pkt_config (
+    .clk(clk), .rst(reset),
+    .strobe(set_stb), .addr(set_addr), .in(set_data),
+    .out({bist_size_ramp, bist_max_pkt_size, bist_max_pkts}),.changed()
+  );
+
+  generate if (DELAY_MODE == "DYNAMIC") begin
+    // SETTING: Delay Register
+    // Fields:
+    // - [15:0]   : Number of cycles to wait between generating consecutive *packets*
+    // - [23:16]  : Number of cycles to wait between consuming consecutive *samples*
+    setting_reg #(
+      .my_addr(SR_BASE + 2), .width(24), .at_reset(24'b0)
+    ) reg_delay (
+      .clk(clk), .rst(reset),
+      .strobe(set_stb), .addr(set_addr), .in(set_data),
+      .out({bist_rx_samp_delay, bist_tx_pkt_delay}),.changed()
+    );
+  end else begin
+    assign {bist_rx_samp_delay, bist_tx_pkt_delay} = 24'h0;
+  end endgenerate
+
+  generate if (SID_MODE == "DYNAMIC") begin
+    // SETTING: CHDR Stream ID Register
+    // Fields:
+    // - [31:0]   : Stream ID to attach to CHDR packets
+    setting_reg #(
+      .my_addr(SR_BASE + 3), .width(32), .at_reset(32'b0)
+    ) reg_sid (
+      .clk(clk), .rst(reset),
+      .strobe(set_stb), .addr(set_addr), .in(set_data),
+      .out(bist_cvita_sid),.changed()
+    );
+  end else begin
+    assign bist_cvita_sid = STATIC_SID;
+  end endgenerate
+
+  //
+  // State
+  //
+  localparam TX_IDLE    = 3'd0;
+  localparam TX_START   = 3'd1;
+  localparam TX_ACTIVE  = 3'd2;
+  localparam TX_GAP     = 3'd3;
+  localparam TX_DONE    = 3'd4;
+  localparam TX_WAIT    = 3'd5;
+
+  localparam RX_IDLE    = 3'd0;
+  localparam RX_ACTIVE  = 3'd1;
+  localparam RX_FAIL    = 3'd2;
+  localparam RX_DONE    = 3'd3;
+  localparam RX_WAIT    = 3'd4;
+
+  reg [2:0]                   tx_state, rx_state;
+  reg [ERR_TIMEOUT_LOG2-1:0]  err_timeout;
+  reg [1:0]                   test_pattern;
+  reg                         rearm_test;
+
+  reg [17:0]  tx_pkt_cnt, rx_pkt_cnt;
+  reg [13:0]  tx_byte_cnt, rx_byte_cnt;
+  reg [23:0]  test_run_cnt;
+  reg [15:0]  tx_delay;
+  reg [7:0]   rx_delay;
+  wire [63:0] tx_cvita_hdr, rx_cvita_hdr;
+
+  wire tx_next_pkt_cond, rx_next_pkt_cond;
+  assign tx_next_pkt_cond = (tx_byte_cnt[12:3] == bist_max_pkt_size[12:3]) ||               //Packet size reaches max OR
+                            (bist_size_ramp && ({7'h0, tx_byte_cnt[13:3]} == tx_pkt_cnt));  //Packet size / 8 == Packet Count
+  assign rx_next_pkt_cond = (rx_byte_cnt[12:3] == bist_max_pkt_size[12:3]) || 
+                            (bist_size_ramp && ({7'h0, rx_byte_cnt[13:3]} == rx_pkt_cnt));
+
+  wire tx_test_done_cond, rx_test_done_cond;
+  assign tx_test_done_cond = (tx_pkt_cnt == bist_max_pkts);
+  assign rx_test_done_cond = (rx_pkt_cnt == bist_max_pkts);
+  
+  reg [63:0] tx_data_next, rx_data_exp;
+  always @(*) begin
+    case (test_pattern)
+      TEST_PATT_ZERO_ONE: begin
+        tx_data_next  <= {8{tx_byte_cnt[3] ? 8'h00 : 8'hFF}};
+        rx_data_exp   <= {8{rx_byte_cnt[3] ? 8'h00 : 8'hFF}};
+      end
+      TEST_PATT_CHECKERBOARD: begin
+        tx_data_next  <= {32{tx_byte_cnt[3] ? 2'b01 : 2'b10}};
+        rx_data_exp   <= {32{rx_byte_cnt[3] ? 2'b01 : 2'b10}};
+      end
+      TEST_PATT_COUNT: begin
+        tx_data_next  <= {8{tx_byte_cnt[10:3]}};
+        rx_data_exp   <= {8{rx_byte_cnt[10:3]}};
+      end
+      TEST_PATT_COUNT_INV: begin
+        tx_data_next  <= {8{(tx_byte_cnt[3] ? 8'hFF : 8'h00) ^ tx_byte_cnt[10:3]}};
+        rx_data_exp   <= {8{(rx_byte_cnt[3] ? 8'hFF : 8'h00) ^ rx_byte_cnt[10:3]}};
+      end
+      default: begin
+        tx_data_next  <= 64'd0;
+        rx_data_exp   <= 64'd0;
+      end
+    endcase
+  end
+
+  //NOTE: We always attach the max size in the packet header for simplicity.
+  //      This will not work with state machines that validate the packet length in the
+  //      header with the tlast position.
+  assign tx_cvita_hdr = {4'h0, tx_pkt_cnt[11:0], 2'b00, bist_max_pkt_size, bist_cvita_sid};
+  assign rx_cvita_hdr = {4'h0, rx_pkt_cnt[11:0], 2'b00, bist_max_pkt_size, bist_cvita_sid};
+
+  reg [63:0] o_tdata_fail;
+  assign status_vtr = {     //Status at the time of failure
+    o_tdata_fail,           //[127:64]
+    test_run_cnt,           //[63:40]
+    rx_data_exp[7:0],       //[39:32]
+    rx_pkt_cnt,             //[31:14]
+    rx_byte_cnt             //[13:0]
+  };
+
+  //-------------------------------------------------------
+  // Transmitter
+  //-------------------------------------------------------
+  always @(posedge clk) begin
+    if (reset | (bist_clear & ~rearm_test)) begin
+      tx_delay      <= 0;
+      tx_pkt_cnt    <= 0;
+      tx_byte_cnt   <= 0;
+      i_tdata       <= 64'h0;
+      i_tlast       <= 1'b0;
+      i_tvalid      <= 1'b0;
+      tx_state      <= TX_IDLE;
+    end else begin
+      case(tx_state)
+        TX_IDLE: begin
+          tx_delay      <= 0;
+          tx_pkt_cnt    <= 1;
+          tx_byte_cnt   <= 0;
+          i_tdata       <= 64'h0;
+          i_tlast       <= 1'b0;
+          i_tvalid      <= 1'b0;
+          // Run when bist_start asserted.
+          if (bist_start | rearm_test) begin
+            tx_state      <= TX_START;
+            test_pattern  <= bist_test_patt;
+          end
+        end // case: TX_IDLE
+
+        // START signal is asserted.
+        // Now need to start transmiting a packet.
+        TX_START: begin
+          // At the next clock edge drive first beat of new packet onto HDR bus.
+          i_tlast     <= 1'b0;
+          i_tvalid    <= 1'b1;
+          tx_byte_cnt <= tx_byte_cnt + 8;
+          i_tdata     <= tx_cvita_hdr;
+          tx_state    <= TX_ACTIVE;
+        end
+
+        // Valid data is (already) being driven onto the CHDR bus.
+        // i_tlast may also be driven asserted if current data count has reached EOP.
+        // Watch i_tready to see when it's consumed.
+        // When packets are consumed increment data counter or transition state if
+        // EOP has sucsesfully concluded.
+        TX_ACTIVE: begin
+          i_tvalid <= 1'b1; // Always assert tvalid
+          if (i_tready) begin
+            i_tdata <= tx_data_next;
+            // Will this next beat be the last in a packet?
+            if (tx_next_pkt_cond) begin
+              tx_byte_cnt <= 0;
+              i_tlast     <= 1'b1;
+              tx_state    <= TX_GAP;
+            end else begin
+              tx_byte_cnt <= tx_byte_cnt + 8;
+              i_tlast     <= 1'b0;
+              tx_state    <= TX_ACTIVE;
+            end
+          end else begin
+            //Keep driving all CHDR bus signals as-is until i_tready is asserted.
+            tx_state <= TX_ACTIVE;
+          end
+        end // case: TX_ACTIVE
+
+        // Force an inter-packet gap between packets in a BIST sequence where tvalid is driven low.
+        // As we leave this state check if all packets in BIST sequence have been generated yet,
+        // and if so go to done state.
+        TX_GAP: begin
+          if (i_tready) begin
+            i_tvalid    <= 1'b0;
+            i_tdata     <= 64'h0;
+            i_tlast     <= 1'b0;
+            tx_pkt_cnt  <= tx_pkt_cnt + 1;
+
+            if (tx_test_done_cond) begin
+              tx_state <= TX_DONE;
+            end else begin
+              tx_state <= TX_WAIT;
+              tx_delay <= bist_tx_pkt_delay;
+            end
+          end else begin // if (i_tready)
+            tx_state <= TX_GAP;
+          end
+        end // case: TX_GAP
+
+        // Simulate inter packet gap in real UHD system
+        TX_WAIT: begin
+          if (tx_delay == 0)
+            tx_state <= TX_START;
+          else begin
+            tx_delay <= tx_delay - 1;
+            tx_state <= TX_WAIT;
+          end
+        end
+
+        // Complete test pattern BIST sequence has been transmitted.
+        // Sit in this state until the RX side consumes all packets except
+        // for when the test is running in continuous mode.
+        TX_DONE: begin
+          i_tvalid  <= 1'b0;
+          i_tlast   <= 1'b0;
+          i_tdata   <= 64'd0;
+
+          if (running & ~rearm_test) begin
+            tx_state <= TX_DONE;
+          end else begin
+            tx_state <= TX_IDLE;
+          end
+        end
+      endcase // case (tx_state)
+    end
+  end
+
+  //-------------------------------------------------------
+  // Receiver
+  //-------------------------------------------------------
+  always @(posedge clk) begin
+    if (reset | (bist_clear & ~rearm_test)) begin
+      rx_delay      <= 0;
+      rx_pkt_cnt    <= 0;
+      rx_byte_cnt   <= 0;
+      o_tdata_fail  <= 64'h0;
+      o_tready      <= 1'b0;
+      error         <= ERR_SUCCESS;
+      done          <= 1'b0;
+      rx_state      <= RX_IDLE;
+      err_timeout   <= {ERR_TIMEOUT_LOG2{1'b0}};
+      test_run_cnt  <= 0;
+    end else begin
+      case(rx_state)
+        RX_IDLE: begin
+          rx_delay      <= 0;
+          rx_pkt_cnt    <= 1;
+          rx_byte_cnt   <= 0;
+          o_tdata_fail  <= 64'h0;
+          o_tready      <= 1'b0;
+          error         <= ERR_SUCCESS;
+          done          <= 1'b0;
+          err_timeout   <= {ERR_TIMEOUT_LOG2{1'b0}};
+          // Not accepting data whilst Idle,
+          // switch to active when packet arrives
+          if (o_tvalid) begin
+            o_tready <= 1'b1;
+            rx_state <= RX_ACTIVE;
+          end else begin
+            rx_state <= RX_IDLE;
+          end
+        end
+
+        RX_ACTIVE: begin
+          o_tready <= 1'b1;
+          if (o_tvalid) begin
+            if (o_tdata != (rx_byte_cnt == 0 ? rx_cvita_hdr : rx_data_exp)) begin
+              $display("axis_test_pattern: o_tdata: %x  !=  expected:  %x @ time: %d", o_tdata, rx_data_exp, $time);
+              error         <= ERR_DATA_MISMATCH;
+              rx_state      <= RX_FAIL;
+              o_tdata_fail  <= o_tdata;
+            end else if (rx_next_pkt_cond) begin
+              // Last not asserted when it should be!
+              if (~(o_tlast === 1)) begin
+                $display("axis_test_pattern: o_tlast not asserted when it should be @ time: %d", $time);
+                error    <= ERR_SIZE_MISMATCH_TOO_LONG;
+                rx_state <= RX_FAIL;
+              end else begin
+                // End of packet, set up to RX next
+                rx_byte_cnt <= 0;
+                rx_pkt_cnt  <= rx_pkt_cnt + 1;
+                rx_delay    <= bist_rx_samp_delay;
+                if (rx_test_done_cond) begin
+                  rx_state      <= rearm_test ? RX_IDLE : RX_DONE;
+                  error         <= ERR_SUCCESS;
+                  test_run_cnt  <= test_run_cnt + 1;
+                end else begin
+                  rx_state      <= RX_WAIT;
+                end
+                o_tready        <= 1'b0;
+              end
+            end else begin
+              // ...last asserted when it should not be!
+              if (~(o_tlast === 0)) begin
+                $display("axis_test_pattern: o_tlast asserted when it should not be @ time: %d", $time);
+                error       <= ERR_SIZE_MISMATCH_TOO_SHORT;
+                rx_state    <= RX_FAIL;
+              end else begin
+                // Still in packet body
+                rx_byte_cnt <= rx_byte_cnt + 8;
+                rx_delay    <= bist_rx_samp_delay;
+                if (bist_rx_samp_delay == 0) begin
+                  rx_state    <= RX_ACTIVE;
+                end else begin
+                  rx_state    <= RX_WAIT;
+                  o_tready    <= 1'b0;
+                end
+              end
+            end
+          end else begin
+            // Nothing to do this cycle
+            rx_state <= RX_ACTIVE;
+          end
+        end // case: RX_ACTIVE
+
+        // To simulate the radio consuming samples at a steady rate set by the decimation
+        // have a programable delay here
+        RX_WAIT: begin
+          if (rx_delay == 0) begin
+            rx_state <= RX_ACTIVE;
+            o_tready <= 1'b1;
+          end else begin
+            rx_delay <= rx_delay - 1;
+            rx_state <= RX_WAIT;
+          end
+        end
+
+        RX_FAIL: begin
+          //The test has failed but the sender still has packets en route
+          //Consume all of them before asserting done. Packets could be
+          //malformed so just blindly consume lines and count cycles of
+          //gaps. If non-valid cycles are more than 2^ERR_TIMEOUT_LOG2 then stop.
+          o_tready <= 1'b1;
+          if (~o_tvalid) begin
+            if (err_timeout == {ERR_TIMEOUT_LOG2{1'b1}}) begin
+              rx_state <= RX_DONE;
+            end
+            err_timeout <= err_timeout + 1;
+          end
+        end
+
+        RX_DONE: begin
+          o_tready    <= 1'b0;
+          done        <= 1'b1;
+          //The only way to exit this state is by asserting bist_clear
+        end
+      endcase // case (rx_state)
+    end
+  end
+
+  //-------------------------------------------------------
+  // Status Monitor
+  //-------------------------------------------------------
+  always @(posedge clk) begin
+    if (reset)
+      running <= 1'b0;
+    else if (tx_state == TX_START)
+      running <= 1'b1;
+    else if (rx_state == RX_DONE)
+      running <= 1'b0;
+  end
+
+  always @(posedge clk) begin
+    if (reset | bist_clear)
+      rearm_test  <= 1'b0;
+    else if (bist_start & bist_cont)
+      rearm_test  <= 1'b1;
+    else if (rx_state == RX_FAIL)
+      rearm_test  <= 1'b0;
+  end
+  
+  //-------------------------------------------------------
+  // Bandwidth Counter
+  //-------------------------------------------------------
+  generate if (BW_COUNTER) begin
+    reg [47:0]         word_count, cyc_count;
+    assign bw_ratio = {word_count, cyc_count};
+
+    //Count number of lines transferred
+    always @(posedge clk) begin
+      if (reset| (bist_clear & ~rearm_test) | bist_start)
+        word_count  <= 48'd0;
+      else if (o_tvalid && rx_state == RX_ACTIVE)
+        word_count  <= word_count + 48'd1;
+    end
+
+    //Count cycles as long as test is running
+    always @(posedge clk) begin
+      if (reset| (bist_clear & ~rearm_test) | bist_start)
+        cyc_count   <= 48'd0;
+      else if (rx_state == RX_ACTIVE || rx_state == RX_WAIT)
+        cyc_count   <= cyc_count + 48'd1;
+    end
+  end else begin
+    assign bw_ratio = 96'h0;
+  end endgenerate
+
+endmodule
diff --git a/fpga/usrp3/lib/axi/axi_defs.v b/fpga/usrp3/lib/axi/axi_defs.v
new file mode 100644
index 000000000..f6f5a1ede
--- /dev/null
+++ b/fpga/usrp3/lib/axi/axi_defs.v
@@ -0,0 +1,40 @@
+//
+// Copyright 2014 Ettus Research LLC
+// Copyright 2018 Ettus Research, a National Instruments Company
+//
+// SPDX-License-Identifier: LGPL-3.0-or-later
+//
+//
+// AXI4 Burst enumeration
+//
+`define AXI4_BURST_FIXED 2'b00
+`define AXI4_BURST_INCR 2'b01
+`define AXI4_BURST_WRAP 2'b10
+`define AXI4_BURST_RSVD 2'b11
+//
+// AXI4 response code enumeration
+//
+`define AXI4_RESP_OKAY 2'b00
+`define AXI4_RESP_EXOKAY 2'b01
+`define AXI4_RESP_SLVERR 2'b10
+`define AXI4_RESP_DECERR 2'b11
+//
+// AXI4 lock enumeration
+//
+`define AXI4_LOCK_NORMAL 1'b0
+`define AXI4_LOCK_EXCLUSIVE 1'b1
+//
+// AXI4 memory attrubutes
+//
+`define AXI4_CACHE_ALLOCATE 4'h8
+`define AXI4_CACHE_OTHER_ALLOCATE 4'h4
+`define AXI4_CACHE_MODIFIABLE 4'h2
+`define AXI4_CACHE_BUFFERABLE 4'h1
+//
+// AXI4 PROT attributes
+//
+`define AXI4_PROT_PRIVILEDGED 3'h1
+`define AXI4_PROT_NON_SECURE 3'h2
+`define AXI4_PROT_INSTRUCTION 3'h4
+
+
diff --git a/fpga/usrp3/lib/axi/axi_dma_fifo.v b/fpga/usrp3/lib/axi/axi_dma_fifo.v
new file mode 100644
index 000000000..3664b43b3
--- /dev/null
+++ b/fpga/usrp3/lib/axi/axi_dma_fifo.v
@@ -0,0 +1,1073 @@
+//
+// Copyright 2015 Ettus Research, a National Instruments Company
+//
+// SPDX-License-Identifier: LGPL-3.0-or-later
+//
+
+//
+// There are various obligations put on this code not present in regular BRAM based FIFO's
+//
+// 1) Bursts are way more efficient, use local small FIFO's to interact with DRAM
+// 2) Never cross a 4KByte address boundary within a single transaction, this is an AXI4 rule.
+// 3) 2^SIZE must be greater than 4KB so that the 4KByte page protection also deals with FIFO wrap corner case.
+//
+module axi_dma_fifo 
+#(
+   parameter SIMULATION       = 0,             // Shorten flush counter for simulation
+   parameter DEFAULT_BASE     = 30'h00000000,
+   parameter DEFAULT_MASK     = 30'hFF000000,
+   parameter DEFAULT_TIMEOUT  = 12'd256,
+   parameter BUS_CLK_RATE     = 32'd166666666,  // Frequency in Hz of bus_clk
+   parameter SR_BASE          = 0,              // Base address for settings registers
+   parameter EXT_BIST         = 0,              // If 1 then instantiate extended BIST with dynamic SID, delays and BW counters
+   parameter MAX_PKT_LEN      = 12              // Log2 of maximum packet length
+) (
+   input bus_clk,
+   input bus_reset, 
+   input dram_clk,
+   input dram_reset,
+   //
+   // AXI Write address channel
+   //
+   output [0 : 0] m_axi_awid,     // Write address ID. This signal is the identification tag for the write address signals
+   output [31 : 0] m_axi_awaddr,  // Write address. The write address gives the address of the first transfer in a write burst
+   output [7 : 0] m_axi_awlen,    // Burst length. The burst length gives the exact number of transfers in a burst.
+   output [2 : 0] m_axi_awsize,   // Burst size. This signal indicates the size of each transfer in the burst. 
+   output [1 : 0] m_axi_awburst,  // Burst type. The burst type and the size information, determine how the address is calculated
+   output [0 : 0] m_axi_awlock,   // Lock type. Provides additional information about the atomic characteristics of the transfer.
+   output [3 : 0] m_axi_awcache,  // Memory type. This signal indicates how transactions are required to progress
+   output [2 : 0] m_axi_awprot,   // Protection type. This signal indicates the privilege and security level of the transaction
+   output [3 : 0] m_axi_awqos,    // Quality of Service, QoS. The QoS identifier sent for each write transaction
+   output [3 : 0] m_axi_awregion, // Region identifier. Permits a single physical interface on a slave to be re-used.
+   output [0 : 0] m_axi_awuser,   // User signal. Optional User-defined signal in the write address channel.
+   output m_axi_awvalid,          // Write address valid. This signal indicates that the channel is signaling valid write addr
+   input m_axi_awready,           // Write address ready. This signal indicates that the slave is ready to accept an address
+   //
+   // AXI Write data channel.
+   //
+   output [63 : 0] m_axi_wdata,   // Write data
+   output [7 : 0] m_axi_wstrb,    // Write strobes. This signal indicates which byte lanes hold valid data.
+   output m_axi_wlast,            // Write last. This signal indicates the last transfer in a write burst
+   output [0 : 0] m_axi_wuser,    // User signal. Optional User-defined signal in the write data channel.
+   output m_axi_wvalid,           // Write valid. This signal indicates that valid write data and strobes are available. 
+   input m_axi_wready,            // Write ready. This signal indicates that the slave can accept the write data.
+   //
+   // AXI Write response channel signals
+   //
+   input [0 : 0] m_axi_bid,       // Response ID tag. This signal is the ID tag of the write response. 
+   input [1 : 0] m_axi_bresp,     // Write response. This signal indicates the status of the write transaction.
+   input [0 : 0] m_axi_buser,     // User signal. Optional User-defined signal in the write response channel.
+   input m_axi_bvalid,            // Write response valid. This signal indicates that the channel is signaling a valid response
+   output m_axi_bready,           // Response ready. This signal indicates that the master can accept a write response
+   //
+   // AXI Read address channel
+   //
+   output [0 : 0] m_axi_arid,     // Read address ID. This signal is the identification tag for the read address group of signals
+   output [31 : 0] m_axi_araddr,  // Read address. The read address gives the address of the first transfer in a read burst
+   output [7 : 0] m_axi_arlen,    // Burst length. This signal indicates the exact number of transfers in a burst.
+   output [2 : 0] m_axi_arsize,   // Burst size. This signal indicates the size of each transfer in the burst.
+   output [1 : 0] m_axi_arburst,  // Burst type. The burst type and the size information determine how the address for each transfer
+   output [0 : 0] m_axi_arlock,   // Lock type. This signal provides additional information about the atomic characteristics
+   output [3 : 0] m_axi_arcache,  // Memory type. This signal indicates how transactions are required to progress 
+   output [2 : 0] m_axi_arprot,   // Protection type. This signal indicates the privilege and security level of the transaction
+   output [3 : 0] m_axi_arqos,    // Quality of Service, QoS. QoS identifier sent for each read transaction.
+   output [3 : 0] m_axi_arregion, // Region identifier. Permits a single physical interface on a slave to be re-used
+   output [0 : 0] m_axi_aruser,   // User signal. Optional User-defined signal in the read address channel.
+   output m_axi_arvalid,          // Read address valid. This signal indicates that the channel is signaling valid read addr
+   input m_axi_arready,           // Read address ready. This signal indicates that the slave is ready to accept an address
+   //
+   // AXI Read data channel
+   //
+   input [0 : 0] m_axi_rid,       // Read ID tag. This signal is the identification tag for the read data group of signals
+   input [63 : 0] m_axi_rdata,    // Read data.
+   input [1 : 0] m_axi_rresp,     // Read response. This signal indicates the status of the read transfer
+   input m_axi_rlast,             // Read last. This signal indicates the last transfer in a read burst.
+   input [0 : 0] m_axi_ruser,     // User signal. Optional User-defined signal in the read data channel.
+   input m_axi_rvalid,            // Read valid. This signal indicates that the channel is signaling the required read data. 
+   output m_axi_rready,           // Read ready. This signal indicates that the master can accept the read data and response
+   //
+   // CHDR friendly AXI stream input
+   //
+   input [63:0] i_tdata,
+   input i_tlast,
+   input i_tvalid,
+   output i_tready,
+   //
+   // CHDR friendly AXI Stream output
+   //
+   output [63:0] o_tdata,
+   output o_tlast,
+   output o_tvalid,
+   input o_tready,
+   //
+   // Settings and Readback
+   //
+   input              set_stb,
+   input [7:0]        set_addr,
+   input [31:0]       set_data,
+   output reg [31:0]  rb_data,
+   //
+   // Debug Bus
+   //
+   output [197:0] debug
+);
+
+   //
+   // We are only solving for width 64bits here, since it's our standard CHDR quanta
+   //
+   localparam DWIDTH = 64;
+   localparam AWIDTH = 30;  //Can address 1GiB of memory
+
+   //
+   // Settings and Readback
+   //
+   wire [2:0]         rb_addr;
+   wire               clear_bclk, flush_bclk;
+   wire               supress_enable_bclk;
+   wire [15:0]        supress_threshold_bclk;
+   wire [11:0]        timeout_bclk;
+   wire [AWIDTH-1:0]  fifo_base_addr_bclk;
+   wire [AWIDTH-1:0]  fifo_addr_mask_bclk;
+   wire [0:0]         ctrl_reserved;
+
+   wire [31:0]  rb_fifo_status;
+   wire [3:0]   rb_bist_status;
+   wire [95:0]  rb_bist_bw_ratio;
+   reg  [31:0]  out_pkt_count = 32'd0;
+
+   localparam RB_FIFO_STATUS    = 3'd0;
+   localparam RB_BIST_STATUS    = 3'd1;
+   localparam RB_BIST_XFER_CNT  = 3'd2;
+   localparam RB_BIST_CYC_CNT   = 3'd3;
+   localparam RB_BUS_CLK_RATE   = 3'd4;
+   localparam RB_OUT_PKT_CNT    = 3'd5;
+
+   // SETTING: Readback Address Register
+   // Fields:
+   // - [2:0]  : Address for readback register
+   //            - 0 = RB_FIFO_STATUS
+   //            - 1 = RB_BIST_STATUS
+   //            - 2 = RB_BIST_XFER_CNT
+   //            - 3 = RB_BIST_CYC_CNT
+   //            - 4 = RB_BUS_CLK_RATE
+   //            - rest reserved
+   setting_reg #(.my_addr(SR_BASE + 0), .awidth(8), .width(3), .at_reset(3'b000)) sr_readback
+     (.clk(bus_clk), .rst(bus_reset),
+      .strobe(set_stb), .addr(set_addr), .in(set_data),
+      .out(rb_addr), .changed());
+
+   // SETTING: FIFO Control Register
+   // Fields:
+   // - [0]     : Clear FIFO and discard stored data
+   // - [1]     : Enable read suppression to prioritize writes
+   // - [2]     : Flush all packets from the FIFO
+   // - [3]     : Reserved
+   // - [15:4]  : Timeout (in memory clock beats) for issuing smaller than optimal bursts
+   // - [31:16] : Read suppression threshold in number of words
+   setting_reg #(.my_addr(SR_BASE + 1), .awidth(8), .width(32), .at_reset({16'h0, DEFAULT_TIMEOUT[11:0], 1'b0, 1'b0, 1'b0, 1'b1})) sr_fifo_ctrl
+     (.clk(bus_clk), .rst(bus_reset),
+      .strobe(set_stb), .addr(set_addr), .in(set_data),
+      .out({supress_threshold_bclk, timeout_bclk, ctrl_reserved, flush_bclk, supress_enable_bclk, clear_bclk}), .changed());
+
+   // SETTING: Base Address for FIFO in memory space
+   // Fields:
+   // - [29:0]  : Base address
+   setting_reg #(.my_addr(SR_BASE + 2), .awidth(8), .width(AWIDTH), .at_reset(DEFAULT_BASE)) sr_fifo_base_addr
+     (.clk(bus_clk), .rst(bus_reset),
+      .strobe(set_stb), .addr(set_addr), .in(set_data),
+      .out(fifo_base_addr_bclk), .changed());
+
+   // SETTING: Address Mask for FIFO in memory space. The mask is ANDed with the base address to define
+   //          a unique address for this FIFO. A zero in the mask signifies that the DRAM FIFO can
+   //          utilize the address bit internally for maintaining FIFO data
+   // Fields:
+   // - [29:0]  : Address mask
+   setting_reg #(.my_addr(SR_BASE + 3), .awidth(8), .width(AWIDTH), .at_reset(DEFAULT_MASK)) sr_fifo_addr_mask
+     (.clk(bus_clk), .rst(bus_reset),
+      .strobe(set_stb), .addr(set_addr), .in(set_data),
+      .out(fifo_addr_mask_bclk), .changed());
+
+   always @(*) begin
+      case(rb_addr)
+         RB_FIFO_STATUS:      rb_data = rb_fifo_status;
+         RB_BIST_STATUS:      rb_data = {(EXT_BIST?1'b1:1'b0), 27'h0, rb_bist_status};
+         RB_BIST_XFER_CNT:    rb_data = rb_bist_bw_ratio[79:48];
+         RB_BIST_CYC_CNT:     rb_data = rb_bist_bw_ratio[31:0];
+         RB_BUS_CLK_RATE:     rb_data = BUS_CLK_RATE;
+         RB_OUT_PKT_CNT:      rb_data = out_pkt_count;
+         default:             rb_data = 32'h0;
+      endcase
+   end
+
+   //
+   // Synchronize settings register values to dram_clk
+   //
+   wire clear;
+   synchronizer #(.INITIAL_VAL(1'b1)) clear_sync_inst (.clk(dram_clk), .rst(1'b0), .in(clear_bclk), .out(clear));
+
+   wire               set_suppress_en;
+   wire [15:0]        set_supress_threshold;
+   wire [11:0]        set_timeout;
+   wire [AWIDTH-1:0]  set_fifo_base_addr, set_fifo_addr_mask, set_fifo_addr_mask_bar;
+
+   wire [(72-AWIDTH-29-1):0]  set_sync_discard0;
+   wire [(72-(2*AWIDTH)-1):0] set_sync_discard1;
+   fifo_short_2clk set_sync_fifo0(
+      .rst(bus_reset),
+      .wr_clk(bus_clk), .din({{(72-AWIDTH-29){1'b0}}, timeout_bclk, supress_enable_bclk, supress_threshold_bclk, fifo_base_addr_bclk}),
+      .wr_en(1'b1), .full(), .wr_data_count(),
+      .rd_clk(dram_clk), .dout({set_sync_discard0, set_timeout, set_suppress_en, set_supress_threshold, set_fifo_base_addr}),
+      .rd_en(1'b1), .empty(), .rd_data_count()
+   );
+   fifo_short_2clk set_sync_fifo1(
+      .rst(bus_reset),
+      .wr_clk(bus_clk), .din({{(72-(2*AWIDTH)){1'b0}}, ~fifo_addr_mask_bclk, fifo_addr_mask_bclk}),
+      .wr_en(1'b1), .full(), .wr_data_count(),
+      .rd_clk(dram_clk), .dout({set_sync_discard1, set_fifo_addr_mask_bar, set_fifo_addr_mask}),
+      .rd_en(1'b1), .empty(), .rd_data_count()
+   );
+
+   //
+   // Input side declarations
+   //
+   localparam [2:0] INPUT_IDLE = 0;
+   localparam [2:0] INPUT1 = 1;
+   localparam [2:0] INPUT2 = 2;
+   localparam [2:0] INPUT3 = 3;   
+   localparam [2:0] INPUT4 = 4;
+   localparam [2:0] INPUT5 = 5;
+   localparam [2:0] INPUT6 = 6;
+
+   reg [2:0]   input_state;
+   reg         input_timeout_triggered;
+   reg         input_timeout_reset;
+   reg [8:0]   input_timeout_count;
+   reg [AWIDTH-1:0]  write_addr;
+   reg         write_ctrl_valid;
+   wire        write_ctrl_ready;
+   reg [7:0]   write_count = 8'd0;
+   reg [8:0]   write_count_plus_one = 9'd1;  // Maintain a +1 version to break critical timing paths
+   reg         update_write;
+
+   //
+   // Output side declarations
+   //
+   localparam [2:0] OUTPUT_IDLE = 0;
+   localparam [2:0] OUTPUT1 = 1;
+   localparam [2:0] OUTPUT2 = 2;
+   localparam [2:0] OUTPUT3 = 3;   
+   localparam [2:0] OUTPUT4 = 4;
+   localparam [2:0] OUTPUT5 = 5;
+   localparam [2:0] OUTPUT6 = 6;
+
+   reg [2:0]   output_state;
+   reg         output_timeout_triggered;
+   reg         output_timeout_reset;
+   reg [8:0]   output_timeout_count;
+   reg [AWIDTH-1:0]  read_addr;
+   reg         read_ctrl_valid;
+   wire        read_ctrl_ready;
+   reg [7:0]   read_count = 8'd0; 
+   reg [8:0]   read_count_plus_one = 9'd1;  // Maintain a +1 version to break critical timing paths
+   reg         update_read;
+   
+   // Track main FIFO active size.
+   reg [AWIDTH-3:0] space, occupied, occupied_minus_one;  // Maintain a -1 version to break critical timing paths
+   reg [AWIDTH-3:0] input_page_boundry, output_page_boundry;  // Cache in a register to break critical timing paths
+
+   // Assign FIFO status bits
+   wire [71:0] status_out_bclk;
+   fifo_short_2clk status_fifo_2clk(
+      .rst(dram_reset),
+      .wr_clk(dram_clk), .din({{(72-(AWIDTH-2)){1'b0}}, occupied}),
+      .wr_en(1'b1), .full(), .wr_data_count(),
+      .rd_clk(bus_clk), .dout(status_out_bclk),
+      .rd_en(1'b1), .empty(), .rd_data_count()
+   );
+   assign rb_fifo_status[31]    = 1'b1;   //DRAM FIFO signature (validates existence of DRAM FIFO)
+   assign rb_fifo_status[30:27] = {o_tvalid, o_tready, i_tvalid, i_tready};   //Ready valid flags
+   assign rb_fifo_status[26:0]  = status_out_bclk[26:0];   //FIFO fullness count in 64bit words (max 27 bits = 1GiB)
+
+   ///////////////////////////////////////////////////////////////////////////////
+   // Inline BIST for production testing
+   //
+   wire       i_tready_int;
+
+   wire [DWIDTH-1:0] i_tdata_fifo;
+   wire       i_tvalid_fifo, i_tready_fifo, i_tlast_fifo;
+
+   wire [DWIDTH-1:0] i_tdata_bist;
+   wire       i_tvalid_bist, i_tready_bist, i_tlast_bist;
+
+   wire [DWIDTH-1:0] o_tdata_int;
+   wire       o_tvalid_int, o_tready_int, o_tlast_int;
+
+   wire [DWIDTH-1:0] o_tdata_fifo;
+   wire       o_tvalid_fifo, o_tready_fifo, o_tlast_fifo;
+
+   wire [DWIDTH-1:0] o_tdata_bist;
+   wire       o_tvalid_bist, o_tready_bist, o_tlast_bist;
+
+   wire [DWIDTH-1:0] o_tdata_gate;
+   wire       o_tvalid_gate, o_tready_gate, o_tlast_gate;
+
+   axi_mux4 #(.PRIO(1), .WIDTH(DWIDTH), .BUFFER(1)) axi_mux (
+      .clk(bus_clk), .reset(bus_reset), .clear(clear_bclk),
+      .i0_tdata(i_tdata), .i0_tlast(i_tlast), .i0_tvalid(i_tvalid), .i0_tready(i_tready_int),
+      .i1_tdata(i_tdata_bist), .i1_tlast(i_tlast_bist), .i1_tvalid(i_tvalid_bist), .i1_tready(i_tready_bist),
+      .i2_tdata({DWIDTH{1'b0}}), .i2_tlast(1'b0), .i2_tvalid(1'b0), .i2_tready(),
+      .i3_tdata({DWIDTH{1'b0}}), .i3_tlast(1'b0), .i3_tvalid(1'b0), .i3_tready(),
+      .o_tdata(i_tdata_fifo), .o_tlast(i_tlast_fifo), .o_tvalid(i_tvalid_fifo), .o_tready(i_tready_fifo)
+   );
+   assign i_tready = i_tready_int & (~clear_bclk);
+
+   wire       bist_running, bist_done;
+   wire [1:0] bist_error;
+   
+   axi_chdr_test_pattern #(
+     .DELAY_MODE(EXT_BIST ? "DYNAMIC" : "STATIC"), 
+     .SID_MODE(EXT_BIST ? "DYNAMIC" : "STATIC"), 
+     .BW_COUNTER(EXT_BIST ? 1 : 0),
+     .SR_BASE(SR_BASE + 4)
+   ) axi_chdr_test_pattern_i (
+      .clk(bus_clk), .reset(bus_reset | clear_bclk),
+      .i_tdata(i_tdata_bist), .i_tlast(i_tlast_bist), .i_tvalid(i_tvalid_bist), .i_tready(i_tready_bist),
+      .o_tdata(o_tdata_bist), .o_tlast(o_tlast_bist), .o_tvalid(o_tvalid_bist), .o_tready(o_tready_bist),
+      .set_stb(set_stb), .set_addr(set_addr), .set_data(set_data),
+      .running(bist_running), .done(bist_done), .error(bist_error), .status_vtr(), .bw_ratio(rb_bist_bw_ratio)
+   );
+   assign rb_bist_status = {bist_error, bist_done, bist_running};
+
+   axi_demux4 #(.ACTIVE_CHAN(4'b0011), .WIDTH(DWIDTH)) axi_demux(
+      .clk(bus_clk), .reset(bus_reset), .clear(clear_bclk),
+      .header(), .dest({1'b0, bist_running}),
+      .i_tdata(o_tdata_fifo), .i_tlast(o_tlast_fifo), .i_tvalid(o_tvalid_fifo), .i_tready(o_tready_fifo),
+      .o0_tdata(o_tdata_gate), .o0_tlast(o_tlast_gate), .o0_tvalid(o_tvalid_gate), .o0_tready(o_tready_gate),
+      .o1_tdata(o_tdata_bist), .o1_tlast(o_tlast_bist), .o1_tvalid(o_tvalid_bist), .o1_tready(o_tready_bist),
+      .o2_tdata(), .o2_tlast(), .o2_tvalid(), .o2_tready(1'b0),
+      .o3_tdata(), .o3_tlast(), .o3_tvalid(), .o3_tready(1'b0)
+   );
+
+   //Insert package gate before output to absorb any intra-packet bubble cycles
+   axi_packet_gate #(.WIDTH(DWIDTH), .SIZE(MAX_PKT_LEN)) out_pkt_gate (
+      .clk(bus_clk), .reset(bus_reset), .clear(clear_bclk),
+      .i_tdata(o_tdata_gate), .i_tlast(o_tlast_gate), .i_tvalid(o_tvalid_gate), .i_tready(o_tready_gate),
+      .i_terror(1'b0),
+      .o_tdata(o_tdata_int), .o_tlast(o_tlast_int), .o_tvalid(o_tvalid_int), .o_tready(o_tready_int)
+   );
+
+   axis_packet_flush #(
+      .WIDTH(DWIDTH), .FLUSH_PARTIAL_PKTS(0), .TIMEOUT_W(1), .PIPELINE("NONE")
+   ) flusher_i (
+      .clk(bus_clk), .reset(bus_reset),
+      .enable(clear_bclk | flush_bclk), .timeout(1'b0), .flushing(), .done(),
+      .s_axis_tdata(o_tdata_int), .s_axis_tlast(o_tlast_int),
+      .s_axis_tvalid(o_tvalid_int), .s_axis_tready(o_tready_int),
+      .m_axis_tdata(o_tdata), .m_axis_tlast(o_tlast),
+      .m_axis_tvalid(o_tvalid), .m_axis_tready(o_tready)
+   );
+
+   always @(posedge bus_clk) begin
+      if (bus_reset) begin
+        out_pkt_count <= 32'd0;
+      end else if (o_tlast_int & o_tvalid_int & o_tready_int) begin
+        out_pkt_count <= out_pkt_count + 32'd1;
+      end
+   end
+
+   //
+   // Buffer input in FIFO's. Embeded tlast signal using ESCape code.
+   //
+
+   wire [DWIDTH-1:0] i_tdata_i0;
+   wire             i_tvalid_i0, i_tready_i0, i_tlast_i0;
+ 
+   wire [DWIDTH-1:0] i_tdata_i1;
+   wire             i_tvalid_i1, i_tready_i1, i_tlast_i1;
+
+   wire [DWIDTH-1:0] i_tdata_i2;
+   wire             i_tvalid_i2, i_tready_i2;
+
+   wire [DWIDTH-1:0] i_tdata_i3;
+   wire             i_tvalid_i3, i_tready_i3;
+
+   wire [DWIDTH-1:0] i_tdata_input;
+   wire             i_tvalid_input, i_tready_input;
+   wire [15:0]      space_input, occupied_input;
+   reg [15:0]       space_input_reg;
+   reg              supress_reads;
+
+
+   ///////////////////////////////////////////////////////////////////////////////
+   
+   wire         write_in, read_in, empty_in, full_in;
+   assign       i_tready_fifo = ~full_in;
+   assign       write_in = i_tvalid_fifo & i_tready_fifo;
+   assign       i_tvalid_i0 = ~empty_in;
+   assign       read_in = i_tvalid_i0 & i_tready_i0;
+   wire [6:0]   discard_i0;
+   
+   fifo_short_2clk fifo_short_2clk_i0 (
+      .rst(bus_reset),
+      .wr_clk(bus_clk),
+      .din({7'h0,i_tlast_fifo,i_tdata_fifo}), // input [71 : 0] din
+      .wr_en(write_in), // input wr_en
+      .full(full_in), // output full
+      .wr_data_count(), // output [9 : 0] wr_data_count
+
+      .rd_clk(dram_clk), // input rd_clk
+      .dout({discard_i0,i_tlast_i0,i_tdata_i0}), // output [71 : 0] dout
+      .rd_en(read_in), // input rd_en
+      .empty(empty_in), // output empty
+      .rd_data_count()  // output [9 : 0] rd_data_count
+   );
+
+   axi_fifo_flop2 #(.WIDTH(DWIDTH+1)) input_pipe_i0
+     (
+      .clk(dram_clk), 
+      .reset(dram_reset), 
+      .clear(clear),
+      //
+      .i_tdata({i_tlast_i0, i_tdata_i0}), 
+      .i_tvalid(i_tvalid_i0), 
+      .i_tready(i_tready_i0),
+      //
+      .o_tdata({i_tlast_i1, i_tdata_i1}), 
+      .o_tvalid(i_tvalid_i1), 
+      .o_tready(i_tready_i1)
+   );
+
+   axi_embed_tlast #(.WIDTH(DWIDTH), .ADD_CHECKSUM(0)) axi_embed_tlast_i (
+      .clk(dram_clk),
+      .reset(dram_reset),
+      .clear(clear),
+      //
+      .i_tdata(i_tdata_i1),
+      .i_tlast(i_tlast_i1),
+      .i_tvalid(i_tvalid_i1),
+      .i_tready(i_tready_i1),
+      //
+      .o_tdata(i_tdata_i2),
+      .o_tvalid(i_tvalid_i2),
+      .o_tready(i_tready_i2)
+   );
+
+   axi_fifo_flop2 #(.WIDTH(DWIDTH)) input_pipe_i1 (
+      .clk(dram_clk), 
+      .reset(dram_reset), 
+      .clear(clear),
+      //
+      .i_tdata(i_tdata_i2), 
+      .i_tvalid(i_tvalid_i2), 
+      .i_tready(i_tready_i2),
+      //
+      .o_tdata(i_tdata_i3), 
+      .o_tvalid(i_tvalid_i3), 
+      .o_tready(i_tready_i3)
+   );
+
+   axi_fifo #(.WIDTH(DWIDTH),.SIZE(10)) fifo_i1 (
+      .clk(dram_clk), 
+      .reset(dram_reset), 
+      .clear(clear),
+      //
+      .i_tdata(i_tdata_i3), 
+      .i_tvalid(i_tvalid_i3), 
+      .i_tready(i_tready_i3),
+      //
+      .o_tdata(i_tdata_input), 
+      .o_tvalid(i_tvalid_input), 
+      .o_tready(i_tready_input),
+      //
+      .space(space_input), 
+      .occupied(occupied_input)
+   );
+
+   //
+   // Monitor occupied_input to deduce when DRAM FIFO is running short of bandwidth and there is a danger of backpressure
+   // passing upstream of the DRAM FIFO.
+   // In this situation supress read requests to the DRAM FIFO so that more bandwidth is available to writes.
+   //
+   always @(posedge dram_clk) 
+      begin
+         space_input_reg <= space_input;
+         if ((space_input_reg < set_supress_threshold[15:0])  && set_suppress_en)
+            supress_reads <= 1'b1;
+         else 
+            supress_reads <= 1'b0;
+      end
+
+   //
+   // Buffer output in 32entry FIFO's. Extract embeded tlast signal.
+   //
+   wire [DWIDTH-1:0] o_tdata_output;
+   wire             o_tvalid_output, o_tready_output;
+   wire [15:0]      space_output, occupied_output;
+
+   wire [DWIDTH-1:0] o_tdata_i0;
+   wire             o_tvalid_i0, o_tready_i0;
+   
+   wire [DWIDTH-1:0] o_tdata_i1;
+   wire             o_tvalid_i1, o_tready_i1;
+   
+   wire [DWIDTH-1:0] o_tdata_i2;
+   wire             o_tvalid_i2, o_tready_i2;
+   
+   wire [DWIDTH-1:0] o_tdata_i3;
+   wire             o_tvalid_i3, o_tready_i3;
+   
+   wire [DWIDTH-1:0] o_tdata_i4;
+   wire             o_tvalid_i4, o_tready_i4, o_tlast_i4;
+
+   wire [DWIDTH-1:0] o_tdata_i5;
+   wire             o_tvalid_i5, o_tready_i5, o_tlast_i5;
+
+   wire             checksum_error;
+
+   axi_fifo #(.WIDTH(DWIDTH),.SIZE(10)) fifo_i2 (
+      .clk(dram_clk), 
+      .reset(dram_reset), 
+      .clear(clear),
+      //
+      .i_tdata(o_tdata_output), 
+      .i_tvalid(o_tvalid_output), 
+      .i_tready(o_tready_output),
+      //
+      .o_tdata(o_tdata_i0), 
+      .o_tvalid(o_tvalid_i0), 
+      .o_tready(o_tready_i0),
+      //
+      .space(space_output), 
+      .occupied(occupied_output)
+   );
+
+   // Place FLops straight after SRAM read access for timing.
+   axi_fifo_flop2 #(.WIDTH(DWIDTH)) output_pipe_i0
+     (
+      .clk(dram_clk), 
+      .reset(dram_reset), 
+      .clear(clear),
+      //
+      .i_tdata(o_tdata_i0), 
+      .i_tvalid(o_tvalid_i0), 
+      .i_tready(o_tready_i0),
+      //
+      .o_tdata(o_tdata_i1), 
+      .o_tvalid(o_tvalid_i1), 
+      .o_tready(o_tready_i1 && ~supress_reads)
+   );
+
+   // Read suppression logic
+   // The CL part of this exists between these
+   // axi_flops 
+   axi_fifo_flop2 #(.WIDTH(DWIDTH)) output_pipe_i1
+     (
+      .clk(dram_clk), 
+      .reset(dram_reset), 
+      .clear(clear),
+      //
+      .i_tdata(o_tdata_i1), 
+      .i_tvalid(o_tvalid_i1 && ~supress_reads), 
+      .i_tready(o_tready_i1),
+      //
+      .o_tdata(o_tdata_i2), 
+      .o_tvalid(o_tvalid_i2), 
+      .o_tready(o_tready_i2)
+   );
+
+   // Pipeline flop before tlast extraction logic
+   axi_fifo_flop2 #(.WIDTH(DWIDTH)) output_pipe_i2
+     (
+      .clk(dram_clk), 
+      .reset(dram_reset), 
+      .clear(clear),
+      //
+      .i_tdata(o_tdata_i2), 
+      .i_tvalid(o_tvalid_i2), 
+      .i_tready(o_tready_i2),
+      //
+      .o_tdata(o_tdata_i3), 
+      .o_tvalid(o_tvalid_i3), 
+      .o_tready(o_tready_i3)
+   );
+
+    axi_extract_tlast #(.WIDTH(DWIDTH), .VALIDATE_CHECKSUM(0)) axi_extract_tlast_i (
+      .clk(dram_clk),
+      .reset(dram_reset),
+      .clear(clear),
+      //
+      .i_tdata(o_tdata_i3),
+      .i_tvalid(o_tvalid_i3),
+      .i_tready(o_tready_i3),
+      //
+      .o_tdata(o_tdata_i4),
+      .o_tlast(o_tlast_i4),
+      .o_tvalid(o_tvalid_i4),
+      .o_tready(o_tready_i4),
+      //
+      .checksum_error()
+   );
+
+   // Pipeline flop after tlast extraction logic
+   axi_fifo_flop2 #(.WIDTH(DWIDTH+1)) output_pipe_i3
+     (
+      .clk(dram_clk), 
+      .reset(dram_reset), 
+      .clear(clear),
+      //
+      .i_tdata({o_tlast_i4,o_tdata_i4}), 
+      .i_tvalid(o_tvalid_i4), 
+      .i_tready(o_tready_i4),
+      //
+      .o_tdata({o_tlast_i5,o_tdata_i5}), 
+      .o_tvalid(o_tvalid_i5), 
+      .o_tready(o_tready_i5)
+   );
+
+   wire         write_out, read_out, empty_out, full_out;
+   assign       o_tready_i5 = ~full_out;
+   assign       write_out = o_tvalid_i5 & o_tready_i5;
+   assign       o_tvalid_fifo = ~empty_out;
+   assign       read_out = o_tvalid_fifo & o_tready_fifo;
+   wire [6:0]   discard_i1;
+   
+   fifo_short_2clk fifo_short_2clk_i1 (
+      .rst(dram_reset),
+      .wr_clk(dram_clk),
+      .din({7'h0,o_tlast_i5,o_tdata_i5}), // input [71 : 0] din
+      .wr_en(write_out), // input wr_en
+      .full(full_out), // output full
+      .wr_data_count(), // output [9 : 0] wr_data_count
+
+      .rd_clk(bus_clk), // input rd_clk
+      .dout({discard_i1,o_tlast_fifo,o_tdata_fifo}), // output [71 : 0] dout
+      .rd_en(read_out), // input rd_en
+      .empty(empty_out), // output empty
+      .rd_data_count()  // output [9 : 0] rd_data_count
+   );
+
+   //
+   // Simple input timeout counter for now.
+   // Timeout count only increments when there is some data waiting to be written.
+   //
+   always @(posedge dram_clk)
+      if (dram_reset | clear) begin
+         input_timeout_count <= 9'd0;
+         input_timeout_triggered <= 1'b0;
+      end else if (input_timeout_reset) begin
+         input_timeout_count <= 9'd0;
+         input_timeout_triggered <= 1'b0;
+     end else if (input_timeout_count == set_timeout[8:0]) begin
+         input_timeout_triggered <= 1'b1;
+     end else if (input_state == INPUT_IDLE) begin
+         input_timeout_count <= input_timeout_count + ((occupied_input != 16'd0) ? 9'd1 : 9'd0);
+     end
+
+   //
+   // Wait for 16 entries in input FIFO to trigger DRAM write burst.
+   // Timeout can also trigger burst so fragments of data are not left to rot in the input FIFO.
+   // Also if enough data is present in the input FIFO to complete a burst upto the edge
+   // of a 4KByte page then immediately start the burst.
+   //
+   always @(posedge dram_clk)
+      if (dram_reset | clear) begin
+         input_state <= INPUT_IDLE;
+         write_addr <= set_fifo_base_addr & set_fifo_addr_mask;
+         input_timeout_reset <= 1'b0;
+         write_ctrl_valid <= 1'b0;
+         write_count <= 8'd0;
+         write_count_plus_one <= 9'd1;
+         update_write <= 1'b0;
+      end else
+         case (input_state)
+         //
+         // INPUT_IDLE.
+         // To start an input transfer to DRAM need:
+         // 1) Space in the DRAM FIFO 
+         // and either
+         // 2) 256 entrys in the input FIFO
+         // or
+         // 3) Timeout waiting for more data.
+         //
+         INPUT_IDLE: begin
+            write_ctrl_valid <= 1'b0;
+            update_write <= 1'b0;
+            if (space[AWIDTH-3:8] != 'd0) begin // (space > 255): Space in the DRAM FIFO
+               if (occupied_input[15:8] != 'd0) begin  // (occupied_input > 255): 256 or more entries in input FIFO
+                  input_state <= INPUT1;
+                  input_timeout_reset <= 1'b1;
+                  // Calculate number of entries remaining until next 4KB page boundry is crossed minus 1.
+                  // Note, units of calculation are 64bit wide words. Address is always 64bit alligned.
+                  input_page_boundry <= {write_addr[AWIDTH-1:12],9'h1ff} - write_addr[AWIDTH-1:3];   
+               end else if (input_timeout_triggered) begin // input FIFO timeout waiting for new data.
+                  input_state <= INPUT2;
+                  input_timeout_reset <= 1'b1;
+                  // Calculate number of entries remaining until next 4KB page boundry is crossed minus 1.
+                  // Note, units of calculation are 64bit wide words. Address is always 64bit alligned.
+                  input_page_boundry <= {write_addr[AWIDTH-1:12],9'h1ff} - write_addr[AWIDTH-1:3];   
+               end else begin
+                  input_timeout_reset <= 1'b0;
+                  input_state <= INPUT_IDLE;
+               end
+            end else begin
+               input_timeout_reset <= 1'b0;
+               input_state <= INPUT_IDLE;
+            end
+         end
+         //
+         // INPUT1.
+         // Caused by input FIFO reaching 256 entries.
+         // Request write burst of lesser of:
+         // 1) Entrys until page boundry crossed
+         // 2) 256.
+         //
+         INPUT1: begin
+            // Replicated write logic to break a read timing critical path for write_count
+            write_count <= (input_page_boundry[11:8] == 4'd0) ? input_page_boundry[7:0] : 8'd255;
+            write_count_plus_one <= (input_page_boundry[11:8] == 4'd0) ? ({1'b0,input_page_boundry[7:0]} + 9'd1) : 9'd256;
+            write_ctrl_valid <= 1'b1;
+            if (write_ctrl_ready)
+               input_state <= INPUT4; // Pre-emptive ACK
+            else
+               input_state <= INPUT3; // Wait for ACK
+         end
+         //
+         // INPUT2.
+         // Caused by timeout of input FIFO. (occupied_input was implicitly less than 256 last cycle)
+         // Request write burst of lesser of:
+         // 1) Entries until page boundry crossed
+         // 2) Entries in input FIFO
+         //
+         INPUT2: begin
+            // Replicated write logic to break a read timing critical path for write_count
+            write_count <= (input_page_boundry < ({3'h0,occupied_input[8:0]} - 12'd1)) ? input_page_boundry[7:0] : (occupied_input[8:0] - 9'd1);
+            write_count_plus_one <= (input_page_boundry < ({3'h0,occupied_input[8:0]} - 12'd1)) ? ({1'b0,input_page_boundry[7:0]} + 9'd1) : occupied_input[8:0];
+            write_ctrl_valid <= 1'b1;
+            if (write_ctrl_ready)
+               input_state <= INPUT4; // Pre-emptive ACK
+            else
+               input_state <= INPUT3; // Wait for ACK
+         end
+         //
+         // INPUT3.
+         // Wait in this state for AXI4_DMA engine to accept transaction.
+         //
+         INPUT3: begin
+            if (write_ctrl_ready) begin
+               write_ctrl_valid <= 1'b0;
+               input_state <= INPUT4; // ACK
+            end else begin
+               write_ctrl_valid <= 1'b1;
+               input_state <= INPUT3; // Wait for ACK
+            end
+         end
+         //
+         // INPUT4.
+         // Wait here until write_ctrl_ready_deasserts.
+         // This is important as the next time it asserts we know that a write response was receieved.
+         INPUT4: begin
+            write_ctrl_valid <= 1'b0;
+            if (!write_ctrl_ready)
+               input_state <= INPUT5; // Move on
+            else
+               input_state <= INPUT4; // Wait for deassert
+         end   
+         //
+         // INPUT5.
+         // Transaction has been accepted by AXI4 DMA engine. Now we wait for the re-assertion
+         // of write_ctrl_ready which signals that the AXI4 DMA engine has receieved a response
+         // for the whole write transaction and we assume that this means it is commited to DRAM.
+         // We are now free to update write_addr pointer and go back to idle state.
+         // 
+         INPUT5: begin
+            write_ctrl_valid <= 1'b0;
+            if (write_ctrl_ready) begin
+               write_addr <= ((write_addr + (write_count_plus_one << 3)) & set_fifo_addr_mask_bar) | (write_addr & set_fifo_addr_mask);
+               input_state <= INPUT6;
+               update_write <= 1'b1;
+            end else begin
+               input_state <= INPUT5;
+            end
+         end
+         //
+         // INPUT6:
+         // Need to let space update before looking if there's more to do.
+         //
+         INPUT6: begin
+            input_state <= INPUT_IDLE;
+            update_write <= 1'b0;
+         end
+
+         default: 
+            input_state <= INPUT_IDLE;
+      endcase // case(input_state)
+
+
+   //
+   // Simple output timeout counter for now
+   //
+   always @(posedge dram_clk)
+      if (dram_reset | clear) begin
+         output_timeout_count <= 9'd0;
+         output_timeout_triggered <= 1'b0;
+      end else if (output_timeout_reset) begin
+         output_timeout_count <= 9'd0;
+         output_timeout_triggered <= 1'b0;
+      end else if (output_timeout_count == set_timeout[8:0]) begin
+         output_timeout_triggered <= 1'b1;
+      end else if (output_state == OUTPUT_IDLE) begin
+         output_timeout_count <= output_timeout_count + ((occupied != 'd0) ? 9'd1 : 9'd0);
+     end
+
+
+   //
+   // Wait for 64 entries in main FIFO to trigger DRAM read burst.
+   // Timeout can also trigger burst so fragments of data are not left to rot in the main FIFO.
+   // Also if enough data is present in the main FIFO to complete a burst upto the edge
+   // of a 4KByte page then immediately start the burst.
+   //
+   always @(posedge dram_clk)
+      if (dram_reset | clear) begin
+         output_state <= OUTPUT_IDLE;
+         read_addr <= set_fifo_base_addr & set_fifo_addr_mask;
+         output_timeout_reset <= 1'b0;
+         read_ctrl_valid <= 1'b0;
+         read_count <= 8'd0;
+         read_count_plus_one <= 9'd1;
+         update_read <= 1'b0;
+      end else
+         case (output_state)
+         //
+         // OUTPUT_IDLE.
+         // To start an output tranfer from DRAM
+         // 1) Space in the small output FIFO 
+         // and either
+         // 2) 256 entrys in the DRAM FIFO
+         // or
+         // 3) Timeout waiting for more data.
+         //
+         OUTPUT_IDLE: begin
+            read_ctrl_valid <= 1'b0;
+            update_read <= 1'b0;
+            if (space_output[15:8] != 'd0) begin // (space_output > 255): Space in the output FIFO.
+               if (occupied[AWIDTH-3:8] != 'd0) begin // (occupied > 255): 64 or more entrys in main FIFO
+                  output_state <= OUTPUT1;
+                  output_timeout_reset <= 1'b1;
+                  // Calculate number of entries remaining until next 4KB page boundry is crossed minus 1.
+                  // Note, units of calculation are 64bit wide words. Address is always 64bit alligned.
+                  output_page_boundry <= {read_addr[AWIDTH-1:12],9'h1ff} - read_addr[AWIDTH-1:3];
+               end else if (output_timeout_triggered) begin // output FIFO timeout waiting for new data.
+                  output_state <= OUTPUT2;
+                  output_timeout_reset <= 1'b1;
+                  // Calculate number of entries remaining until next 4KB page boundry is crossed minus 1.
+                  // Note, units of calculation are 64bit wide words. Address is always 64bit alligned.
+                  output_page_boundry <= {read_addr[AWIDTH-1:12],9'h1ff} - read_addr[AWIDTH-1:3];
+               end else begin
+                  output_timeout_reset <= 1'b0;
+                  output_state <= OUTPUT_IDLE;
+               end
+            end else begin
+               output_timeout_reset <= 1'b0;
+               output_state <= OUTPUT_IDLE;
+            end
+         end // case: OUTPUT_IDLE
+         //
+         // OUTPUT1.
+         // Caused by main FIFO reaching 256 entries.
+         // Request read burst of lesser of lesser of:
+         // 1) Entrys until page boundry crossed
+         // 2) 256.
+         //
+         OUTPUT1: begin
+            // Replicated write logic to break a read timing critical path for read_count
+            read_count <= (output_page_boundry[11:8] == 4'd0) ? output_page_boundry[7:0] : 8'd255;
+            read_count_plus_one <= (output_page_boundry[11:8] == 4'd0) ? ({1'b0,output_page_boundry[7:0]} + 9'd1) : 9'd256;
+            read_ctrl_valid <= 1'b1;
+            if (read_ctrl_ready)
+               output_state <= OUTPUT4; // Pre-emptive ACK
+            else
+               output_state <= OUTPUT3; // Wait for ACK
+         end
+         //
+         // OUTPUT2.
+         // Caused by timeout of main FIFO
+         // Request read burst of lesser of:
+         // 1) Entries until page boundry crossed
+         // 2) Entries in main FIFO
+         //
+         OUTPUT2: begin
+            // Replicated write logic to break a read timing critical path for read_count
+            read_count <= (output_page_boundry < occupied_minus_one) ? output_page_boundry[7:0] : occupied_minus_one[7:0];
+            read_count_plus_one <= (output_page_boundry < occupied_minus_one) ? ({1'b0,output_page_boundry[7:0]} + 9'd1) : {1'b0, occupied[7:0]};
+            read_ctrl_valid <= 1'b1;
+            if (read_ctrl_ready)
+               output_state <= OUTPUT4; // Pre-emptive ACK
+            else
+               output_state <= OUTPUT3; // Wait for ACK
+         end
+         //
+         // OUTPUT3.
+         // Wait in this state for AXI4_DMA engine to accept transaction.
+         //
+         OUTPUT3: begin
+            if (read_ctrl_ready) begin
+               read_ctrl_valid <= 1'b0;
+               output_state <= OUTPUT4; // ACK
+            end else begin
+               read_ctrl_valid <= 1'b1;
+               output_state <= OUTPUT3; // Wait for ACK
+            end
+         end
+         //
+         // OUTPUT4.
+         // Wait here unitl read_ctrl_ready_deasserts.
+         // This is important as the next time it asserts we know that a read response was receieved.
+         OUTPUT4: begin
+            read_ctrl_valid <= 1'b0;
+            if (!read_ctrl_ready)
+               output_state <= OUTPUT5; // Move on
+            else
+               output_state <= OUTPUT4; // Wait for deassert
+         end   
+         //
+         // OUTPUT5.
+         // Transaction has been accepted by AXI4 DMA engine. Now we wait for the re-assertion
+         // of read_ctrl_ready which signals that the AXI4 DMA engine has receieved a last signal and good response
+         // for the whole read transaction.
+         // We are now free to update read_addr pointer and go back to idle state.
+         // 
+         OUTPUT5: begin
+            read_ctrl_valid <= 1'b0;
+            if (read_ctrl_ready) begin
+               read_addr <= ((read_addr + (read_count_plus_one << 3)) & set_fifo_addr_mask_bar) | (read_addr & set_fifo_addr_mask);
+               output_state <= OUTPUT6;
+               update_read <= 1'b1;
+            end else begin
+               output_state <= OUTPUT5;
+            end
+         end // case: OUTPUT5
+         //
+         // OUTPUT6.
+         // Need to get occupied value updated before checking if there's more to do.
+         //
+         OUTPUT6: begin
+            update_read <= 1'b0;
+            output_state <= OUTPUT_IDLE;
+         end
+
+         default: 
+            output_state <= OUTPUT_IDLE;
+       endcase // case(output_state)
+
+   //
+   // Count number of used entries in main DRAM FIFO.
+   // Note that this is expressed in units of 64bit wide words.
+   //
+   always @(posedge dram_clk)
+      if (dram_reset | clear) begin
+         occupied <= 'd0;
+         occupied_minus_one <= {(AWIDTH-2){1'b1}};
+      end else begin
+         occupied <= occupied + (update_write ? write_count_plus_one : 9'd0) - (update_read ? read_count_plus_one : 9'd0);
+         occupied_minus_one <= occupied_minus_one + (update_write ? write_count_plus_one : 9'd0) - (update_read ? read_count_plus_one : 9'd0);
+      end
+
+   always @(posedge dram_clk)
+      if (dram_reset | clear)
+         space <= set_fifo_addr_mask_bar[AWIDTH-1:3] & ~('d63); // Subtract 64 from space to make allowance for read/write reordering in DRAM controller
+      else
+         space <= space - (update_write ? write_count_plus_one : 9'd0) + (update_read ? read_count_plus_one : 9'd0);
+
+   //
+   // Instamce of axi_dma_master
+   //
+   axi_dma_master axi_dma_master_i
+   (
+      .aclk(dram_clk), // input aclk
+      .areset(dram_reset | clear), // input aresetn
+      // Write control
+      .m_axi_awid(m_axi_awid), // input [0 : 0] m_axi_awid
+      .m_axi_awaddr(m_axi_awaddr), // input [31 : 0] m_axi_awaddr
+      .m_axi_awlen(m_axi_awlen), // input [7 : 0] m_axi_awlen
+      .m_axi_awsize(m_axi_awsize), // input [2 : 0] m_axi_awsize
+      .m_axi_awburst(m_axi_awburst), // input [1 : 0] m_axi_awburst
+      .m_axi_awvalid(m_axi_awvalid), // input m_axi_awvalid
+      .m_axi_awready(m_axi_awready), // output m_axi_awready
+      .m_axi_awlock(m_axi_awlock),
+      .m_axi_awcache(m_axi_awcache),
+      .m_axi_awprot(m_axi_awprot),
+      .m_axi_awqos(m_axi_awqos),
+      .m_axi_awregion(m_axi_awregion),
+      .m_axi_awuser(m_axi_awuser),
+      // Write Data
+      .m_axi_wdata(m_axi_wdata), // input [63 : 0] m_axi_wdata
+      .m_axi_wstrb(m_axi_wstrb), // input [7 : 0] m_axi_wstrb
+      .m_axi_wlast(m_axi_wlast), // input m_axi_wlast
+      .m_axi_wvalid(m_axi_wvalid), // input m_axi_wvalid
+      .m_axi_wready(m_axi_wready), // output m_axi_wready
+      .m_axi_wuser(m_axi_wuser),
+      // Write Response
+      .m_axi_bid(m_axi_bid), // output [0 : 0] m_axi_bid
+      .m_axi_bresp(m_axi_bresp), // output [1 : 0] m_axi_bresp
+      .m_axi_bvalid(m_axi_bvalid), // output m_axi_bvalid
+      .m_axi_bready(m_axi_bready), // input m_axi_bready
+      .m_axi_buser(m_axi_buser),
+      // Read Control
+      .m_axi_arid(m_axi_arid), // input [0 : 0] m_axi_arid
+      .m_axi_araddr(m_axi_araddr), // input [31 : 0] m_axi_araddr
+      .m_axi_arlen(m_axi_arlen), // input [7 : 0] m_axi_arlen
+      .m_axi_arsize(m_axi_arsize), // input [2 : 0] m_axi_arsize
+      .m_axi_arburst(m_axi_arburst), // input [1 : 0] m_axi_arburst
+      .m_axi_arvalid(m_axi_arvalid), // input m_axi_arvalid
+      .m_axi_arready(m_axi_arready), // output m_axi_arready
+      .m_axi_arlock(m_axi_arlock),
+      .m_axi_arcache(m_axi_arcache),
+      .m_axi_arprot(m_axi_arprot),
+      .m_axi_arqos(m_axi_arqos),
+      .m_axi_arregion(m_axi_arregion),
+      .m_axi_aruser(m_axi_aruser),
+      // Read Data
+      .m_axi_rid(m_axi_rid), // output [0 : 0] m_axi_rid
+      .m_axi_rdata(m_axi_rdata), // output [63 : 0] m_axi_rdata
+      .m_axi_rresp(m_axi_rresp), // output [1 : 0] m_axi_rresp
+      .m_axi_rlast(m_axi_rlast), // output m_axi_rlast
+      .m_axi_rvalid(m_axi_rvalid), // output m_axi_rvalid
+      .m_axi_rready(m_axi_rready), // input m_axi_rready
+      .m_axi_ruser(m_axi_ruser),
+      //
+      // DMA interface for Write transaction
+      //
+      .write_addr({{(32-AWIDTH){1'b0}}, write_addr}),       // Byte address for start of write transaction (should be 64bit alligned)
+      .write_count(write_count),       // Count of 64bit words to write.
+      .write_ctrl_valid(write_ctrl_valid),
+      .write_ctrl_ready(write_ctrl_ready),
+      .write_data(i_tdata_input),
+      .write_data_valid(i_tvalid_input),
+      .write_data_ready(i_tready_input),
+      //
+      // DMA interface for Read
+      //
+      .read_addr({{(32-AWIDTH){1'b0}}, read_addr}),       // Byte address for start of read transaction (should be 64bit alligned)
+      .read_count(read_count),       // Count of 64bit words to read.
+      .read_ctrl_valid(read_ctrl_valid),
+      .read_ctrl_ready(read_ctrl_ready),
+      .read_data(o_tdata_output),
+      .read_data_valid(o_tvalid_output),
+      .read_data_ready(o_tready_output),
+      //
+      // Debug
+      //
+      .debug()
+   );
+
+  //ila_axi_dma_fifo inst_ila (
+  //  .clk(ce_clk), // input wire clk
+  //  .probe0(rb_bist_status), // input wire [3:0]  probe0  channel 0
+  //  .probe1(), // input wire [3:0]  probe0  channel 0
+  //);
+
+
+
+
+ endmodule // axi_dma_fifo
+
diff --git a/fpga/usrp3/lib/axi/axi_dma_master.v b/fpga/usrp3/lib/axi/axi_dma_master.v
new file mode 100644
index 000000000..59e2e97a7
--- /dev/null
+++ b/fpga/usrp3/lib/axi/axi_dma_master.v
@@ -0,0 +1,548 @@
+//
+// Copyright 2014 Ettus Research LLC
+// Copyright 2018 Ettus Research, a National Instruments Company
+//
+// SPDX-License-Identifier: LGPL-3.0-or-later
+//
+
+`include "axi_defs.v"
+
+`define DEBUG if (0)
+
+module axi_dma_master #(
+   parameter AWIDTH = 32,
+   parameter DWIDTH = 64
+) (
+   input aclk,                    // Global AXI clock
+   input areset,                 // Global AXI reset
+   //
+   // AXI Write address channel
+   //
+   output [0 : 0] m_axi_awid,     // Write address ID. This signal is the identification tag for the write address signals
+   output reg [AWIDTH-1 : 0] m_axi_awaddr,  // Write address. The write address gives the address of the first transfer in a write burst
+   output reg [7 : 0] m_axi_awlen,    // Burst length. The burst length gives the exact number of transfers in a burst.
+   output [2 : 0] m_axi_awsize,   // Burst size. This signal indicates the size of each transfer in the burst. 
+   output [1 : 0] m_axi_awburst,  // Burst type. The burst type and the size information, determine how the address is calculated
+   output [0 : 0] m_axi_awlock,   // Lock type. Provides additional information about the atomic characteristics of the transfer.
+   output [3 : 0] m_axi_awcache,  // Memory type. This signal indicates how transactions are required to progress
+   output [2 : 0] m_axi_awprot,   // Protection type. This signal indicates the privilege and security level of the transaction
+   output [3 : 0] m_axi_awqos,    // Quality of Service, QoS. The QoS identifier sent for each write transaction
+   output [3 : 0] m_axi_awregion, // Region identifier. Permits a single physical interface on a slave to be re-used.
+   output [0 : 0] m_axi_awuser,   // User signal. Optional User-defined signal in the write address channel.
+   output reg m_axi_awvalid,      // Write address valid. This signal indicates that the channel is signaling valid write addr
+   input m_axi_awready,           // Write address ready. This signal indicates that the slave is ready to accept an address
+   //
+   // AXI Write data channel.
+   //
+   output [DWIDTH-1 : 0] m_axi_wdata,   // Write data
+   output [DWIDTH/8-1 : 0] m_axi_wstrb,    // Write strobes. This signal indicates which byte lanes hold valid data.
+   output reg m_axi_wlast,        // Write last. This signal indicates the last transfer in a write burst
+   output m_axi_wuser,            // User signal. Optional User-defined signal in the write data channel.
+   output m_axi_wvalid,           // Write valid. This signal indicates that valid write data and strobes are available. 
+   input m_axi_wready,            // Write ready. This signal indicates that the slave can accept the write data.
+   //
+   // AXI Write response channel signals
+   //
+   input [0 : 0] m_axi_bid,       // Response ID tag. This signal is the ID tag of the write response. 
+   input [1 : 0] m_axi_bresp,     // Write response. This signal indicates the status of the write transaction.
+   input [0 : 0] m_axi_buser,     // User signal. Optional User-defined signal in the write response channel.
+   input m_axi_bvalid,            // Write response valid. This signal indicates that the channel is signaling a valid response
+   output reg m_axi_bready,       // Response ready. This signal indicates that the master can accept a write response
+   //
+   // AXI Read address channel
+   //
+   output [0 : 0] m_axi_arid,     // Read address ID. This signal is the identification tag for the read address group of signals
+   output reg [AWIDTH-1 : 0] m_axi_araddr,  // Read address. The read address gives the address of the first transfer in a read burst
+   output reg [7 : 0] m_axi_arlen,    // Burst length. This signal indicates the exact number of transfers in a burst.
+   output [2 : 0] m_axi_arsize,   // Burst size. This signal indicates the size of each transfer in the burst.
+   output [1 : 0] m_axi_arburst,  // Burst type. The burst type and the size information determine how the address for each transfer
+   output [0 : 0] m_axi_arlock,   // Lock type. This signal provides additional information about the atomic characteristics
+   output [3 : 0] m_axi_arcache,  // Memory type. This signal indicates how transactions are required to progress 
+   output [2 : 0] m_axi_arprot,   // Protection type. This signal indicates the privilege and security level of the transaction
+   output [3 : 0] m_axi_arqos,    // Quality of Service, QoS. QoS identifier sent for each read transaction.
+   output [3 : 0] m_axi_arregion, // Region identifier. Permits a single physical interface on a slave to be re-used
+   output [0 : 0] m_axi_aruser,   // User signal. Optional User-defined signal in the read address channel.
+   output reg m_axi_arvalid,          // Read address valid. This signal indicates that the channel is signaling valid read addr
+   input m_axi_arready,           // Read address ready. This signal indicates that the slave is ready to accept an address
+   //
+   // AXI Read data channel
+   //
+   input [0 : 0] m_axi_rid,       // Read ID tag. This signal is the identification tag for the read data group of signals
+   input [DWIDTH-1 : 0] m_axi_rdata,    // Read data.
+   input [1 : 0] m_axi_rresp,     // Read response. This signal indicates the status of the read transfer
+   input m_axi_rlast,             // Read last. This signal indicates the last transfer in a read burst.
+   input [0 : 0] m_axi_ruser,     // User signal. Optional User-defined signal in the read data channel.
+   input m_axi_rvalid,            // Read valid. This signal indicates that the channel is signaling the required read data. 
+   output m_axi_rready,           // Read ready. This signal indicates that the master can accept the read data and response
+   //
+   // DMA interface for Write transaction
+   //
+   input [AWIDTH-1:0] write_addr,       // Byte address for start of write transaction (should be 64bit alligned)
+   input [7:0] write_count,       // Count of 64bit words to write. (minus one)
+   input write_ctrl_valid,
+   output reg write_ctrl_ready,
+   input [DWIDTH-1:0] write_data,
+   input write_data_valid,
+   output write_data_ready,
+   //
+   // DMA interface for Read
+   //
+   input [AWIDTH-1:0] read_addr,       // Byte address for start of read transaction (should be 64bit alligned)
+   input [7:0] read_count,       // Count of 64bit words to read.
+   input read_ctrl_valid,
+   output reg read_ctrl_ready,
+   output [DWIDTH-1:0] read_data,
+   output read_data_valid,
+   input read_data_ready,
+   //
+   // Debug Bus
+   //
+   output [31:0] debug
+   
+   );
+ 
+   
+   localparam AW_IDLE = 0;
+   localparam WAIT_AWREADY = 1;
+   localparam WAIT_BVALID = 2;
+   localparam AW_ERROR = 3;
+   
+   reg [1:0] write_addr_state;  
+   reg [7:0] write_data_count;         // Count write transfers.
+   reg 	     enable_data_write;
+   
+   localparam DW_IDLE = 0;
+   localparam DW_RUN = 1;
+   localparam DW_LAST = 2;
+   
+   reg [1:0]  write_data_state;
+   
+   localparam AR_IDLE = 0;
+   localparam WAIT_ARREADY = 1;
+   localparam WAIT_READ_DONE = 2;
+   localparam AR_ERROR = 3;
+   
+   reg [1:0]  read_addr_state;
+
+   localparam DR_IDLE = 0;
+   localparam DR_RUN = 1;
+   localparam DR_WAIT_ERROR = 2;
+   localparam DR_ERROR = 3;
+
+   reg [1:0]  read_data_state;
+   reg [7:0]  read_data_count;
+   reg 	      enable_data_read;
+
+   ///////////////////////////
+   // DEBUG
+   ///////////////////////////
+   assign debug= {24'h0,write_addr_state[1:0],write_data_state[1:0],read_addr_state[1:0],read_data_state[1:0]};
+   
+   
+   //
+   //
+   //
+   
+   
+   
+   
+   /////////////////////////////////////////////////////////////////////////////////
+   //
+   // AXI Write address channel
+   //
+   /////////////////////////////////////////////////////////////////////////////////
+   assign m_axi_awid = 1'b0;
+   assign m_axi_awsize = $clog2(DWIDTH/8);
+   assign m_axi_awburst = `AXI4_BURST_INCR;
+   assign m_axi_awlock = `AXI4_LOCK_NORMAL;
+   assign m_axi_awcache = `AXI4_CACHE_ALLOCATE | `AXI4_CACHE_OTHER_ALLOCATE | `AXI4_CACHE_MODIFIABLE | `AXI4_CACHE_BUFFERABLE;
+   assign m_axi_awprot = `AXI4_PROT_NON_SECURE;
+   assign m_axi_awqos = 4'h0;
+   assign m_axi_awregion = 4'h0;
+   assign m_axi_awuser = 1'b0;
+
+   
+   //
+   // AXI Write address state machine
+   //
+   always @(posedge aclk)
+     if (areset) begin
+	write_ctrl_ready <= 1'b0;
+	write_addr_state <= AW_IDLE;
+	m_axi_awaddr <= {AWIDTH{1'b0}};
+	m_axi_awlen[7:0] <= 8'h0;
+	m_axi_awvalid <= 1'b0;
+	m_axi_bready <= 1'b0;
+     end else
+       case (write_addr_state)
+	 //
+	 // AW_IDLE
+	 // We are ready to accept a new write transaction.
+	 //
+	 AW_IDLE: begin
+	    // Premptively accept new write transaction since we are idle.
+	    write_ctrl_ready <= 1'b1;
+	    // No need to be waiting for a response while idle.
+	    m_axi_bready <= 1'b0;
+	    // If we are offered a new transaction then.....
+	    if (write_ctrl_valid) begin
+	       // Drive all the relevent AXI4 write address channel signals next cycle.
+	       m_axi_awaddr <= write_addr;
+	       m_axi_awlen[7:0] <= {write_count};
+	       m_axi_awvalid <= 1'b1;
+	       // If the AXI4 write channel is pre-emptively accepting the transaction...
+	       if (m_axi_awready == 1'b1) begin
+		  // ...go straight to looking for a transaction response...
+		  `DEBUG $display("WRITE TRANSACTION: ADDR: %x  LEN: %x @ time %d",write_addr,write_count,$time);	 		  
+		  write_addr_state <= WAIT_BVALID;
+		  m_axi_bready <= 1'b1;
+	       end else begin
+                  // ...otherwise wait to get the transaction accepted.
+		  write_addr_state <= WAIT_AWREADY;
+	       end	       
+	    end
+	 end
+	 //
+	 // WAIT_AWREADY
+	 // Waiting for AXI4 slave to accept new write transaction.
+	 //
+	 WAIT_AWREADY: begin
+	    write_ctrl_ready <= 1'b0;
+	    // If the AXI4 write channel is accepting the transaction...
+	    if (m_axi_awready == 1'b1) begin
+	       // ...go to looking for a transaction response...
+	       write_addr_state <= WAIT_BVALID;
+	       m_axi_awvalid <= 1'b0;
+	       m_axi_bready <= 1'b1;
+	       `DEBUG $display("WRITE TRANSACTION: ADDR: %x  LEN: %x @ time %d",m_axi_awaddr,m_axi_awlen[7:0],$time);	       
+	    end else begin
+               // ...otherwise wait to get the trasaction accepted.
+	       write_addr_state <= WAIT_AWREADY;
+	    end
+	 end // case: WAIT_AWREADY
+	 //
+	 // WAIT_BVALID
+	 // Write transaction has been accepted, now waiting for a response to signal it's sucsesful.
+	 // Ignoring ID tag for the moment
+	 //
+	 WAIT_BVALID: begin
+	    write_ctrl_ready <= 1'b0;
+	    m_axi_awvalid <= 1'b0;
+	    // Wait for response channel to signal how write transaction went down....
+	    if (m_axi_bvalid == 1'b1) begin
+	       if ((m_axi_bresp == `AXI4_RESP_OKAY) ||  (m_axi_bresp == `AXI4_RESP_EXOKAY)) begin
+		  // ....it went well, we are ready to start something new.
+		  write_addr_state <= AW_IDLE;
+		  m_axi_bready <= 1'b0;
+		  write_ctrl_ready <= 1'b1; // Ready to run again as soon as we hit idle.
+	       end else if ((m_axi_bresp == `AXI4_RESP_SLVERR) || (m_axi_bresp == `AXI4_RESP_DECERR)) begin
+		  // ....things got ugly, retreat to an error stat and wait for intervention.
+		  write_addr_state <= AW_ERROR;
+		  m_axi_bready <= 1'b0;
+	       end	       
+	    end else begin
+	       write_addr_state <= WAIT_BVALID;
+	       m_axi_bready <= 1'b1;
+	    end
+	 end // case: WAIT_BVALID
+	 //
+	 // AW_ERROR
+	 // Something bad happened, going to need external intervention to restore a safe state.
+	 //
+	 AW_ERROR: begin
+	    write_ctrl_ready <= 1'b0;
+	    write_addr_state <= AW_ERROR;
+	    m_axi_awaddr <= {AWIDTH{1'b0}};
+	    m_axi_awlen[7:0] <= 8'h0;
+	    m_axi_awvalid <= 1'b0;
+	    m_axi_bready <= 1'b0;
+	 end
+       endcase // case(write_addr_state)
+
+   /////////////////////////////////////////////////////////////////////////////////
+   //
+   // AXI Write data channel
+   //
+   /////////////////////////////////////////////////////////////////////////////////
+   assign m_axi_wstrb = {DWIDTH/8{1'b1}};
+   assign m_axi_wuser = 1'b0;
+
+   //
+   // AXI Write data state machine
+   //
+   always @(posedge aclk)
+     if (areset) begin
+	write_data_state <= AW_IDLE;
+	write_data_count <= 1;
+	enable_data_write <= 1'b0;
+	m_axi_wlast <= 1'b0;
+	
+     end else
+       case (write_data_state)
+	 //
+	 // DW_IDLE
+	 // Sit in this state until presented with the control details of a new write transaction.
+	 //
+	 DW_IDLE: begin
+	    write_data_count <= 1;
+	    m_axi_wlast <= 1'b0;
+	    
+	    if (write_ctrl_valid && write_ctrl_ready) begin	       
+	       enable_data_write <= 1'b1;
+	       if (write_count[7:0] == 8'h0) begin
+		  // Single transfer transaction
+		  write_data_state <= DW_LAST;
+		  m_axi_wlast <= 1'b1;
+	       end else begin
+		  write_data_state <= DW_RUN;
+	       end
+	    end else begin
+	       write_data_state <= DW_IDLE;
+	    end
+	 end
+	 //
+	 // DW_RUN
+	 //
+	 DW_RUN : begin
+	    enable_data_write <= 1'b1;
+	    m_axi_wlast <= 1'b0;
+	    
+	    if (write_data_valid && m_axi_wready) begin
+	      // Single write transfer
+	       write_data_count <= write_data_count + 1;
+	    
+	       if (write_data_count == m_axi_awlen[7:0]) begin
+		  write_data_state <= DW_LAST;
+		  m_axi_wlast <= 1'b1;
+	       end else begin
+		  write_data_state <= DW_RUN;
+	       end
+	    end else begin
+	       write_data_state <= DW_RUN;
+	    end	    
+	 end
+	 //
+	 // DW_LAST
+	 //
+	 DW_LAST: begin
+	    if (write_data_valid && m_axi_wready) begin
+	       enable_data_write <= 1'b0;
+	       write_data_state <= DW_IDLE;
+	       m_axi_wlast <= 1'b0;
+	    end else begin
+	       enable_data_write <= 1'b1;
+	       write_data_state <= DW_LAST;
+	       m_axi_wlast <= 1'b1;
+	    end
+	 end // case: DW_LAST
+	 //
+	 default:
+	   write_data_state <= DW_IDLE;
+	 
+       endcase // case(write_data_state)
+   
+	       
+   assign m_axi_wdata = write_data;
+   assign m_axi_wvalid = enable_data_write && write_data_valid;
+   assign write_data_ready = enable_data_write && m_axi_wready;
+
+   /////////////////////////////////////////////////////////////////////////////////
+   //
+   // AXI Read address channel
+   //
+   /////////////////////////////////////////////////////////////////////////////////
+   assign m_axi_arid = 1'b0;
+   assign m_axi_arsize = $clog2(DWIDTH/8);
+   assign m_axi_arburst = `AXI4_BURST_INCR;
+   assign m_axi_arlock = `AXI4_LOCK_NORMAL;
+   assign m_axi_arcache = `AXI4_CACHE_ALLOCATE | `AXI4_CACHE_OTHER_ALLOCATE | `AXI4_CACHE_MODIFIABLE | `AXI4_CACHE_BUFFERABLE;
+   assign m_axi_arprot = `AXI4_PROT_NON_SECURE;
+   assign m_axi_arqos = 4'h0;
+   assign m_axi_arregion = 4'h0;
+   assign m_axi_aruser = 1'b0;
+
+
+   //
+   // AXI Read address state machine
+   //
+   always @(posedge aclk)
+     if (areset) begin
+	read_ctrl_ready <= 1'b0;
+	read_addr_state <= AR_IDLE;
+	m_axi_araddr <= {AWIDTH{1'b0}};
+	m_axi_arlen[7:0] <= 8'h0;
+	m_axi_arvalid <= 1'b0;
+     end else
+       case (read_addr_state)
+	 //
+	 // AR_IDLE
+	 // We are ready to accept a new read transaction.
+	 //
+	 AR_IDLE: begin
+	    // Premptively accept new read transaction since we are idle.
+	    read_ctrl_ready <= 1'b1;
+	    // If we are offered a new transaction then.....
+	    if (read_ctrl_valid) begin
+	       // Drive all the relevent AXI4 read address channel signals next cycle.
+	       m_axi_araddr <= read_addr;
+	       m_axi_arlen[7:0] <= {read_count};
+	       m_axi_arvalid <= 1'b1;
+	       // If the AXI4 read channel is pre-emptively accepting the transaction...
+	       if (m_axi_arready == 1'b1) begin
+		  // ...go straight to looking for the transaction to complete
+		   `DEBUG $display("READ TRANSACTION: ADDR: %x  LEN: %x @ time %d",read_addr,read_count,$time);
+		  read_addr_state <= WAIT_READ_DONE;
+	       end else begin
+                  // ...otherwise wait to get the transaction accepted.
+		  read_addr_state <= WAIT_ARREADY;
+	       end	       
+	    end
+	 end
+	 //
+	 // WAIT_ARREADY
+	 // Waiting for AXI4 slave to accept new read transaction.
+	 //
+	 WAIT_ARREADY: begin
+	    read_ctrl_ready <= 1'b0;
+	    // If the AXI4 read channel is accepting the transaction...
+	    if (m_axi_arready == 1'b1) begin
+	       // ...go to looking for the transaction to complete...
+	       read_addr_state <= WAIT_READ_DONE;
+	       m_axi_arvalid <= 1'b0;
+	        `DEBUG $display("READ TRANSACTION: ADDR: %x  LEN: %x @ time %d",m_axi_araddr,m_axi_arlen[7:0],$time);	       
+	    end else begin
+               // ...otherwise wait to get the trasaction accepted.
+	       read_addr_state <= WAIT_ARREADY;
+	    end
+	 end // case: WAIT_ARREADY
+	 //
+	 // WAIT_READ_DONE
+	 // Read transaction has been accepted, now waiting for the data transfer to complete
+	 // Ignoring ID tag for the moment
+	 //
+	 WAIT_READ_DONE: begin
+	    read_ctrl_ready <= 1'b0;
+	    m_axi_arvalid <= 1'b0;
+	    // Wait for read transaction to complete
+	    if (read_data_state == DR_IDLE) begin
+		 // ....it went well, we are ready to start something new.
+	       read_addr_state <= AR_IDLE;
+	       read_ctrl_ready <= 1'b1; // Ready to run again as soon as we hit idle.
+	    end else if (read_data_state == DR_ERROR) begin
+		 // ....things got ugly, retreat to an error stat and wait for intervention.
+	       read_addr_state <= AR_ERROR;
+	    end else begin
+	       read_addr_state <= WAIT_READ_DONE;
+	    end
+	 end // case: WAIT_BVALID
+	 //
+	 // AR_ERROR
+	 // Something bad happened, going to need external intervention to restore a safe state.
+	 //
+	 AR_ERROR: begin
+	    read_ctrl_ready <= 1'b0;
+	    read_addr_state <= AR_ERROR;
+	    m_axi_araddr <= {AWIDTH{1'b0}};
+	    m_axi_arlen[7:0] <= 8'h0;
+	    m_axi_arvalid <= 1'b0;
+	 end
+       endcase // case(read_addr_state)
+
+   /////////////////////////////////////////////////////////////////////////////////
+   //
+   // AXI Read data channel
+   //
+   /////////////////////////////////////////////////////////////////////////////////
+  
+
+   //
+   // AXI Read data state machine
+   //
+   always @(posedge aclk)
+     if (areset) begin
+	read_data_state <= AR_IDLE;
+	read_data_count <= 0;
+	enable_data_read <= 1'b0;
+	
+     end else
+       case (read_data_state)
+	 //
+	 // DR_IDLE
+	 // Sit in this state until presented with the control details of a new read transaction.
+	 //
+	 DR_IDLE: begin
+	    read_data_count <= 0;
+	    
+	    if (read_ctrl_valid && read_ctrl_ready) begin	       
+	       enable_data_read <= 1'b1;
+	       read_data_state <= DR_RUN;			   
+	    end else begin
+	       read_data_state <= DR_IDLE;
+	    end
+	 end
+	 //
+	 // DR_RUN
+	 // Sit here counting read transfers. If any have error's shift to error state.
+	 //
+	 DR_RUN : begin
+	    enable_data_read <= 1'b1;
+	     
+	    if (read_data_ready && m_axi_rvalid) begin
+	      // Single read transfer
+	       read_data_count <= read_data_count + 1;
+	       if ((m_axi_rresp == `AXI4_RESP_SLVERR) ||  (m_axi_rresp == `AXI4_RESP_DECERR)) begin
+		  if (m_axi_rlast) begin
+		     read_data_state <= DR_ERROR;
+		  end else begin
+		     read_data_state <= DR_WAIT_ERROR;
+		  end
+	       end else if (m_axi_rlast) begin // Implicitly good response signalled this transfer.
+		  if (read_data_count == m_axi_arlen[7:0]) begin
+		     read_data_state <= DR_IDLE;
+		  end else begin
+		     read_data_state <= DR_ERROR;
+		  end
+	       end else begin
+		  read_data_state <= DR_RUN;
+	       end
+	    end else begin
+	       read_data_state <= DR_RUN;
+	    end
+	 end
+	 //
+	 // DR_WAIT_ERROR
+	 // Something bad happened, wait for last signalled in this burst
+	 //
+	 DR_WAIT_ERROR: begin
+	    if (read_data_ready && m_axi_rvalid && m_axi_rlast) begin
+	       enable_data_read <= 1'b0;
+	       read_data_state <= DR_ERROR;
+	    end else begin
+	       enable_data_read <= 1'b1;
+	       read_data_state <= DR_WAIT_ERROR;
+	    end
+	 end // case: DR_WAIT_ERROR
+	 //
+	 // DR_ERROR
+	 // Something bad happened, going to need external intervention to restore a safe state.
+	 //
+	 DR_ERROR: begin
+	    enable_data_read <= 1'b0;
+	    read_data_state <= DR_ERROR;
+	 end // case: DR_ERROR
+	 
+
+       endcase // case(read_data_state)
+   
+	       
+   assign read_data = m_axi_rdata;
+   assign m_axi_rready = enable_data_read && read_data_ready;
+   assign read_data_valid = enable_data_read && m_axi_rvalid;
+			   
+endmodule // axi_dma_master
+
+	      
+	    
+	 
+   
+	      
\ No newline at end of file
diff --git a/fpga/usrp3/lib/axi/axi_dummy.v b/fpga/usrp3/lib/axi/axi_dummy.v
new file mode 100644
index 000000000..5ba430fc4
--- /dev/null
+++ b/fpga/usrp3/lib/axi/axi_dummy.v
@@ -0,0 +1,85 @@
+//
+// Copyright 2015 Ettus Research
+// Copyright 2018 Ettus Research, a National Instruments Company
+//
+// SPDX-License-Identifier: LGPL-3.0-or-later
+//
+
+module axi_dummy
+(
+  // sys connect
+  input         s_axi_aclk,
+  input         s_axi_areset,
+
+  // axi4 lite slave port
+  input [31:0]  s_axi_awaddr,
+  input         s_axi_awvalid,
+  output        s_axi_awready,
+
+  input [31:0]  s_axi_wdata,
+  input [3:0]   s_axi_wstrb,
+  input         s_axi_wvalid,
+  output        s_axi_wready,
+
+  output [1:0]  s_axi_bresp,
+  output        s_axi_bvalid,
+  input         s_axi_bready,
+
+  input [31:0]  s_axi_araddr,
+  input         s_axi_arvalid,
+  output        s_axi_arready,
+
+  output [31:0] s_axi_rdata,
+  output [1:0]  s_axi_rresp,
+  output        s_axi_rvalid,
+  input         s_axi_rready
+);
+  parameter DEC_ERR = 1'b1;
+
+  localparam IDLE              = 3'b001;
+  localparam READ_IN_PROGRESS  = 3'b010;
+  localparam WRITE_IN_PROGRESS = 3'b100;
+
+  reg [2:0] state;
+
+  always @ (posedge s_axi_aclk) begin
+    if (s_axi_areset) begin
+      state <= IDLE;
+    end
+    else case (state)
+
+    IDLE: begin
+      if (s_axi_arvalid)
+        state <= READ_IN_PROGRESS;
+      else if (s_axi_awvalid)
+        state <= WRITE_IN_PROGRESS;
+    end
+
+    READ_IN_PROGRESS: begin
+      if (s_axi_rready)
+        state <= IDLE;
+    end
+
+    WRITE_IN_PROGRESS: begin
+      if (s_axi_bready)
+        state <= IDLE;
+    end
+
+    default: begin
+      state <= IDLE;
+    end
+
+    endcase
+  end
+
+  assign s_axi_awready = (state == IDLE);
+  assign s_axi_wready  = (state == WRITE_IN_PROGRESS);
+  assign s_axi_bvalid  = (state == WRITE_IN_PROGRESS);
+
+  assign s_axi_arready = (state == IDLE);
+  assign s_axi_rdata = 32'hdead_ba5e;
+  assign s_axi_rvalid  = (state == READ_IN_PROGRESS);
+  assign s_axi_rresp = DEC_ERR ? 2'b11 : 2'b00;
+  assign s_axi_bresp = DEC_ERR ? 2'b11 : 2'b00;
+
+endmodule
diff --git a/fpga/usrp3/lib/axi/axi_embed_tlast.v b/fpga/usrp3/lib/axi/axi_embed_tlast.v
new file mode 100644
index 000000000..bc14043e9
--- /dev/null
+++ b/fpga/usrp3/lib/axi/axi_embed_tlast.v
@@ -0,0 +1,132 @@
+//
+// Copyright 2014 Ettus Research LLC
+// Copyright 2018 Ettus Research, a National Instruments Company
+//
+// SPDX-License-Identifier: LGPL-3.0-or-later
+//
+//
+// AXI stream neds N+1 bits to transmit packets of N bits so that the LAST bit can be represented.
+// LAST occurs relatively infrequently and can be synthesized by using an in-band ESC code to generate
+// a multi-word sequence to encode it (and the escape character when it appears as data input).
+//
+// 0x1234567887654321 with last becomes
+// 0xDEADBEEFFEEDCAFE 0x0000000000000001 0x1234567887654321
+//
+// 0xDEADBEEFFEEDCAFE with last becomes
+// 0xDEADBEEFFEEDCAFE 0x0000000000000001 0xDEADBEEFFEEDCAFE
+//
+// 0xDEADBEEFFEEDCAFE without last becomes
+// 0xDEADBEEFFEEDCAFE 0x0000000000000000 0xDEADBEEFFEEDCAFE
+//
+
+module axi_embed_tlast #(
+  parameter WIDTH=64,
+  parameter ADD_CHECKSUM=0
+) (
+   input clk,
+   input reset,
+   input clear,
+   //
+   input [WIDTH-1:0] i_tdata,
+   input i_tlast,
+   input i_tvalid,
+   output i_tready,
+   //
+   output reg [WIDTH-1:0] o_tdata,
+   output o_tvalid,
+   input o_tready
+);
+
+   localparam PASS = 0;
+   localparam ZERO = 1;
+   localparam ONE = 2;
+   localparam ESCAPE = 3;
+
+   localparam IDLE = 0;
+   localparam LAST = 1;
+   localparam ESC = 2;
+   localparam FINISH = 3;
+   
+   reg [1:0]  state, next_state;
+
+   reg [1:0]  select;
+
+   wire [31:0] checksum;
+   generate if (ADD_CHECKSUM == 1) begin
+      reg [31:0] checksum_reg;
+      always @(posedge clk) begin
+         if (reset | clear) begin
+            checksum_reg <= 0;
+         end else if (i_tready && i_tvalid && i_tlast) begin
+            checksum_reg <= 0;
+         end else if (i_tready && i_tvalid) begin
+            checksum_reg <= checksum_reg ^ i_tdata[31:0] ^ i_tdata[63:32];
+         end
+      end
+      assign checksum = checksum_reg;
+   end else begin
+      assign checksum = 32'h0;
+   end endgenerate
+
+   always @(posedge clk) 
+      if (reset | clear) begin
+         state <= IDLE;
+      end else begin if (o_tready)
+         state <= next_state;
+      end 
+
+   always @(*) begin
+      case(state)
+         IDLE: begin
+            if (i_tlast && i_tvalid) begin
+               next_state = LAST;
+               select = ESCAPE;
+            end else if ((i_tdata == 64'hDEADBEEFFEEDCAFE) && i_tvalid) begin
+               next_state = ESC;
+               select = ESCAPE;
+            end else begin
+               next_state = IDLE;
+               select = PASS;
+            end
+         end // case: IDLE
+
+         LAST: begin
+            select = ONE;
+            next_state = FINISH;
+         end
+
+         ESC: begin
+            select = ZERO;
+            next_state = FINISH;
+         end
+
+         FINISH: begin
+            select = PASS;
+            if (i_tvalid)
+               next_state = IDLE;
+            else
+               next_state = FINISH;
+         end
+      endcase // case(state)
+   end // always @ (*)
+
+   //
+   // Muxes
+   //
+   always @*
+      begin
+         case(select)
+            PASS:   o_tdata = i_tdata;
+            ZERO:   o_tdata = 0;
+            ONE:    o_tdata = {checksum[31:0],32'h1};
+            ESCAPE: o_tdata = 64'hDEADBEEFFEEDCAFE;
+         endcase // case(select)
+      end
+
+   assign o_tvalid = (select == PASS) ? i_tvalid : 1'b1;
+   assign i_tready = (select == PASS) ? o_tready : 1'b0;
+
+endmodule // axi_embed_tlast
+
+
+
diff --git a/fpga/usrp3/lib/axi/axi_embed_tlast_tkeep.v b/fpga/usrp3/lib/axi/axi_embed_tlast_tkeep.v
new file mode 100644
index 000000000..4adb91fa6
--- /dev/null
+++ b/fpga/usrp3/lib/axi/axi_embed_tlast_tkeep.v
@@ -0,0 +1,152 @@
+//
+// Copyright 2019 Ettus Research, a National Instruments Brand
+//
+// SPDX-License-Identifier: LGPL-3.0-or-later
+//
+// Module: axi_embed_tlast_tkeep
+//
+// Description:
+//
+//   This module takes the TLAST and TKEEP values of an AXI-Stream interface
+//   and embeds them into the data stream. This allows a data pipe to be used
+//   that isn't wide enough for the TDATA, TLAST,and TKEEP to be passed through
+//   in parallel. Since TLAST and TKEEP are only usually needed for one word
+//   per packet, this also reduces the amount of memory required to store a
+//   packet. Note that this module only supports TKEEP at the end of a packet
+//   when TLAST is asserted. See also axi_extract_tlast_tkeep.
+//
+//   This embedding is accomplished by using an escape sequence using the word
+//   0xDEADBEEF as the escape code. If TLAST and TKEEP are both 0 (the usual
+//   case) then no escape sequence is used. Any word that has "DEADBEEF" in the
+//   most significant position is considered an escape word. The least
+//   significant bits of the escape word contain the TKEEP and TLAST bits. The
+//   word following the escape word is the normal data word associated with
+//   those TLAST and TKEEP values.
+//
+//   Here are some examples for the case where DATA_W = 64
+//
+//     0x1234567887654321 with TLAST=0 and TKEEP=0 becomes
+//     0x1234567887654321
+//
+//     0x1234567887654321 with TLAST=1 and TKEEP=0 becomes
+//     0xDEADBEEF00000001 0x1234567887654321
+//
+//     0x1234567887654321 with TLAST=1 and TKEEP=2 becomes
+//     0xDEADBEEF00000005 0x1234567887654321
+//
+//     0x1234567887654321 with TLAST=0 and TKEEP=1 becomes
+//     0x1234567887654321 (because TKEEP is ignored when TLAST=0)
+//
+//     0xDEADBEEFFEEDCAFE without TLAST=0 and TKEEP=0 becomes
+//     0xDEADBEEF00000000 0xDEADBEEFFEEDCAFE
+//
+//     0xDEADBEEFFEEDCAFE with TLAST=0 and TKEEP=1 becomes
+//     0xDEADBEEF00000002 0xDEADBEEFFEEDCAFE
+//
+
+module axi_embed_tlast_tkeep #(
+  parameter DATA_W = 64,
+  parameter KEEP_W = DATA_W/8
+) (
+  input clk,
+  input rst,
+
+  // Input AXI-Stream
+  input  [DATA_W-1:0] i_tdata,
+  input  [KEEP_W-1:0] i_tkeep,
+  input               i_tlast,
+  input               i_tvalid,
+  output              i_tready,
+
+  // Output AXI-Stream
+  output reg [DATA_W-1:0] o_tdata,
+  output                  o_tvalid,
+  input                   o_tready
+);
+
+  localparam                  ESC_WORD_W = 32;
+  localparam [ESC_WORD_W-1:0] ESC_WORD   = 'hDEADBEEF;
+
+
+  //---------------------------------------------------------------------------
+  // Parameter Checking
+  //---------------------------------------------------------------------------
+
+  if (DATA_W < ESC_WORD_W+KEEP_W+1) begin : gen_assertion
+    // Cause an error if DATA_W is not large enough.
+    DATA_W_is_not_large_enough_to_store_escape_code_TKEEP_and_TLAST();
+  end
+
+
+  //---------------------------------------------------------------------------
+  // State Machine
+  //---------------------------------------------------------------------------
+
+  localparam PASS   = 0;
+  localparam ESCAPE = 1;
+
+  localparam ST_IDLE = 0;
+  localparam ST_DATA = 1;
+
+  reg [0:0] state = ST_IDLE;
+  reg [0:0] next_state;
+
+  reg [0:0] select;
+
+  always @(posedge clk) begin
+    if (rst) begin
+      state <= ST_IDLE;
+    end else begin if (o_tready)
+      state <= next_state;
+    end
+  end
+
+  always @(*) begin
+    case(state)
+      ST_IDLE: begin
+        if (i_tlast && i_tvalid) begin
+          next_state = ST_DATA;
+          select     = ESCAPE;
+        end else if ((i_tdata[DATA_W-1 -: ESC_WORD_W] == ESC_WORD) && i_tvalid) begin
+          next_state = ST_DATA;
+          select     = ESCAPE;
+        end else begin
+          next_state = ST_IDLE;
+          select     = PASS;
+        end
+      end
+
+      ST_DATA: begin
+        select = PASS;
+        if (i_tvalid) begin
+          next_state = ST_IDLE;
+        end else begin
+          next_state = ST_DATA;
+        end
+      end
+    endcase
+  end
+
+
+  //---------------------------------------------------------------------------
+  // Output Multiplexers
+  //---------------------------------------------------------------------------
+
+  always @(*) begin
+    case(select)
+      PASS   : begin
+        o_tdata = i_tdata;
+      end
+      ESCAPE : begin
+        o_tdata                         = {DATA_W{1'b0}};
+        o_tdata[DATA_W-1 -: ESC_WORD_W] = ESC_WORD;
+        o_tdata[       1 +:     KEEP_W] = i_tkeep;
+        o_tdata[       0 +:          1] = i_tlast;
+      end
+    endcase
+  end
+
+  assign o_tvalid = (select == PASS) ? i_tvalid : 1'b1;
+  assign i_tready = (select == PASS) ? o_tready : 1'b0;
+
+endmodule
diff --git a/fpga/usrp3/lib/axi/axi_extract_tlast.v b/fpga/usrp3/lib/axi/axi_extract_tlast.v
new file mode 100644
index 000000000..9afef3c35
--- /dev/null
+++ b/fpga/usrp3/lib/axi/axi_extract_tlast.v
@@ -0,0 +1,147 @@
+//
+// Copyright 2014 Ettus Research LLC
+// Copyright 2018 Ettus Research, a National Instruments Company
+//
+// SPDX-License-Identifier: LGPL-3.0-or-later
+//
+//
+// AXI stream neds N+1 bits to transmit packets of N bits so that the LAST bit can be represented.
+// LAST occurs relatively infrequently and can be synthesized by using an in-band ESC code to generate
+// a multi-word sequence to encode it (and the escape character when it appears as data input).
+//
+// 0x1234567887654321 with last becomes
+// 0xDEADBEEFFEEDCAFE 0x0000000000000001 0x1234567887654321
+//
+// 0xDEADBEEFFEEDCAFE with last becomes
+// 0xDEADBEEFFEEDCAFE 0x0000000000000001 0xDEADBEEFFEEDCAFE
+//
+// 0xDEADBEEFFEEDCAFE without last becomes
+// 0xDEADBEEFFEEDCAFE 0x0000000000000000 0xDEADBEEFFEEDCAFE
+//
+
+module axi_extract_tlast #(
+  parameter WIDTH=64,
+  parameter VALIDATE_CHECKSUM=0
+) (
+   input clk,
+   input reset,
+   input clear,
+   //
+   input [WIDTH-1:0] i_tdata,
+   input i_tvalid,
+   output reg i_tready,
+   //
+   output [WIDTH-1:0] o_tdata,
+   output reg o_tlast,
+   output reg o_tvalid,
+   input  o_tready,
+   //
+   output reg checksum_error
+);
+
+   reg [1:0] state, next_state;
+   
+   localparam IDLE = 0;
+   localparam EXTRACT1 = 1;
+   localparam EXTRACT2 = 2;
+   localparam EXTRACT3 = 3;
+   
+   assign     o_tdata = i_tdata;
+
+   reg        checksum_error_pre;
+   reg [31:0] checksum, old_checksum;
+
+   always @(posedge clk) 
+      if (reset | clear) begin
+         checksum <= 0;
+         old_checksum <= 0;
+      end else if (VALIDATE_CHECKSUM && o_tready && i_tvalid && o_tlast) begin
+         checksum <= 0;
+         old_checksum <= 0;
+      end else if (VALIDATE_CHECKSUM && i_tready && i_tvalid && (state == IDLE)) begin
+         checksum <= checksum ^ i_tdata[31:0] ^ i_tdata[63:32];
+         old_checksum <= checksum;
+      end
+
+   always @(posedge clk)
+      checksum_error <= checksum_error_pre;
+
+   always @(posedge clk) 
+      if (reset | clear) begin
+         state <= IDLE;
+      end else begin
+         state <= next_state;
+      end 
+
+   always @(*) begin
+      checksum_error_pre = 0;
+      case(state)
+         //
+         // Search for Escape sequence "0xDEADBEEFFEEDCAFE"
+         // If ESC found don't pass data downstream but transition to next state.
+         // else pass data downstream.
+         //
+         IDLE: begin
+            o_tlast = 1'b0;
+            if ((i_tdata == 64'hDEADBEEFFEEDCAFE) && i_tvalid) begin
+               next_state = EXTRACT1;
+               o_tvalid = 1'b0;
+               i_tready = 1'b1;
+            end else begin
+               next_state = IDLE;
+               o_tvalid = i_tvalid;
+               i_tready = o_tready;
+            end // else: !if((i_tdata == 'hDEADBEEFFEEDCAFE) && i_tvalid)
+         end // case: IDLE
+         //
+         // Look at next data. If it's a 0x1 then o_tlast should be asserted with next data word.
+         // if it's 0x0 then it signals emulation of the Escape code in the original data stream
+         // and we should just pass the next data word through unchanged with no o_tlast indication.
+         //
+         EXTRACT1: begin
+            o_tvalid = 1'b0;
+            i_tready = 1'b1;
+            o_tlast = 1'b0;
+            if (i_tvalid) begin
+               if (i_tdata[31:0] == 'h1) begin
+                  if (VALIDATE_CHECKSUM && (old_checksum != i_tdata[63:32])) 
+                     checksum_error_pre = 1'b1;
+                  next_state = EXTRACT2;
+               end else  begin
+                  // We assume emulation and don't look for illegal codes.
+                  next_state = EXTRACT3;
+               end // else: !if(i_tdata == 'h1)
+            end else begin // if (i_tvalid)
+               next_state = EXTRACT1;
+            end // else: !if(i_tvalid)
+         end // case: EXTRACT1
+         //
+         // Assert o_tlast with data word.
+         //
+         EXTRACT2: begin
+            o_tvalid = i_tvalid;
+            i_tready = o_tready;
+            o_tlast = 1'b1;
+            if (i_tvalid & o_tready)
+               next_state = IDLE;
+            else
+               next_state = EXTRACT2;
+         end
+         //
+         // Emulation, don't assert o_tlast with dataword.
+         //
+         EXTRACT3: begin
+            o_tvalid = i_tvalid;
+            i_tready = o_tready;
+            o_tlast = 1'b0;
+            if (i_tvalid & o_tready)
+               next_state = IDLE;
+            else
+               next_state = EXTRACT2;
+         end
+      endcase // case(state)
+   end
+
+endmodule // axi_extract_tlast
+
+  
\ No newline at end of file
diff --git a/fpga/usrp3/lib/axi/axi_extract_tlast_tkeep.v b/fpga/usrp3/lib/axi/axi_extract_tlast_tkeep.v
new file mode 100644
index 000000000..4d9178052
--- /dev/null
+++ b/fpga/usrp3/lib/axi/axi_extract_tlast_tkeep.v
@@ -0,0 +1,128 @@
+//
+// Copyright 2019 Ettus Research, a National Instruments Brand
+//
+// SPDX-License-Identifier: LGPL-3.0-or-later
+//
+// Module: axi_extract_tlast_tkeep
+//
+// Description:
+//
+//   This module extracts the TLAST and TKEEP values that were embedded by the
+//   axi_embed_tlast_tkeep module. See axi_embed_tlast_tkeep for a description
+//   of how the data is encoded.
+//
+//   Here are some extraction examples for DATA_W = 64.
+//
+//     0x1234567887654321 becomes
+//     0x1234567887654321 (no changes)
+//
+//     0xDEADBEEF00000001 0x1234567887654321 becomes
+//     0x1234567887654321 with TLAST=1 and TKEEP=0
+//
+//     0xDEADBEEF00000005 0x1234567887654321 becomes
+//     0x1234567887654321 with TLAST=1 and TKEEP=2
+//
+//     0xDEADBEEF00000000 0xDEADBEEFFEEDCAFE
+//     0xDEADBEEFFEEDCAFE without TLAST=0 and TKEEP=0 becomes
+//
+//     0xDEADBEEF00000002 0xDEADBEEFFEEDCAFE
+//     0xDEADBEEFFEEDCAFE with TLAST=0 and TKEEP=1 becomes
+//
+
+module axi_extract_tlast_tkeep #(
+   parameter DATA_W = 64,
+   parameter KEEP_W = DATA_W  /8
+) (
+   input clk,
+   input rst,
+
+   // Input AXI-Stream
+   input      [DATA_W-1:0] i_tdata,
+   input                   i_tvalid,
+   output reg              i_tready,
+
+   // Output AXI-Stream
+   output reg [DATA_W-1:0] o_tdata,
+   output reg [KEEP_W-1:0] o_tkeep,
+   output reg              o_tlast,
+   output reg              o_tvalid,
+   input                   o_tready
+);
+
+  localparam                  ESC_WORD_W = 32;
+  localparam [ESC_WORD_W-1:0] ESC_WORD   = 'hDEADBEEF;
+
+
+  //---------------------------------------------------------------------------
+  // TKEEP and TLAST Holding Register
+  //---------------------------------------------------------------------------
+
+  reg              save_flags;
+  reg              tlast_saved;
+  reg [KEEP_W-1:0] tkeep_saved;
+
+  always @(posedge clk) begin
+    if (save_flags) begin
+      // Save the TLAST and TKEEP values embedded in the escape word
+      tlast_saved <= i_tdata[0];
+      tkeep_saved <= i_tdata[1 +: KEEP_W];
+    end
+  end
+
+
+  //--------------------------------------------------------------------------
+  // State Machine
+  //--------------------------------------------------------------------------
+
+  localparam ST_IDLE = 0;
+  localparam ST_DATA = 1;
+
+  reg [0:0] state = ST_IDLE;
+  reg [0:0] next_state;
+
+  always @(posedge clk) begin
+    if (rst) begin
+      state <= ST_IDLE;
+    end else begin
+      state <= next_state;
+    end
+  end
+
+  always @(*) begin
+    // Default assignments (pass through)
+    o_tdata    = i_tdata;
+    o_tlast    = 1'b0;
+    o_tkeep    = {KEEP_W{1'b1}};
+    save_flags = 1'b0;
+    next_state = state;
+    o_tvalid   = i_tvalid;
+    i_tready   = o_tready;
+
+    case(state)
+      //
+      // Search for escape code. If found don't pass data downstream but
+      // transition to next state. Otherwise, pass data downstream.
+      //
+      ST_IDLE: begin
+        if ((i_tdata[DATA_W-1 -: ESC_WORD_W] == ESC_WORD) && i_tvalid) begin
+          save_flags = 1'b1;
+          next_state = ST_DATA;
+          o_tvalid   = 1'b0;
+          i_tready   = 1'b1;
+        end
+      end
+
+      //
+      // Output data word with the saved TLAST and TKEEP values
+      //
+      ST_DATA: begin
+        o_tlast = tlast_saved;
+        o_tkeep = tkeep_saved;
+        if (i_tvalid & o_tready) begin
+          next_state = ST_IDLE;
+        end
+      end
+    endcase
+  end
+
+endmodule
diff --git a/fpga/usrp3/lib/axi/axi_fast_extract_tlast.v b/fpga/usrp3/lib/axi/axi_fast_extract_tlast.v
new file mode 100644
index 000000000..59e925caf
--- /dev/null
+++ b/fpga/usrp3/lib/axi/axi_fast_extract_tlast.v
@@ -0,0 +1,193 @@
+//
+// Copyright 2014 Ettus Research LLC
+// Copyright 2018 Ettus Research, a National Instruments Company
+//
+// SPDX-License-Identifier: LGPL-3.0-or-later
+//
+//
+// Ultra fast critical path FIFO.
+// Only 2 entrys but no combinatorial feed through paths
+//
+
+
+module axi_fast_extract_tlast
+  #(parameter WIDTH=64)
+  (
+   input clk,
+   input reset,
+   input clear,
+   //
+   input [WIDTH-1:0] i_tdata,
+   input i_tvalid,
+   output reg i_tready,
+   //
+   output [WIDTH-1:0] o_tdata,
+   output o_tlast,
+   output reg o_tvalid,
+   input  o_tready
+   );
+
+   reg [WIDTH:0] data_reg1, data_reg2;
+
+   reg [1:0] 	   fifo_state;
+   
+   localparam EMPTY = 0;
+   localparam HALF = 1;
+   localparam FULL = 2;
+
+   reg [1:0]  extract_state;
+   
+   localparam IDLE = 0;
+   localparam EXTRACT1 = 1;
+   localparam EXTRACT2 = 2;
+   localparam EXTRACT3 = 3;
+
+   
+   always @(posedge clk) 
+     if (reset | clear) begin
+	fifo_state <= EMPTY;
+     end else begin
+	case (fifo_state)
+	  // Nothing in either register.
+	  // Upstream can always push data to us.
+	  // Downstream has nothing to take from us.
+	  EMPTY: begin
+	     if ((extract_state == IDLE) && (i_tdata == 64'hDEADBEEFFEEDCAFE) && i_tvalid) begin
+		// Embeded escpae code received.
+		extract_state <= EXTRACT1;
+		i_tready <= 1'b1;
+		o_tvalid <= 1'b0;
+		fifo_state <= EMPTY;
+	     end else if ((extract_state == EXTRACT1) && i_tvalid) begin
+		// Now work out if its a genuine embeded tlast or emulation.
+		i_tready <= 1'b1;
+		o_tvalid <= 1'b0;
+		fifo_state <= EMPTY;
+		if (i_tdata[31:0] == 'h1) begin
+		   extract_state <= EXTRACT2;
+		end else begin
+		   extract_state <= EXTRACT3;
+		end
+	     end else if ((extract_state == EXTRACT2) && i_tvalid) begin
+		// Extract tlast.
+		data_reg1 <= {1'b1,i_tdata};
+		i_tready <= 1'b1;
+		o_tvalid <= 1'b1;
+		fifo_state <= HALF;
+		extract_state <= IDLE;
+	     end else if (i_tvalid) begin
+		// Get here both for normal data and for EXTRACT3 emulation data.
+		data_reg1 <= {1'b0,i_tdata};
+		fifo_state <= HALF;
+		extract_state <= IDLE;
+		i_tready <= 1'b1;
+		o_tvalid <= 1'b1;
+	     end else begin
+		// Nothing to do.
+		fifo_state <= EMPTY;
+		i_tready <= 1'b1;
+		o_tvalid <= 1'b0;
+	     end
+	  end
+	  // First Register Full.
+	  // Upstream can always push data to us.
+	  // Downstream can always read from us.
+	  HALF: begin
+	     if ((extract_state == IDLE) && (i_tdata == 64'hDEADBEEFFEEDCAFE) && i_tvalid) begin
+		// Embeded escpae code received.
+		extract_state <= EXTRACT1;
+		if (o_tready) begin
+		   // If meanwhile we get read then go empty...
+		   i_tready <= 1'b1;
+		   o_tvalid <= 1'b0;
+		   fifo_state <= EMPTY;
+		end else begin
+		   // ...else stay half full.
+		   fifo_state <= HALF;
+		   i_tready <= 1'b1;
+		   o_tvalid <= 1'b1;
+		end
+	     end else if ((extract_state == EXTRACT1) && i_tvalid) begin
+		// Now work out if its a genuine embeded tlast or emulation.
+		if (i_tdata[31:0] == 'h1) begin
+		   extract_state <= EXTRACT2;
+		end else begin
+		   extract_state <= EXTRACT3;
+		end
+		if (o_tready) begin
+		   // If meanwhile we get read then go empty...
+		   i_tready <= 1'b1;
+		   o_tvalid <= 1'b0;
+		   fifo_state <= EMPTY;
+		end else begin
+		   // ...else stay half full.
+		   fifo_state <= HALF;
+		   i_tready <= 1'b1;
+		   o_tvalid <= 1'b1;
+		end		
+	     end else if ((extract_state == EXTRACT2) && i_tvalid) begin
+		// Extract tlast.
+		data_reg1 <= {1'b1,i_tdata};
+		extract_state <= IDLE;
+		if (o_tready) begin
+		   // We get read and writen same cycle...
+		   i_tready <= 1'b1;
+		   o_tvalid <= 1'b1;
+		   fifo_state <= HALF;
+		end else begin
+		   // ...or we get written and go full.
+		   data_reg2 <= data_reg1;
+		   i_tready <= 1'b0;
+		   o_tvalid <= 1'b1;
+		   fifo_state <= FULL;
+		end	
+	     end else if (i_tvalid) begin
+		// Get here both for normal data and for EXTRACT3 emulation data.
+		data_reg1 <= {1'b0,i_tdata};
+		extract_state <= IDLE;
+		if (o_tready) begin
+		   // We get read and writen same cycle...
+		   fifo_state <= HALF;
+		   i_tready <= 1'b1;
+		   o_tvalid <= 1'b1;
+		end else begin
+		   // ...or we get written and go full.
+		   data_reg2 <= data_reg1;
+		   i_tready <= 1'b0;
+		   o_tvalid <= 1'b1;
+		   fifo_state <= FULL;
+		end	
+	     end else if (o_tready) begin // if (i_tvalid)
+		// Only getting read this cycle so go empty
+		fifo_state <= EMPTY;
+		i_tready <= 1'b1;
+		o_tvalid <= 1'b0;
+	     end else begin
+		// Absolutley nothing happens, everything stays the same.
+		fifo_state <= HALF;
+		i_tready <= 1'b1;
+		o_tvalid <= 1'b1;
+	     end
+	  end // case: HALF
+	  // Both Registers Full.
+	  // Upstream can not push to us in this fifo_state.
+	  // Downstream can always read from us.
+	  FULL: begin
+	     if (o_tready) begin
+		fifo_state <= HALF;
+		i_tready <= 1'b1;
+		o_tvalid <= 1'b1;
+	     end
+	     else begin
+		fifo_state <= FULL;
+		i_tready <= 1'b0;
+		o_tvalid <= 1'b1;
+	     end
+	  end
+	endcase // case(fifo_state)
+     end // else: !if(reset | clear)
+   
+   assign {o_tlast,o_tdata} = (fifo_state == FULL) ? data_reg2 : data_reg1;
+   
+
+endmodule // axi_fast_extract_tlast
diff --git a/fpga/usrp3/lib/axi/axi_fast_fifo.v b/fpga/usrp3/lib/axi/axi_fast_fifo.v
new file mode 100644
index 000000000..5ff303b11
--- /dev/null
+++ b/fpga/usrp3/lib/axi/axi_fast_fifo.v
@@ -0,0 +1,108 @@
+//
+// Copyright 2014 Ettus Research LLC
+// Copyright 2018 Ettus Research, a National Instruments Company
+//
+// SPDX-License-Identifier: LGPL-3.0-or-later
+//
+//
+// Ultra fast critical path FIFO.
+// Only 2 entrys but no combinatorial feed through paths
+//
+
+
+module axi_fast_fifo
+  #(parameter WIDTH=64)
+  (
+   input clk,
+   input reset,
+   input clear,
+   //
+   input [WIDTH-1:0] i_tdata,
+   input i_tvalid,
+   output reg i_tready,
+   //
+   output [WIDTH-1:0] o_tdata,
+   output reg o_tvalid,
+   input  o_tready
+   );
+
+   reg [WIDTH-1:0] data_reg1, data_reg2;
+
+   reg [1:0] 	   state;
+   
+   localparam EMPTY = 0;
+   localparam HALF = 1;
+   localparam FULL = 2;
+   
+   always @(posedge clk) 
+     if (reset | clear) begin
+	state <= EMPTY;
+	data_reg1 <= 0;
+	data_reg2 <= 0;
+	o_tvalid <= 1'b0;
+	i_tready <= 1'b0;
+	
+     end else begin
+	case (state)
+	  // Nothing in either register.
+	  // Upstream can always push data to us.
+	  // Downstream has nothing to take from us.
+	  EMPTY: begin
+	     if (i_tvalid) begin
+		data_reg1 <= i_tdata;
+		state <= HALF;
+		i_tready <= 1'b1;
+		o_tvalid <= 1'b1;
+	     end else begin
+		state <= EMPTY;
+		i_tready <= 1'b1;
+		o_tvalid <= 1'b0;
+	     end
+	  end
+	  // First Register Full.
+	  // Upstream can always push data to us.
+	  // Downstream can always read from us.
+	  HALF: begin
+	     if (i_tvalid && o_tready) begin
+		data_reg1 <= i_tdata;
+		state <= HALF;
+		i_tready <= 1'b1;
+		o_tvalid <= 1'b1;
+	     end else if (i_tvalid) begin
+		data_reg1 <= i_tdata;
+		data_reg2 <= data_reg1;
+		state <= FULL;
+		i_tready <= 1'b0;
+		o_tvalid <= 1'b1;
+	     end else if (o_tready) begin
+		state <= EMPTY;
+		i_tready <= 1'b1;
+		o_tvalid <= 1'b0;
+	     end else begin
+		state <= HALF;
+		i_tready <= 1'b1;
+		o_tvalid <= 1'b1;
+	     end
+	  end // case: HALF
+	  // Both Registers Full.
+	  // Upstream can not push to us in this state.
+	  // Downstream can always read from us.
+	  FULL: begin
+	     if (o_tready) begin
+		state <= HALF;
+		i_tready <= 1'b1;
+		o_tvalid <= 1'b1;
+	     end
+	     else begin
+		state <= FULL;
+		i_tready <= 1'b0;
+		o_tvalid <= 1'b1;
+	     end
+	  end
+	endcase // case(state)
+     end // else: !if(reset | clear)
+
+   assign o_tdata = (state == FULL) ? data_reg2 : data_reg1;
+   
+
+endmodule // axi_fast_fifo
diff --git a/fpga/usrp3/lib/axi/axi_replay.v b/fpga/usrp3/lib/axi/axi_replay.v
new file mode 100644
index 000000000..49e4318c5
--- /dev/null
+++ b/fpga/usrp3/lib/axi/axi_replay.v
@@ -0,0 +1,867 @@
+//
+// Copyright 2017 Ettus Research, A National Instruments Company
+//
+// SPDX-License-Identifier: LGPL-3.0
+//
+// Module: axi_replay.v
+// Description:
+//
+// This block implements the state machine and control logic for recording and 
+// playback of AXI-Stream data, using a DMA-accessible memory as a buffer.
+
+
+module axi_replay #(
+  parameter DATA_WIDTH  = 64,
+  parameter ADDR_WIDTH  = 32, // Byte address width used by DMA master
+  parameter COUNT_WIDTH = 8   // Length of counters used to connect to the DMA 
+                              // master's read and write interfaces.
+) (
+  input wire clk,
+  input wire rst,  // Synchronous to clk
+
+  //---------------------------------------------------------------------------
+  // Settings Bus
+  //---------------------------------------------------------------------------
+
+  input  wire        set_stb,
+  input  wire [ 7:0] set_addr,
+  input  wire [31:0] set_data,
+  output reg  [31:0] rb_data,
+  input  wire [ 7:0] rb_addr,
+
+  //---------------------------------------------------------------------------
+  // AXI Stream Interface
+  //---------------------------------------------------------------------------
+
+  // Input
+  input  wire [DATA_WIDTH-1:0] i_tdata,
+  input  wire                  i_tvalid,
+  input  wire                  i_tlast,
+  output wire                  i_tready,
+
+  // Output
+  output wire [DATA_WIDTH-1:0] o_tdata,
+  output wire                  o_tvalid,
+  output wire                  o_tlast,
+  input  wire                  o_tready,
+
+  //---------------------------------------------------------------------------
+  // DMA Interface
+  //---------------------------------------------------------------------------
+
+  // Write interface
+  output reg  [ ADDR_WIDTH-1:0] write_addr,       // Byte address for start of write
+                                                  // transaction (64-bit aligned).
+  output reg  [COUNT_WIDTH-1:0] write_count,      // Count of 64-bit words to write, minus 1.
+  output reg                    write_ctrl_valid,
+  input  wire                   write_ctrl_ready,
+  output wire [ DATA_WIDTH-1:0] write_data,
+  output wire                   write_data_valid,
+  input  wire                   write_data_ready,
+
+  // Read interface
+  output reg  [ ADDR_WIDTH-1:0] read_addr,       // Byte address for start of read
+                                                 // transaction (64-bit aligned).
+  output reg  [COUNT_WIDTH-1:0] read_count,      // Count of 64-bit words to read, minus 1.
+  output reg                    read_ctrl_valid,
+  input  wire                   read_ctrl_ready,
+  input  wire [ DATA_WIDTH-1:0] read_data,
+  input  wire                   read_data_valid,
+  output wire                   read_data_ready
+);
+
+  //---------------------------------------------------------------------------
+  // Constants
+  //---------------------------------------------------------------------------
+
+  // Size constants
+  localparam CMD_WIDTH   = 32;           // Command width
+  localparam LINES_WIDTH = 28;           // Width of cmd_num_lines
+  localparam WORD_SIZE   = DATA_WIDTH/8; // Size of DATA_WIDTH in bytes
+
+  // Register offsets
+  localparam [7:0] SR_REC_BASE_ADDR    = 128;
+  localparam [7:0] SR_REC_BUFFER_SIZE  = 129;
+  localparam [7:0] SR_REC_RESTART      = 130;
+  localparam [7:0] SR_REC_FULLNESS     = 131;
+  localparam [7:0] SR_PLAY_BASE_ADDR   = 132;
+  localparam [7:0] SR_PLAY_BUFFER_SIZE = 133;
+  localparam [7:0] SR_RX_CTRL_COMMAND  = 152; // Same offset as radio
+  localparam [7:0] SR_RX_CTRL_HALT     = 155; // Same offset as radio
+  localparam [7:0] SR_RX_CTRL_MAXLEN   = 156; // Same offset as radio
+
+
+  // Memory buffering parameters:
+  //
+  // Log base 2 of the depth of the input and output FIFOs to use. The FIFOs 
+  // should be large enough to store more than a complete burst 
+  // (MEM_BURST_SIZE). A size of 9 (512 64-bit words) is one 36-kbit BRAM.
+  localparam REC_FIFO_ADDR_WIDTH  = 9;  // Log2 of input/record FIFO size
+  localparam PLAY_FIFO_ADDR_WIDTH = 9;  // Log2 of output/playback FIFO size
+  //
+  // Amount of data to buffer before writing to RAM. This should be a power of 
+  // two so that it evenly divides the AXI_ALIGNMENT requirement. It also must 
+  // not exceed 2**COUNT_WIDTH (the maximum count allowed by DMA master).
+  localparam MEM_BURST_SIZE = 2**COUNT_WIDTH;  // Size in DATA_WIDTH-sized words
+  //
+  // AXI alignment requirement (4096 bytes) in DATA_WIDTH-bit words
+  localparam AXI_ALIGNMENT = 4096 / WORD_SIZE;
+  //
+  // Clock cycles to wait before writing something less than MEM_BURST_SIZE 
+  // to memory.
+  localparam DATA_WAIT_TIMEOUT = 31;  
+
+
+  //---------------------------------------------------------------------------
+  // Signals
+  //---------------------------------------------------------------------------
+
+  // Command wires
+  wire                   cmd_send_imm_cf, cmd_chain_cf, cmd_reload_cf, cmd_stop_cf;
+  wire [LINES_WIDTH-1:0] cmd_num_lines_cf;
+
+  // Settings registers signals
+  wire [ ADDR_WIDTH-1:0] rec_base_addr_sr;    // Byte address
+  wire [ ADDR_WIDTH-1:0] rec_buffer_size_sr;  // Size in bytes
+  wire [ ADDR_WIDTH-1:0] play_base_addr_sr;   // Byte address
+  wire [ ADDR_WIDTH-1:0] play_buffer_size_sr; // Size in bytes
+  reg                    rec_restart;
+  reg                    rec_restart_clear;
+  wire [  CMD_WIDTH-1:0] command;
+  wire                   command_valid;
+  reg                    play_halt;
+  reg                    play_halt_clear;
+  wire [COUNT_WIDTH:0]   play_max_len_sr;
+
+  // Command FIFO
+  wire cmd_fifo_valid;
+  reg  cmd_fifo_ready;
+
+  // Record Data FIFO (Input)
+  wire [DATA_WIDTH-1:0] rec_fifo_o_tdata;
+  wire                  rec_fifo_o_tvalid;
+  wire                  rec_fifo_o_tready;
+  wire [          15:0] rec_fifo_occupied;
+
+  // Playback Data FIFO (Output)
+  wire [DATA_WIDTH-1:0] play_fifo_i_tdata; 
+  wire                  play_fifo_i_tvalid;
+  wire                  play_fifo_i_tready;
+  wire [          15:0] play_fifo_space;    // Free space in play_axi_fifo
+
+  // Buffer usage registers
+  reg [ADDR_WIDTH-1:0] rec_buffer_avail;  // Amount of free buffer space in words
+  reg [ADDR_WIDTH-1:0] rec_buffer_used;   // Amount of occupied buffer space in words
+
+
+  //---------------------------------------------------------------------------
+  // Registers
+  //---------------------------------------------------------------------------
+
+  // Record Base Address Register. Address is a byte address. This must be a 
+  // multiple of 8 bytes.
+  setting_reg #(
+    .my_addr (SR_REC_BASE_ADDR),
+    .width   (ADDR_WIDTH)
+  ) sr_rec_base_addr (
+    .clk     (clk),
+    .rst     (rst),
+    .strobe  (set_stb),
+    .addr    (set_addr),
+    .in      (set_data),
+    .out     (rec_base_addr_sr),
+    .changed ()
+  );
+
+
+  // Record Buffer Size Register. This indicates the portion of the RAM 
+  // allocated to the record buffer, in bytes. This should be a multiple of 8 
+  // bytes.
+  setting_reg #(
+    .my_addr (SR_REC_BUFFER_SIZE),
+    .width   (ADDR_WIDTH)
+  ) sr_rec_buffer_size (
+    .clk     (clk),
+    .rst     (rst),
+    .strobe  (set_stb),
+    .addr    (set_addr),
+    .in      (set_data),
+    .out     (rec_buffer_size_sr),
+    .changed ()
+  );
+
+
+  // Playback Base Address Register. Address is a byte address. This must be a 
+  // multiple of the 8 bytes.   
+  setting_reg #(
+    .my_addr (SR_PLAY_BASE_ADDR),
+    .width   (ADDR_WIDTH)
+  ) sr_play_base_addr (
+    .clk     (clk),
+    .rst     (rst),
+    .strobe  (set_stb),
+    .addr    (set_addr),
+    .in      (set_data),
+    .out     (play_base_addr_sr),
+    .changed ()
+  );
+
+
+  // Playback Buffer Size Register. This indicates the portion of the RAM 
+  // allocated to the record buffer, in bytes. This should be a multiple of 8 
+  // bytes.
+  setting_reg #(
+    .my_addr (SR_PLAY_BUFFER_SIZE),
+    .width   (ADDR_WIDTH)
+  ) sr_play_buffer_size (
+    .clk     (clk),
+    .rst     (rst),
+    .strobe  (set_stb),
+    .addr    (set_addr),
+    .in      (set_data),
+    .out     (play_buffer_size_sr),
+    .changed ()
+  );
+
+
+  // Record Buffer Restart Register. Software must write to this register after 
+  // updating the base address or buffer size. A write to this register means 
+  // we need to stop any recording in progress and reset the record buffers 
+  // according to the current buffer base address and size registers.
+  always @(posedge clk)
+  begin : sr_restart
+    if(rst) begin
+      rec_restart <= 1'b0;
+    end else begin
+      if(set_stb & (set_addr == SR_REC_RESTART)) begin
+        rec_restart <= 1'b1;
+      end else if (rec_restart_clear) begin
+        rec_restart <= 1'b0;
+      end
+    end
+  end
+
+
+  // Halt Register. A write to this register stops any replay operation as soon 
+  // as the current DRAM transaction completes.
+  always @(posedge clk)
+  begin : sr_halt
+    if(rst) begin
+      play_halt <= 1'b0;
+    end else begin
+      if(set_stb & (set_addr == SR_RX_CTRL_HALT)) begin
+        play_halt <= 1'b1;
+      end else if (play_halt_clear) begin
+        play_halt <= 1'b0;
+      end
+    end
+  end
+
+
+  // Play Command Register
+  //
+  // This register mirrors the behavior of the RFNoC RX radio block. All 
+  // commands are queued up in the replay command FIFO. The fields are as 
+  // follows.
+  //
+  //   send_imm    [31]  Send command immediately (don't use time).
+  //   
+  //   chain       [30]  When done with num_lines, immediately run next command.
+  //   
+  //   reload      [29]  When done with num_lines, rerun the same command if 
+  //                     cmd_chain is set and no new command is available.
+  //   
+  //   stop        [28]  When done with num_lines, stop transferring if 
+  //                     cmd_chain is set.
+  //   
+  //   num_lines [27:0]  Number of samples to transfer to/from block.
+  //
+  setting_reg #(
+    .my_addr (SR_RX_CTRL_COMMAND),
+    .width   (CMD_WIDTH)
+  ) sr_command (
+    .clk     (clk),
+    .rst     (rst),
+    .strobe  (set_stb),
+    .addr    (set_addr),
+    .in      (set_data),
+    .out     (command),
+    .changed (command_valid)
+  );
+
+
+  // Max Length Register. This register sets the number of words for the 
+  // maximum packet size.
+  setting_reg #(
+    .my_addr (SR_RX_CTRL_MAXLEN),
+    .width   (COUNT_WIDTH+1),
+    .at_reset({1'b1, {COUNT_WIDTH{1'b0}}})
+  ) sr_max_len (
+    .clk     (clk),
+    .rst     (rst),
+    .strobe  (set_stb),
+    .addr    (set_addr),
+    .in      (set_data),
+    .out     (play_max_len_sr),
+    .changed ()
+  );
+
+
+  // Implement register read
+  always @(*) begin
+    case (rb_addr)
+      SR_REC_BASE_ADDR    : rb_data = rec_base_addr_sr;
+      SR_REC_BUFFER_SIZE  : rb_data = rec_buffer_size_sr;
+      SR_REC_FULLNESS     : rb_data = rec_buffer_used * WORD_SIZE;
+      SR_PLAY_BASE_ADDR   : rb_data = play_base_addr_sr;
+      SR_PLAY_BUFFER_SIZE : rb_data = play_buffer_size_sr;
+      SR_RX_CTRL_MAXLEN   : rb_data = play_max_len_sr;
+      default             : rb_data = 32'h0;
+    endcase
+  end
+
+
+  //---------------------------------------------------------------------------
+  // Playback Command FIFO
+  //---------------------------------------------------------------------------
+  //
+  // This block queues up commands for playback control.
+  //
+  //---------------------------------------------------------------------------
+
+  axi_fifo_short #(
+    .WIDTH (CMD_WIDTH)
+  ) command_fifo (
+    .clk      (clk),
+    .reset    (rst),
+    .clear    (play_halt_clear),
+    .i_tdata  (command),
+    .i_tvalid (command_valid),
+    .i_tready (),
+    .o_tdata  ({cmd_send_imm_cf, cmd_chain_cf, cmd_reload_cf, cmd_stop_cf, cmd_num_lines_cf}),
+    .o_tvalid (cmd_fifo_valid),
+    .o_tready (cmd_fifo_ready),
+    .occupied (),
+    .space    ()
+  );
+
+
+  //---------------------------------------------------------------------------
+  // Record Input Data FIFO
+  //---------------------------------------------------------------------------
+  //
+  // This FIFO stores data to be recording into the RAM buffer.
+  //
+  //---------------------------------------------------------------------------
+
+  axi_fifo #(
+    .WIDTH (DATA_WIDTH),
+    .SIZE  (REC_FIFO_ADDR_WIDTH)
+  ) rec_axi_fifo (
+    .clk      (clk),
+    .reset    (rst),
+    .clear    (1'b0),
+    //
+    .i_tdata  (i_tdata),
+    .i_tvalid (i_tvalid),
+    .i_tready (i_tready),
+    //
+    .o_tdata  (rec_fifo_o_tdata),
+    .o_tvalid (rec_fifo_o_tvalid),
+    .o_tready (rec_fifo_o_tready),
+    //
+    .space    (),
+    .occupied (rec_fifo_occupied)
+  );
+
+
+  //---------------------------------------------------------------------------
+  // Record State Machine
+  //---------------------------------------------------------------------------
+
+  // FSM States
+  localparam REC_WAIT_FIFO       = 0;
+  localparam REC_CHECK_ALIGN     = 1;
+  localparam REC_DMA_REQ         = 2;
+  localparam REC_WAIT_DMA_START  = 3;
+  localparam REC_WAIT_DMA_COMMIT = 4;
+
+  // State Signals
+  reg [2:0] rec_state;
+
+  // Registers
+  reg [ADDR_WIDTH-1:0] rec_base_addr;   // Last base address pulled from settings register
+  reg [ADDR_WIDTH-1:0] rec_buffer_size; // Last buffer size pulled from settings register
+  reg [ADDR_WIDTH-1:0] rec_addr;        // Current offset into record buffer
+  reg [ADDR_WIDTH-1:0] rec_size;        // Number of words to transfer next
+  reg [ADDR_WIDTH-1:0] rec_size_0;      // Pipeline stage for computation of rec_size
+
+  reg signed [ADDR_WIDTH:0] rec_size_aligned; // rec_size reduced to not cross 4k boundary
+
+  // Timer to count how many cycles we've been waiting for new data
+  reg [$clog2(DATA_WAIT_TIMEOUT+1)-1:0] rec_wait_timer;
+  reg                                   rec_wait_timeout;
+
+  always @(posedge clk) begin
+    if (rst) begin
+      rec_state        <= REC_WAIT_FIFO;
+      rec_addr         <= 0;
+      write_ctrl_valid <= 1'b0;
+
+      rec_buffer_avail <= 0;
+      rec_buffer_used  <= 0;
+      rec_wait_timer   <= 0;
+      rec_wait_timeout <= 0;
+
+    end else begin
+
+      // Default assignments
+      rec_restart_clear <= 1'b0;
+
+      // Update wait timer
+      if (i_tvalid || !rec_fifo_occupied) begin
+        // If a new word is presented to the input FIFO, or the FIFO is empty, 
+        // then reset the timer.
+        rec_wait_timer   <= 0;
+        rec_wait_timeout <= 1'b0;
+      end else if (rec_fifo_occupied) begin
+        // If no new word is written, but there's data in the FIFO, update the 
+        // timer. Latch timeout condition when we reach out limit.
+        rec_wait_timer <= rec_wait_timer + 1;
+
+        if (rec_wait_timer == DATA_WAIT_TIMEOUT) begin
+          rec_wait_timeout <= 1'b1;
+        end
+      end
+
+      // Pre-calculate the aligned size
+      rec_size_aligned <= $signed(AXI_ALIGNMENT) - $signed(rec_addr & (AXI_ALIGNMENT-1));
+
+      //
+      // State logic
+      //
+      case (rec_state)
+
+        REC_WAIT_FIFO : begin
+          // Wait until there's enough data to initiate a transfer from the 
+          // FIFO to the RAM.
+
+          // Check if a restart was requested on the record interface
+          if (rec_restart) begin
+            rec_restart_clear <= 1'b1;
+
+            // Latch the new register values. We don't want them to change 
+            // while we're running.
+            rec_base_addr   <= rec_base_addr_sr;
+            rec_buffer_size <= rec_buffer_size_sr / WORD_SIZE;   // Store size in words
+
+            // Reset counters and address any time we update the buffer size or 
+            // base address.
+            rec_buffer_avail <= rec_buffer_size_sr / WORD_SIZE;  // Store size in words
+            rec_buffer_used  <= 0;
+            rec_addr         <= rec_base_addr_sr;
+
+          // Check if there's room left in the record RAM buffer
+          end else if (rec_buffer_used < rec_buffer_size) begin
+            // See if we can transfer a full burst
+            if (rec_fifo_occupied >= MEM_BURST_SIZE && rec_buffer_avail >= MEM_BURST_SIZE) begin
+              rec_size_0 <= MEM_BURST_SIZE;
+              rec_state  <= REC_CHECK_ALIGN;
+
+            // Otherwise, if we've been waiting a long time, see if we can 
+            // transfer less than a burst.
+            end else if (rec_fifo_occupied > 0 && rec_wait_timeout) begin
+              rec_size_0 <= (rec_fifo_occupied <= rec_buffer_avail) ?
+                            rec_fifo_occupied : rec_buffer_avail;
+              rec_state  <= REC_CHECK_ALIGN;
+            end
+          end
+        end
+
+        REC_CHECK_ALIGN : begin
+          // Check the address alignment, since AXI requires that an access not 
+          // cross 4k boundaries (boo), and the axi_dma_master doesn't handle 
+          // this automatically (boo again). 
+          rec_size <= ($signed({1'b0,rec_size_0}) > rec_size_aligned) ? 
+                      rec_size_aligned : rec_size_0;
+
+          // DMA interface is ready, so transaction will begin
+          rec_state <= REC_DMA_REQ;
+        end
+
+        REC_DMA_REQ : begin
+          // The write count written to the DMA engine should be 1 less than 
+          // the number of words you want to write (not the number of bytes).
+          write_count <= rec_size - 1;
+
+          // Create the physical RAM byte address by combining the address and 
+          // base address.
+          write_addr <= rec_addr;
+
+          // Once the interface is ready, make the DMA request
+          if (write_ctrl_ready) begin
+            // Request the write transaction
+            write_ctrl_valid <= 1'b1;
+            rec_state        <= REC_WAIT_DMA_START;
+          end
+        end
+
+        REC_WAIT_DMA_START : begin
+          // Wait until DMA interface deasserts ready, indicating it has 
+          // started on the request.
+          write_ctrl_valid <= 1'b0;
+          if (!write_ctrl_ready) begin
+            rec_state <= REC_WAIT_DMA_COMMIT;
+          end
+        end
+
+        REC_WAIT_DMA_COMMIT : begin
+          // Wait for the DMA interface to reassert write_ctrl_ready, which 
+          // signals that the DMA engine has received a response for the whole 
+          // write transaction and (we assume) it has been committed to RAM. 
+          // After this, we can update the write address and start the next 
+          // transaction.
+          if (write_ctrl_ready) begin
+             rec_addr         <= rec_addr + (rec_size * WORD_SIZE);
+             rec_buffer_used  <= rec_buffer_used + rec_size;
+             rec_buffer_avail <= rec_buffer_avail - rec_size;
+             rec_state        <= REC_WAIT_FIFO;
+          end
+        end
+
+        default : begin
+          rec_state <= REC_WAIT_FIFO;
+        end
+
+      endcase
+    end
+  end
+
+  // Connect output of record FIFO to input of DMA write interface
+  assign write_data        = rec_fifo_o_tdata;
+  assign write_data_valid  = rec_fifo_o_tvalid;
+  assign rec_fifo_o_tready = write_data_ready;
+
+
+  //---------------------------------------------------------------------------
+  // Playback State Machine
+  //---------------------------------------------------------------------------
+
+  // FSM States
+  localparam PLAY_IDLE            = 0;
+  localparam PLAY_WAIT_DATA_READY = 1;
+  localparam PLAY_SIZE_CALC       = 2;
+  localparam PLAY_DMA_REQ         = 3;
+  localparam PLAY_WAIT_DMA_START  = 4;
+  localparam PLAY_WAIT_DMA_COMMIT = 5;
+  localparam PLAY_DONE_CHECK      = 6;
+
+  // State Signals
+  reg [2:0] play_state;
+
+  // Registers
+  reg [ADDR_WIDTH-1:0] play_base_addr;    // Last base address pulled from settings register
+  reg [ADDR_WIDTH-1:0] play_buffer_size;  // Last buffer size pulled from settings register
+  reg [ADDR_WIDTH-1:0] play_addr;         // Current byte offset into record buffer
+  reg [ADDR_WIDTH-1:0] play_addr_0;       // Pipeline stage for computing play_addr
+  reg [ADDR_WIDTH-1:0] play_addr_1;       // Pipeline stage for computing play_addr
+  reg [ADDR_WIDTH-1:0] play_buffer_end;   // Address of location after end of buffer
+  reg [ADDR_WIDTH-1:0] max_dma_size;      // Maximum size of next transfer, in words
+  //
+  reg [LINES_WIDTH-1:0] cmd_num_lines;        // Copy of cmd_num_lines from last command
+  reg [LINES_WIDTH-1:0] play_words_remaining; // Number of lines left to read for command
+  reg                   cmd_chain;            // Copy of cmd_chain from last command
+  reg                   cmd_reload;           // Copy of cmd_reload from last command
+
+  reg play_full_burst_avail;     // True if we there's a full burst to read
+  reg play_buffer_avail_nonzero; // True if > 0
+  reg cmd_num_lines_cf_nonzero;  // True if > 0
+  reg max_dma_size_ok;           // True if it's OK to read max_dma_size
+
+  reg [ADDR_WIDTH-1:0] max_dma_size_m1;         // max_dma_size - 1
+  reg [ADDR_WIDTH-1:0] play_words_remaining_m1; // play_words_remaining - 1
+
+  reg [ADDR_WIDTH-1:0] play_buffer_avail;   // Number of words left to read in record buffer
+  reg [ADDR_WIDTH-1:0] play_buffer_avail_0; // Pipeline stage for computing play_buffer_avail
+
+  always @(posedge clk)
+  begin
+    if (rst) begin
+      play_state     <= PLAY_IDLE;
+      cmd_fifo_ready <= 1'b0;
+
+    end else begin
+      
+      // Calculate how many words are left to read from the record buffer
+      play_full_burst_avail     <= (play_buffer_avail >= MEM_BURST_SIZE);
+      play_buffer_avail_nonzero <= (play_buffer_avail > 0);
+      cmd_num_lines_cf_nonzero  <= (cmd_num_lines_cf > 0);
+      play_buffer_end           <= play_base_addr_sr + play_buffer_size_sr;
+
+      // Default values
+      cmd_fifo_ready  <= 1'b0;
+      read_ctrl_valid <= 1'b0;
+      play_halt_clear <= 1'b0;
+
+      //
+      // State logic
+      //
+      case (play_state)
+        PLAY_IDLE : begin
+          // Always start reading at the start of the record buffer
+          play_addr <= play_base_addr_sr;
+
+          // Save off command info, in case we need to repeat the command
+          cmd_num_lines    <= cmd_num_lines_cf;
+          cmd_reload       <= cmd_reload_cf;
+          cmd_chain        <= cmd_chain_cf;
+
+          // Save the buffer info so it doesn't update during playback
+          play_base_addr    <= play_base_addr_sr;
+          play_buffer_size  <= play_buffer_size_sr;
+          play_buffer_avail <= play_buffer_size_sr / WORD_SIZE;
+
+          // Wait until we receive a command and we have enough data recorded 
+          // to honor it.
+          if (cmd_fifo_valid && ~play_halt_clear) begin
+            // Load the number of word remaining to complete this command
+            play_words_remaining <= cmd_num_lines_cf;
+
+            // We don't support time yet, so we require send_imm to do 
+            // anything. Also, we can't do anything until we have data recorded.
+            if (cmd_stop_cf) begin
+              // Do nothing, except clear command from the FIFO
+              cmd_fifo_ready <= 1'b1;
+            end else if (cmd_send_imm_cf
+                         && play_buffer_avail_nonzero 
+                         && cmd_num_lines_cf_nonzero) begin
+              // Dequeue the command from the FIFO
+              cmd_fifo_ready <= 1'b1;
+
+              play_state <= PLAY_WAIT_DATA_READY;
+            end
+          end else if (play_halt) begin
+            // In case we get a HALT after a command has finished
+            play_halt_clear <= 1'b1;
+          end
+        end
+
+        PLAY_WAIT_DATA_READY : begin
+          // Save the maximum size we can read from RAM
+          max_dma_size  <= play_full_burst_avail ? MEM_BURST_SIZE : play_buffer_avail;
+
+          // Check if we got a halt command while waiting
+          if (play_halt) begin
+            play_halt_clear <= 1'b1;
+            play_state      <= PLAY_IDLE;
+
+          // Wait for output FIFO to empty sufficiently so we can read an 
+          // entire burst at once. This may be more space than needed, but we 
+          // won't know the exact size until the next state.
+          end else if (play_fifo_space >= MEM_BURST_SIZE) begin
+            play_state <= PLAY_SIZE_CALC;
+          end
+        end
+
+        PLAY_SIZE_CALC : begin
+          // Do some intermediate calculations to determine what the read_count 
+          // should be.
+          play_words_remaining_m1 <= play_words_remaining-1;
+          max_dma_size_m1         <= max_dma_size-1;
+          max_dma_size_ok         <= play_words_remaining >= max_dma_size;
+          play_state              <= PLAY_DMA_REQ;
+        end
+
+        PLAY_DMA_REQ : begin
+          // Load the size of the next read into a register. We try to read the 
+          // max amount available (up to the burst size) or however many words 
+          // are needed to reach the end of the RAM buffer.
+          //
+          // The read count written to the DMA engine should be 1 less than the 
+          // number of words you want to read (not the number of bytes).
+          read_count <= max_dma_size_ok ? max_dma_size_m1 : play_words_remaining_m1;
+
+          // Load the address to read. Note that we don't do an alignment check 
+          // since we assume that multiples of MEM_BURST_SIZE meet the 
+          // AXI_ALIGNMENT requirement.
+          read_addr <= play_addr;
+
+          // Request the read transaction as soon as DMA interface is ready
+          if (read_ctrl_ready) begin
+            read_ctrl_valid <= 1'b1;
+            play_state      <= PLAY_WAIT_DMA_START;
+          end
+        end
+
+        PLAY_WAIT_DMA_START : begin
+          // Wait until DMA interface deasserts ready, indicating it has 
+          // started on the request.
+          read_ctrl_valid <= 1'b0;
+          if (!read_ctrl_ready) begin
+            // Update values for next transaction
+            play_addr_0          <= play_addr + ({{(ADDR_WIDTH-COUNT_WIDTH){1'b0}}, read_count} + 1) * WORD_SIZE;
+            play_words_remaining <= play_words_remaining - ({1'b0, read_count} + 1);
+            play_buffer_avail_0  <= play_buffer_avail - ({1'b0, read_count} + 1);
+
+            play_state <= PLAY_WAIT_DMA_COMMIT;
+          end
+        end
+
+        PLAY_WAIT_DMA_COMMIT : begin
+          // Wait for the DMA interface to reassert read_ctrl_ready, which 
+          // signals that the DMA engine has received a response for the whole 
+          // read transaction.
+          if (read_ctrl_ready) begin
+            // Check if we need to wrap the address for the next transaction
+            if (play_addr_0 >= play_buffer_end) begin
+              play_addr_1       <= play_base_addr_sr;
+              play_buffer_avail <= play_buffer_size_sr / WORD_SIZE;
+            end else begin
+              play_addr_1       <= play_addr_0;
+              play_buffer_avail <= play_buffer_avail_0;
+            end
+
+            play_state <= PLAY_DONE_CHECK;
+          end
+        end
+
+        PLAY_DONE_CHECK : begin
+          play_addr <= play_addr_1;
+
+          // Check if we have more data to transfer for this command
+          if (play_words_remaining) begin
+            play_state <= PLAY_WAIT_DATA_READY;
+
+          // Check if we're chaining
+          end else if (cmd_chain) begin
+            // Check if there's a new command waiting
+            if (cmd_fifo_valid) begin
+              // Load the next command. Note that we don't reset the playback 
+              // address when commands are chained together.
+              play_words_remaining <= cmd_num_lines_cf;
+              cmd_num_lines        <= cmd_num_lines_cf;
+              cmd_reload           <= cmd_reload_cf;
+              cmd_chain            <= cmd_chain_cf;
+
+              // Dequeue the command from the FIFO
+              cmd_fifo_ready <= 1'b1;
+
+              // Stop if it's a stop command, otherwise restart
+              if (cmd_stop_cf) begin
+                play_state <= PLAY_IDLE;
+              end else begin
+                play_state <= PLAY_WAIT_DATA_READY;
+              end
+
+            // Check if we need to restart the previous command
+            end else if (cmd_reload) begin
+              play_words_remaining <= cmd_num_lines;
+              play_state           <= PLAY_WAIT_DATA_READY;
+            end
+          // Nothing left to do
+          end else begin
+            play_state <= PLAY_IDLE;
+          end
+        end
+      endcase
+
+    end
+  end
+
+  // Connect output of DMA master to playback data FIFO
+  assign play_fifo_i_tdata  = read_data;
+  assign play_fifo_i_tvalid = read_data_valid;
+  assign read_data_ready    = play_fifo_i_tready;
+
+
+  //---------------------------------------------------------------------------
+  // TLAST Generation
+  //---------------------------------------------------------------------------
+  //
+  // This block monitors the signals to/from the DMA master and generates the 
+  // TLAST signal. We assert TLAST at the end of every read transaction and 
+  // after every play_max_len_sr words, so that no packets are longer than the 
+  // length indicated by the max_len register.
+  //
+  // The timing of this block relies on the fact that read_ctrl_ready is not 
+  // reasserted by the DMA master until after TLAST gets asserted.
+  //
+  //---------------------------------------------------------------------------
+
+  reg [COUNT_WIDTH-1:0] read_counter;
+  reg [COUNT_WIDTH-1:0] length_counter;
+  reg                   play_fifo_i_tlast;
+
+  always @(posedge clk)
+  begin
+    if (rst) begin
+      play_fifo_i_tlast <= 1'b0;
+    end else begin
+      // Check if we're requesting a read transaction
+      if (read_ctrl_valid && read_ctrl_ready) begin
+        // Initialize read_counter for new transaction
+        read_counter   <= read_count;
+        length_counter <= play_max_len_sr;
+
+        // If read_count is 0, then the first word is also the last word
+        if (read_count == 0) begin
+          play_fifo_i_tlast <= 1'b1;
+        end
+
+      // Track the number of words read out by DMA master
+      end else if (read_data_valid && read_data_ready) begin
+        read_counter   <= read_counter - 1;
+        length_counter <= length_counter - 1;        
+
+        // Check if the word currently being output is the last word of a 
+        // packet, which means we need to clear tlast. 
+        if (play_fifo_i_tlast) begin
+          // But make sure that the next word isn't also the last of a DMA 
+          // burst, for which we will need to keep tlast asserted.
+          if (read_counter != 1) begin
+            play_fifo_i_tlast <= 1'b0;
+          end
+          
+          // Restart length counter
+          length_counter <= play_max_len_sr;
+
+        // Check if the next word to be output should be the last of a packet.
+        end else if (read_counter == 1 || length_counter == 2) begin
+          play_fifo_i_tlast <= 1'b1;
+        end 
+      end
+
+    end
+  end
+
+
+  //---------------------------------------------------------------------------
+  // Playback Output Data FIFO 
+  //---------------------------------------------------------------------------
+  //
+  // This FIFO buffers data that has been read out of RAM as part of a playback 
+  // operation.
+  //
+  //---------------------------------------------------------------------------
+
+  axi_fifo #(
+    .WIDTH (DATA_WIDTH+1),
+    .SIZE  (PLAY_FIFO_ADDR_WIDTH)
+  ) play_axi_fifo (
+    .clk      (clk),
+    .reset    (rst),
+    .clear    (1'b0),
+    //
+    .i_tdata  ({play_fifo_i_tlast, play_fifo_i_tdata}),
+    .i_tvalid (play_fifo_i_tvalid),
+    .i_tready (play_fifo_i_tready),
+    //
+    .o_tdata  ({o_tlast, o_tdata}),
+    .o_tvalid (o_tvalid),
+    .o_tready (o_tready),
+    //
+    .space    (play_fifo_space),
+    .occupied ()
+  );
+
+endmodule
\ No newline at end of file
diff --git a/fpga/usrp3/lib/axi/axi_strip_preamble.v b/fpga/usrp3/lib/axi/axi_strip_preamble.v
new file mode 100644
index 000000000..b4a911880
--- /dev/null
+++ b/fpga/usrp3/lib/axi/axi_strip_preamble.v
@@ -0,0 +1,296 @@
+//
+// Copyright 2016 Ettus Research LLC
+// Copyright 2018 Ettus Research, a National Instruments Company
+//
+// SPDX-License-Identifier: LGPL-3.0-or-later
+//
+// Strips preamble, EOP, and CRC/num_words check
+// <preamble> <packet> <EOP> [control_chksum,word_count,payload_chksum] 
+// <preamble>   = 64'h9E6774129E677412
+// <EOP>        = 64'h2A1D632F2A1D632F
+
+module axi_strip_preamble #(
+  parameter WIDTH=64,
+  parameter MAX_PKT_SIZE=512 //Set to 128 in sim to fill up buffers faster to help try and trigger more fail cases. 
+) (
+   input clk,
+   input reset,
+   input clear,
+   //
+   input [WIDTH-1:0] i_tdata,
+   input i_tvalid,
+   output i_tready,
+   //
+   output [WIDTH-1:0] o_tdata,
+   output o_tlast,
+   output o_tvalid,
+   input  o_tready,
+   //
+   output pkt_dropped,
+   output crc_err,
+   output crit_error
+);
+
+    function [0:0] cvita_get_has_time;
+        input [63:0] header;
+        cvita_get_has_time = header[61];
+    endfunction
+    
+   //State machine info
+   reg [1:0] state, next_state;
+   
+   localparam IDLE = 0;
+   localparam CHECK_HDR = 1; 
+   localparam PASS = 2;
+   localparam CHECK_CRC = 3;
+   
+   localparam PAYLOAD_WORDCOUNT_WIDTH = 16;
+   localparam PAYLOAD_CHKSUM_WIDTH = 32;
+   localparam CONTROL_CHKSUM_WIDTH = 16;  
+
+   //Note that held_word is required when EOP is detected
+   //so that we can rewrite into memory the last word + last bit
+   reg [WIDTH-1:0] held_word;
+   reg [WIDTH-1:0] held_word_r;
+   always @(posedge clk) begin
+    if(i_tvalid && i_tready) begin
+        held_word <= i_tdata;
+        held_word_r <= held_word;
+    end
+   end   
+   
+   //Look for next word that specifies if frame has timestamp
+   reg [PAYLOAD_WORDCOUNT_WIDTH-1:0] cntrl_length = 16'd2;
+   always @(posedge clk) begin 
+     if ((next_state == CHECK_HDR || state == CHECK_HDR) && i_tvalid)
+         cntrl_length <= cvita_get_has_time(i_tdata) ? 16'd2 : 16'd1;
+   end
+
+   reg [PAYLOAD_WORDCOUNT_WIDTH-1:0] word_count;
+   wire det_preamble =  (i_tdata == 64'h9E6774129E677412);
+   wire det_eop =       (i_tdata == 64'h2A1D632F2A1D632F);
+
+   wire [PAYLOAD_CHKSUM_WIDTH-1:0] payload_chksum;
+   wire [CONTROL_CHKSUM_WIDTH-1:0] control_chksum;
+   
+   // Payload LFSR. Must hold LFSR once detected EOP so checksum does not keep updating after EOP
+   // Note the payload LFSR also includes the EOP in its checksum
+   crc_xnor #(.INPUT_WIDTH(WIDTH), .OUTPUT_WIDTH(PAYLOAD_CHKSUM_WIDTH)) payload_chksum_gen (
+      .clk(clk), .rst(word_count<=cntrl_length), .hold(~(i_tready && i_tvalid) || det_eop || state == CHECK_CRC),
+      .input_data(i_tdata), .crc_out(payload_chksum)
+   );
+   
+   // Control LFSR. Varies in size based on whether the control information includes a timestamp
+   // Hold the LFSR once the control word(s) have been parsed
+   crc_xnor #(.INPUT_WIDTH(WIDTH), .OUTPUT_WIDTH(CONTROL_CHKSUM_WIDTH)) control_chksum_gen (
+      .clk(clk), .rst(word_count=='d0), .hold(~(i_tready && i_tvalid) || word_count>=cntrl_length),
+      .input_data(i_tdata), .crc_out(control_chksum)
+   );    
+   
+   //Good frame is when the word_count is correct and the control checksum passes. 
+   //Allows passthrough of payloads with bit errors to reduce overall dropped frame rate
+   wire frame_good = (word_count == i_tdata[47:32]) && (control_chksum == i_tdata[63:48]) && state == CHECK_CRC;
+   
+   //CRC error only increments when the state machine makes it to CHECK_CRC state
+   //It will not increment if a preamble or eop is detected outside of IDLE
+   wire payload_crc_check = (payload_chksum == i_tdata[31:0]) && state == CHECK_CRC;
+   assign crc_err = (~frame_good || ~payload_crc_check) && state == CHECK_CRC && i_tvalid;
+
+   //Increment word_count for payload and EOP
+   always @(posedge clk) begin
+      if (state == IDLE || pkt_dropped) begin
+         word_count <= 0;
+      end else if ((state == PASS || state == CHECK_HDR) && i_tready && i_tvalid) begin
+         word_count <= word_count+1'b1;
+      end
+   end
+
+   always @(posedge clk) begin
+      if (reset | clear) begin
+         state <= IDLE;
+      end else begin
+         state <= next_state;
+      end
+   end
+   
+   //Only drop packet if preamble detected outside of idle or bad frame was detected during CRC check
+   assign pkt_dropped = ((state != IDLE) && det_preamble && i_tvalid) || ((state == CHECK_CRC) && ~frame_good && i_tvalid);
+
+   //When preamble is missing or has bit error, state machine stays in IDLE
+   //When EOP is missing or has bit error, either the next preamble is detected and resets logic
+   //or state machine exits on next EOP and fails CRC check.
+   //For cables with very high BER its possible for the write buffer to fill up which causes a critical error and resets everything
+   always @(*) begin
+      case(state)
+         IDLE: begin
+            if (det_preamble && i_tvalid) //Preamble detected so check to see if timestamp is part of header
+               next_state = CHECK_HDR;
+            else
+               next_state = IDLE;
+         end 
+         
+         //Check incoming word to see if frame will have timestamp
+         CHECK_HDR: begin
+            if(crit_error) begin //Critical error so reset SM
+                next_state = IDLE; 
+            end else if(~det_preamble && i_tvalid && i_tready) begin //Found control word so go to normal pass state
+                next_state = PASS; 
+            end else begin
+                next_state = CHECK_HDR;
+            end
+         end             
+         
+         //Note if early preamble is detected in PASS state everything is reset for the next frame
+         PASS: begin
+            if(crit_error) begin //Critical error so reset SM
+                next_state = IDLE;
+            end else if(det_preamble && i_tvalid) begin //Saw preamble so drop packet and start over
+                next_state = CHECK_HDR;  
+            end else if(det_eop && i_tvalid && i_tready) begin //Saw EOP so check for crc on next word
+                next_state = CHECK_CRC; 
+            end else begin
+                next_state = PASS;
+            end
+         end     
+         
+         //Check for crc and go to idle or go back to pass if another preamble is detected
+         CHECK_CRC: begin
+             if(crit_error) begin //Critical error so reset SM
+                 next_state = IDLE; 
+             end else if(det_preamble && i_tvalid) begin //Saw preamble so drop packet and start over
+                 next_state = CHECK_HDR; 
+             end else if(i_tvalid) begin //Got word which should've been the CRC
+                 next_state = IDLE;
+             end else begin
+                 next_state = CHECK_CRC;
+             end
+          end
+         
+         default: begin
+            next_state = IDLE;
+         end
+
+      endcase
+   end 
+
+   wire [WIDTH-1:0] buf_tdata;
+   wire buf_tlast, buf_tvalid, buf_tready, buf_empty;
+   reg buf_full = 1'b0;
+   wire [$clog2(MAX_PKT_SIZE)-1:0] valid_rd_addr;
+   reg buf_empty_r;
+
+   assign mem_tvalid = (state == PASS || state == CHECK_HDR) ? (i_tvalid && ~pkt_dropped) : 1'b0;
+   assign i_tready = (state == PASS || state == CHECK_HDR) ?  buf_tready : 1'b1;
+   
+   assign crit_error = buf_full && buf_empty; //This should never happen, if it does that indicates poor BER over Aurora or packet size too large
+   
+   /////////////////////////////////////////////////
+   //Fifo to store incoming packets
+   //The write pntr rewinds whenever an error occurs
+   /////////////////////////////////////////////////
+   
+   wire int_tready;
+
+   reg [$clog2(MAX_PKT_SIZE)-1:0] wr_addr, prev_wr_addr, rd_addr, old_rd_addr;
+   reg [$clog2(MAX_PKT_SIZE):0] in_pkt_cnt, out_pkt_cnt;
+   wire read         = ~buf_empty && (int_tready || buf_empty_r); //Read from buffer if its no longer empty to prime output reg
+   wire almost_full  = (wr_addr == valid_rd_addr-1'b1); //We need to look at the masked rd_addr in case its 1 ahead
+
+   assign buf_tready   = ~buf_full;
+   wire write        = mem_tvalid && buf_tready && ~det_eop;
+   
+   //If frame was good we need to go back and rewrite the last word and set the last bit
+   wire [WIDTH:0] int_write_data = (frame_good) ? {1'b1,held_word_r} : {1'b0,i_tdata};
+   wire [$clog2(MAX_PKT_SIZE)-1:0] int_wr_addr = (frame_good) ? wr_addr-1 : wr_addr;
+   
+   //BRAM inferred
+   wire [WIDTH:0] buf_data;
+    ram_2port #(.DWIDTH(WIDTH+1), .AWIDTH($clog2(MAX_PKT_SIZE))) pkt_buf
+      (.clka(clk), .ena(1'b1), .wea(1'b1), .addra(int_wr_addr),
+       .dia(int_write_data), .doa(),
+       .clkb(clk), .enb(read), .web(1'b0), .addrb(rd_addr), .dib(),
+       .dob(buf_data));
+
+   // Write logic
+   always @(posedge clk) begin
+        
+        // Rewind logic
+        if(pkt_dropped || crit_error)
+            wr_addr <= prev_wr_addr;
+        else if(write)
+            wr_addr <= wr_addr + 1'b1;
+        
+        if (almost_full) begin
+            if (write && ~read) begin
+                buf_full       <= 1'b1;
+            end
+        end else begin
+            if (~write && read) begin
+                buf_full       <= 1'b0;
+            end
+        end
+            
+        if (frame_good) begin
+            in_pkt_cnt   <= in_pkt_cnt + 1'b1;
+            prev_wr_addr <= wr_addr;
+        end
+           
+        if (reset || clear) begin
+            wr_addr       <= 0;
+            prev_wr_addr  <= 0;
+            in_pkt_cnt    <= 0;
+        end
+        
+        if(reset || clear || crit_error) begin
+            buf_full          <= 1'b0;
+        end
+   end
+
+   // Read logic. Hold data if pkt_count is equal
+   assign buf_empty = in_pkt_cnt == out_pkt_cnt;
+   reg last_word;
+   
+   //Use current read addr only if read is enabled
+   assign valid_rd_addr = (read) ? rd_addr : old_rd_addr;
+   
+   assign buf_tvalid = ~buf_empty_r && ~(last_word && buf_empty);
+   
+   assign buf_tdata = buf_data[WIDTH-1:0];
+   assign buf_tlast = buf_data[WIDTH];
+
+   always @(posedge clk) begin
+     buf_empty_r <= buf_empty;
+     
+     if (read) old_rd_addr <= rd_addr; //Keeps track of last valid rd_addr
+     
+     //Last word has two possibilities
+     //If buffer empty then we need to rewind rd_addr and mask reading from buffer
+     //If buffer is not empty continue with rd_addr and continue reading from buffer
+     last_word <= buf_tvalid && int_tready && buf_tlast;
+     
+     //Need to rewind rd_addr since it incremented one too far
+     //This means there will be one cycle where rd_addr is ahead of where it should be
+     //Other logic that uses rd_addr will have it masked for that cycle
+     if (last_word && buf_empty)    rd_addr      <= rd_addr - 1;
+     else if (read)                 rd_addr      <= rd_addr + 1;
+     
+     // Prevent output until we have a full packet
+     if (buf_tvalid && int_tready && buf_tlast) begin
+       out_pkt_cnt  <= out_pkt_cnt + 1'b1;
+     end
+     
+     if (reset || clear) begin
+       old_rd_addr <= 0;
+       rd_addr     <= 0;
+       out_pkt_cnt <= 0;
+     end
+   end
+   
+   assign o_tlast = buf_tlast;
+   assign o_tdata = buf_tdata;
+   assign o_tvalid = buf_tvalid;
+   assign int_tready = o_tready;
+
+endmodule
+
+  
diff --git a/fpga/usrp3/lib/axi/axi_to_strobed.v b/fpga/usrp3/lib/axi/axi_to_strobed.v
new file mode 100644
index 000000000..9703be5bf
--- /dev/null
+++ b/fpga/usrp3/lib/axi/axi_to_strobed.v
@@ -0,0 +1,52 @@
+//
+// Copyright 2016 Ettus Research
+// Copyright 2018 Ettus Research, a National Instruments Company
+//
+// SPDX-License-Identifier: LGPL-3.0-or-later
+//
+// Convert AXI Stream to a strobed interface.
+// Note: Not especially useful if simply wanting to set
+//       
+
+module axi_to_strobed #(
+  parameter WIDTH = 32,
+  parameter FIFO_SIZE = 1,
+  parameter MIN_RATE = 256
+)(
+  input clk, input reset, input clear,
+  input [$clog2(MIN_RATE):0] out_rate,  // Number of clock cycles between strobes
+  input ready,
+  output error,                           // Output strobe but no data
+  input [WIDTH-1:0] i_tdata, input i_tvalid, input i_tlast, output i_tready,
+  output out_stb, output out_last, output [WIDTH-1:0] out_data
+);
+
+  reg strobe;
+  wire valid;
+  reg [$clog2(MIN_RATE):0] counter = 1;
+  always @(posedge clk) begin
+    if (reset | clear) begin
+      strobe    <= 1'b0;
+      counter   <= 1;
+    end else if (ready) begin
+      if (counter >= out_rate) begin
+        strobe  <= 1'b1;
+        counter <= 1;
+      end else begin
+        strobe  <= 1'b0;
+        counter <= counter + 1'b1;
+      end
+    end else begin
+      strobe <= 1'b0;
+    end
+  end
+
+  axi_fifo #(.WIDTH(WIDTH+1), .SIZE(FIFO_SIZE)) axi_fifo (
+    .clk(clk), .reset(reset), .clear(clear),
+    .i_tdata({i_tlast,i_tdata}), .i_tvalid(i_tvalid), .i_tready(i_tready),
+    .o_tdata({out_last,out_data}), .o_tvalid(valid), .o_tready(strobe),
+    .space(), .occupied());
+
+  assign out_stb = valid & strobe;
+  assign error = ~valid & strobe;
+endmodule
\ No newline at end of file
diff --git a/fpga/usrp3/lib/axi/axis_data_swap.v b/fpga/usrp3/lib/axi/axis_data_swap.v
new file mode 100644
index 000000000..2408ab6c6
--- /dev/null
+++ b/fpga/usrp3/lib/axi/axis_data_swap.v
@@ -0,0 +1,125 @@
+//
+// Copyright 2019 Ettus Research, A National Instruments Company
+//
+// SPDX-License-Identifier: LGPL-3.0-or-later
+//
+// Module: axis_data_swap
+// Description:
+//  A generic data swapper module for AXI-Stream. The contents of
+//  tdata are swapped based on the tswap signal. For each bit 'i'
+//  in tswap, adjacent words of width 2^i are swapped if tswap[i]
+//  is high. For example, if tswap[3] = 1, then each byte in tdata
+//  will be swapped with its adjacent neighbor. It is permissible
+//  for tswap to change for each transfer in an AXIS packet.
+//  Swapping can also be configured to be static (zero logic) by
+//  setting DYNAMIC = 0. To reduce area, certain swap stages can
+//  even be disabled. For example, if STAGES_EN[2:0] is set to 0
+//  then the lowest granularity for swaps will be a byte.
+//
+// Parameters:
+//   - DATA_W: Width of the tdata bus in bits
+//   - USER_W: Width of the tuser bus in bits
+//   - STAGES_EN: Which swap stages are enabled.
+//   - DYNAMIC: Dynamic swapping enabled (use tswap)
+//
+// Signals:
+//   - s_axis_*: The input AXI stream
+//   - m_axis_*: The output AXI stream
+//
+
+module axis_data_swap #(
+  parameter integer              DATA_W    = 256,
+  parameter integer              USER_W    = 1,
+  parameter [$clog2(DATA_W)-1:0] STAGES_EN = 'hFFFFFFFF, //@HACK: Vivado does not allow $clog2 in value of this expr
+  parameter [0:0]                DYNAMIC   = 1
+)(
+  // Clock and Reset
+  input  wire                      clk,
+  input  wire                      rst,
+  // Input AXIS
+  input  wire [DATA_W-1:0]         s_axis_tdata,
+  input  wire [$clog2(DATA_W)-2:0] s_axis_tswap,
+  input  wire [USER_W-1:0]         s_axis_tuser,
+  input  wire                      s_axis_tlast,
+  input  wire                      s_axis_tvalid,
+  output wire                      s_axis_tready,
+  // Output AXIS                   
+  output wire [DATA_W-1:0]         m_axis_tdata,
+  output wire [USER_W-1:0]         m_axis_tuser,
+  output wire                      m_axis_tlast,
+  output wire                      m_axis_tvalid,
+  input  wire                      m_axis_tready
+);
+
+  parameter SWAP_STAGES = $clog2(DATA_W);
+  parameter SWAP_W      = $clog2(DATA_W)-1;
+  genvar s, w;
+
+  wire [DATA_W-1:0] stg_tdata [0:SWAP_STAGES], stg_tdata_swp[0:SWAP_STAGES], stg_tdata_res[0:SWAP_STAGES];
+  wire [SWAP_W-1:0] stg_tswap [0:SWAP_STAGES];
+  wire [USER_W-1:0] stg_tuser [0:SWAP_STAGES];
+  wire              stg_tlast [0:SWAP_STAGES];
+  wire              stg_tvalid[0:SWAP_STAGES];
+  wire              stg_tready[0:SWAP_STAGES];
+
+  // Connect input and output to stage wires
+  generate
+    assign stg_tdata [0] = s_axis_tdata;
+    assign stg_tswap [0] = s_axis_tswap;
+    assign stg_tuser [0] = s_axis_tuser;
+    assign stg_tlast [0] = s_axis_tlast;
+    assign stg_tvalid[0] = s_axis_tvalid;
+    assign s_axis_tready = stg_tready[0];
+
+    assign m_axis_tdata            = stg_tdata [SWAP_STAGES];
+    assign m_axis_tuser            = stg_tuser [SWAP_STAGES];
+    assign m_axis_tlast            = stg_tlast [SWAP_STAGES];
+    assign m_axis_tvalid           = stg_tvalid[SWAP_STAGES];
+    assign stg_tready[SWAP_STAGES] = m_axis_tready;
+  endgenerate
+
+  // Instantiate AXIS flip-flops for each stage
+  generate
+    for (s = 0; s < SWAP_STAGES; s=s+1) begin
+      if (STAGES_EN[SWAP_STAGES-s-1]) begin
+        // Swap Logic
+        for (w = 0; w < (1<<s); w=w+1) begin
+          assign stg_tdata_swp[s][(w*(DATA_W/(1<<s)))+:(DATA_W/(1<<s))] =
+            stg_tdata[s][(((1<<s)-w-1)*(DATA_W/(1<<s)))+:(DATA_W/(1<<s))];
+        end
+        if (DYNAMIC) begin
+          // Honor tswap in DYNAMIC mode.
+          // Also add a flip_flop to break the long start-to-end critical path
+          assign stg_tdata_res[s] = (s > 0 && stg_tswap[s][SWAP_W-s]) ?
+            stg_tdata_swp[s] : stg_tdata[s];
+          // Flip-flop
+          axi_fifo_flop #(.WIDTH(DATA_W+SWAP_W+USER_W+1)) reg_i (
+            .clk(clk), .reset(rst), .clear(1'b0),
+            .i_tdata({stg_tlast[s], stg_tuser[s], stg_tswap[s], stg_tdata_res[s]}),
+            .i_tvalid(stg_tvalid[s]), .i_tready(stg_tready[s]),
+            .o_tdata({stg_tlast[s+1], stg_tuser[s+1], stg_tswap[s+1], stg_tdata[s+1]}),
+            .o_tvalid(stg_tvalid[s+1]), .o_tready(stg_tready[s+1]),
+            .occupied(), .space()
+          );
+        end else begin
+          // Static swapping logic
+          assign stg_tdata [s+1] = stg_tdata_swp[s];
+          assign stg_tswap [s+1] = stg_tswap    [s];
+          assign stg_tuser [s+1] = stg_tuser    [s];
+          assign stg_tlast [s+1] = stg_tlast    [s];
+          assign stg_tvalid[s+1] = stg_tvalid   [s];
+          assign stg_tready[s]   = stg_tready   [s+1];
+        end
+      end else begin
+        // Skip this stage
+        assign stg_tdata [s+1] = stg_tdata [s];
+        assign stg_tswap [s+1] = stg_tswap [s];
+        assign stg_tuser [s+1] = stg_tuser [s];
+        assign stg_tlast [s+1] = stg_tlast [s];
+        assign stg_tvalid[s+1] = stg_tvalid[s];
+        assign stg_tready[s]   = stg_tready[s+1];
+      end
+    end
+  endgenerate
+
+endmodule // axis_data_swap
diff --git a/fpga/usrp3/lib/axi/axis_downsizer.v b/fpga/usrp3/lib/axi/axis_downsizer.v
new file mode 100644
index 000000000..aa8426e5f
--- /dev/null
+++ b/fpga/usrp3/lib/axi/axis_downsizer.v
@@ -0,0 +1,96 @@
+//
+// Copyright 2018 Ettus Research, A National Instruments Company
+//
+// SPDX-License-Identifier: LGPL-3.0-or-later
+//
+// Module: axis_downsizer
+// Description: 
+//   An AXI-Stream width conversion module that narrows the input
+//   sample with by a factor of RATIO.
+//   NOTE: This module has end-to-end combanitorial paths. For a
+//         pipelined version, please use axis_width_conv
+//
+// Parameters:
+//   - OUT_DATA_W: The bitwidth of the output data bus. The width of the
+//                 input data bus is OUT_DATA_W*RATIO
+//   - OUT_USER_W: The bitwidth of the output user bus. The width of the
+//                 input user bus is OUT_USER_W*RATIO
+//   - RATIO: The downsizing ratio
+//
+// Signals:
+//   - s_axis_* : Input sample stream (AXI-Stream)
+//   - m_axis_* : Output sample stream (AXI-Stream)
+
+module axis_downsizer #(
+  parameter OUT_DATA_W  = 32,
+  parameter OUT_USER_W  = 1,
+  parameter RATIO       = 4
+)(
+  // Clock, reset and settings
+  input  wire                          clk,              // Clock
+  input  wire                          reset,            // Reset
+  // Data In (AXI-Stream)              
+  input  wire [(OUT_DATA_W*RATIO)-1:0] s_axis_tdata,     // Input stream tdata
+  input  wire [(OUT_USER_W*RATIO)-1:0] s_axis_tuser,     // Input stream tuser
+  input  wire [RATIO-1:0]              s_axis_tkeep,     // Input stream tkeep
+  input  wire                          s_axis_tlast,     // Input stream tlast
+  input  wire                          s_axis_tvalid,    // Input stream tvalid
+  output wire                          s_axis_tready,    // Input stream tready
+  // Data Out (AXI-Stream)             
+  output wire [OUT_DATA_W-1:0]         m_axis_tdata,     // Output stream tdata
+  output wire [OUT_USER_W-1:0]         m_axis_tuser,     // Output stream tuser
+  output wire                          m_axis_tlast,     // Output stream tlast
+  output wire                          m_axis_tvalid,    // Output stream tvalid
+  input  wire                          m_axis_tready     // Output stream tready
+);
+
+  genvar i;
+  generate if (RATIO != 1) begin
+    // Constants
+    localparam [$clog2(RATIO)-1:0] SEL_FIRST  = 'd0;
+    localparam [$clog2(RATIO)-1:0] SEL_LAST   = RATIO-1;
+    localparam [RATIO-1:0]         KEEP_FIRST = {{(RATIO-1){1'b0}}, 1'b1};
+    localparam [RATIO-1:0]         KEEP_ALL   = {(RATIO){1'b1}};
+
+    // Keep a binary-coded and one-hot version of the current
+    // section of the input that is being processed.
+    reg [$clog2(RATIO)-1:0] select = SEL_FIRST;
+    reg [RATIO-1:0]         keep   = KEEP_FIRST;
+
+    // State machine to drive the select bits for the
+    // input selection MUX.
+    always @(posedge clk) begin
+      if (reset) begin
+        select <= SEL_FIRST;
+        keep   <= KEEP_FIRST;
+      end else if (m_axis_tvalid & m_axis_tready) begin
+        select <= (select == SEL_LAST || m_axis_tlast) ? SEL_FIRST  : (select + 'd1);
+        keep   <= (keep   == KEEP_ALL || m_axis_tlast) ? KEEP_FIRST : {keep[RATIO-2:0], 1'b1};
+      end
+    end
+
+    // The input selection MUX
+    wire [OUT_DATA_W-1:0] in_data[0:RATIO-1];
+    wire [OUT_USER_W-1:0] in_user[0:RATIO-1];
+    for (i = 0; i < RATIO; i=i+1) begin
+      assign in_data[i] = s_axis_tdata[i*OUT_DATA_W+:OUT_DATA_W];
+      assign in_user[i] = s_axis_tuser[i*OUT_USER_W+:OUT_USER_W];
+    end
+    assign m_axis_tdata   = in_data[select];
+    assign m_axis_tuser   = in_user[select];
+    assign m_axis_tlast   = s_axis_tlast && (keep == s_axis_tkeep);
+    assign m_axis_tvalid  = s_axis_tvalid;
+    assign s_axis_tready  = m_axis_tvalid && m_axis_tready && ((keep == KEEP_ALL) || m_axis_tlast);
+
+  end else begin  // if (RATIO != 1)
+
+    // Passthrough
+    assign m_axis_tdata   = s_axis_tdata;
+    assign m_axis_tuser   = s_axis_tuser;
+    assign m_axis_tlast   = s_axis_tlast;
+    assign m_axis_tvalid  = s_axis_tvalid;
+    assign s_axis_tready  = m_axis_tready;
+
+  end endgenerate
+
+endmodule // axis_downsizer
diff --git a/fpga/usrp3/lib/axi/axis_packet_flush.v b/fpga/usrp3/lib/axi/axis_packet_flush.v
new file mode 100644
index 000000000..f8b57e0a0
--- /dev/null
+++ b/fpga/usrp3/lib/axi/axis_packet_flush.v
@@ -0,0 +1,148 @@
+//
+// Copyright 2018-2019 Ettus Research, A National Instruments Company
+//
+// SPDX-License-Identifier: LGPL-3.0-or-later
+//
+// Module: axis_packet_flush
+// Description:
+//   When this module is inserted in an AXI-Stream link, it allows
+//   the client to flip a bit to make the stream lossy. When enable=1
+//   all data coming through the input is dropped. This module can
+//   start and stop flushing at packet boundaries to ensure no partial
+//   packets are introduces into the stream. Set FLUSH_PARTIAL_PKTS = 1
+//   to disable that behavior. An optional timeout can be set to 
+//   determine if flushing was done (without turning it off).
+//
+// Parameters:
+//   - WIDTH: The bitwidth of the AXI-Stream bus
+//   - TIMEOUT_W: Width of the timeout counter
+//   - FLUSH_PARTIAL_PKTS: Start flusing immediately even if a packet is in flight
+//   - PIPELINE: Which ports to pipeline? {NONE, IN, OUT, INOUT}
+//
+// Signals:
+//   - s_axis_*  : Input AXI-Stream
+//   - m_axis_*  : Output AXI-Stream
+//   - enable    : Enable flush mode
+//   - timeout   : Flush timeout (# of cycles of inactivity until done)
+//   - flushing  : The module is currently flushing
+//   - done      : Finished flushing (but is still active)
+
+module axis_packet_flush #(
+  parameter WIDTH               = 64,
+  parameter TIMEOUT_W           = 32,
+  parameter FLUSH_PARTIAL_PKTS  = 0,
+  parameter PIPELINE            = "NONE"
+)(
+  // Clock and reset
+  input  wire                 clk,
+  input  wire                 reset,
+  // Control and status       
+  input  wire                 enable,
+  input  wire [TIMEOUT_W-1:0] timeout,
+  output wire                 flushing,
+  output reg                  done = 1'b0,
+  // Input stream             
+  input  wire [WIDTH-1:0]     s_axis_tdata,
+  input  wire                 s_axis_tlast,
+  input  wire                 s_axis_tvalid,
+  output wire                 s_axis_tready,
+  // Output stream            
+  output wire [WIDTH-1:0]     m_axis_tdata,
+  output wire                 m_axis_tlast,
+  output wire                 m_axis_tvalid,
+  input  wire                 m_axis_tready
+);
+
+  //----------------------------------------------
+  // Pipeline Logic
+  //----------------------------------------------
+
+  wire [WIDTH-1:0] i_pipe_tdata,  o_pipe_tdata;
+  wire             i_pipe_tlast,  o_pipe_tlast;
+  wire             i_pipe_tvalid, o_pipe_tvalid;
+  wire             i_pipe_tready, o_pipe_tready;
+
+  generate
+    if (PIPELINE == "IN" || PIPELINE == "INOUT") begin
+      axi_fifo_flop2 #(.WIDTH(WIDTH+1)) in_pipe_i (
+        .clk(clk), .reset(reset), .clear(1'b0),
+        .i_tdata({s_axis_tlast, s_axis_tdata}), .i_tvalid(s_axis_tvalid), .i_tready(s_axis_tready),
+        .o_tdata({i_pipe_tlast, i_pipe_tdata}), .o_tvalid(i_pipe_tvalid), .o_tready(i_pipe_tready),
+        .space(), .occupied()
+      );
+    end else begin
+      assign {i_pipe_tlast, i_pipe_tdata, i_pipe_tvalid} = {s_axis_tlast, s_axis_tdata, s_axis_tvalid}; 
+      assign s_axis_tready = i_pipe_tready;
+    end
+
+    if (PIPELINE == "OUT" || PIPELINE == "INOUT") begin
+      axi_fifo_flop2 #(.WIDTH(WIDTH+1)) out_pipe_i (
+        .clk(clk), .reset(reset), .clear(1'b0),
+        .i_tdata({o_pipe_tlast, o_pipe_tdata}), .i_tvalid(o_pipe_tvalid), .i_tready(o_pipe_tready),
+        .o_tdata({m_axis_tlast, m_axis_tdata}), .o_tvalid(m_axis_tvalid), .o_tready(m_axis_tready),
+        .space(), .occupied()
+      );
+    end else begin
+      assign {m_axis_tlast, m_axis_tdata, m_axis_tvalid} = {o_pipe_tlast, o_pipe_tdata, o_pipe_tvalid};
+      assign o_pipe_tready = m_axis_tready;
+    end
+  endgenerate
+
+  //----------------------------------------------
+  // Flushing Logic
+  //----------------------------------------------
+
+  // Shortcuts
+  wire xfer_stb = i_pipe_tvalid & i_pipe_tready;
+  wire pkt_stb  = xfer_stb & i_pipe_tlast;
+
+  // Packet boundary detector
+  reg mid_pkt = 1'b0;
+  always @(posedge clk) begin
+    if (reset) begin
+      mid_pkt <= 1'b0;
+    end else if (xfer_stb) begin
+      mid_pkt <= ~pkt_stb;
+    end
+  end
+
+  // Flush startup state machine
+  reg active  = 1'b0;
+  always @(posedge clk) begin
+    if (reset) begin
+      active <= 1'b0;
+    end else begin
+      if (enable & (pkt_stb | (~mid_pkt & ~xfer_stb))) begin
+        active <= 1'b1;
+      end else if (~enable) begin
+        active <= 1'b0;
+      end
+    end
+  end
+  assign flushing = (FLUSH_PARTIAL_PKTS == 0) ? active : enable;
+
+  // Flush done detector based on timeout
+  reg [TIMEOUT_W-1:0] cyc_to_go = {TIMEOUT_W{1'b1}};
+  wire done_tmp = (cyc_to_go == {TIMEOUT_W{1'b0}});
+  always @(posedge clk) begin
+    if (reset | ~enable) begin
+      cyc_to_go <= {TIMEOUT_W{1'b1}};
+      done <= 1'b0;
+    end else if (enable & ~active) begin
+      cyc_to_go <= timeout;
+    end else begin
+      if (~done_tmp) begin
+        cyc_to_go <= xfer_stb ? timeout : (cyc_to_go - 1'b1);
+      end
+      done <= done_tmp;
+    end
+  end
+
+  // When flushing, drop all input data and quiet output data
+  // When no flushing, pass data without interruption
+  assign o_pipe_tdata  = i_pipe_tdata;
+  assign o_pipe_tlast  = i_pipe_tlast;
+  assign o_pipe_tvalid = flushing ? 1'b0 : i_pipe_tvalid;
+  assign i_pipe_tready = flushing ? 1'b1 : o_pipe_tready;
+
+endmodule
\ No newline at end of file
diff --git a/fpga/usrp3/lib/axi/axis_shift_register.v b/fpga/usrp3/lib/axi/axis_shift_register.v
new file mode 100644
index 000000000..4b3c9f4de
--- /dev/null
+++ b/fpga/usrp3/lib/axi/axis_shift_register.v
@@ -0,0 +1,209 @@
+//
+// Copyright 2018 Ettus Research, A National Instruments Company
+//
+// SPDX-License-Identifier: LGPL-3.0-or-later
+//
+// Module: axis_shift_register
+// Description:
+//   This module implements a chain of flip-flops in connected
+//   using AXI-Stream. It can be used in the following ways:
+//   * As a AXI-Stream shift register. The tready path is 
+//     combinatorial from the output to the input so backpressure
+//     is immediate. The same behavior makes this module non-ideal
+//     to actually break timing critical paths.
+//   * An AXI-Stream wrapper module for a multi-cycle operation
+//     with clock-enables. This can most commonly be used with DSP
+//     operations like filters. Enable the sideband datapath to 
+//     let the module handle handshaking while processing samples
+//     outside it.
+//
+// Parameters:
+//   - WIDTH: The bitwidth of a sample on the data bus.
+//   - NSPC: The number of parallel samples per cycle to process. The
+//       total width of the data bus will be WIDTH*NSPC.
+//   - LATENCY: Number of stages in the shift register
+//   - SIDEBAND_DATAPATH: If SIDEBAND_DATAPATH==1 then tdata is managed
+//       outside this module and imported from s_sideband_data. 
+//       If SIDEBAND_DATAPATH=0, then tdata is managed internally and 
+//       the sideband signals are unused.
+//       Useful when using this module to manage a DSP pipeline where the
+//       data could be changing in each stage.
+//   - GAPLESS: After the shift register has filled up, should gaps be
+//       allowed? If set to 1, then if s_axis_tvalid goes low then the 
+//       pipeline will stall and all bits in stage_stb will immediately go low 
+//       to ensure all stages in the shift register have valid data.
+//       NOTE: This GAPLESS=1 will not allow the final "LATENCY" samples
+//       to exit the shift register.
+//   - PIPELINE: Which ports to pipeline? {NONE, IN, OUT, INOUT}
+//
+// Signals:
+//   - s_axis_* : Input sample stream (AXI-Stream)
+//   - m_axis_* : Output sample stream (AXI-Stream)
+//   - stage_stb : Transfer strobe for each stage 
+//   - stage_eop : Transfer end-of-packet out. bit[i] = stage[i]
+//   - m_sideband_data : Sideband data out for external consumer
+//   - m_sideband_keep : Sideband keep signal out for external consumer
+//   - s_sideband_data : Sideband data in from external producer
+
+module axis_shift_register #(
+  parameter WIDTH             = 32,
+  parameter NSPC              = 1,
+  parameter LATENCY           = 3,
+  parameter SIDEBAND_DATAPATH = 0,
+  parameter GAPLESS           = 0,
+  parameter PIPELINE          = "NONE"
+)(
+  // Clock, reset and settings
+  input  wire                     clk,              // Clock
+  input  wire                     reset,            // Reset
+  // Serial Data In (AXI-Stream)              
+  input  wire [(WIDTH*NSPC)-1:0]  s_axis_tdata,     // Input stream tdata
+  input  wire [NSPC-1:0]          s_axis_tkeep,     // Input stream tkeep (used as a sample qualifier)
+  input  wire                     s_axis_tlast,     // Input stream tlast
+  input  wire                     s_axis_tvalid,    // Input stream tvalid
+  output wire                     s_axis_tready,    // Input stream tready
+  // Serial Data Out (AXI-Stream)             
+  output wire [(WIDTH*NSPC)-1:0]  m_axis_tdata,     // Output stream tdata
+  output wire [NSPC-1:0]          m_axis_tkeep,     // Output stream tkeep (used as a sample qualifier)
+  output wire                     m_axis_tlast,     // Output stream tlast
+  output wire                     m_axis_tvalid,    // Output stream tvalid
+  input  wire                     m_axis_tready,    // Output stream tready
+  // Signals for the sideband data path                     
+  output wire [LATENCY-1:0]       stage_stb,        // Transfer strobe out. bit[i] = stage[i]
+  output wire [LATENCY-1:0]       stage_eop,        // Transfer end-of-packet out. bit[i] = stage[i]
+  output wire [(WIDTH*NSPC)-1:0]  m_sideband_data,  // Sideband data out for external consumer
+  output wire [NSPC-1:0]          m_sideband_keep,  // Sideband keep signal out for external consumer
+  input  wire [(WIDTH*NSPC)-1:0]  s_sideband_data   // Sideband data in from external producer
+);
+  // Shift register width depends on whether the datapath is internal
+  localparam SHREG_WIDTH     = SIDEBAND_DATAPATH[0] ? (NSPC + 1) : ((WIDTH*NSPC) + NSPC + 1);
+  localparam SHREG_TLAST_LOC = SHREG_WIDTH-1;
+  localparam SHREG_TKEEP_HI  = SHREG_WIDTH-2;
+  localparam SHREG_TKEEP_LO  = SHREG_WIDTH-NSPC-1;
+
+  //----------------------------------------------
+  // Pipeline Logic
+  // (fifo_flop2 is used because it breaks timing
+  //  path going both ways: valid and ready)
+  //----------------------------------------------
+  wire [(WIDTH*NSPC)-1:0] i_tdata,  o_tdata;
+  wire [NSPC-1:0]         i_tkeep,  o_tkeep;
+  wire                    i_tlast,  o_tlast;
+  wire                    i_tvalid, o_tvalid;
+  wire                    i_tready, o_tready;
+
+  generate
+    // Input pipeline register if requested
+    if (PIPELINE == "IN" || PIPELINE == "INOUT") begin
+      axi_fifo_flop2 #(.WIDTH((WIDTH*NSPC) + NSPC + 1)) in_pipe_i (
+        .clk(clk), .reset(reset), .clear(1'b0),
+        .i_tdata({s_axis_tlast, s_axis_tkeep, s_axis_tdata}),
+        .i_tvalid(s_axis_tvalid), .i_tready(s_axis_tready),
+        .o_tdata({i_tlast, i_tkeep, i_tdata}), .o_tvalid(i_tvalid), .o_tready(i_tready),
+        .space(), .occupied()
+      );
+    end else begin
+      assign {i_tlast, i_tkeep, i_tdata} = {s_axis_tlast, s_axis_tkeep, s_axis_tdata};
+      assign i_tvalid = s_axis_tvalid;
+      assign s_axis_tready = i_tready;
+    end
+
+    // Output pipeline register if requested
+    if (PIPELINE == "OUT" || PIPELINE == "INOUT") begin
+      axi_fifo_flop2 #(.WIDTH((WIDTH*NSPC) + NSPC + 1)) out_pipe_i (
+        .clk(clk), .reset(reset), .clear(1'b0),
+        .i_tdata({o_tlast, o_tkeep, o_tdata}), .i_tvalid(o_tvalid), .i_tready(o_tready),
+        .o_tdata({m_axis_tlast, m_axis_tkeep, m_axis_tdata}),
+        .o_tvalid(m_axis_tvalid), .o_tready(m_axis_tready),
+        .space(), .occupied()
+      );
+    end else begin
+      assign {m_axis_tlast, m_axis_tkeep, m_axis_tdata} = {o_tlast, o_tkeep, o_tdata};
+      assign m_axis_tvalid = o_tvalid;
+      assign o_tready = m_axis_tready;
+    end
+  endgenerate
+
+  assign m_sideband_data = i_tdata;
+  assign m_sideband_keep = i_tkeep;
+
+  //----------------------------------------------
+  // Shift register stages
+  //----------------------------------------------
+  genvar i;
+  generate
+    if (GAPLESS == 0) begin
+      // Individual stage wires
+      wire [SHREG_WIDTH-1:0]  stg_tdata [0:LATENCY];
+      wire                    stg_tvalid[0:LATENCY];
+      wire                    stg_tready[0:LATENCY];
+      // Shift register input
+      assign stg_tdata[0] = SIDEBAND_DATAPATH[0] ? {i_tlast, i_tkeep} : {i_tlast, i_tkeep, i_tdata};
+      assign stg_tvalid[0] = i_tvalid;
+      assign i_tready = stg_tready[0];
+      // Shift register output
+      assign o_tlast = stg_tdata[LATENCY][SHREG_TLAST_LOC];
+      assign o_tkeep = stg_tdata[LATENCY][SHREG_TKEEP_HI:SHREG_TKEEP_LO];
+      assign o_tdata = SIDEBAND_DATAPATH[0] ? s_sideband_data : stg_tdata[LATENCY][(WIDTH*NSPC)-1:0];
+      assign o_tvalid = stg_tvalid[LATENCY];
+      assign stg_tready[LATENCY] = o_tready;
+  
+      for (i = 0; i < LATENCY; i=i+1) begin: stages
+        axi_fifo_flop #(.WIDTH(SHREG_WIDTH)) reg_i (
+          .clk(clk), .reset(reset), .clear(1'b0),
+          .i_tdata(stg_tdata[i  ]), .i_tvalid(stg_tvalid[i  ]), .i_tready(stg_tready[i  ]),
+          .o_tdata(stg_tdata[i+1]), .o_tvalid(stg_tvalid[i+1]), .o_tready(stg_tready[i+1]),
+          .occupied(), .space()
+        );
+        assign stage_stb[i] = stg_tvalid[i] & stg_tready[i];
+        assign stage_eop[i] = stage_stb[i] & stg_tdata[i][SHREG_TLAST_LOC];
+      end
+    end else begin // if (GAPLESS == 0)
+      wire [(WIDTH*NSPC)-1:0] o_tdata_fifo;
+      wire [NSPC-1:0]         o_tkeep_fifo;
+      wire                    o_tlast_fifo, o_tvalid_fifo, o_tready_fifo;
+
+      // Shift register to hold valids
+      reg  [LATENCY-1:0]     stage_valid = {LATENCY{1'b0}};
+      // Shift register to hold data/last
+      reg  [SHREG_WIDTH-1:0] stage_shreg[0:LATENCY-1];
+      wire [SHREG_WIDTH-1:0] shreg_input = SIDEBAND_DATAPATH[0] ? {i_tlast, i_tkeep} : {i_tlast, i_tkeep, i_tdata};
+      wire                   shreg_ce = i_tready & i_tvalid;
+
+      assign i_tready      = o_tready_fifo;
+      assign o_tvalid_fifo = stage_valid[LATENCY-1] & shreg_ce;
+      assign o_tlast_fifo  = stage_shreg[LATENCY-1][SHREG_TLAST_LOC];
+      assign o_tkeep_fifo  = stage_shreg[LATENCY-1][SHREG_TKEEP_HI:SHREG_TKEEP_LO];
+      assign o_tdata_fifo  = SIDEBAND_DATAPATH[0] ? s_sideband_data : stage_shreg[LATENCY-1][(WIDTH*NSPC)-1:0];
+
+      for (i = 0; i < LATENCY; i=i+1) begin
+        // Initialize shift register
+        initial begin
+          stage_shreg[i] <= {SHREG_WIDTH{1'b0}};
+        end
+        // Shift register logic
+        always @(posedge clk) begin
+          if (reset) begin
+            stage_shreg[i] <= {SHREG_WIDTH{1'b0}};
+            stage_valid[i] <= 1'b0;
+          end else if (shreg_ce) begin
+            stage_shreg[i] <= (i == 0) ? shreg_input : stage_shreg[i-1];
+            stage_valid[i] <= (i == 0) ? 1'b1        : stage_valid[i-1];
+          end
+        end
+        // Outputs
+        assign stage_stb[i] = ((i == 0) ? 1'b1 : stage_valid[i-1]) & shreg_ce;
+        assign stage_eop[i] = stage_stb[i] & ((i == 0) ? i_tlast : stage_shreg[i-1][SHREG_TLAST_LOC]);
+      end
+
+      // The "gapless" logic violates AXI-Stream by having an o_tready -> o_tvalid dependency, 
+      // so we add a FIFO downstream to prevent deadlocks.
+      axi_fifo #(.WIDTH((WIDTH*NSPC) + NSPC + 1), .SIZE($clog2(LATENCY))) out_fifo_i (
+        .clk(clk), .reset(reset), .clear(1'b0),
+        .i_tdata({o_tlast_fifo, o_tkeep_fifo, o_tdata_fifo}), .i_tvalid(o_tvalid_fifo), .i_tready(o_tready_fifo),
+        .o_tdata({o_tlast, o_tkeep, o_tdata}), .o_tvalid(o_tvalid), .o_tready(o_tready),
+        .space(), .occupied()
+      );
+    end
+  endgenerate
+endmodule // axis_shift_register
diff --git a/fpga/usrp3/lib/axi/axis_upsizer.v b/fpga/usrp3/lib/axi/axis_upsizer.v
new file mode 100644
index 000000000..07e313e2d
--- /dev/null
+++ b/fpga/usrp3/lib/axi/axis_upsizer.v
@@ -0,0 +1,104 @@
+//
+// Copyright 2018 Ettus Research, A National Instruments Company
+//
+// SPDX-License-Identifier: LGPL-3.0-or-later
+//
+// Module: axis_upsizer
+// Description: 
+//   An AXI-Stream width conversion module that widens the input
+//   sample with by a factor of RATIO.
+//   NOTE: This module has end-to-end combanitorial paths. For a
+//         pipelined version, please use axis_width_conv
+//
+// Parameters:
+//   - IN_DATA_W: The bitwidth of the input data bus. The width of the
+//                output data bus is IN_DATA_W*RATIO
+//   - IN_USER_W: The bitwidth of the input user bus. The width of the
+//                output user bus is IN_USER_W*RATIO
+//   - RATIO: The upsizing ratio
+//
+// Signals:
+//   - s_axis_* : Input sample stream (AXI-Stream)
+//   - m_axis_* : Output sample stream (AXI-Stream)
+
+module axis_upsizer #(
+  parameter IN_DATA_W = 32,
+  parameter IN_USER_W = 1,
+  parameter RATIO     = 4
+)(
+  // Clock, reset and settings
+  input  wire                         clk,              // Clock
+  input  wire                         reset,            // Reset
+  // Data In (AXI-Stream)              
+  input  wire [IN_DATA_W-1:0]         s_axis_tdata,     // Input stream tdata
+  input  wire [IN_USER_W-1:0]         s_axis_tuser,     // Input stream tuser
+  input  wire                         s_axis_tlast,     // Input stream tlast
+  input  wire                         s_axis_tvalid,    // Input stream tvalid
+  output wire                         s_axis_tready,    // Input stream tready
+  // Data Out (AXI-Stream)             
+  output wire [(IN_DATA_W*RATIO)-1:0] m_axis_tdata,     // Output stream tdata
+  output wire [(IN_USER_W*RATIO)-1:0] m_axis_tuser,     // Output stream tuser
+  output wire [RATIO-1:0]             m_axis_tkeep,     // Output stream tkeep
+  output wire                         m_axis_tlast,     // Output stream tlast
+  output wire                         m_axis_tvalid,    // Output stream tvalid
+  input  wire                         m_axis_tready     // Output stream tready
+);
+
+  genvar i;
+  generate if (RATIO != 1) begin
+    // Constants
+    localparam [$clog2(RATIO)-1:0] SEL_FIRST  = 'd0;
+    localparam [$clog2(RATIO)-1:0] SEL_LAST   = RATIO-1;
+    localparam [RATIO-1:0]         KEEP_FIRST = {{(RATIO-1){1'b0}}, 1'b1};
+    localparam [RATIO-1:0]         KEEP_ALL   = {(RATIO){1'b1}};
+  
+    // Keep a binary-coded and one-hot version of the current
+    // section of the output that is being processed.
+    reg [$clog2(RATIO)-1:0] select = SEL_FIRST;
+    reg [RATIO-1:0]         keep   = KEEP_FIRST;
+    // Cached data. Incomplete output word.
+    reg [IN_DATA_W-1:0] cached_data[0:RATIO-2];
+    reg [IN_USER_W-1:0] cached_user[0:RATIO-2];
+
+    // State machine to drive the select bits for the
+    // output DEMUX.
+    always @(posedge clk) begin
+      if (reset) begin
+        select <= SEL_FIRST;
+        keep   <= KEEP_FIRST;
+      end else if (s_axis_tvalid & s_axis_tready) begin
+        select <= (select == SEL_LAST || s_axis_tlast) ? SEL_FIRST  : (select + 'd1);
+        keep   <= (keep   == KEEP_ALL || s_axis_tlast) ? KEEP_FIRST : {keep[RATIO-2:0], 1'b1};
+        cached_data[select] <= s_axis_tdata;
+        cached_user[select] <= s_axis_tuser;
+      end
+    end
+
+    // The output DEMUX
+    for (i = 0; i < RATIO; i=i+1) begin
+      if (i == SEL_LAST) begin
+        assign m_axis_tdata[(i*IN_DATA_W)+:IN_DATA_W] = s_axis_tdata;
+        assign m_axis_tuser[(i*IN_USER_W)+:IN_USER_W] = s_axis_tuser;
+      end else begin
+        assign m_axis_tdata[(i*IN_DATA_W)+:IN_DATA_W] = keep[i+1] ? cached_data[i] : s_axis_tdata;
+        assign m_axis_tuser[(i*IN_USER_W)+:IN_USER_W] = keep[i+1] ? cached_user[i] : s_axis_tuser;
+      end
+    end
+    assign m_axis_tkeep   = keep;
+    assign m_axis_tlast   = s_axis_tlast;
+    assign m_axis_tvalid  = s_axis_tvalid & ((keep == KEEP_ALL) | s_axis_tlast);
+    assign s_axis_tready  = m_axis_tvalid ? m_axis_tready : s_axis_tvalid;
+
+  end else begin  // if (RATIO != 1)
+
+    // Passthrough
+    assign m_axis_tdata   = s_axis_tdata;
+    assign m_axis_tuser   = s_axis_tuser;
+    assign m_axis_tkeep   = 1'b1;
+    assign m_axis_tlast   = s_axis_tlast;
+    assign m_axis_tvalid  = s_axis_tvalid;
+    assign s_axis_tready  = m_axis_tready;
+
+  end endgenerate
+
+endmodule // axis_upsizer
diff --git a/fpga/usrp3/lib/axi/axis_width_conv.v b/fpga/usrp3/lib/axi/axis_width_conv.v
new file mode 100644
index 000000000..2cf19ece8
--- /dev/null
+++ b/fpga/usrp3/lib/axi/axis_width_conv.v
@@ -0,0 +1,232 @@
+//
+// Copyright 2018 Ettus Research, A National Instruments Company
+//
+// SPDX-License-Identifier: LGPL-3.0-or-later
+//
+// Module: axis_width_conv
+// Description: 
+//   An AXI-Stream width conversion module that can convert from
+//   an arbitrary input width to an arbitrary output width. The
+//   module also supports an optional clock crossing. Data bits
+//   are grouped into words which will be rearranged by this module.
+//   The contents of a word are not rearranged.
+//   Example (WORD_W=4, IN_WORDS=4, OUT_WORDS=6):
+//    Input  : 3_1_2_0, x_6_5_4           (comma-delimited packets)
+//    Output : 5_4_3_2_1_0, x_x_x_x_x_6   (comma-delimited packets)
+//   NOTE: The use of tkeep in this module is a slight deviation from
+//         the AXI standard where the bits are "byte qualifiers". In
+//         this module, tkeep is a "word qualifier" where the width
+//         of a word can be arbitrary. If WORD_W = 8, the behavior
+//         of this module is identical to an AXI width converter. 
+//
+// Parameters:
+//   - WORD_W: Bitwidth of a word
+//   - IN_WORDS: Number of words in the input stream
+//   - OUT_WORDS: Number of words in the output stream
+//   - SYNC_CLKS: Are s_axis_aclk and m_axis_aclk synchronous to each other?
+//   - PIPELINE: Which ports to pipeline? {NONE, IN, OUT, INOUT}
+//
+// Signals:
+//   - s_axis_* : Input sample stream (AXI-Stream)
+//   - m_axis_* : Output sample stream (AXI-Stream)
+
+module axis_width_conv #(
+  parameter WORD_W    = 8,
+  parameter IN_WORDS  = 4,
+  parameter OUT_WORDS = 6,
+  parameter SYNC_CLKS = 0,
+  parameter PIPELINE  = "NONE"
+)(
+  // Data In (AXI-Stream)                
+  input  wire                          s_axis_aclk,    // Input stream Clock
+  input  wire                          s_axis_rst,     // Input stream Reset
+  input  wire [(IN_WORDS*WORD_W)-1:0]  s_axis_tdata,   // Input stream tdata
+  input  wire [IN_WORDS-1:0]           s_axis_tkeep,   // Input stream tkeep
+  input  wire                          s_axis_tlast,   // Input stream tlast
+  input  wire                          s_axis_tvalid,  // Input stream tvalid
+  output wire                          s_axis_tready,  // Input stream tready
+  // Data Out (AXI-Stream)             
+  input  wire                          m_axis_aclk,    // Output stream Clock
+  input  wire                          m_axis_rst,     // Output stream Reset
+  output wire [(OUT_WORDS*WORD_W)-1:0] m_axis_tdata,   // Output stream tdata
+  output wire [OUT_WORDS-1:0]          m_axis_tkeep,   // Output stream tkeep
+  output wire                          m_axis_tlast,   // Output stream tlast
+  output wire                          m_axis_tvalid,  // Output stream tvalid
+  input  wire                          m_axis_tready   // Output stream tready
+);
+
+  //----------------------------------------------
+  // Pipeline Logic
+  //----------------------------------------------
+  // Add optional input and output pipeline stages
+
+  wire [(IN_WORDS*WORD_W)-1:0]  i_tdata;
+  wire [IN_WORDS-1:0]           i_tkeep;
+  wire                          i_tlast, i_tvalid, i_tready;
+  wire [(OUT_WORDS*WORD_W)-1:0] o_tdata;
+  wire [OUT_WORDS-1:0]          o_tkeep;
+  wire                          o_tlast, o_tvalid, o_tready;
+
+  generate
+    if (PIPELINE == "IN" || PIPELINE == "INOUT") begin
+      axi_fifo_flop2 #(.WIDTH((IN_WORDS*(WORD_W+1))+1)) in_pipe_i (
+        .clk(s_axis_aclk), .reset(s_axis_rst), .clear(1'b0),
+        .i_tdata({s_axis_tlast, s_axis_tkeep, s_axis_tdata}),
+        .i_tvalid(s_axis_tvalid), .i_tready(s_axis_tready),
+        .o_tdata({i_tlast, i_tkeep, i_tdata}),
+        .o_tvalid(i_tvalid), .o_tready(i_tready),
+        .space(), .occupied()
+      );
+    end else begin
+      assign {i_tlast, i_tkeep, i_tdata} = {s_axis_tlast, s_axis_tkeep, s_axis_tdata};
+      assign i_tvalid = s_axis_tvalid;
+      assign s_axis_tready = i_tready;
+    end
+
+    if (PIPELINE == "OUT" || PIPELINE == "INOUT") begin
+      axi_fifo_flop2 #(.WIDTH((OUT_WORDS*(WORD_W+1))+1)) out_pipe_i (
+        .clk(m_axis_aclk), .reset(m_axis_rst), .clear(1'b0),
+        .i_tdata({o_tlast, o_tkeep, o_tdata}),
+        .i_tvalid(o_tvalid), .i_tready(o_tready),
+        .o_tdata({m_axis_tlast, m_axis_tkeep, m_axis_tdata}),
+        .o_tvalid(m_axis_tvalid), .o_tready(m_axis_tready),
+        .space(), .occupied()
+      );
+    end else begin
+      assign {m_axis_tlast, m_axis_tkeep, m_axis_tdata} = {o_tlast, o_tkeep, o_tdata};
+      assign m_axis_tvalid = o_tvalid;
+      assign o_tready = m_axis_tready;
+    end
+  endgenerate
+
+  //----------------------------------------------
+  // Intermediate Data Bus
+  //----------------------------------------------
+  // To perform an M to N width conversion, we first
+  // convert from M to LCM(M, N), then to N
+
+  // Function to compute the least common multiple
+  // of two numbers (parameters or localparams only)
+  function integer lcm;
+    input integer a;
+    input integer b;
+    integer x, y, swap;
+    reg done;
+  begin
+    done = 1'b0;
+    x = a;
+    y = b;
+    while (!done) begin
+      if (x < y) begin
+        swap = x; 
+        x = y; 
+        y = swap; 
+      end else if (y != 0) begin
+        x = x - y;
+      end else begin
+        done = 1'b1; 
+      end
+    end
+    // x is the greatest common divisor
+    // LCM = (a*b)/GCD
+    lcm = (a*b)/x;
+  end
+  endfunction
+
+  // Intermediate bus parameters
+  localparam integer INT_KEEP_W     = lcm(IN_WORDS, OUT_WORDS);
+  localparam integer INT_DATA_W     = INT_KEEP_W * WORD_W;
+  localparam integer UPSIZE_RATIO   = INT_KEEP_W / IN_WORDS;
+  localparam integer DOWNSIZE_RATIO = INT_KEEP_W / OUT_WORDS;
+
+  wire [INT_DATA_W-1:0] fifo_i_tdata, fifo_o_tdata;
+  wire [INT_KEEP_W-1:0] fifo_i_tkeep, fifo_o_tkeep;
+  wire                  fifo_i_tlast, fifo_i_tvalid, fifo_i_tready;
+  wire                  fifo_o_tlast, fifo_o_tvalid, fifo_o_tready;
+
+  // Skip the intermediate FIFO if
+  // - The input and output clocks are the same
+  // - The upsizer is effectively a passthrough and input registering is requested
+  // - The downsizer is effectively a passthrough and output registering is requested
+  localparam [0:0] SKIP_FIFO = (SYNC_CLKS == 1) && (
+      ((PIPELINE == "IN"  || PIPELINE == "INOUT") && (UPSIZE_RATIO == 1)) ||
+      ((PIPELINE == "OUT" || PIPELINE == "INOUT") && (DOWNSIZE_RATIO == 1))
+    );
+  localparam FIFO_SIZE = 1;
+
+  //----------------------------------------------
+  // In => Upsizer => FIFO => Downsizer => Out
+  //----------------------------------------------
+
+  wire [INT_KEEP_W-1:0]     up_keep_flat;
+  wire [UPSIZE_RATIO-1:0]   up_keep_keep;
+  wire [DOWNSIZE_RATIO-1:0] down_keep_keep;
+
+  axis_upsizer #(
+    .IN_DATA_W(IN_WORDS*WORD_W), .IN_USER_W(IN_WORDS),
+    .RATIO(UPSIZE_RATIO)
+  ) upsizer_i (
+    .clk(s_axis_aclk), .reset(s_axis_rst),
+    .s_axis_tdata(i_tdata), .s_axis_tuser(i_tkeep),
+    .s_axis_tlast(i_tlast), .s_axis_tvalid(i_tvalid), .s_axis_tready(i_tready),
+    .m_axis_tdata(fifo_i_tdata), .m_axis_tuser(up_keep_flat), .m_axis_tkeep(up_keep_keep),
+    .m_axis_tlast(fifo_i_tlast), .m_axis_tvalid(fifo_i_tvalid), .m_axis_tready(fifo_i_tready)
+  );
+
+  // tkeep unmasking logic after upsizer
+  genvar i;
+  generate for (i = 0; i < INT_KEEP_W; i = i + 1) begin
+    // tkeep is assumed to be valid only when tlast is asserted
+    // otherwise it is 1
+    assign fifo_i_tkeep[i] = ~fifo_i_tlast |
+      (up_keep_keep[i/IN_WORDS] ? up_keep_flat[i] : 1'b0);
+  end endgenerate
+
+  generate
+    if (SKIP_FIFO) begin
+      assign fifo_o_tdata  = fifo_i_tdata;
+      assign fifo_o_tkeep  = fifo_i_tkeep;
+      assign fifo_o_tlast  = fifo_i_tlast;
+      assign fifo_o_tvalid = fifo_i_tvalid;
+      assign fifo_i_tready = fifo_o_tready;
+    end else begin
+      if (SYNC_CLKS) begin
+        axi_fifo #(.WIDTH(INT_DATA_W+INT_KEEP_W+1), .SIZE(FIFO_SIZE)) fifo_i (
+          .clk(s_axis_aclk), .reset(s_axis_rst), .clear(1'b0),
+          .i_tdata({fifo_i_tlast, fifo_i_tkeep, fifo_i_tdata}),
+          .i_tvalid(fifo_i_tvalid), .i_tready(fifo_i_tready),
+          .o_tdata({fifo_o_tlast, fifo_o_tkeep, fifo_o_tdata}),
+          .o_tvalid(fifo_o_tvalid), .o_tready(fifo_o_tready),
+          .space(), .occupied()
+        );
+      end else begin
+        axi_fifo_2clk #(.WIDTH(INT_DATA_W+INT_KEEP_W+1), .SIZE(FIFO_SIZE)) fifo_i (
+          .reset(s_axis_rst),
+          .i_aclk(s_axis_aclk),
+          .i_tdata({fifo_i_tlast, fifo_i_tkeep, fifo_i_tdata}),
+          .i_tvalid(fifo_i_tvalid), .i_tready(fifo_i_tready),
+          .o_aclk(m_axis_aclk),
+          .o_tdata({fifo_o_tlast, fifo_o_tkeep, fifo_o_tdata}),
+          .o_tvalid(fifo_o_tvalid), .o_tready(fifo_o_tready)
+        );
+      end
+    end
+  endgenerate
+
+  // tkeep masking logic after downsizer
+  generate for (i = 0; i < DOWNSIZE_RATIO; i = i + 1) begin
+    assign down_keep_keep[i] = |fifo_o_tkeep[i*OUT_WORDS+:OUT_WORDS];
+  end endgenerate
+
+  axis_downsizer #(
+    .OUT_DATA_W(OUT_WORDS*WORD_W), .OUT_USER_W(OUT_WORDS),
+    .RATIO(DOWNSIZE_RATIO)
+  ) downsizer_i (
+    .clk(m_axis_aclk), .reset(m_axis_rst),
+    .s_axis_tdata(fifo_o_tdata), .s_axis_tuser(fifo_o_tkeep), .s_axis_tkeep(down_keep_keep),
+    .s_axis_tlast(fifo_o_tlast), .s_axis_tvalid(fifo_o_tvalid), .s_axis_tready(fifo_o_tready),
+    .m_axis_tdata(o_tdata), .m_axis_tuser(o_tkeep),
+    .m_axis_tlast(o_tlast), .m_axis_tvalid(o_tvalid), .m_axis_tready(o_tready)
+  );
+
+endmodule // axis_width_conv
diff --git a/fpga/usrp3/lib/axi/crc_xnor.v b/fpga/usrp3/lib/axi/crc_xnor.v
new file mode 100644
index 000000000..c0923be66
--- /dev/null
+++ b/fpga/usrp3/lib/axi/crc_xnor.v
@@ -0,0 +1,57 @@
+//
+// Copyright 2017 Ettus Research LLC
+// Copyright 2018 Ettus Research, a National Instruments Company
+//
+// SPDX-License-Identifier: LGPL-3.0-or-later
+//
+// Generates an LFSR based on a given seed value
+// Note that not all length LFSRs are supported in the file
+// For xnor LSFR equations please refer to following link:
+// https://www.xilinx.com/support/documentation/application_notes/xapp210.pdf
+
+// All indexing will be from 1 to match indexing used in table from app note above
+
+
+module crc_xnor #(
+  parameter INPUT_WIDTH=64,
+  parameter OUTPUT_WIDTH=8
+) (
+   input clk,
+   input [INPUT_WIDTH:1] input_data,
+   input rst,
+   input hold,
+   output [OUTPUT_WIDTH:1] crc_out
+);
+
+    wire [INPUT_WIDTH:1] current_lfsr;
+    reg  [INPUT_WIDTH:1] current_lfsr_r;
+
+    // LFSR based on table given by Xilinx
+    generate if (INPUT_WIDTH == 64) begin
+        assign current_lfsr[1] = current_lfsr_r[64] ^ current_lfsr_r[63] ^ current_lfsr_r[61] ^ current_lfsr_r[60];
+        assign current_lfsr[INPUT_WIDTH:2] = current_lfsr_r[INPUT_WIDTH-1:1]; 
+    end else begin
+        fake_error_thrower invalid_width_parameter();
+    end endgenerate
+
+   always @(posedge clk) begin
+      if (rst) begin
+         current_lfsr_r <= input_data;
+      end else if(~hold) begin
+         current_lfsr_r <= current_lfsr ^ input_data;
+      end
+   end
+
+    // Sum reduce based on output width
+    generate if(INPUT_WIDTH == 64 && OUTPUT_WIDTH == 16) begin
+        assign crc_out =    current_lfsr_r[INPUT_WIDTH:INPUT_WIDTH/4*3+1]+current_lfsr_r[INPUT_WIDTH/4*3:INPUT_WIDTH/4*2+1]+
+                            current_lfsr_r[INPUT_WIDTH/4*2:INPUT_WIDTH/4+1]+current_lfsr_r[INPUT_WIDTH/4:1];
+    end else if(INPUT_WIDTH == 64 && OUTPUT_WIDTH == 32) begin
+        assign crc_out =    current_lfsr_r[INPUT_WIDTH:INPUT_WIDTH/2+1]+current_lfsr_r[INPUT_WIDTH/2:1];
+    end else begin
+        fake_error_thrower invalid_width_parameter();
+    end endgenerate
+      
+   
+
+endmodule 
diff --git a/fpga/usrp3/lib/axi/strobed_to_axi.v b/fpga/usrp3/lib/axi/strobed_to_axi.v
new file mode 100644
index 000000000..378fdc3ab
--- /dev/null
+++ b/fpga/usrp3/lib/axi/strobed_to_axi.v
@@ -0,0 +1,22 @@
+//
+// Copyright 2016 Ettus Research
+// Copyright 2018 Ettus Research, a National Instruments Company
+//
+// SPDX-License-Identifier: LGPL-3.0-or-later
+//
+
+module strobed_to_axi #(
+  parameter WIDTH = 32,
+  parameter FIFO_SIZE = 1
+)(
+  input clk, input reset, input clear,
+  input in_stb, input [WIDTH-1:0] in_data, input in_last,
+  output [WIDTH-1:0] o_tdata, output o_tlast, output o_tvalid, input o_tready
+);
+
+  axi_fifo #(.WIDTH(WIDTH+1), .SIZE(FIFO_SIZE)) axi_fifo (
+    .clk(clk), .reset(reset), .clear(clear),
+    .i_tdata({in_last,in_data}), .i_tvalid(in_stb), .i_tready(),
+    .o_tdata({o_tlast,o_tdata}), .o_tvalid(o_tvalid), .o_tready(o_tready),
+    .space(), .occupied());
+endmodule
\ No newline at end of file