Replacing the checker with your own data sink
Playback sends a recording back into the fabric: the command builder issues NVMe Read
commands into the ring in DDR, the ring reader streams the completed chunks, in order, into
the egress FIFO of nvrec_core, and from there to the user_data_sink hierarchy. On the
reference design that hierarchy holds fdrec_check, which verifies the test pattern. For a
real application (a DAC, a transmitter, a stimulus generator) you replace it with your own
logic. nvrec_core, the NVMe Host Accelerator and the driver stay the same.
The hierarchy pins and the stream contract are those of
fpga-drive-recorder, except for
the meaning of TLAST (one chunk instead of one DMA buffer) and the priming, which the
driver does for you here.
Hierarchy pins
Pin |
Direction |
Width |
Description |
|---|---|---|---|
|
in |
1 |
Sink clock. This is the same clock as |
|
in |
1 |
Active-low reset, synchronous to |
|
in |
AXI4-Stream |
|
|
in |
1 |
|
|
in |
1 |
|
|
in |
32 |
|
|
out |
64 each |
Read back as the |
The stream contract
Data: each beat is 32 bytes of the recording (64 on
vck190_fmcp1), little-endian —TDATA[7:0]is the first byte of each beat, exactly asnvrecrecorded it. The file header is not played; the stream starts with the first recorded beat. A recording made with another beat size (16 bytes on the 128-bit revision, 32 on a 256-bit build) is the same byte stream and plays the same.TLASTmarks the last beat of each chunk (one NVMe command, 1 MB or the drives’ MDTS). It is not a frame marker of your data; ignore it unless you want chunk boundaries.Backpressure is allowed. Deassert
TREADYwhenever you are not ready. Nothing is lost: the egress FIFO (4096 beats: 128 KB of 32-byte beats on the Zynq UltraScale+ targets, 256 KB of 64-byte beats onvck190_fmcp1) fills, the ring reader waits, and the slots it has not read yet stay occupied, so no new Read command is issued into them. Data only arrives as fast as the SSDs can deliver it, so the playback rate is set by the sink.Underflow. A sink that must consume at a fixed rate (a DAC) sees
TVALIDlow when the SSDs cannot keep up. Your logic decides what to output then (repeat the last sample, output zero, …).fdrec_checkcounts these events inCHK_UNDERFLOWS: ticks of its rate throttle on whichTVALIDwas low, after the first beat, and only when more data followed — the idle time after the end of a playback is not an underflow. A playback of a TPG recording is clean whenCHK_ERRORS= 0,CHK_UNDERFLOWS= 0 andCHK_GAP_BEATSequals the drop count in the file header (compared in 16-byte words when the recording was made with another beat size;nvplaydoes that). The checker checks every 16-byte word of a beat and counts a beat with any bad word once inCHK_ERRORS.Priming. At the start of a playback the SSDs complete the first Read commands out of order and with latency peaks of several milliseconds, so a sink that consumes from the first beat at a fixed rate would underflow. When
nvplayruns with the checker (the default), the driver keepsCHK_CTRL.ENABLElow until the ring has filled (every slot read, or the whole playback, orplay_prefill_ms= 2 s at most) and then sets it. Usechk_enableas your sink’s run control and you get the same priming for free. (A fabric-only playback,nvplay --ring-only, has no SSD to wait for: there the driver sets it once the egress FIFO is half full.)Stopping. If your sink stops consuming while a playback is in progress, the reader stalls. Set
EGR_CTRL.FLUSHto let it finish: beats are then accepted and discarded (counted inEGR_DISCARD).
Rate throttle (fdrec_check)
CHK_RATE_INC is a Q1.31 fraction of one beat per snk_clk cycle, exactly like
TPG_RATE_INC: 0x8000_0000 = ready on every clock (the default: 8 GB/s with 32-byte beats,
16 GB/s with 64-byte beats, at 250 MHz), larger values clamp. rate_bps = CHK_RATE_INC / 2^31 × SRC_CLK_HZ × BEAT_BYTES. A 31-bit fractional
accumulator asserts TREADY on every carry, so the long-term rate is exact. While
CHK_CTRL.ENABLE is 0, TREADY is 0.
Clocking and width
The read side of the egress FIFO runs on nvrec_core’s src_clk, so the sink shares the
source’s clock. If your sink needs a different clock, put an independent-clock AXI4-Stream
FIFO (axis_data_fifo with IS_ACLK_ASYNC = 1) inside user_data_sink between S_AXIS and
your logic. For a narrower sink, use an axis_dwidth_converter (the beat size, 32 or 64
bytes → your width, for example 16 bytes for a sink written for the 128-bit interface of the
previous revision) in
the hierarchy; with backpressure allowed it works without any special care. The width of
the whole datapath is the nvrec_data_width block-design variable ("data_width" of the
target in config/data.json: 128, 256 or 512), see
Custom data source.
Step by step
In the block-design script of your target’s device family (
Vivado/src/bd/bd_zynqmp.tclfor Zynq UltraScale+,Vivado/src/bd/bd_versal.tclfor Versal), find theuser_data_sinkhierarchy. Replacecreate_bd_cell -type module -reference fdrec_check user_data_sink/fdrec_check_0and its connections with your own IP or module reference. Keep the hierarchy pins; tie thechk_*counter outputs to 0 (anxlconstantof width 64) if you have nothing to report. Usingchk_enableas your sink’s run control is a good idea: it lets the driver start your sink after the ring has primed, and stop it.Put your RTL in
Vivado/src/hdl/(top module in a.vfile, as for the source).Rebuild with
./build.sh xsa --target <target>and check timing, then rebuild the Linux image (./build.sh yocto --target <target>).Play a recording with
nvplay --no-check <file>.--no-checkmakes the driver leave the checker registers alone: it neither primes nor enables the sink, andnvplaydoes not compare the checker counters, so the exit code only tells you whether the playback was complete; underflow detection is up to your sink. If your sink useschk_enable, set it by hand (sudo nvregs wr core 0x060 1, theCHK_CTRLregister), otherwise the stream stalls andnvplaygives up after--timeoutseconds with exit code 1. Alternatively, drive thechk_beats/chk_errors/chk_gaps/chk_gap_beats/chk_underflowsoutputs from your sink with the checker’s meaning and runnvplaywithout--no-check: the driver then primes and enables your sink, andnvplayjudges the playback from your counters.