FPGA Drive NVMe Offload Recorder

This is the documentation for the FPGA Drive NVMe Offload Recorder reference design: recording of data generated in the FPGA fabric to NVMe SSDs, and playback into the fabric, with the NVMe commands built in the fabric and queued by the AMD NVMe Host Accelerator IP – no CPU in the data path. For the FPGA Drive FMC Gen4 and the M.2 M-key Stack FMC.

User Guide

• Description
  • What the design does
  • Positioning
  • Hardware platforms
  • IP cores
  • M.2 slots
  • Software
• Requirements
  • NVMe Host Accelerator license
  • List of supported boards
• Supported carrier boards
  • List of supported boards
  • Board specific notes
  • Unlisted boards
• Supported SSDs
  • Compatibility
  • SSDs tested with this design
  • Choosing SSDs for recording
• Build instructions
  • Source code
  • License requirements
  • Target designs
  • Which machine can build what
  • Cross-platform build runner
• Architecture
  • Block diagram (Zynq UltraScale+)
  • The record path
  • The playback path
  • The CPU command injector
  • Address map (Zynq UltraScale+)
  • Clocks and resets
  • Interrupts
  • Versal designs
  • Resource usage
• Yocto
  • Requirements
  • How to build
  • How the image is put together
  • Boot from SD card
  • Log in
  • Next steps
• Test the design in Linux
  • 1. Check the PCIe links and the SSDs
  • 2. Check the recorder driver
  • 3. Prepare the recording volume
  • 4. Record
  • 5. Verify
  • 6. Copy the recording off the board
  • 7. Play back
  • 8. When the Host Accelerator stops (evaluation license)
  • 9. Find the sustained recording rate
• Filesystem and striping
  • Why exFAT
  • The recording volume
  • How nvrec makes a contiguous file (and its limits)
  • md RAID0 layout math
• Apps
  • Quick start
  • nvsetup-raid0.sh
  • nvrec
  • nvplay
  • nvverify
  • nvbench.sh
  • nvbench-suite.sh
  • nvregs
• Benchmarks
  • Method
  • Fixture
  • Results
  • The SLC cache
• Comparison with fpga-drive-recorder
  • At a glance
  • What differs, and why it matters
  • Discussion
• Known limitations
  • Positioning
  • Licensing
  • Recording model
  • SSDs
  • Data path
  • An interrupted recording
  • Software
  • Platforms and tools
• Troubleshooting
  • The Host Accelerator stopped (exit code 5)
  • Extended recordings on the evaluation core
  • SSDs and PCIe
  • Recording
  • Playback
  • After a power cycle
  • Boot and SD card
  • Build failures
• Revision History
  • v1.0 (2025.2), October 2026

Reference

• Register map
  • Conventions
  • nvrec_core: fdrec-compatible registers (0x000-0x0BF)
  • nvrec_core: the NVMe offload engine (0x100-0x3FF)
  • AMD NVMe Host Accelerator (nvmeha)
  • Host Accelerator timeout guard
  • Interrupts (Zynq UltraScale+)
  • Changelog
• Kernel driver
  • Why the stock nvme driver is kept off the SSDs
  • Controller bring-up
  • Block devices /dev/nvrecXn1
  • Control device /dev/nvrec
  • Device tree
  • sysfs and module parameters
  • Building the module by hand

Customizing the design

• Replacing the test pattern generator with your own data source
  • The insertion point
  • Clocking
  • Data width
  • What about the TPG controls?
  • Step by step
• Replacing the checker with your own data sink
  • Hierarchy pins
  • The stream contract
  • Rate throttle (fdrec_check)
  • Clocking and width
  • Step by step