High Speed Data acquisition over HDMI ===================================== :author: Steve Markgraf :copyright: 2024 by Steve Markgraf (License: CC-BY-SA) :backend: slidy :max-width: 45em == Interface limitations for (PC) data acquisition [role=""] * 480 MBit/s USB 2.0 is limited to ~45 MByte/s ** Multiple USB 2.0 devices on different ports don't always have full bandwidth * 1 GBit/s Ethernet also maxes out at ~115 MByte/s * 5 GBit/s USB 3.0 is a huge step in hardware complexity == Small FPGA boards [role=""] image::tang_nano_boards.jpg[width="45%",align="center"] * Sipeed Tang nano series (4K/9K/20K) * Cost: $15 - $30 == USB 3 interface solutions [role=""] For capturing high speed data streams, the following USB 3 interface solutions exist: * Cypress/Infineon FX3, FX4 * FTDI FT60x * WCH CH569 * Lattice CrossLinkU-NX (yet to be released) [role="incremental"] -- Drawbacks: * Not cheap * Need many IO pins -- == USB 3 interface solutions [role=""] For capturing high speed data streams, the following USB 3 interface solutions exist: * Cypress/Infineon FX3, FX4 * FTDI FT60x * WCH CH569 * Lattice CrossLinkU-NX (yet to be released) * *MacroSilicon MS2130/MS2131 USB 3.0 HDMI grabber* == Devices based on MS2130 image::ms2130_devices.jpg[width="45%",align="center"] [role="incremental"] * Watch out for "U3" marking, older and cheaper adapters use USB 2.0-only MS2109! == MS2130 block diagram image::ms2130_block_edit.png[width="90%"] [small]#Source: MacroSilicon/UltraSemi# == MS2130 technical details * Input can be RGB, YUV, also supports HDR * Integrates video scaling * Output via USB is fixed to raw 4:2:2 YUV (or MJPEG) * Output is a configured framerate, irregardless of the input framerate == Chroma subsampling image::chroma_subsampling.svg[width="65%"] [small]#Source: https://commons.wikimedia.org/wiki/File:Common_chroma_subsampling_ratios_YCbCr_CORRECTED.svg# == HDMI high level block diagram * TMDS: Transition-Minimized Differential Signaling, 8b10b encoded image::hdmi_high_level.svg[width="60%"] [small]#Source: https://www.ti.com/lit/an/slla367/slla367.pdf# == HDMI YCbCr 4:2:2 channel mapping image::hdmi_ycbcr_mapping.svg[width="95%"] [small]#Source: https://engineering.purdue.edu/ece477/Archive/2012/Spring/S12-Grp10/Datasheets/CEC_HDMI_Specification.pdf# == HDMI data islands image::hdmi_infoframes.png[width="85%"] [small]#Source: Essentials of HDMI 2.1 Protocols, Teledyne Lecroy, https://docplayer.net/169677532-Essentials-of-hdmi-2-1-protocols.html# == MS2130 video processing image::pixel.png[width="60%"] == MS2130 "transparent" mode [role=""] [[app-listing]] [source,c] ---- /* force YCbCr 4:2:2/YUV input, default is 0x04 (RGB) */ ms2130_write_reg(0xf039, 0x06); ms2130_write_reg(0xf6b0, 0x00); /* disable sharpening */ /* disable luma processing -> bypass UVC brightness/contrast control */ ms2130_write_reg(0xf6be, 0x11); /* disable luma horizontal scaling/subpixel interpolation */ ms2130_write_reg(0xf65c, 0x10); /* disable luma vertical scaling/subpixel interpolation */ ms2130_write_reg(0xf65e, 0x10); /* disable chroma processing -> bypass UVC hue/saturation control */ ms2130_write_reg(0xf6bf, 0x11); ms2130_write_reg(0xf600, 0x80); /* disable chroma interpolation */ ---- == hsdaoh Block Diagram [role=""] image::fpga_block.svg[width="100%",align="center"] == Frame format [role=""] image::frame_format.svg[width="85%",align="center"] == Demo: How does it look like? == Benchmark [cols="1,1,1,1,1"] |=== |Board|FPGA|Speed grade|MB/s|Price USD |Tang Nano 4K |GW1NSR |C7/I6 |119 |15 |Tang Nano 9K |GW1NR |C6/I5 |102 |16 |Tang Nano 20K |GW2AR |C8/I7 |184 |30 |Tang Primer 20K |GW2A |C8/I7 |184 |35 |Tang Primer 25K |GW5A |C1/I0 |178 |43 |EBAZ4205 Zynq 7010 |XC7Z010 |-1 |156 | 16-20 |=== * Maximum of MS2130: 4K 18fps ** 3840 × 2160 × 2 × 18 = 298.5 MB/s (284.8 MiB/s) == hsdaoh + AD9226 board * Attach a cheap 12 Bit, 65 MSPS ADC board image::ad9226.jpg[width="60%",align="center"] == Are you going to build an SDR? * Why not? * Use a DVB-S2 zero IF tuner IC (80 MHz bandwidth) * Cheap 10 Bit 105 MSPS ADC (AD9218) * First test: RDA5815S (250 - 2300 MHz) * Second test: Rafael Micro RT720 (~180 - 2450 MHz) == SDR Prototype image::sdr_prototype.jpg[width="75%",align="center"] == hsdaohSDR image::rt7x0_pcb.jpg[width="40%",align="center"] == hsdaohSDR image::hsdaoh_sdr.jpg[width="45%",align="center"] == Demo: hsdaohSDR * 8 Bit samples: 90 MHz bandwidth * 10 Bit samples: 72 MHz bandwidth {empty} + {empty} + * Let's look at some signals with fosphor == Outlook/further ideas * Use open source FPGA toolchain (Yosys + nextpnr-himbaechel + apicula) ** Currently not yet working, as CLKDIV primitive is missing * The reverse direction might work with osmo-fl2k adapters with HDMI out {empty} + {empty} + {empty} + * Other ideas (logic analyzer, ...) == EOF * More information: * https://github.com/steve-m/libhsdaoh * https://github.com/steve-m/hsdaoh-fpga Thanks to: * Sameer Puri for the HDMI IP core ** https://github.com/hdl-util/hdmi * Bertold Van den Bergh for ms-tools ** https://github.com/BertoldVdb/ms-tools * Questions?