CUDA in Open Source? FFmpeg integrates NVIDIA’s hardware frame interpolation after a three-year hurdle
[10:58 Tue,1.September 2026 � by Rudi Schmidts] ���
We hope we understood everything correctly, but it looks like—after a three-year licensing dance—the frame interpolation feature for NVIDIA GPUs has now been officially integrated into the master branch of FFmpeg. The "fruc_vulkan" filter can now finally use the hardware-based optical flow unit (NVOFA) of modern NVIDIA graphics cards to upscale low-framerate video content (e.g., 24fps) to higher rates (48fps, 60fps)—and with a significant speed advantage over software-based solutions like `vf_minterpolate`.
As early as 2023, developer Philip Langdale submitted a CUDA-based version called `vf_nvoffruc`. However, this failed due to the licensing terms of the proprietary NVIDIA Optical Flow SDK: using it required the SDK for both building and executing the binaries, and redistribution was not permitted, which was understandably a dealbreaker for FFmpeg.
The Vulkan version that has now been merged bypasses this problem by exclusively using the open Vulkan extension `VK_NV_optical_flow`, which NVIDIA released in February 2023 with SDK 5.0. This eliminates the dependency on NVIDIA's own SDK libraries; the filter works solely through the GPU driver.
The irony here is that the FRUC (Frame Rate Up-Conversion) function still utilizes the dedicated NVOFA hardware built into GPUs starting from the Turing generation (RTX 20 series), just without the CUDA SDK. It calculates pixel-accurate motion vectors between two frames in order to synthesize intermediate frames. With the Ada Lovelace generation (RTX 40 series), the accelerator was further optimized, which, according to NVIDIA, led to a doubling of speed and improved image quality.
However, the filter is limited to NVIDIA GPUs starting from the Ampere generation (RTX 30 series), as complete support for the Vulkan extension is only available from this generation onward. AMD and Intel GPUs are still left out because they do not offer hardware support.
At the same time, incidentally, a branch featuring extensive RTX video filters (including RTX Dynamic Vibrance, DLPP Super Resolution, RTX TrueHDR, and RTX VSR) was also integrated into FFmpeg, but removed again the very next morning. All of these filters are based on NVIDIA's proprietary AI models and CUDA—and thus likely stumbled over the same licensing hurdle as Langdale's first attempt three years ago.
For media players like mpv, which have been demanding support for NVOFA-based FRUC since December 2022, this integration is a decisive step. Users can now benefit from smoother video playback without having to rely on compute-intensive software solutions or proprietary SDKs. And this integration shows that open interfaces like Vulkan can bring even manufacturer-specific hardware features into the open-source mainstream—provided the manufacturers clear the path. It just looks a bit strange for NVIDIA that their hardware has to take the detour via Vulkan. Perhaps NVIDIA should rethink its CUDA licenses with regard to open-source projects, because in the long run, such protective licenses certainly benefit the competition more.
