TesseraFlow: Scalable Video Diffusion
for Dynamic Medical Imaging Inverse Problems

Atakan Topcu1, Mohammadhossein Momeni1, Alfredo De Goyeneche1,2, Michael Lustig1
1University of California, Berkeley 2Voio Inc.
In submission

TesseraFlow turns one standard PROPELLER MRI scan into a 512×512 video with one frame per blade (every 0.58 s), resolving bowel peristalsis that the conventional reconstruction averages into a single image. Left: each blade with parallel imaging (GRAPPA); middle: TesseraFlow; right: the scanner's image from all 92 blades. Bottom: TesseraFlow along the amber line over the 53 s scan. Shown at three times real time.

Abstract

Generative priors are state-of-the-art for modeling medical images in image reconstruction tasks, but reconstruction with video priors takes tens of seconds to minutes per frame. Latent priors inherit the errors of their autoencoder, the memory of pixel-space whole-frame priors grows with the frame size, and pixel-space patch priors are slow for two reasons. First, their deep networks process every frame at full resolution at every sampling step, even though early steps resolve only low frequencies. Second, their samplers spend many network evaluations per sampling step on overlapping windows, multiple re-noising rounds, or backpropagation through the prior.

We introduce TesseraFlow, a complex-valued pixel-space flow prior that reduces the first cost. A global encoder reads a low-resolution view of every frame once per sampling step. A single shallow decoder operates on this view during early steps and on full-resolution space-time tiles during later steps, so no network processes a whole frame at full resolution. Our sampler, ReTile, addresses the second reason. We use randomized grid shifting at every sampling step along with re-noising, which acts as a stochastic gradient instead of computing overlapping tiles, so a single network pass per sampling step suffices.

On its own, ReTile speeds up an existing global-context patch prior by 6–9×, and our prior adds a further 3–5× speedup. TesseraFlow reconstructs about one 256×256 frame per second, 20–367× faster than patch and latent video priors, and can match or exceed the best generative baseline on cardiac, brain, and abdominal MRI and lung CT, at a fraction of its GPU time. We demonstrate that our framework makes generative video reconstruction more practical for dynamic MRI and CT.

Method

TesseraFlow and ReTile: one sampling step

Prior. The information that ties a region to the rest of the video, such as anatomy, contrast, and motion state, is often low-frequency and inexpensive to represent, whereas full-resolution local detail is not. TesseraFlow models the two separately, without an autoencoder. A global encoder reads a low-resolution view of every frame once per sampling step and produces features that all tiles share. A shallow local decoder predicts the velocity of the generative flow from these features. Because flow sampling resolves low frequencies first, the decoder runs only on the low-resolution views during the early, coarse steps, and on full-resolution space-time tiles during the late steps.

Sampler. ReTile evaluates the prior on one non-overlapping space-time tiling per sampling step. At every step, the tile grid moves to a new random offset, the predicted clean video is corrected against the measurements by conjugate gradients, and it is re-noised with fresh noise. The shifting seams average out over the steps instead of accumulating, so no overlapping windows are needed, and no gradient passes through the prior.

Results

TesseraFlow results overview
(a) One test frame per dataset: the zero-filled reconstruction (CT: filtered backprojection) with its sampling pattern, and the magnitude and phase of TesseraFlow, with PSNR (dB) / SSIM and time per frame. (b) Quality against time on brain MRI at 60 and 120 network evaluations and as published. (c) In vivo scan at native 512×512: each blade with GRAPPA and with TesseraFlow over all 92 frames.
Qualitative comparison
Qualitative comparison, one frame per dataset, zoomed in on the moving anatomy (TesseraFlow: 120 steps, one sample). Last column: the reference with the zoomed region and the line of the space–time strip. Numbers: frame PSNR (dB) / SSIM.
Tiling ablation
One randomly shifted tiling per sampling step matches overlapping consensus at a tenth of the time; fixed tiles err at their edges.
Native resolution
At the native 512×512 grid, TesseraFlow's memory barely grows with the frame size, while a whole-frame network nearly fills a 96 GB GPU.

In Vivo PROPELLER Scans

The abdominal model, trained at 256×256, is applied unchanged to raw PROPELLER data at the scanner's native 512×512 grid. Each frame uses only the data of its own blade. There is no reference for the moving anatomy; the paper checks the reconstructions against blades withheld from the data consistency and across independent samples.

The video at the top of this page is a breath-hold scan with 92 blades in 53 s. Below is a second exam, applied with the same model and settings.

Second exam: breath-hold, pelvis level, 78 blades in 39 s

Each blade with GRAPPA | TesseraFlow | conventional all-blade reconstruction, a zoom on the bowel, and TesseraFlow along the amber line over time; three times real time.