每日 arXiv 论文简报

Key-value (KV) cache growth is a major bottleneck in autoregressive decoding, as memory and bandwidth scale linearly with context length. Existing KV eviction methods often rely on static heuristics or proxy scores, which poorly track future token utility and cause brittle eviction as relevance shifts. To address this, we introduce KVpop, which learns a fixed-budget KV eviction policy by directly supervising the keep-or-drop decision. The scorer is trained against a novel future-attention target, computed efficiently without materializing dense attention maps. We further introduce a delayed memory-based scorer that, uniquely among learned eviction methods, defers scoring for a fixed number of steps to exploit near-future context. On AIME and HMMT mathematical reasoning, KVpop retains 98% of full-attention performance on Qwen3-4B at 75% KV cache compression and 97% at 88% compression, consistently outperforming established eviction baselines. Qwen3-8B shows even stronger results, reaching near-full teacher performance. These results show that supervising eviction with future-attention signals cuts memory costs while maintaining quality.

键值（KV）缓存增长是自回归解码中的一个主要瓶颈，因为内存和带宽随上下文长度线性增长。现有的KV缓存驱逐方法通常依赖静态启发式方法或代理分数，这些方法难以追踪未来token的效用，并导致当相关性发生变化时驱逐行为脆弱。为了解决这一问题，我们提出了KVpop，它通过直接监督保留或丢弃决策来学习固定预算的KV缓存驱逐策略。评分器针对一种创新的未来注意力目标进行训练，该目标可在不实例化密集注意力图的情况下高效计算。我们进一步引入了一种基于延迟记忆的评分器，这在学习式驱逐方法中独一无二，它会延迟固定步数的评分以利用近未来上下文。在AIME和HMMT数学推理任务上，KVpop在Qwen3-4B模型上于75% KV缓存压缩率下保持了98%的全注意力性能，在88%压缩率下保持97%的性能，持续优于已有的驱逐基线方法。Qwen3-8B取得了更显著的结果，达到了接近完整教师模型的性能。这些结果表明，通过未来注意力信号监督驱逐决策能够在保持质量的同时大幅降低内存成本。

Accurate and timely channel state information (CSI) is essential for next-generation wireless systems, yet existing works treat CSI compression and CSI prediction as separate problems, both in academia and in current 3GPP studies. Consequently, channel aging remains insufficiently addressed within standardized CSI feedback pipelines. In this article, we propose a unified compression-prediction framework that integrates Contrastive Predictive Coding (CPC) directly into the 3GPP-compliant CSI compression architecture. Instead of predicting high-dimensional CSI matrices, our approach forecasts future latent representations and jointly optimizes reconstruction fidelity and temporal predictive coherence via a combined 1-SGCS and InfoNCE objective. This design enables temporal representation learning without increasing feedback overhead. We present two variants: CPC-before-Compression, which performs autoregressive modeling on encoded features prior to quantization, and CPC-after-Compression, which shifts temporal modeling to the base-station to reduce the complexity of users' devices. Evaluations on 3GPP-compliant datasets from Nokia, Oppo, and CATT show that CPC-before-Compression achieves over 90% reconstruction accuracy with 32x lower decoder GFLOPs than the 3GPP baseline, while CPC-after-Compression preserves an identical encoder footprint and the same 64-bit feedback overhead. By unifying compression and prediction within a standardized pipeline, the proposed framework provides an age-aware, computationally efficient CSI feedback solution. The source code is publicly available at: https://github.com/AhmedRadwan02/cpc-3gpp

准确且及时的信道状态信息（CSI）对下一代无线系统至关重要，然而现有研究，无论是学术界还是当前的3GPP研究，均将CSI压缩与CSI预测视为独立问题。因此，信道老化问题在标准化的CSI反馈流程中仍未得到充分解决。本文提出了一种统一的压缩-预测框架，将对比预测编码（CPC）直接集成到符合3GPP标准的CSI压缩架构中。我们的方法不是直接预测高维CSI矩阵，而是对未来潜在表示进行预测，并通过结合1-SGCS和InfoNCE目标函数联合优化重建保真度和时间预测一致性。该设计使得时间表示学习能够在不增加反馈开销的情况下实现。我们提出了两种变体：压缩前CPC（在量化前对编码特征进行自回归建模）和压缩后CPC（将时间建模移至基站端以降低用户设备复杂度）。在来自Nokia、Oppo和CATT的3GPP合规数据集上的评估表明，压缩前CPC在实现超过90%重建精度的同时，解码器GFLOPs比3GPP基线低32倍；而压缩后CPC则保持相同的编码器规模和64位反馈开销。通过在标准化流程中统一压缩与预测，所提出的框架提供了一种具有时效感知能力且计算效率高的CSI反馈解决方案。源代码公开可访问于：https://github.com/AhmedRadwan02/cpc-3gpp

Autoregressive (AR) models based on next-scale prediction have emerged as a powerful tool for image generation, but they face a critical weakness: information inconsistencies between patches across timesteps introduced by progressive resolution scaling. These inconsistencies scatter guidance signals, causing them to drift away from salient regions within the image and leaving behind ambiguous, unfaithful features during sampling. We tackle this challenge with Information-Grounding Guidance (IGG), a novel framework that anchors guidance to semantically important tokens via an attention-based dynamic weighting formulation, consequently ensuring that guidance and semantic contents remain tightly aligned. Across both class-conditioned and text-to-image generation tasks, IGG delivers sharper, more coherent, and semantically grounded images, demonstrating its efficacy for correcting AR-based methods. Our code is available at https://github.com/dnngky/infoground-guidance.

基于下一尺度预测的自回归模型已成为图像生成的强大工具，但其面临一个关键弱点：渐进分辨率缩放引入的跨时间步patch间的信息不一致问题。这些不一致导致引导信号分散，偏离图像中的显著区域，在采样过程中留下模糊、不忠实的特征。我们提出信息锚定引导（Information-Grounding Guidance，IGG）来解决这一挑战，这是一种通过基于注意力的动态加权公式将引导锚定在语义重要token上的新框架，从而确保引导与语义内容紧密对齐。在类别条件图像生成和文本到图像生成任务中，IGG生成了更清晰、更连贯且语义更扎实的图像，证明了其在纠正自回归方法方面的有效性。代码已开源于 https://github.com/dnngky/infoground-guidance。

Recent advances in video diffusion have been driven by scaling transformer-based architectures to billions of parameters, substantially improving visual fidelity and motion coherence. In contrast, existing mobile video diffusion models remain limited to relatively small parameter budgets, typically 0.4-1.8B, restricting generation quality. In this work, we show that high-quality mobile video generation does not require small models. Instead, we demonstrate that a server-scale 5B-parameter video diffusion transformer can be deployed efficiently on memory-constrained mobile hardware through recurrent reformulation and structured compression. Starting from Wan2.2-5B, we rely on a recurrence distillation framework that converts video generation into a chunk-wise autoregressive process with constant-memory attention computation. Combined with causal linear attention, the model operates as an RNN at inference time while preserving temporal coherence across chunks. We further propose a learnable attention head pruning method based on binary per-head gates optimized end-to-end using a noise-biased sparsity objective and distillation-based finetuning. Together with sampling-step distillation and memory-optimized VAE decoding, MobileWan becomes the first 5B-scale video diffusion model deployable on a commercial mobile device. Our system generates 5-second 480x832 videos at 16 FPS in 20 seconds end-to-end latency, achieving a VBench score of 83.79 and establishing a new state of the art in mobile video generation. Project page: https://qualcomm-ai-research.github.io/mobilewan

近年来，视频扩散模型的进展得益于将基于Transformer的架构扩展至数十亿参数，显著提升了视觉保真度和运动连贯性。相比之下，现有的移动视频扩散模型仍受限于相对较小的参数规模，通常为0.4-1.8B，制约了生成质量。本工作表明，高质量移动视频生成并不需要小型模型。相反，我们证明了通过循环重构和结构化压缩，5B参数的服务器级视频扩散Transformer可以高效部署在内存受限的移动硬件上。以Wan2.2-5B为起点，我们采用循环蒸馏框架，将视频生成分解为分块自回归过程，并实现常量内存的注意力计算。结合因果线性注意力，模型在推理时如同RNN一般运行，同时保持跨分块的时间一致性。我们进一步提出基于二元逐头门控的可学习注意力头剪枝方法，采用噪声偏向的稀疏目标进行端到端优化，并结合基于蒸馏的微调。结合采样步蒸馏和内存优化VAE解码，MobileWan成为首个可在商业移动设备上部署的5B规模视频扩散模型。我们的系统在20秒端到端延迟内生成5秒480x832分辨率16FPS的视频，达到VBench分数83.79，树立了移动视频生成的新标杆。项目主页：https://qualcomm-ai-research.github.io/mobilewan

Game worlds have traditionally been built through labor-intensive production pipelines, making them costly to develop, difficult to customization, and expensive to modify after deployment. Recent advances in video world models offer a fundamentally different paradigm. Rather than explicitly authoring every component of a virtual environment, these models autoregressively synthesize future observations conditioned on the current world state and user interactions, enabling playable worlds to be generated online. Trained on both gameplay recordings and real-world videos, they can capture diverse visual appearances and physical dynamics, opening new opportunities for interactive applications beyond gaming, including embodied intelligence. In this paper, we present \textbf{AlayaWorld}, a full-stack open-source framework for building interactive generative worlds. AlayaWorld enables open-ended real-time interaction, allowing users to freely navigate and perform diverse actions such as combat, spell casting, and monster summoning. The framework unifies the complete development-from data preparation model architecture, model training, inference acceleration, and deployment-within a modular and extensible architecture. Alongside the framework, we release reproducible pipelines, reference implementations, evaluation tools, and comprehensive documentation, establishing a practical foundation for future research and real-time applications of generative world models.

传统游戏世界通过劳动密集型生产流程构建，导致开发成本高昂、定制困难、部署后修改代价巨大。视频世界模型的最新进展提供了一种根本不同的范式。这些模型不再显式编写虚拟环境的每个组件，而是基于当前世界状态和用户交互条件自回归地合成未来观测，使可玩世界能够在线生成。它们在游戏录屏和真实世界视频上进行训练，能够捕捉多样化的视觉外观和物理动力学，为游戏之外的交互应用（包括具身智能）开辟了新的研究机会。本文中，我们提出AlayaWorld，这是一个用于构建交互式生成世界的全栈开源框架。AlayaWorld支持开放的实时交互，允许用户自由导航并执行战斗、施法、怪物召唤等多种动作。该框架将完整开发流程——从数据准备、模型架构、模型训练、推理加速到部署——统一于一个模块化且可扩展的架构中。除了框架本身，我们还发布了可复现的流程管道、参考实现、评估工具和全面的文档，为生成世界模型的未来研究和实时应用奠定了实践基础。

Evaluating end-to-end autonomous driving (E2E-AD) remains challenging, as existing driving simulation methods often trade off closed-loop interactivity (e.g., CARLA) and real-world visual fidelity (e.g., nuScenes). We present \textbf{\emph{Point as Skeleton}}, a generative sensor simulation framework for state-updated autoregressive driving video generation, in which an autoregressive generator synthesizes visual observations from step-wise updated ego states, actor states, scene maps, and point-cloud skeleton conditions. To support closed-loop rollout, we introduce Reset-and-Roll, which adapts rolling diffusion inference to simulation by preventing future-conditioned latent states from being committed across simulation steps. To stabilize error accumulation during step-wise autoregressive rollout, we introduce point-cloud skeletons that decouple foreground and background assets and project them into camera-view painted-point and template-depth conditions, providing appearance and geometric cues. We further implement a nuPlan-based renderer-level closed-loop generative interface for evaluating generation under ego deviations from the original log. Experiments on nuScenes and nuPlan show that \textit{Point as Skeleton} improves autoregressive generation quality during closed-loop rollout, demonstrating its potential for visually faithful closed-loop driving simulation. The code is available at https://github.com/krauwu/point-as-skeleton.

端到端自动驾驶（E2E-AD）的评估仍具挑战性，因为现有驾驶仿真方法通常在闭环交互性（如CARLA）和真实世界视觉保真度（如nuScenes）之间难以兼顾。我们提出点即骨架（Point as Skeleton），一个用于状态更新自回归驾驶视频生成的生成式传感器仿真框架，其中自回归生成器根据逐步更新的自车状态、参与者状态、场景地图和点云骨架条件来合成视觉观测结果。为支持闭环 rollout，我们引入Reset-and-Roll方法，通过防止未来条件的潜在状态在仿真步骤之间被提交，将滚动扩散推理适配到仿真中。为稳定逐步自回归rollout过程中的误差累积，我们引入点云骨架来解耦前景和背景资产，并将它们投影到相机视角的绘制点和模板深度条件中，提供外观和几何线索。我们进一步实现了基于nuPlan的渲染器级闭环生成接口，用于在自车偏离原始日志的情况下评估生成质量。在nuScenes和nuPlan上的实验表明，点即骨架（Point as Skeleton）在闭环rollout期间提升了自回归生成质量，展示了其在视觉保真闭环驾驶仿真中的潜力。代码可访问 https://github.com/krauwu/point-as-skeleton。

While diffusion models dominate the field of visual generation, they are computationally inefficient, applying a uniform computational effort regardless of different complexity. In contrast, autoregressive (AR) models are inherently complexity-aware, as evidenced by their variable likelihoods, but are often hindered by lossy discrete tokenization and error accumulation. In this work, we introduce Generative Refinement Networks (GRN), a next-generation visual synthesis paradigm that addresses these issues. At its core, GRN addresses the discrete tokenization bottleneck through a theoretically near-lossless Hierarchical Binary Quantization (HBQ), achieving a reconstruction quality comparable to continuous counterparts. Built upon HBQ's latent space, GRN fundamentally upgrades AR generation with a global refinement mechanism that progressively perfects and corrects artworks -- like a human artist painting. Besides, GRN integrates an entropy-guided sampling strategy, enabling complexity-aware, adaptive-step generation without compromising visual quality. On the ImageNet benchmark, GRN establishes new records in image reconstruction (0.56 rFID) and class-conditional image generation (1.81 gFID). We also scale GRN to more challenging text-to-image and text-to-video generation, delivering superior performance on an equivalent scale. We release all models and code to foster further research on GRN.

尽管扩散模型在视觉生成领域占据主导地位，但它们计算效率低下，无论样本复杂度如何都采用统一的计算量。相比之下，自回归（AR）模型本身具有复杂度感知能力，这一点从其可变的似然值可以得到验证，但其发展常受限于有损的离散tokenization和误差累积问题。本工作提出了生成式精炼网络（GRN），一种新一代视觉合成范式，用以解决上述问题。GRN的核心是通过理论近无损的层次二元量化（HBQ）来解决离散tokenization瓶颈，达到与连续方法相当的重建质量。基于HBQ的潜在空间，GRN从根本上升级了自回归生成能力，采用全局精炼机制逐步完善和修正艺术作品——如同人类画家作画一般。此外，GRN集成了熵引导采样策略，实现了复杂度感知的自适应步长生成，同时不影响视觉质量。在ImageNet基准上，GRN在图像重建（0.56 rFID）和类别条件图像生成（1.81 gFID）方面创下了新纪录。我们还将GRN扩展到更具挑战性的文本到图像和文本到视频生成任务，在同等规模下提供了卓越的性能。我们发布了所有模型和代码，以促进GRN的进一步研究。

The increasing uncertainty from flexible demand and renewable generation has made distributionally robust optimization (DRO) an important tool for robust power system dispatch. DRO relies on forecast scenarios to construct ambiguity sets, but conventional scenario generation pipelines are often trained in an accuracy-oriented manner and may neglect spatial correlations among uncertainties. This mismatch can produce ambiguity sets that are statistically plausible but suboptimal for downstream operation. This work proposes a decision-focused generative framework for correlated scenario generation in DRO-based dispatch. Instead of training generative models solely to fit the historical uncertainty distribution, the proposed framework optimizes generated scenarios according to their induced downstream operational cost. The proposed framework is tailored to mainstream generative models, including variational autoencoders, generative adversarial networks, and diffusion models, while capturing the joint distribution of uncertainties across buses. To improve computational tractability, we further develop a differentiable scenario selector that selects decision-relevant scenarios from a generated pool and can be trained within the same decision-focused pipeline. Case studies demonstrate that the proposed framework effectively reduces 0.80%-2.02% operational cost across different generative models compared to accuracy-oriented methods.

灵活需求与可再生能源带来的不确定性日益增加，使得分布鲁棒优化（DRO）成为电网鲁棒调度的关键工具。DRO依赖于预测场景来构建模糊集，但传统场景生成流程通常以准确性为导向进行训练，可能忽视不确定性之间的空间相关性。这种差异会导致生成的模糊集虽然统计上合理，但对下游运行却并非最优。针对这一问题，本文提出了一种面向决策的相关场景生成框架。该框架不再仅以拟合历史不确定性分布为目标，而是根据生成场景所引发的下游运营成本进行优化。该框架适用于主流生成模型，包括变分自编码器（VAE）、生成对抗网络（GAN）和扩散模型，同时能够捕捉各母线间不确定性的联合分布。为提高计算效率，本文进一步开发了可微场景选择器，可从生成的场景池中选取与决策相关的场景，并可在同一决策导向流程中训练。案例研究表明，与准确性导向的方法相比，所提框架在不同生成模型下可有效降低0.80%至2.02%的运营成本。

Diffusion and flow matching models generate high-quality samples, but their ODE samplers often need tens to hundreds of neural function evaluations (NFEs). This remains a practical challenge for released checkpoints, since many accelerators require additional design choices and training cost through retraining, distillation, or trajectory redesign. We investigate a different route based on $x$-prediction. During sampling, standard affine probability paths already expose $x_0$ information: an intermediate state and its path velocity determine a principled estimate of the clean sample. We formalize this property as \textbf{endpoint decodability} and show that the decoder is the minimum-MSE estimator $\mathbb{E}[x_0\mid x_t]$ under the usual $\ell_2$ objective. This yields \textbf{Truncated Jump Sampling} (TJS): stop the ODE at an early-exit time $t^*$ and return the decoded $x_0$. TJS requires no retraining, distillation, or architecture change. Across SDXL, SD3.5M, Z-Image-Turbo, and three class-conditional benchmarks, it reduces NFEs by 20--70\% with near-matched quality. The analysis also shows why endpoint prediction can work without straightening the trajectory, providing inference acceleration without trajectory redesign.

扩散模型和流匹配模型能够生成高质量样本，但其ODE采样器通常需要数十至数百次神经函数评估（NFEs）。对于已发布的模型检查点而言，这仍是一个实际挑战，因为许多加速方法需要额外设计选择以及重新训练、蒸馏或轨迹重新设计的训练成本。我们研究了一条基于x预测的不同路径。在采样过程中，标准的仿射概率路径已经暴露了x0信息：中间状态及其路径速度决定了清洁样本的原则性估计。我们将这一性质形式化为端点可解码性，并证明在常用的ℓ2目标下，解码器是最小均方误差估计器E[x0|xt]。这产生了截断跳跃采样（TJS）：在提前退出时间t*处停止ODE并返回解码后的x0。TJS无需重新训练、蒸馏或架构变更。在SDXL、SD3.5M、Z-Image-Turbo以及三个类别条件基准数据集上的实验表明，它在质量接近的情况下将NFEs减少了20%至70%。该分析还说明了为什么端点预测可以在不拉直轨迹的情况下工作，从而实现无需轨迹重新设计的推理加速。

Concept unlearning in text-to-image diffusion models is critical for safe and practical deployment: with rising privacy concerns, copyright disputes, trademark constraints, and safety regulations, deployed systems must be able to suppress unwanted concepts after training. Existing methods often remove the target concept effectively, but practical unlearning also requires an equally fundamental property: the unlearned model should retain quality, diversity, and semantic coverage on benign generation. The gold standard is a retain-only model trained from scratch without the unwanted data. However, common erasure objectives do not specify which post-unlearning distribution should approximate this reference, leaving retention as an implicit consequence of the update rule. We propose TILDE, TILt-based Distributional Erasure, which formulates concept unlearning as a distributional alignment problem: the desired target is the minimum-deviation conditional distribution from the pretrained model under a forgetting constraint. This energy-tilted, anchor-free target suppresses concept-expressing images while preserving benign relative mass for each prompt. We instantiate this principle with residual $\nabla$-GFlowNet training, which learns the score correction induced by the forget energy relative to the pretrained diffusion model. Across objects, artistic styles, and characters, TILDE achieves strong forgetting while improving retention and distributional fidelity over prior baselines.

概念遗忘在文本到图像扩散模型中对于安全实际部署至关重要：随着隐私担忧、版权争议、商标约束和安全法规的日益增加，部署后的系统必须在训练后能够抑制不需要的概念。现有方法虽然能有效移除目标概念，但实际应用还需满足一个同等重要的基本要求：遗忘后的模型应在良性生成中保持质量、多样性和语义覆盖范围。黄金标准是从头开始训练的不含无关数据的保留模型。然而，常见的擦除目标并未明确遗忘后应该近似哪种分布，使得保留成为更新规则的隐含结果。我们提出TILDE（基于倾斜的分布擦除），将概念遗忘表述为分布对齐问题：期望目标是遗忘约束下与预训练模型偏差最小的条件分布。这种能量倾斜、无锚点的目标在抑制概念表达图像的同时，为每个提示词保留良性的相对质量。我们通过残差∇-GFlowNet训练来实现这一原则，学习遗忘能量相对于预训练扩散模型所诱导的分数校正。在物体、艺术风格和角色等各类概念上，TILDE在实现强遗忘的同时，相比先前基线方法提升了保留效果和分布保真度。

We present PACR-Video, a parameter-efficient framework for multi-shot long video extrapolation that preserves recurring entities, scene structure, visual style, and causal progression without full generator fine-tuning. PACR-Video keeps a text-to-video diffusion transformer frozen and augments it with low-rank temporal adapters conditioned by learned shot-role prompt tokens. To maintain long-horizon coherence, it builds a recursive prompt bank that stores compact entity, location, action, and style prompts from previous shots, then routes them through adapter gates according to predicted narrative dependencies. A Shot-Local/Story-Global tuning objective combines next-shot reconstruction, cross-shot identity contrast, and prompt sparsity regularization, while an adapter composition schedule balances early-shot visual consistency with later-shot event progression and viewpoint change. Across six multi-shot and long-video benchmarks, PACR-Video outperforms text-to-video, tuning-based, memory-augmented, streaming, and recursive-context baselines on distributional quality, semantic alignment, identity consistency, temporal smoothness, motion stability, transition coherence, and human preference. These results show that compact prompt routing and lightweight temporal adaptation provide sufficient controllable capacity for stable long video extrapolation.

我们提出了PACR-Video，这是一个用于多镜头长视频外推的参数高效框架，能够在无需完全微调生成器的情况下保留 recurring 实体、场景结构、视觉风格和因果进展。PACR-Video保持文生视频扩散变换器冻结，并利用由学习到的镜头角色提示标记调节的低秩时序适配器进行增强。为保持长程连贯性，它构建了一个递归提示库，存储来自之前镜头的紧凑实体、位置、动作和风格提示，然后根据预测的叙事依赖关系通过适配器门路由它们。镜头局部/故事全局调优目标结合了下一镜头重建、跨镜头身份对比和提示稀疏正则化，而适配器组合调度则平衡了早期镜头的视觉一致性与后期镜头的事件进展和视角变化。在六个多镜头和长视频基准测试中，PACR-Video在分布质量、语义对齐、身份一致性、时序平滑度、运动稳定性、过渡连贯性和人类偏好方面均优于文生视频、基于调优、记忆增强、流式和递归上下文基线。这些结果表明，紧凑提示路由和轻量级时序适配为稳定的长视频外推提供了足够的可控容量。

Denoising graphs is a fundamental problem in graph learning and the core operation of graph diffusion models. Attention-based architectures like graph transformers have recently shown promise in denoising graphs. However, our principled understanding of attention-based graph denoising remains limited, making it unclear whether standard attention is the right mechanism for this task. Here we show that, under a denoising objective, linear attention is suboptimal and can only learn an average spectral denoising filter over the training distribution. This creates a fundamental limitation as graphs often vary spectrally across the distribution. To overcome this limitation, we introduce Spectral Attention, which directly utilizes the input graph spectrum and provably outperforms linear attention by a margin governed by the spectral diversity of the distribution. We then derive Graph Convolutional Attention (GCA), a practical and permutation-equivariant realization of this idea that implements spectral denoising through graph-filtered queries and keys. For stochastic block models, GCA provably matches the idealized Spectral Attention mechanism. We further show that the softmax operation, that follows the attention, provides additional denoising by approximately projecting noisy eigenvectors onto the clean eigenspace. Empirically, replacing linear attention with GCA consistently improves graph denoising and diffusion on synthetic and real datasets, with gains strongly correlated with spectral diversity. In DiGress, GCA matches standard graph-transformer performance without computing expensive structural features, and when combined with the recently proposed PEARL positional encodings, avoids explicit eigendecomposition computations resulting in faster inference without degrading quality. The code can be found here: github.com/shervinkhalafi/graph_conv_att

图去噪是图学习中的基础问题，也是图扩散模型的核心操作。基于注意力的架构如图神经网络transformer近期在图去噪任务中展现出潜力。然而，我们对基于注意力的图去噪的原理性理解仍然有限，这使得标准注意力机制是否是该任务的最佳选择尚不明确。本研究表明，在去噪目标下，线性注意力是次优的，只能学习训练分布上的平均谱去噪滤波器。这产生了一个根本性局限，因为图在分布中往往存在谱变化。为克服这一局限，本文提出了谱注意力，该方法直接利用输入图的谱，并通过谱多样性决定的边际优于线性注意力。随后，本文推导出了图卷积注意力（GCA），这是该思想的一种实用且置换等变的实现，通过图过滤的查询和键来实现谱去噪。对于随机块模型，GCA可证明匹配理想化的谱注意力机制。本文进一步表明，注意力之后的softmax操作通过将噪声特征向量近似投影到干净的特征空间来提供额外的去噪效果。实证结果表明，在合成数据集和真实数据集上，用GCA替换线性注意力始终能提升图去噪和扩散效果，且提升幅度与谱多样性高度相关。在DiGress中，GCA无需计算昂贵结构特征即可匹配标准图transformer性能，而当与近期提出的PEARL位置编码结合时，可避免显式特征分解计算，从而实现更快的推理且不损失质量。代码可访问：github.com/shervinkhalafi/graph_conv_att

We argue that training autoencoders to reconstruct inputs from noised versions of their encodings, when combined with perceptually motivated losses, yields encodings that are structured according to a perceptual hierarchy. We demonstrate the emergence of this hierarchy by showing that, after training an audio autoencoder in this manner, perceptually salient information is captured in coarser representation structures than with conventional training. Furthermore, we show that such perceptual hierarchies improve latent diffusion decoding in the context of estimating pitch surprisal in music and predicting EEG-brain responses to music listening. In both cases, our results surpass those of previous methods. Pretrained weights are available on github.com/CPJKU/pa-audioic.

我们认为，训练自编码器从其编码的噪声版本重建输入，结合感知动机损失，可产生根据感知层次结构组织的编码。我们通过以下方式证明该层次结构的出现：以这种方式训练音频自编码器后，感知显著信息比传统训练更早地出现在更粗粒度的表征结构中。此外，我们证明这种感知层次结构改善了潜在扩散解码，在估计音乐音高信息量以及预测音乐聆听时脑电响应方面均是如此。在这两种情况下，我们的结果均优于先前方法。预训练权重可访问 github.com/CPJKU/pa-audioic。

Diffusion Transformers (DiTs) and related flow-based architectures are now among the strongest text-to-image generators, yet the internal mechanisms through which prompts shape image semantics remain poorly understood. In this work, we study massive activations: a small subset of hidden-state channels whose responses are consistently much larger than the rest. We show that, despite their sparsity, these few channels effectively draw the whole picture, in three complementary senses. First, they are functionally critical: a controlled disruption probe that zeroes the massive channels causes a sharp collapse in generation quality, while disrupting an equally-sized set of low-statistic channels has marginal effect. Second, they are spatially organized: restricting image-stream tokens to massive channels and clustering them yields coherent partitions that closely align with the main subject and salient regions, exposing a structured spatial code hidden inside an apparently outlier-like subspace. Third, they are transferable: transporting massive activations from one prompt-conditioned trajectory into another, shifts the final image toward the source prompt while preserving substantial content from the target, producing localized semantic interpolation rather than unstructured pixel blending. We exploit this property in two use cases: text-conditioned and image-conditioned semantic transport, where massive activations transport enables prompt interpolation and subject-driven generation without any additional training. Together, these results recast massive activations not as activation anomalies, but as a sparse prompt-conditioned carrier subspace that organizes and controls semantic information in modern DiT models.

扩散Transformer（DiT）及相关流架构目前已成为最强的文本到图像生成器之一，然而提示词塑造图像语义的内部机制仍未被深入理解。本研究聚焦于大规模激活：一小部分隐藏状态通道，其响应值持续显著高于其他通道。研究表明，尽管这些通道数量稀少，却在三个互补层面有效地绘制了整幅图像。首先，它们在功能上至关重要：受控干扰实验表明，将大规模通道置零会导致生成质量急剧下降，而干扰等量的低统计量通道则影响甚微。其次，它们在空间上具有组织性：限制图像流token至大规模通道并进行聚类，可产生与主要目标及显著区域紧密对齐的连贯分区，揭示了看似异常值的子空间内隐藏的结构化空间编码。第三，它们具有可迁移性：将大规模激活从一条提示条件轨迹迁移到另一条，可将最终图像向源提示方向偏移，同时保留目标图像的实质性内容，产生局部语义插值而非无结构的像素混合。本研究利用这一特性开发了两个应用场景：文本条件语义迁移和图像条件语义迁移，其中大规模激活迁移无需任何额外训练即可实现提示插值和主体驱动生成。综上，这些研究结果将大规模激活重新定位为一种稀疏的提示条件载码子空间，其在现代DiT模型中组织并控制语义信息，而非被视为激活异常。

Challenges remain in ego-centric 3D scene generation due to limited view overlap and the dominant influence of individual perspectives on scene interpretation. These factors hinder the creation of viewpoint-consistent and semantically aligned visual content, as well as the construction of accurate geometric structures. In this paper, we propose CGGS, a text-to-3D framework aiming to enhance 3D-content-awareness and address geometric distortions in ego-centric scene generation. Firstly, the Ego-centric Generator is proposed by fine-tuning a Multi-View Latent Diffusion Model with consistency-augmented loss to generate consistent, high-fidelity 2D content aligned with textual descriptions. Then, Layout Decorator leverages optical flow and point-track correspondence to estimate depth, therefore producing dense point clouds as coarse layouts from the ego-centric 2D priors. Building on this initialization, Geometric Refiner is proposed to enhance 3D Gaussian reconstruction via an entropy-based Mutual Information Depth Loss (MID) combined with a hierarchical optimization scheme for improving visual quality and geometric structure. Comprehensive experiments demonstrate that CGGS outperforms previous methods in generating coherent and accurate text-driven 3D scenes. Project page: https://cggs-26.github.io/cggs26/.

第一人称视角3D场景生成仍面临诸多挑战，主要包括视角重叠有限以及个人视角对场景理解的主导影响。这些因素阻碍了视角一致且语义对齐的视觉内容的创建，以及准确几何结构的构建。本文提出CGGS，一个旨在增强3D内容感知并解决第一人称视角场景生成中几何失真的文本到3D框架。首先，本文提出第一人称视角生成器，通过一致性增强损失对多视图潜在扩散模型进行微调，以生成与文本描述一致的高保真2D内容。随后，布局装饰器利用光流和点轨迹对应关系估计深度，从而从第一人称视角的2D先验生成稠密点云作为粗略布局。在此初始化基础上，本文提出几何细化器，通过基于熵的互信息深度损失（MID）结合分层优化方案来增强3D高斯重建，从而提升视觉质量和几何结构。大量实验表明，CGGS在生成连贯且准确的文本驱动3D场景方面优于先前方法。项目页面：https://cggs-26.github.io/cggs26/

We introduce the first multiplayer world model for highly dynamic environments governed by complex physical interactions. Whereas single-player world models treat the other agents as part of the environment, ours conditions on the action streams of multiple agents, learning to attribute changes in the scene to the correct player and to stay coherent under arbitrary combinations of their actions. We study this problem in the game of Rocket League, where players compete and cooperate under fast, tightly coupled dynamics. Trained on 10,000 hours of gameplay collected with publicly available bots, our 5-billion-parameter latent diffusion model generates four-player matches in real time, producing 20 frames per second on a single Nvidia B200 GPU. Although trained only on short clips, its rollouts stay stable far beyond the training horizon: distributional quality holds steady out to five minutes, the longest horizon we measure, and in practice we observe rollouts continuing for hours with no sign of collapse. We systematically investigate the central design choices: the video codec, the generative objective, and the multiplayer conditioning scheme. In addition, we characterize how behavior changes with model and data scale, including the capabilities that emerge and the failure modes that persist. We further develop targeted evaluations that probe the model's physical understanding rather than visual appearance alone. To support continued research on multiplayer world models, we release our dataset, our full training and inference codebase, and a live demo.

我们首次为受复杂物理交互驱动的高动态环境引入了多人世界模型。相较于将其他智能体视为环境一部分的单人世界模型，我们的模型以多个智能体的动作流为条件，学习将场景变化归因于正确的玩家，并在任意动作组合下保持一致性。我们以《火箭联盟》（Rocket League）游戏为研究场景，玩家在快速、紧耦合的动力学下进行竞争与合作。我们的潜扩散模型拥有50亿参数，基于使用公开可用机器人收集的10000小时游戏数据进行训练，可在单块Nvidia B200 GPU上实时生成四人比赛，输出20帧/秒。尽管仅在短片段上训练，其展开远超出训练范围仍保持稳定：分布质量在五分钟（我们测量的最长时间范围）内保持稳定，而在实际应用中我们观察到展开可持续数小时而未出现崩溃。我们系统地研究了核心设计选择：视频编解码器、生成目标以及多人条件方案。此外，我们还分析了行为如何随模型和数据规模变化，包括涌现的能力和持续存在的失败模式。我们进一步开发了针对性评估，以探测模型对物理理解的能力而非仅关注视觉效果。为支持多人世界模型的持续研究，我们发布了数据集、完整的训练和推理代码库以及在线演示。

Autonomous vehicles (AVs) face increasing threats from vandalism-induced occlusion attacks (VOAs) that compromise camera-based perception. While detection frameworks can identify vandalized images, restoring camera-stream utility after physical occlusion remains underexplored. This paper presents present the Recovery and Enhancement of Vandalized Images for Vision Excellence (REVIVE) framework, a vandalism recovery pipeline integrating: (1) binary VOA detection, (2) multi-class VOA pattern identification, (3) EfficientNet-based U-Net segmentation, and (4) type-aware recovery using Bootstrapping Language-Image Pre-training (BLIP)-guided Stable Diffusion inpainting, direct pixel replacement, or adaptive median filtering. Stable Diffusion shows variable reconstruction performance (per-pattern SSIM 0.667-0.867, PSNR 15.4-26.7dB) across VOA patterns, while aligned direct pixel replacement achieves near-identical reconstruction under the aligned-reference condition. On 500 tracked clean/vandalized image pairs, unrecovered VOAs reduce YOLOv8l object-detection recall to 0.588, while direct pixel replacement restores recall to 0.967 and F1-score to 0.970 under that aligned-reference condition. LaMa, Telea, and Navier-Stokes baselines improve image similarity but provide more limited downstream detection recovery, and Stable Diffusion is treated as an asynchronous recovery branch subject to a quality gate rather than a blocking real-time perception step. We evaluate a reference-available quality gate that filters recovered candidates before downstream use: without it, type-aware routing degrades per-image recall to 0.304, whereas with it, recall returns to 0.608, at or above the unrecovered baseline, ensuring the forwarded stream is never worse than the unrecovered frame. REVIVE therefore, provides a structured recovery framework from VOAs in AVs.

自动驾驶汽车（AV）面临日益严峻的vandalism导致的遮挡攻击（VOA）威胁，这类攻击会危及基于摄像头的感知系统。现有的检测框架可以识别受损图像，但在物理遮挡后恢复摄像头数据流的功能仍缺乏深入研究。本文提出了视觉卓越受损图像恢复与增强（REVIVE）框架，该vandalism恢复管道集成了以下模块：（1）二分类VOA检测，（2）多类VOA模式识别，（3）基于EfficientNet的U-Net分割，以及（4）类型感知的恢复模块，采用BLIP引导的Stable Diffusion修复、直接像素替换或自适应中值滤波。Stable Diffusion在不同VOA模式上表现出不稳定的重建性能（各模式SSIM为0.667-0.867，PSNR为15.4-26.7dB），而在对齐参考条件下，直接像素替换可实现近乎一致的重建效果。在500对追踪的清洁/受损图像对上，未恢复的VOA导致YOLOv8l目标检测召回率降至0.588，而在该对齐参考条件下，直接像素替换将召回率恢复至0.967，F1分数提升至0.970。LaMa、Telea和Navier-Stokes基线方法提高了图像相似性，但对下游检测恢复的提升有限；Stable Diffusion作为异步恢复分支受到质量门控约束，而非阻塞式实时感知步骤。本文评估了一种参考可用的质量门控机制，用于在下行使用前过滤恢复候选：缺乏该机制时，类型感知路由会导致每张图像召回率降至0.304；而配备该机制后，召回率恢复至0.608，达到或超过未恢复基线，确保转发至下游的数据流不会劣于未恢复帧。因此，REVIVE为自动驾驶汽车中的VOA提供了结构化的恢复框架。

Images acquired in surveillance environments often suffer from conditions such as low resolution, variations in pose, irregular illumination, and occlusions. Due to the low quality of these images, face recognition algorithms often struggle. This major limitation can be addressed by employing super-resolution techniques that enhance the details of the image. However, due to the high degree of difficulty of the problem, most super-resolution algorithms tend to cause distortions in the image and in the individual's identity. Thus, additional information must be incorporated into the processing to improve recognition robustness. In this regard, surveillance cameras can capture multiple images, even at low quality, and the data extracted from these images, such as consecutive video frames, can significantly enhance both super-resolution and facial recognition. In this work, we introduce FASR++, a diffusion-model-based super-resolution algorithm. It leverages a reference low-resolution image and features extracted from multiple auxiliary low-quality images to generate a super-resolved output, minimizing distortions in the individual&x27;s identity. Our approach recovers facial features without explicitly providing soft attributes or computing a function gradient to guide the reconstruction process. FASR++ generates high-quality images that can considerably improve performance in face recognition tasks when used as a pre-processing step. We validate our approach on two standard face recognition datasets and attain state-of-the-art results for verification, face recognition, and image quality metrics such as PSNR, SSIM, and LPIPS.

监控环境下获取的图像经常受到低分辨率、姿态变化、不规则光照和遮挡等因素的影响。由于这些图像质量较低，人脸识别算法往往难以发挥作用。这一主要局限可以通过采用超分辨率技术来增强图像细节加以解决。然而，由于问题难度较高，大多数超分辨率算法容易导致图像和个体身份的失真。因此，需要在处理过程中引入额外信息以提高识别的鲁棒性就此而言，监控摄像头可以捕获多幅图像（即使质量较低），并且从这些图像中提取的数据（如连续视频帧）可以显著提升超分辨率和人脸识别的性能。本研究提出了FASR++，一种基于扩散模型的超分辨率算法。它利用一张参考低分辨率图像和从多幅辅助低质量图像中提取的特征来生成超分辨率输出，最大程度减少个体身份的失真。我们的方法无需显式提供软属性或计算函数梯度来引导重建过程，即可恢复人脸特征。FASR++生成的高质量图像作为预处理步骤可显著提升人脸识别任务的性能。我们在两个标准人脸识别数据集上验证了我们的方法，并在验证、人脸识别以及PSNR、SSIM和LPIPS等图像质量指标上达到了最先进的结果。

Generating temporally continuous and socially coherent human motion from text remains a fundamental challenge, particularly in realistic streams where people act alone, enter interactions, and later disengage. Most existing methods generate fixed-length motion clips under static agent configurations, which makes them brittle to solo-social transitions and unsuitable for incremental generation over long horizons. We propose ARMS, an Anchor-Relational Motion Streaming framework that unifies solo motion and human-human interaction within a single causal generative process. ARMS introduces a dynamics-asymmetric representation that decouples per-person temporal evolution from inter-person alignment via a partner-referenced relative-translation term, enabling seamless switching of social coupling without sacrificing long-horizon stability or spatial consistency between agents. On top of a causal latent space, a causal relational diffusion model progressively refines motion segment by segment using only past context, capturing both intra-person temporal dependencies and inter-person relations. Mode-aware relational gating activates or masks cross-agent connections, allowing the same model to support both solo and interaction generation. Experiments show that ARMS improves transition smoothness and social coherence compared to interaction-centric baselines, while also achieving competitive results on human-human interaction benchmarks.

从文本生成时间连续且社会一致的人类运动仍是一项基础挑战，尤其是在现实场景中人们独自行动、进入互动随后又脱离互动的运动流。现有的多数方法在静态智能体配置下生成固定长度的运动片段，导致其对独处-社交转换过于脆弱，且不适合在长时间范围内进行增量生成。我们提出ARMS，一种锚点关系运动流框架，将独处运动和人-人交互统一于单个因果生成过程中。ARMS引入了一种动力学非对称表示，通过伙伴参考的相对平移项将每个人员的时间演化与人际对齐解耦，使得社交耦合的无缝切换不会牺牲长期稳定性或智能体间的空间一致性。在因果潜在空间之上，因果关系扩散模型仅利用过去上下文逐步精炼运动片段，同时捕捉人物内时间依赖和人物间关系。模式感知关系门控激活或屏蔽跨智能体连接，使同一模型能够同时支持独处和交互生成。实验表明，与以交互为中心的基线方法相比，ARMS提升了转换流畅度和社交一致性，同时在人-人交互基准测试中也取得了具有竞争力的结果。

Synthesizing physically plausible human-scene interactions (HSI) remains a critical challenge in computer vision and the development of human avatars. Although recent generative models enable diverse motion synthesis, they suffer from an inductive bias referred to as semantic-geometric entanglement. Because spatial constraints often strongly correlate with specific actions in training data, monolithic models will learn the shortcut bias, aggressively overriding the semantic intent when faced with strict geometric cues. Furthermore, this entanglement exacerbates physical hallucinations, such as body-scene penetrations. To address these limitations, we propose DeSeG, a hierarchical framework that explicitly decouples semantic intent from geometric constraints. First, we introduce a Residual Semantic Planner that encodes textual instructions and canonicalized goal voxels into a compact latent space, enabling fine-grained semantic control independent of spatial trajectories. Second, we propose a physics regularized diffusion executor that incorporates differentiable repulsive potential fields directly into the diffusion objective, enforcing collision-aware motion generation. Extensive experiments on the Lingo dataset demonstrate that DeSeG achieves state-of-the-art performance, reducing mean scene penetration by 47% and improving semantic alignment by 29% over the SOTA baselines.

合成物理上可信的人体-场景交互（HSI）仍然是计算机视觉和虚拟数字人开发中的关键挑战。尽管当前的生成模型能够实现多样化的运动合成，但它们存在一种被称为语义-几何纠缠的归纳偏置问题。由于空间约束通常与训练数据中的特定动作高度相关，单一模型会学习到捷径偏置，在面对严格的几何线索时会激进地覆盖语义意图。此外，这种纠缠还会加剧物理幻觉问题，例如身体与场景之间的穿透。为解决这些局限性，我们提出了DeSeG，一个将语义意图与几何约束显式解耦的分层框架。首先，我们引入了一个残差语义规划器，将文本指令和规范化的目标体素编码到紧凑的潜在空间中，实现独立于空间轨迹的细粒度语义控制。其次，我们提出了一个物理正则化扩散执行器，将可微排斥势场直接融入扩散目标中，强制执行碰撞感知的运动生成。在Lingo数据集上的大量实验表明，DeSeG实现了最先进的性能，与现有最优方法相比，平均场景穿透减少了47%，语义对齐提升了29%。

Deep neural networks trained with Empirical Risk Minimization (ERM) often fail under distribution shifts because they exploit spurious correlations between object labels and background context. Recent generative approaches address this issue by creating counterfactual images with altered contexts, but typically use these samples as standard data augmentation, leaving the model free to retain background-sensitive representations. We propose a two-stage framework that uses generative intervention to explicitly learn background-invariant visual representations. First, we isolate the foreground object using zero-shot segmentation and generate context-shifted variants with a structure-preserving diffusion model, preserving object identity while varying the surrounding environment. We then introduce Cross-Variant Self-Supervised Learning, where variants of the same object under different backgrounds form positive pairs in a contrastive objective. This encourages the encoder to align object-centric representations while suppressing background-specific cues. Then, we fine-tune the pretrained encoder using an ERM warm-up followed by GroupDRO with layer-wise learning rates. Experiments on distribution-shift benchmarks demonstrate best worst-group performance, achieving 92.5% on Waterbirds, 81.7% on MetaShift, and 87.4% on NICO++. Code: https://github.com/surajyadav-research/GRSSL

使用经验风险最小化（ERM）训练的深度神经网络在面对分布偏移时往往表现不佳，因为它们会利用目标标签与背景上下文之间的虚假相关性。现有的生成式方法通过创建改变背景的反事实图像来解决这一问题，但通常将这些样本作为标准数据增强使用，使得模型仍可保留对背景敏感的表示。我们提出了一个两阶段框架，利用生成式干预显式学习背景不变的视觉表示。首先，我们使用零样本分割分离前景目标，并利用结构保持扩散模型生成上下文变换的变体，在改变周围环境的同时保持目标身份一致。随后，我们引入跨变体自监督学习，将在不同背景下同一目标的变体在对比目标中构成正样本对。这促使编码器对齐以目标为中心的表示，同时抑制背景特定的线索。然后，我们使用ERM预热后接组分布鲁棒优化（GroupDRO）和分层学习率对预训练编码器进行微调。在分布偏移基准测试上的实验表明，我们的方法取得了最佳的最差组性能，在Waterbirds上达到92.5%，在MetaShift上达到81.7%，在NICO++上达到87.4%。代码：https://github.com/surajyadav-research/GRSSL

Text-to-image diffusion models have achieved high visual fidelity and broad adoption, but remain vulnerable to safety violations when adversaries exploit them to synthesize illicit content. Existing alignment paradigms, from input sanitization to structural feature pruning, are largely organized around unsafe concepts explicitly exposed during filtering, editing, or localization. This leaves a blind spot for visual synonym attacks (VSA), a jailbreak where benign-looking prompts elicit prohibited imagery through implicit visual associations. As a result, current defenses face a safety-utility dilemma: they may either under-mitigate VSA threats or over-suppress visually similar benign concepts. The core challenge is that VSA hides the unsafe target at the textual surface while revealing it through generation-time visual-semantic convergence. In this work, we therefore shift from static suppression of pre-specified unsafe concepts to dynamic tracing of how unsafe semantics emerge during generation. Our mechanistic analysis shows that VSA and explicit unsafe prompts converge through sparse semantic-injecting attention heads, which serve as inference-time bottlenecks for prohibited visual semantics. Based on this insight, we propose AEGIS (Adaptive Evasion Guard via Identification and Steering), an inference-time defense that applies similarity-aware repulsion only at the identified vulnerable heads. Evaluated against 16 baselines, AEGIS improves both safety and utility. On SD 1.4, it reduces ASR to $\mathbf{0.00}/\mathbf{0.03}$ for in-domain violence/nudity VSA and achieves ASRs $\le \mathbf{0.09}$ on out-of-domain explicit and adversarial attacks. It preserves benign fidelity, avoids suppressing hard-negative concepts, and transfers to SD 2.1 and FLUX.1 after re-identifying the critical heads for each backbone.

文生图扩散模型已实现较高的视觉保真度并获得广泛应用，但在对抗者利用其合成非法内容时仍存在安全漏洞。现有的对齐范式（包括输入净化、结构特征剪枝等）主要围绕在过滤、编辑或定位过程中显式暴露的不安全概念展开。这为视觉同义词攻击（VSA）留下了一个盲区——这是一种越狱攻击，其中看似无害的提示通过隐含的视觉关联来唤起被禁止的图像。因此，当前防御面临安全-效用困境：它们要么对VSA威胁 Mitigation 不充分，要么过度抑制视觉相似的良性概念。核心挑战在于VSA在文本表面隐藏不安全目标，而在生成时通过视觉-语义收敛将其揭示出来。本工作因此从静态抑制预定义不安全概念转向动态追踪不安全语义在生成过程中的涌现。我们的机制分析表明，VSA与显式不安全提示通过稀疏语义注入注意力头收敛，这些头是禁止性视觉语义在推理时的瓶颈。基于这一见解，我们提出了AEGIS（通过识别与引导的自适应规避防护），这是一种推理时防御，仅在已识别的脆弱头上应用相似度感知的排斥。在16个基线模型上进行的评估表明，AEGIS同时提升了安全性和效用。在SD 1.4上，它将域内暴力/裸露VSA的攻击成功率分别降低至0.00/0.03，并在域外显式和对抗性攻击上达到≤0.09的攻击成功率。它保持了良性保真度，避免了抑制困难负例概念，并可迁移至SD 2.1和FLUX.1（在为每个骨干网络重新识别关键头后）。

Recent diffusion-based generative models have shown impressive performance in image generation and editing. However, due to memory limitations and the high cost of collecting high-resolution training images, existing methods are typically restricted to inputs with linear resolutions below 1K. In contrast, photos captured by modern mobile devices often reach linear resolutions up to 8K, revealing a significant gap between current capabilities and real-world demands. Simply upscaling low-resolution edited results often results in visually enlarged but blurry images that lack fine details. This paper introduces UltraDiffEdit, a novel, tuning-free image editing framework that extends off-the-shelf latent diffusion models (LDMs) to ultrahigh resolutions. UltraDiffEdit employs a multi-scale progressive editing strategy, iteratively blending high-resolution edited content with unedited areas in a coarse-to-fine manner. We employ multi-patch encoding to preserve both edited and unedited visual details within the latent space. To mitigate editing artifacts, our global-local consistency denoising technique consistently integrates edited and unedited latent features, ensuring smooth transition at editing boundaries from the latent representation to the final image. We also introduce a patch-based hybrid sampling approach that captures local, intermediate, and global features, ensuring semantic coherence and enhancing fine detail during denoising. We conduct extensive experiments demonstrating UltraDiffEdit's superior editing quality and flexibility: it can handle image resolutions up to 8K using only a single NVIDIA GeForce RTX 3090 GPU. The source code is publicly available at https://github.com/LonglongaaaGo/UltraDiffEdit.

近年来，基于扩散的生成模型在图像生成与编辑领域展现出卓越的性能。然而，由于内存限制以及高分辨率训练图像采集成本高昂，现有方法通常仅能处理线性分辨率低于1K的输入图像。相比之下，现代移动设备拍摄的照片线性分辨率可高达8K，这表明当前能力与实际需求之间存在显著差距。简单地放大低分辨率编辑结果往往会产生视觉尺寸增大但细节模糊的图像，缺乏精细纹理。本文提出UltraDiffEdit，这是一种创新的免调参图像编辑框架，能够将现成的潜在扩散模型（LDMs）扩展至超高分辨率处理。UltraDiffEdit采用多尺度渐进式编辑策略，以从粗到精的方式迭代混合高分辨率编辑内容与未编辑区域。我们采用多patch编码技术以在潜在空间中保留编辑与未编辑区域的视觉细节。为抑制编辑伪影，我们的全局-局部一致性去噪方法持续整合编辑与未编辑的潜在特征，确保从潜在表示到最终图像在编辑边界处实现平滑过渡。我们还提出一种基于patch的混合采样方法，该方法能够捕捉局部、中间层和全局特征，确保语义一致性并增强去噪过程中的细节表现。我们进行了大量实验，结果表明UltraDiffEdit具有卓越的编辑质量和灵活性：仅需一块NVIDIA GeForce RTX 3090 GPU即可处理最高8K分辨率的图像。源代码已在https://github.com/LonglongaaaGo/UltraDiffEdit公开发布。

Image outpainting extends an image beyond its original borders, requiring seamless style integration and globally coherent scene completion. Building on the success of diffusion models, recent methods have achieved substantial improvements in visual quality. In practice, however, high-resolution outpainting is commonly performed via progressive expansion around a fixed source image, particularly in artwork scenarios. Despite this progress, existing approaches still suffer from three key limitations: (i) the absence of a reliable global planning mechanism, which leads to structural instability and error accumulation at high resolutions; (ii) limited spatial controllability beyond text prompts, making it difficult to place objects at user-specified locations; and (iii) high inference latency caused by inherently sequential patch generation. To address these issues, we propose a global blueprint-guided two-stage diffusion framework for layout-controllable high-resolution outpainting with efficient parallel synthesis. In Stage 1, we generate a low-resolution global blueprint using a layout adapter that injects bounding-box conditions into a Stable Diffusion inpainting backbone, producing a globally consistent structural plan while extracting global guidance features. In Stage 2, we synthesize high-resolution local patches in parallel by injecting the blueprint-derived global guidance and initializing each patch from the blueprint using the low-frequency preservation property of forward diffusion. This design eliminates sequential dependency while maintaining global coherence. Extensive experiments on large-scale artwork datasets demonstrate improved visual fidelity, stronger semantic consistency, and substantially reduced inference time compared to prior baselines, while uniquely supporting explicit layout control for artwork outpainting.

图像外扩旨在将图像扩展至其原始边界之外，要求实现无缝的风格集成与全局一致的场景完成。建立在扩散模型成功的基础之上，现有方法已在视觉质量方面取得了实质性提升。然而在实际应用中，高分辨率外扩通常围绕固定源图像进行渐进式扩展，尤其是在艺术作品场景中。尽管取得了上述进展，现有方法仍存在三个关键局限：（i）缺乏可靠的全局规划机制，导致高分辨率下的结构不稳定性和误差累积；（ii）除文本提示外空间可控性有限，难以将对象放置在用户指定的位置；（iii）固有的顺序分块生成导致推理延迟较高。针对这些问题，我们提出了一种全局蓝图引导的两阶段扩散框架，实现布局可控的高分辨率图像外扩与高效的并行合成。在第一阶段，我们使用布局适配器将边界框条件注入Stable Diffusion修复主干，生成低分辨率全局蓝图，在提取全局引导特征的同时产生全局一致的结构规划。在第二阶段，我们通过注入蓝图导出的全局引导特征，并利用前向扩散的低频保持特性从蓝图初始化各分块，并行合成高分辨率局部区域。该设计消除了顺序依赖性，同时保持了全局一致性。在大规模艺术作品数据集上的广泛实验表明，与现有基线方法相比，本方法在视觉保真度、语义一致性方面均有提升，且推理时间大幅降低，同时首次实现了艺术作品外扩的显式布局控制。

Recent advances in video diffusion have been driven by scaling transformer-based architectures to billions of parameters, substantially improving visual fidelity and motion coherence. In contrast, existing mobile video diffusion models remain limited to relatively small parameter budgets, typically 0.4-1.8B, restricting generation quality. In this work, we show that high-quality mobile video generation does not require small models. Instead, we demonstrate that a server-scale 5B-parameter video diffusion transformer can be deployed efficiently on memory-constrained mobile hardware through recurrent reformulation and structured compression. Starting from Wan2.2-5B, we rely on a recurrence distillation framework that converts video generation into a chunk-wise autoregressive process with constant-memory attention computation. Combined with causal linear attention, the model operates as an RNN at inference time while preserving temporal coherence across chunks. We further propose a learnable attention head pruning method based on binary per-head gates optimized end-to-end using a noise-biased sparsity objective and distillation-based finetuning. Together with sampling-step distillation and memory-optimized VAE decoding, MobileWan becomes the first 5B-scale video diffusion model deployable on a commercial mobile device. Our system generates 5-second 480x832 videos at 16 FPS in 20 seconds end-to-end latency, achieving a VBench score of 83.79 and establishing a new state of the art in mobile video generation. Project page: https://qualcomm-ai-research.github.io/mobilewan

近年来，视频扩散模型的进展得益于将基于Transformer的架构扩展至数十亿参数，显著提升了视觉保真度和运动连贯性。相比之下，现有的移动视频扩散模型仍受限于相对较小的参数规模，通常为0.4-1.8B，制约了生成质量。本工作表明，高质量移动视频生成并不需要小型模型。相反，我们证明了通过循环重构和结构化压缩，5B参数的服务器级视频扩散Transformer可以高效部署在内存受限的移动硬件上。以Wan2.2-5B为起点，我们采用循环蒸馏框架，将视频生成分解为分块自回归过程，并实现常量内存的注意力计算。结合因果线性注意力，模型在推理时如同RNN一般运行，同时保持跨分块的时间一致性。我们进一步提出基于二元逐头门控的可学习注意力头剪枝方法，采用噪声偏向的稀疏目标进行端到端优化，并结合基于蒸馏的微调。结合采样步蒸馏和内存优化VAE解码，MobileWan成为首个可在商业移动设备上部署的5B规模视频扩散模型。我们的系统在20秒端到端延迟内生成5秒480x832分辨率16FPS的视频，达到VBench分数83.79，树立了移动视频生成的新标杆。项目主页：https://qualcomm-ai-research.github.io/mobilewan

The future of World Models depends not only on scaling model capability, but also on scaling practicality and inference efficiency. High-frame-rate inference enables responsive perception, planning, and control in real-world autonomous systems. To this end, we present MoWorld, a cost-effective yet high-performance Flash World Model with an end-to-end framework spanning data generation, pre-training, distillation, and efficient inference, enabling up to 50 FPS real-time interaction with cinematic visual quality without the need of high-end GPUs. To enable large-scale real-world deployment, MoWorld jointly optimizes model capability and cost throughout the entire development pipeline. Specifically, unlike existing approaches that primarily rely on large-scale video corpora, MoWorld is built upon a scalable 3D-native data engine accumulated from our large-scale 3D vision and generative modeling pipeline, enabling the efficient construction of geometrically consistent training data across diverse real-world and synthetic environments. Based on this foundation, a curriculum cross-frame pre-training strategy for stable and scalable World Model learning, an efficient denoising-step distillation algorithm to reduce diffusion training cost, and a mixed-precision parallel inference framework for low-cost real-time deployment. MoWorld is the first real-time interactive World Model built on the Neural Processing Unit (NPU) and can achieves up to 50 FPS in such the devices, enabling practical and efficient deployment at scale. Comprehensive evaluations demonstrate that MoWorld achieves leading performance; notably, its average inference cost is only 30\%-50\% of that of existing World Models, providing a practical foundation for large-scale real-world applications of World Models. We also demonstrate diverse applications of MoWorld.

世界模型的未来不仅取决于模型能力的规模化，还取决于实用性和推理效率的规模化。高帧率推理能够在真实世界自治系统中实现响应式感知、规划与控制。为此，我们提出MoWorld，一个成本效益高但性能卓越的极速世界模型，具有涵盖数据生成、预训练、蒸馏和高效推理的端到端框架，能够以电影级视觉质量实现高达50 FPS的实时交互，且无需高端GPU。为支持大规模真实世界部署，MoWorld在整个开发流程中联合优化模型能力与成本。具体而言，与主要依赖大规模视频语料库的现有方法不同，MoWorld基于可扩展的3D原生数据引擎构建，该引擎源自我们的大规模3D视觉与生成建模流程，能够跨多样真实世界和合成环境高效构建几何一致的训练数据。在此基础上，我们提出了用于稳定且可扩展世界模型学习的课程跨帧预训练策略、用于降低扩散训练成本的高效去噪步蒸馏算法，以及用于低成本实时部署的混合精度并行推理框架。MoWorld是首个在神经处理单元（NPU）上运行的实时交互世界模型，在此类设备上能达到50 FPS的性能，实现了大规模的实际高效部署。全面评估表明，MoWorld取得了领先性能；值得注意的是，其平均推理成本仅为现有世界模型的30%至50%，为世界模型的大规模真实世界应用提供了实用基础。我们还展示了MoWorld的多样化应用。

Incomplete Multi-View Clustering addresses the problem of clustering multi-modal data when certain views are missing. Recent end-to-end generative approaches leverage diffusion models to recover missing views via stochastic noise-to-data trajectories. While expressive, such mechanisms are not explicitly designed for clustering, as they initialize from cluster-agnostic noise and rely on stochastic denoising dynamics. In this work, we revisit probability path design in end-to-end generative IMVC. We introduce a flow-matching framework with a linear interpolation path between paired view representations, that replaces diffusion with probability flows between observed and missing views. We provide a formal analysis showing that deterministic ODE flows are inherently better aligned with clustering objectives than diffusion-based stochastic trajectories, especially in terms of transport mechanisms that respect class-conditional data distributions and maintain cluster consistency in finite-step regimes. Building upon this insight, we develop an end-to-end IMVC architecture that integrates straight-path flow-matching view completion with cluster-level and entropy-based alignment to enforce cross-view clustering consistency. Extensive experiments on standard IMVC benchmarks demonstrate that the proposed framework establishes new state-of-the-art performance.

不完全多视角聚类旨在处理某些视角缺失的多模态数据的聚类问题。近期提出的端到端生成方法利用扩散模型通过随机噪声到数据轨迹来恢复缺失视角。然而，尽管这类方法具有较强的表达能力，却并非专门为聚类任务设计，因为它们从与聚类无关的噪声开始初始化，并依赖随机去噪动力学机制。在本工作中，我们重新审视端到端生成式不完全多视角聚类中的概率路径设计。我们引入了一种流匹配框架，该框架使用配对视角表示之间的线性插值路径，用概率流替代扩散机制。我们进行了形式化分析，表明确定性常微分方程流在本质上比基于扩散的随机轨迹更好地与聚类目标对齐，尤其是在传输机制能够尊重类条件数据分布并在有限步条件下保持聚类一致性方面。基于这一洞察，我们开发了一种端到端不完全多视角聚类架构，该架构将直线流匹配视角补全与聚类级和基于熵的对齐相结合，以增强跨视角聚类一致性。在标准不完全多视角聚类基准数据集上的大量实验表明，所提出的框架达到了新的最先进性能。

Diffusion models generate high-quality images, but their inference cost comes from two sources: large denoising networks and repeated denoising steps. Existing compression pipelines usually attack these costs separately. Pruning reduces the network, but most pruning methods still rely on a long post-pruning retraining stage to recover a many-step sampler. Step distillation reduces the number of denoising steps, but it usually assumes a student that can already follow the teacher well enough to receive useful distillation gradients. This paper asks whether post-pruning retraining can be replaced by step distillation. We find that the direct replacement fails: after pruning an EDM2-XS teacher, starting SiDA from the pruned checkpoint produces unusable samples. We introduce a short teacher-alignment repair stage as a bridge between pruning and step distillation. The bridge matches the pruned generator to the teacher on noisy real-image latents, then hands the repaired checkpoint to one-step distillation. On ImageNet-512, the original EDM2-XS baseline uses 124.713M parameters and 63 network evaluations, reaching an FID of 3.53. With a suitable distillation objective, our 20% pruned one-step generator uses 98.826M parameters and one network evaluation, reaching an FID of 3.12. With 30% pruning, the model uses 88.029M parameters and one network evaluation, with an FID of 4.26.

扩散模型能够生成高质量图像，但其推理成本来自两个方面：大型去噪网络和重复的去噪步骤。现有压缩流水线通常分别处理这些成本。剪枝虽然可以减小网络规模，但大多数剪枝方法仍需依赖较长的剪枝后重训练阶段来恢复多步采样器。步骤蒸馏可以减少去噪步骤数，但它通常假设学生模型已经能够很好地跟随教师模型以接收有效的蒸馏梯度。本文探讨是否可以用步骤蒸馏替代剪枝后的重训练。研究发现直接替换不可行：在对EDM2-XS教师模型进行剪枝后，从剪枝检查点启动SiDA会产生无用的样本。本文引入了一个短暂的教师对齐修复阶段作为剪枝与步骤蒸馏之间的桥梁。该桥梁在含噪真实图像潜在表示上使剪枝后的生成器与教师模型对齐，然后将修复后的检查点交给一步蒸馏。在ImageNet-512上，原始EDM2-XS基线使用1.24713亿参数和63次网络评估，达到了3.53的FID。使用合适的蒸馏目标，我们的20%剪枝一步生成器使用9882.6万参数和1次网络评估，达到了3.12的FID。在30%剪枝下，模型使用8802.9万参数和1次网络评估，FID为4.26。

Video face restoration (VFR) aims to recover high-quality and temporally consistent facial details from severely degraded video sequences; however, existing methods still struggle to balance spatial fidelity and temporal coherence under complex degradations. To address this, we propose FADRA, a frequency-aware diffusion framework with iterative residual adaptation specifically tailored for robust VFR. We first leverage the strong temporal consistency of a pre-trained text-to-video diffusion model and introduce lightweight LoRA adapters together with a Low-Quality (LQ) Pixel-Alignment Feature Fusion module to efficiently adapt the frozen generative prior to the VFR task. To further adapt the frozen diffusion backbone to the downstream VFR task beyond LoRA-based adaptation, we introduce a Repeated Residual Adaptation Head (RRAH) for step-wise residual refinement after the diffusion backbone. To make this refinement explicitly guided by the degraded observation, RRAH further takes the LQ latent together with the current velocity prediction as input, allowing the model to repeatedly revisit LQ cues and predict residual updates at each flow-matching step. This LQ-guided repeated residual adaptation helps recover fine facial details while preserving the inherent temporal priors of the pre-trained model. Furthermore, to ensure the structural integrity of perceptually important details, we introduce a Frequency-Aware Loss that provides explicit supervision across multiple spectral bands, emphasizing visually sensitive frequency components that are crucial for perceptual quality and prone to temporal jittering. Extensive experiments demonstrate that FADRA recovers better facial structures and produces more temporally consistent videos than state-of-the-art methods, leading to clear gains in both quantitative metrics and visual perception.

视频人脸修复旨在从严重退化的视频序列中恢复高质量且时间一致的人脸细节，然而现有方法在复杂退化条件下仍难以平衡空间保真度与时间连贯性。为此，我们提出FADRA，一种专门为鲁棒视频人脸修复设计的频率感知扩散框架，采用迭代残差自适应方法。我们首先利用预训练文本到视频扩散模型的强时间一致性，并引入轻量级LoRA适配器与低质量像素对齐特征融合模块，以有效将冻结的生成先验适配到视频人脸修复任务。为了在LoRA自适应基础上进一步将冻结的扩散主干适配到下游视频人脸修复任务，我们在扩散主干后引入重复残差自适应头部进行逐步残差细化。为了使该细化过程明确受到退化观测的引导，重复残差自适应头部进一步将低质量潜码与当前速度预测作为输入，使模型能够在每个流匹配步骤中反复访问低质量线索并预测残差更新。这种低质量引导的重复残差自适应有助于恢复精细的人脸细节，同时保留预训练模型固有的时间先验。此外，为确保感知重要细节的结构完整性，我们引入频率感知损失函数，在多个频谱带上提供显式监督，强调对感知质量至关重要且易发生时间抖动的敏感频率分量。大量实验表明，FADRA能够恢复更好的人脸结构并生成时间更一致的视频，在定量指标和视觉感知方面均优于现有最先进方法。

Evaluating end-to-end autonomous driving (E2E-AD) remains challenging, as existing driving simulation methods often trade off closed-loop interactivity (e.g., CARLA) and real-world visual fidelity (e.g., nuScenes). We present \textbf{\emph{Point as Skeleton}}, a generative sensor simulation framework for state-updated autoregressive driving video generation, in which an autoregressive generator synthesizes visual observations from step-wise updated ego states, actor states, scene maps, and point-cloud skeleton conditions. To support closed-loop rollout, we introduce Reset-and-Roll, which adapts rolling diffusion inference to simulation by preventing future-conditioned latent states from being committed across simulation steps. To stabilize error accumulation during step-wise autoregressive rollout, we introduce point-cloud skeletons that decouple foreground and background assets and project them into camera-view painted-point and template-depth conditions, providing appearance and geometric cues. We further implement a nuPlan-based renderer-level closed-loop generative interface for evaluating generation under ego deviations from the original log. Experiments on nuScenes and nuPlan show that \textit{Point as Skeleton} improves autoregressive generation quality during closed-loop rollout, demonstrating its potential for visually faithful closed-loop driving simulation. The code is available at https://github.com/krauwu/point-as-skeleton.

端到端自动驾驶（E2E-AD）的评估仍具挑战性，因为现有驾驶仿真方法通常在闭环交互性（如CARLA）和真实世界视觉保真度（如nuScenes）之间难以兼顾。我们提出点即骨架（Point as Skeleton），一个用于状态更新自回归驾驶视频生成的生成式传感器仿真框架，其中自回归生成器根据逐步更新的自车状态、参与者状态、场景地图和点云骨架条件来合成视觉观测结果。为支持闭环 rollout，我们引入Reset-and-Roll方法，通过防止未来条件的潜在状态在仿真步骤之间被提交，将滚动扩散推理适配到仿真中。为稳定逐步自回归rollout过程中的误差累积，我们引入点云骨架来解耦前景和背景资产，并将它们投影到相机视角的绘制点和模板深度条件中，提供外观和几何线索。我们进一步实现了基于nuPlan的渲染器级闭环生成接口，用于在自车偏离原始日志的情况下评估生成质量。在nuScenes和nuPlan上的实验表明，点即骨架（Point as Skeleton）在闭环rollout期间提升了自回归生成质量，展示了其在视觉保真闭环驾驶仿真中的潜力。代码可访问 https://github.com/krauwu/point-as-skeleton。

Tracking the six-degree-of-freedom (6-DoF) pose of objects and surfaces from monocular video is a long-standing problem in computer vision. To tackle this problem, existing methods require inputs beyond the video itself-such as 3D models, depth maps, object masks, or task-specific learned features-and they struggle with textureless, transparent, reflective, or deformable surfaces. Here, we introduce ProxyPose, which recasts 6-DoF pose tracking as video-to-video translation. Given only a video and a single marked pixel in the first frame, a fine-tuned video diffusion model translates the input into a proxy video-a synthetic video depicting a colored polyhedron undergoing the same local rigid-body motion as the surface region at the marked pixel. Because the proxy's geometry and appearance are known by construction, recovering its full 6-DoF trajectory reduces to classical pose estimation with off-the-shelf solvers. This formulation leverages large-scale video pre-training to absorb the hardest aspects of pose tracking-handling challenging materials, occlusions, and deformations-into the translation step, while operating at the pixel level with no assumptions about object identity, boundaries, or global rigidity. ProxyPose achieves state-of-the-art 6-DoF pose tracking accuracy without the additional inputs required by competing methods and after fine-tuning the video model only on synthetic data. We further demonstrate that ProxyPose extends to face tracking, camera pose estimation, and challenging in-the-wild scenes that are beyond the reach of existing approaches. Project page: https://ruihangzhang97.github.io/proxypose/.

从单目视频中跟踪物体和表面的六自由度（6-DoF）姿态是计算机视觉中的一个长期问题。为解决这一问题，现有方法需要输入超出视频本身的内容——如三维模型、深度图、目标掩码或任务特定的学习特征——且难以处理无纹理、透明、反光或可变形表面。本研究提出ProxyPose，将6-DoF姿态跟踪重新表述为视频到视频翻译任务。给定一段视频和第一帧中的单个标记像素，经过微调的视频扩散模型将输入翻译为代理视频——一段合成视频，描绘一个彩色多面体在与标记像素所在表面区域相同的局部刚体运动下运动。由于代理的几何形状和外观是已知构建的，恢复其完整6-DoF轨迹可简化为使用现成求解器的经典姿态估计问题。该方法利用大规模视频预训练，将姿态跟踪中最困难的部分——处理具有挑战性的材质、遮挡和变形——吸收到翻译步骤中，同时在像素级别上运行，不对目标身份、边界或全局刚性做任何假设。ProxyPose在无需竞争方法所需额外输入的情况下实现了最先进的6-DoF姿态跟踪精度，且仅在合成数据上对视频模型进行了微调。本研究进一步证明ProxyPose可扩展至人脸跟踪、相机姿态估计以及现有方法难以处理的挑战性真实场景。项目主页：https://ruihangzhang97.github.io/proxypose/

Diffusion models have become a dominant paradigm for high-quality generative modeling, while post-training is essential for adapting them to diverse downstream applications. However, post-training of large diffusion models is still challenging due to the prohibitive memory footprints and slow training speed, which existing parameter-efficient fine-tuning methods only partially address. To overcome these limitations, we propose FourTune, an efficient post-training framework for diffusion models based on an end-to-end W4A4G4 paradigm. FourTune introduces a triple-branch hybrid pipeline that augments the standard LoRA architecture with a frozen numerical stabilizer to isolate quantization-sensitive outliers, enabling stable training under native 4-bit computation. In addition, FourTune employs hardware-efficient block-wise quantization and customized fused kernels to support efficient quantized backpropagation and reduce memory bandwidth overhead. Across customization, reinforcement learning, and distillation tasks, FourTune matches the quality of full-precision fine-tuning. On FLUX.1-dev (12B), FourTune reduces memory overhead by 2.25$\times$ and increases end-to-end training throughput by 2.27$\times$ compared to BF16 LoRA.

扩散模型已成为高质量生成建模的主流范式，而后训练对于将其适配到不同的下游应用至关重要。然而，大型扩散模型的后训练仍具有挑战性，因为其内存占用过高且训练速度缓慢，现有的参数高效微调方法只能部分解决这些问题。为了克服这些局限性，我们提出了FourTune，一个基于端到端W4A4G4范式的扩散模型高效后训练框架。FourTune引入了一种三分支混合管道，通过在标准LoRA架构中增添加固的数值稳定器来隔离量化敏感的离群值，从而在原生4比特计算下实现稳定训练。此外，FourTune采用硬件高效的块量化方法和定制化的融合内核，以支持高效的量化反向传播并降低内存带宽开销。在定制化、强化学习和蒸馏任务中，FourTune达到了与全精度微调相当的质量水平。在FLUX.1-dev（12B）上，相比BF16 LoRA，FourTune将内存开销降低了2.25倍，并将端到端训练吞吐量提升了2.27倍。

Talking head generation has progressed rapidly from landmark- and GAN-based facial animation to diffusion models, neural rendering, 3D-aware avatars, and foundation-model-assisted systems. This progress has enabled increasingly realistic audio-, image-, and video-driven talking heads, but it has also made the field difficult to navigate because methods differ substantially in their inputs, assumptions, controllability, temporal stability, computational cost, and risks of misuse. This survey provides a critical review of talking head generation techniques, organizing the literature into four broad families: image-driven, audio-driven, video-driven, and 3D/neural-rendering-based approaches. For each family, we discuss the underlying technical ideas, representative methods, strengths, limitations, datasets, and evaluation practices. Beyond cataloguing prior work, we analyse the persistent gap between commonly reported quantitative metrics and perceptual quality, and compare publicly available models in terms of inference time, memory requirements, and human-rated visual quality. We also examine emerging trends, including diffusion-based generation, 3D-aware representation learning, controllable emotional expression, real-time deployment, and the growing importance of provenance, watermarking, and deepfake detection. Finally, we identify open challenges around robust evaluation, identity preservation, lip synchronisation, temporal consistency, demographic fairness, computational efficiency, and responsible use. This review aims to provide researchers and practitioners with a structured and up-to-date map of the talking head generation landscape, while highlighting the technical and societal questions that should shape future work.

虚拟人脸生成技术已从基于关键点和生成对抗网络的面部动画快速发展至扩散模型、神经渲染、3D感知虚拟形象以及基础模型辅助系统。这一进展使得基于音频、图像和视频驱动的虚拟人脸日益逼真，但由于不同方法在输入、假设、可控性、时间稳定性、计算成本和滥用风险等方面存在显著差异，该领域变得难以全面把握。本综述对虚拟人脸生成技术进行了系统性评述，将现有研究归纳为四大类：图像驱动方法、音频驱动方法、视频驱动方法和基于3D/神经渲染的方法。针对每一类别，我们讨论了其底层技术思想、代表性方法、优势与局限、数据集以及评估实践。除了梳理既往工作，我们还分析了常见定量指标与感知质量之间持续存在的差距，并从推理时间、内存需求和人类主观视觉质量等方面对公开可用的模型进行了比较。我们还探讨了新兴趋势，包括基于扩散的生成、3D感知表示学习、可控情感表达、实时部署，以及来源追溯、水印和深度伪造检测日益增长的重要性。最后，我们识别了开放性挑战，涵盖鲁棒评估、身份保持、唇形同步、时间一致性、人口统计公平性、计算效率和负责任使用等方面。本综述旨在为研究人员和从业者提供虚拟人脸生成领域的结构化最新全景图，同时突出应塑造未来工作的技术和社会问题。

While diffusion models dominate the field of visual generation, they are computationally inefficient, applying a uniform computational effort regardless of different complexity. In contrast, autoregressive (AR) models are inherently complexity-aware, as evidenced by their variable likelihoods, but are often hindered by lossy discrete tokenization and error accumulation. In this work, we introduce Generative Refinement Networks (GRN), a next-generation visual synthesis paradigm that addresses these issues. At its core, GRN addresses the discrete tokenization bottleneck through a theoretically near-lossless Hierarchical Binary Quantization (HBQ), achieving a reconstruction quality comparable to continuous counterparts. Built upon HBQ's latent space, GRN fundamentally upgrades AR generation with a global refinement mechanism that progressively perfects and corrects artworks -- like a human artist painting. Besides, GRN integrates an entropy-guided sampling strategy, enabling complexity-aware, adaptive-step generation without compromising visual quality. On the ImageNet benchmark, GRN establishes new records in image reconstruction (0.56 rFID) and class-conditional image generation (1.81 gFID). We also scale GRN to more challenging text-to-image and text-to-video generation, delivering superior performance on an equivalent scale. We release all models and code to foster further research on GRN.

尽管扩散模型在视觉生成领域占据主导地位，但它们计算效率低下，无论样本复杂度如何都采用统一的计算量。相比之下，自回归（AR）模型本身具有复杂度感知能力，这一点从其可变的似然值可以得到验证，但其发展常受限于有损的离散tokenization和误差累积问题。本工作提出了生成式精炼网络（GRN），一种新一代视觉合成范式，用以解决上述问题。GRN的核心是通过理论近无损的层次二元量化（HBQ）来解决离散tokenization瓶颈，达到与连续方法相当的重建质量。基于HBQ的潜在空间，GRN从根本上升级了自回归生成能力，采用全局精炼机制逐步完善和修正艺术作品——如同人类画家作画一般。此外，GRN集成了熵引导采样策略，实现了复杂度感知的自适应步长生成，同时不影响视觉质量。在ImageNet基准上，GRN在图像重建（0.56 rFID）和类别条件图像生成（1.81 gFID）方面创下了新纪录。我们还将GRN扩展到更具挑战性的文本到图像和文本到视频生成任务，在同等规模下提供了卓越的性能。我们发布了所有模型和代码，以促进GRN的进一步研究。

Low-Rank Adaptation (LoRA) has become a widely used mechanism for customizing diffusion models, enabling users to inject new visual concepts or styles through lightweight parameter updates. However, LoRAs can memorize training images, causing generated outputs to reproduce copyrighted or sensitive content. This risk is particularly concerning in LoRA-sharing ecosystems, where users distribute trained LoRAs without releasing the underlying training data. Existing approaches for mitigating memorization rely on access to the training pipeline, training data, or control over the inference process, making them difficult to apply when only the released LoRA weights are available. We propose \textbf{Base-Anchored Filtering (BAF)}, a training-free and data-free framework for post-hoc memorization mitigation in diffusion LoRAs. BAF decomposes LoRA updates into spectral channels and measures their alignment with the principal subspace of the pretrained backbone. Channels strongly aligned with this subspace are retained as generalizable adaptations, while weakly aligned channels are suppressed as potential carriers of memorized content. Experiments on multiple datasets and diffusion backbones demonstrate that BAF consistently reduces memorization while preserving or even improving generation quality. Our code is available in the supplementary material.

低秩适应（LoRA）已成为定制扩散模型的广泛使用机制，使用户能够通过轻量级参数更新注入新的视觉概念或风格。然而，LoRA可以记忆训练图像，导致生成输出复制受版权保护或敏感内容。这种风险在LoRA共享生态系统中尤为突出，用户在发布训练好的LoRA时不提供底层训练数据。现有的记忆化缓解方法依赖于访问训练流程、训练数据或对推理过程的控制，因此当仅有发布的LoRA权重可用时，这些方法难以应用。我们提出了无训练、无数据的LoRA记忆化事后缓解框架——基锚过滤（Base-Anchored Filtering，BAF）。BAF将LoRA更新分解为频谱通道，并测量它们与预训练主干网络主子空间的对齐程度。与该子空间强对齐的通道被保留为可泛化的适应，而弱对齐的通道被抑制为记忆内容的潜在载体。在多个数据集和扩散主干网络上的实验表明，BAF在保持甚至提升生成质量的同时持续减少记忆化。我们的代码在补充材料中提供。

Vision Transformers (ViTs) are known to exhibit high-norm patch-token outliers that degrade feature map quality, a problem effectively mitigated by register tokens. As diffusion models increasingly adopt transformer architectures and move toward pixel-space training, they become closer in form to ViTs, raising the question of whether register tokens are also useful for Diffusion Transformers (DiTs). In this work, we show that DiTs differ from ViTs in a key respect: they do not exhibit patch-token outliers but still benefit from registers. Interestingly, registers are more effective in pixel-space DiTs than in latent-space DiTs. By analyzing intermediate representations, we find that register tokens produce cleaner feature maps at high noise levels, which may contribute to their effectiveness in pixel-space generation. We further observe that recent pixel-space DiT architectures implicitly incorporate register-like mechanisms, which may partially account for their strong empirical performance. Motivated by these observations, we propose Register Guidance, a technique that amplifies the contribution of register tokens responsible for improving visual structure and coherence.

视觉Transformer（ViT）以高范数补丁标记异常值为特征，这种异常会降低特征图质量，而寄存器标记能够有效缓解这一问题。随着扩散模型越来越多地采用Transformer架构并转向像素空间训练，其形式与ViT越来越相似，这就引发了一个问题：寄存器标记是否对扩散Transformer（DiT）同样有效？本研究发现，DiT与ViT在一个关键方面存在差异：DiT虽然不会出现补丁标记异常值，但仍然能从寄存器中获益。有趣的是，像素空间DiT比潜空间DiT从寄存器中获益更多。通过分析中间表示，我们发现寄存器标记在高噪声水平下能够产生更清晰的特征图，这可能解释了它们在像素空间生成中的有效性。我们进一步观察到，近期像素空间DiT架构隐式地包含了类似寄存器的机制，这可能部分解释了其强大的实证性能。基于这些观察，我们提出了寄存器引导（Register Guidance）技术，该技术能够放大对改善视觉结构和一致性有贡献的寄存器标记的作用。

Recent advances in continuous conditional generative modeling, including Continuous conditional Generative Adversarial Network (CcGAN) and Continuous Conditional Diffusion Model (CCDM), estimate high-dimensional data distributions conditioned on scalar regression labels such as angles, ages, or temperatures. However, fixed-size vicinal training in CcGAN can be sensitive to non-uniform label densities, whereas CCDM relies on computationally expensive iterative sampling. To address these issues, we propose CcGAN-AVAR, an imbalance-aware extension of CcGAN that combines soft/hybrid adaptive vicinity with auxiliary discriminator-guided regularization. The adaptive vicinity constructs a label-dependent local radius according to the available samples around each target condition, and the multi-task discriminator supplies both a regression signal for label consistency and a density-ratio-estimation signal for distribution matching. We further provide a theoretical interpretation characterizing how adaptive vicinal weighting affects the local bias-variance behavior of the discriminator target, how hybrid truncation reduces objective-level cross-condition mixing, and how the density-ratio-based generator penalty approximates a Pearson Chi-square discrepancy up to the estimation error of the density-ratio branch. Extensive experiments on four datasets, including the newly constructed imbalanced RC-49-I, covering resolutions from 64x64 to 256x256 across eleven settings, demonstrate that CcGAN-AVAR obtains strong generation quality and label consistency while preserving the one-step sampling efficiency of GANs, achieving 300x--2000x faster inference than CCDM.

连续条件生成建模的最新进展，包括连续条件生成对抗网络（CcGAN）和连续条件扩散模型（CCDM），能够对基于标量回归标签（如角度、年龄或温度）的条件进行高维数据分布估计。然而，CcGAN中的固定大小邻域训练对非均匀标签密度较为敏感，而CCDM则依赖于计算成本较高的迭代采样。为解决这些问题，我们提出了CcGAN-AVAR，这是一种对CcGAN的抗不平衡扩展，结合了软/混合自适应邻域与辅助判别器引导的正则化方法。自适应邻域根据每个目标条件周围可用样本构建标签相关的局部半径，多任务判别器同时提供用于标签一致性的回归信号和用于分布匹配的密度比估计信号。我们进一步提供了理论解释，阐述了自适应邻域权重如何影响判别器目标的局部偏差-方差行为、混合截断如何减少目标级跨条件混合，以及基于密度比的生成器惩罚如何在密度比分支的估计误差范围内近似Pearson卡方散度。在四个数据集上进行了广泛实验，包括新构建的不平衡RC-49-I数据集，分辨率覆盖64×64至256×256的十一个设置，结果表明CcGAN-AVAR在保持GAN单步采样效率的同时获得了优异的生成质量和标签一致性，其推理速度比CCDM快300倍至2000倍。

Diffusion models rely heavily on explicit timestep embeddings to modulate the denoising process across various noise scales. In this work, we challenge the necessity of these temporal signals by analyzing their impact on U-Net and Diffusion Transformer architectures. Beyond empirical evidence, we provide a theoretical framework demonstrating that, under certain conditions, the global minimizer of the diffusion training objective can be achieved without explicit timestep conditioning. Our findings reveal a surprising robustness when timestep embeddings are completely removed. Extensive ablation studies on the CelebA and CIFAR-10 datasets show that these time-agnostic models can maintain high structural fidelity and even surpass their conditioned counterparts in competitive metrics, including FID, precision, and recall. Our analysis suggests these architectures can implicitly infer noise scales from the corrupted input under specific assumptions, rendering explicit temporal conditioning redundant. This study challenges long-standing temporal conditioning paradigms and paves the way for more efficient and structurally focused generative architectures.

扩散模型严重依赖显式的时间步嵌入来调节不同噪声尺度下的去噪过程。本研究通过分析时间信号对U-Net和Diffusion Transformer架构的影响，挑战了这些时间信号的必要性。除了实证证据外，我们还提供了一个理论框架，证明在某些条件下，扩散训练目标的全局最小化器可以在无需显式时间步条件的情况下实现。我们的发现揭示了时间步嵌入被完全移除时模型仍具有惊人的鲁棒性。在CelebA和CIFAR-10数据集上的广泛消融实验表明，这些时间无关模型能够保持较高的结构保真度，并在竞争性指标（包括FID、精确度和召回率）上甚至超越其带条件的对应模型。我们的分析表明，在某些假设下，这些架构能够从受损输入中隐式推断出噪声尺度，从而使显式的时间条件变得冗余。本研究挑战了长期存在的时间条件范式，为更高效、更专注于结构的生成架构开辟了道路。

Vertigo Vertigo is a scene-for-scene AI reconstruction of Hitchcock's Vertigo (1958), generated from only 2.78% of the original film&x27;s frames. Using this sparse set of keyframe anchors, we perform first-last frame interpolation via a large video diffusion model to predict the intervening sequences. Vertigo is itself a film about the obsessive reconstruction of an artificial ideal; Vertigo Vertigo extends this logic to the material of the film, treating the canonical text as a probe for the normative conventions of classical cinema encoded within generative systems. Evaluated through computational analysis and critical feedback from media theorists (Lev Manovich, Shane Denson, Kevin L. Ferguson), the artifact demonstrates remarkable structural fidelity: 73.1% of frames are recognizable as plausible renditions of Vertigo and only 3.6% fail catastrophically. This fidelity suggests that cinematic norms are deeply compressed within the model's latent priors. Aesthetically, the reconstruction is rendered as an unstable overlay between the original film and its predictive shadow, fueling a persistent doubt in the viewer&x27;s perception of authenticity -- a 21st-century vertigo. The work argues that generative media is not a paradigm shift from cinema but an acceleration of its logic of desire and false authenticity, extending from classical Hollywood through to the predictive media environments now reshaping contemporary perception.

《迷魂记迷魂记》是一部逐场景的人工智能重建作品，生成仅使用了希区柯克1958年电影《迷魂记》2.78%的帧。利用这一稀疏的关键帧锚点集，我们通过大型视频扩散模型进行首尾帧插值，以预测中间序列。《迷魂记》本身就是一部关于执着重建人工理想的电影；《迷魂记迷魂记》将这一逻辑延伸至电影本身，将这部经典文本作为探针，探测生成系统内所编码的古典电影规范惯例。通过计算分析以及媒体理论家（Lev Manovich、Shane Denson、Kevin L. Ferguson）的批评反馈，该作品展示了显著的结构保真度：73.1%的帧可被识别为《迷魂记》的可信呈现，仅有3.6%出现严重失败。这种保真度表明电影规范深藏于模型的潜在先验之中。在美学上，重建呈现出原电影与其预测阴影之间的不稳定叠加，激发了观众对真实性的持续质疑——一种21世纪的眩晕感。该作品认为生成媒体并非电影范式的转变，而是其欲望与虚假真实性逻辑的加速，从古典好莱坞延伸至正在重塑当代感知的预测性媒体环境。

Gaussian-based image representations effectively model image content using compact parametric primitives while preserving high visual fidelity, yet storing a large number of floating-point parameters per primitive degrades rate-distortion efficiency at higher fidelity targets. To improve the rate-distortion performance in Gaussian representation, we present our Cluster-Guided Vector Quantization (CGVQ), a Gaussian primitive based image compression method. Our key idea is to partition Gaussian parameters further into homogeneous groups prior to quantization, enabling higher compression efficiency and accurate parameter reconstruction. In practice, our extensive experiments show that CGVQ decreases the bpp by 20% with respect to our baseline, while maintaining on-par visual quality

基于高斯的图像表示通过紧凑的参数化基元有效建模图像内容，同时保持较高的视觉保真度，然而每个基元存储大量浮点参数会在较高保真度目标下降低率失真效率。为了提升高斯表示中的率失真性能，我们提出了聚类引导向量量化（CGVQ），这是一种基于高斯基元的图像压缩方法。我们的核心思路是在量化前将高斯参数进一步划分为同质组，从而实现更高的压缩效率和精确的参数重建。在实际应用中，我们的广泛实验表明，与基线相比，CGVQ在保持相当视觉质量的同时，将bpp降低了20%。

Diffusion models generate high-quality images, but their inference cost comes from two sources: large denoising networks and repeated denoising steps. Existing compression pipelines usually attack these costs separately. Pruning reduces the network, but most pruning methods still rely on a long post-pruning retraining stage to recover a many-step sampler. Step distillation reduces the number of denoising steps, but it usually assumes a student that can already follow the teacher well enough to receive useful distillation gradients. This paper asks whether post-pruning retraining can be replaced by step distillation. We find that the direct replacement fails: after pruning an EDM2-XS teacher, starting SiDA from the pruned checkpoint produces unusable samples. We introduce a short teacher-alignment repair stage as a bridge between pruning and step distillation. The bridge matches the pruned generator to the teacher on noisy real-image latents, then hands the repaired checkpoint to one-step distillation. On ImageNet-512, the original EDM2-XS baseline uses 124.713M parameters and 63 network evaluations, reaching an FID of 3.53. With a suitable distillation objective, our 20% pruned one-step generator uses 98.826M parameters and one network evaluation, reaching an FID of 3.12. With 30% pruning, the model uses 88.029M parameters and one network evaluation, with an FID of 4.26.

扩散模型能够生成高质量图像，但其推理成本来自两个方面：大型去噪网络和重复的去噪步骤。现有压缩流水线通常分别处理这些成本。剪枝虽然可以减小网络规模，但大多数剪枝方法仍需依赖较长的剪枝后重训练阶段来恢复多步采样器。步骤蒸馏可以减少去噪步骤数，但它通常假设学生模型已经能够很好地跟随教师模型以接收有效的蒸馏梯度。本文探讨是否可以用步骤蒸馏替代剪枝后的重训练。研究发现直接替换不可行：在对EDM2-XS教师模型进行剪枝后，从剪枝检查点启动SiDA会产生无用的样本。本文引入了一个短暂的教师对齐修复阶段作为剪枝与步骤蒸馏之间的桥梁。该桥梁在含噪真实图像潜在表示上使剪枝后的生成器与教师模型对齐，然后将修复后的检查点交给一步蒸馏。在ImageNet-512上，原始EDM2-XS基线使用1.24713亿参数和63次网络评估，达到了3.53的FID。使用合适的蒸馏目标，我们的20%剪枝一步生成器使用9882.6万参数和1次网络评估，达到了3.12的FID。在30%剪枝下，模型使用8802.9万参数和1次网络评估，FID为4.26。