每日 arXiv 论文简报

We present Chatterbox-Flash, a zero-shot text-to-speech model obtained by fine-tuning a pretrained autoregressive TTS decoder into a block-diffusion decoder, enabling parallel token generation within each block while retaining block-by-block streaming. We find that naively transferring mainstream block-diffusion decoding to discrete speech tokens degrades quality, as a long-tail token distribution biases parallel position selection toward a few high-frequency tokens. To mitigate this without architectural modification, we introduce two inference-time techniques: prior-calibrated scoring, which subtracts the block-level marginal token distribution, and an early-decoding schedule, which adaptively terminates iteration based on calibrated confidence. On standard zero-shot TTS benchmarks, Chatterbox-Flash attains high-fidelity synthesis comparable to strong autoregressive and non-autoregressive baselines, while supporting streaming inference with time-to-first-packet on par with streaming AR systems and substantially lower real-time factor. Code and audio samples are available at https://github.com/resemble-ai/chatterbox-flash.

我们提出Chatterbox-Flash，这是一个零样本文本转语音模型，通过将预训练的自回归TTS解码器微调为块扩散解码器实现，能够在每个块内并行生成token，同时保持逐块流式推理。我们发现，直接将主流块扩散解码方法迁移到离散语音token会导致质量下降，因为长尾token分布会使并行位置选择偏向少数高频token。为在不做架构修改的情况下解决这一问题，我们引入两种推理时技术：先验校准评分（减去块级边缘token分布）和早期解码调度（基于校准置信度自适应终止迭代）。在标准零样本TTS基准上，Chatterbox-Flash实现了与强自回归和非自回归基线相当的高保真合成，同时支持流式推理，首包时间与流式自回归系统相当，且实时因子显著更低。代码和音频样本见 https://github.com/resemble-ai/chatterbox-flash。

While text-to-3D generation has advanced rapidly, achieving high geometric fidelity at low inference cost remains challenging. Existing text-to-3D methods either decode discrete shape tokens autoregressively or iteratively refine global 3D representations with diffusion or flow models. However, autoregressive decoding is sequential and cannot revise errors, whereas diffusion and flow-matching models repeatedly process the full representation, making high-quality generation increasingly expensive. In this paper, we propose Block3D, a block-wise diffusion framework that partitions the discrete shape-token sequence into contiguous blocks, generates the blocks autoregressively, and jointly denoises all tokens within the current block. To alleviate error accumulation, we introduce confidence-guided intra-block correction, which revises low-confidence tokens before each block is finalized. On a held-out set from TRELLIS-500K, Block3D reduces mean end-to-end generation time from 25.71 seconds to 4.99 seconds, achieving a $5.15\times$ speedup over the fine-tuned autoregressive baseline without sacrificing geometric fidelity.

尽管文本到3D生成技术发展迅速，但在低推理成本下实现高几何保真度仍然具有挑战性。现有的文本到3D方法要么自回归地解码离散形状token，要么使用扩散模型或流匹配模型迭代优化全局3D表示。然而，自回归解码是顺序执行的且无法修正错误，而扩散和流匹配模型需要重复处理完整表示，导致高质量生成的计算成本不断增加。本文提出Block3D，一个分块扩散框架，该框架将离散形状token序列划分为连续块，自回归地生成块，并联合去噪当前块内的所有token。为缓解误差累积，我们引入置信度引导的块内修正方法，在每个块最终确定前修正低置信度token。在TRELLIS-500K的保留测试集上，Block3D将平均端到端生成时间从25.71秒减少到4.99秒，相较于微调后的自回归基线实现了5.15倍的加速，同时不牺牲几何保真度。

Meshes are among the most common 3D scene representations, but directly generating meshes is challenging because the representation contains important symmetries, including permutation invariance of faces and vertices. MeshFlow learns to generate triangle meshes directly as triangle soups, avoiding the need to serialize meshes into long autoregressive sequences. We adopt equivariant optimal-transport flow matching models that respect the key symmetries of triangle soups: arbitrary permutations of faces and permutations of the vertices within each face. Toward this goal, we propose a simple yet effective modification to the Diffusion Transformer architecture, resulting in a scalable network capable of modeling a velocity field while maintaining the desired equivariance. We further introduce an optimal-transport-based training objective that improves convergence by eliminating supervision signals that violate these symmetries. MeshFlow achieves mesh quality comparable to state-of-the-art autoregressive mesh generators while providing about an 18$\times$ speedup during inference. Project page is at https://qiisun.github.io/MeshFlow/.

网格是最常见的3D场景表示形式之一，但直接生成网格具有挑战性，因为该表示包含重要的对称性，包括面和顶点的置换不变性。MeshFlow 学习将三角形网格直接生成为无序三角形集合，避免了将网格序列化为冗长的自回归序列。我们采用等变最优传输流匹配模型，该模型尊重三角形集合的关键对称性：面的任意置换以及每个面内顶点的置换。为此，我们对 Diffusion Transformer 架构提出了一种简单而有效的修改方案，构建了一个可扩展的网络，能够在保持所需等变性的同时建模速度场。我们进一步引入了一种基于最优传输的训练目标，通过消除违反这些对称性的监督信号来改善收敛性。MeshFlow 实现了与最先进的自回归网格生成器相当的网格质量，同时在推理过程中提供了约 18 倍的加速。项目页面位于 https://qiisun.github.io/MeshFlow/。

Diffusion models (DMs) have emerged as powerful generative priors for MRI reconstruction with promising results. Yet DM-based methods require extensive iterative refinement, limiting their practical deployment. Consistency models (CMs) provide a compelling alternative, aiming to map out the diffusion trajectory in a single pass, enabling faster generation. In this work, we propose CM-RED, a novel MRI reconstruction method that integrates a pretrained CM into the regularization by denoising (RED) scheme. Our method builds on accelerated proximal gradient RED (RED-APG), and further incorporates controlled noise injection during the update steps to enhance generative diversity and accelerate convergence. Extensive experiments on the fastMRI knee and brain datasets demonstrate that CM-RED achieves high-quality reconstructions across multiple anatomies, contrast weights, acceleration factors, and undersampling patterns, using only 4 network function evaluations (NFEs). The proposed method consistently outperforms existing DM- and CM-based approaches in both quantitative metrics and visual fidelity, and exhibits strong robustness to hyperparameter variations, highlighting CM-RED as an efficient and effective generative framework for accelerated MRI reconstruction. The source code and pretrained models are publicly available at https://github.com/MerveGulle/CM-RED.

扩散模型（Diffusion Models, DMs）作为强大的生成先验，已在MRI重建中展现出良好的应用前景。然而，基于DM的方法需要进行大量的迭代细化，限制了其实际部署。一致性模型（Consistency Models, CMs）提供了一种极具吸引力的替代方案，旨在单次传递中绘制扩散轨迹，从而实现更快的生成。在本工作中，我们提出了CM-RED，这是一种将预训练CM集成到去噪正则化（Regularization by Denoising, RED）方案中的新型MRI重建方法。我们的方法基于加速近端梯度RED（RED-APG），并进一步在更新步骤中引入受控噪声注入，以增强生成多样性并加速收敛。在fastMRI膝盖和脑部数据集上的大量实验表明，CM-RED仅需4次网络函数评估（Network Function Evaluations, NFEs），即可在多种解剖结构、对比度权重、加速因子和欠采样模式下实现高质量重建。该方法在定量指标和视觉保真度方面始终优于现有的基于DM和CM的方法，并对超参数变化展现出较强的鲁棒性，凸显了CM-RED作为加速MRI重建的高效且有效的生成框架。源代码和预训练模型已公开访问：https://github.com/MerveGulle/CM-RED。

Temporal Knowledge Graph (TKG) extrapolation seeks to infer future facts from time-varying relational histories. Recent diffusion-based approaches improve uncertainty modeling through generative denoising, but their aggregated conditioning on subject histories may insufficiently distinguish query-specific evidence from non-salient historical facts, thereby diluting target-discriminative signals. To bridge this gap, we propose FreqDiff, a Frequency-aware Diffusion framework for TKG extrapolation. Specifically, FreqDiff formulates future object prediction as query-slot denoising and develops a dual-stream denoiser that integrates temporal dependency modeling with context-aware spectral calibration. The spectral branch synthesizes history-conditioned filters from learnable bases to adaptively re-calibrate denoising representations, while a frequency-domain regularizer is proposed to align the denoised target with the gold object in spectral space. Experiments on four public TKG benchmarks demonstrate that FreqDiff achieves state-of-the-art performance.

时序知识图谱（TKG）外推旨在从随时间变化的关系统历史中推断未来事实。近期基于扩散的方法通过生成式去噪改进了不确定性建模，但它们对主体历史的聚合条件化可能无法充分区分查询特定证据与非显著历史事实，从而削弱了目标判别信号。为弥合这一差距，我们提出了FreqDiff，一个用于TKG外推的频率感知扩散框架。具体而言，FreqDiff将未来对象预测形式化为查询槽去噪，并开发了一个双流去噪器，将时序依赖建模与上下文感知谱校准相结合。谱分支从可学习基中合成历史条件化滤波器，以自适应地重新校准去噪表示，同时提出一个频域正则化器，以在谱空间中对齐去噪目标与标准对象。在四个公开TKG基准数据集上的实验表明，FreqDiff达到了最先进的性能。

The matrix-aware optimizer Muon improves large model training by balancing updates across singular directions, yet its scaling behavior and end-to-end efficiency on large Diffusion Transformers (DiTs) remain unclear. We first establish Muon's scaling behavior on DiTs from 1.3B to 15B parameters, showing that its optimization and generative quality advantages over AdamW persist across model scales. However, at scale, the 5-step Newton--Schulz iteration (NS5) performed at every optimization step, together with full-momentum materialization, introduces substantial computation and communication overhead that can offset Muon&x27;s step-efficiency advantage. We introduce \emph{Periodic Row-wise Muon}, which performs a full NS5 spectral update once every \(K\) steps and applies a low compute and communication cost row-wise constrained update based on the current momentum at the remaining steps. We further co-design a distributed implementation that operates directly on sharded momentum during non-refresh steps and accelerates spectral refreshes through bucketed all-gather and communication--computation overlap. Across all scales, Muon improves the best observed generative quality over AdamW by 12.9--19.1\%. Compared with vanilla Muon, Periodic Row-wise Muon remains within 0.5\% in best generative quality on the 1.3B--4B models and improves it by 4.5\% at 9B. It reduces optimizer time by 46.9--54.3\%, end-to-end step time by 15.7--24.3\%, and logical communication volume by 66.7\%, while reaching its respective best generative quality with 33.7--64.8\% less active training time. These results show that Periodic Row-wise Muon preserves Muon's generative quality advantage while translating it into end-to-end training efficiency for large DiTs.

矩阵感知优化器Muon通过在奇异方向上平衡更新来提升大模型训练效果，但其在大型扩散变换器（DiTs）上的规模化行为和端到端效率尚不明确。我们首先在参数量从1.3B到15B的DiTs上确立了Muon的规模化行为，表明其相对于AdamW的优化和生成质量优势在不同模型规模下均能保持。然而，在大规模场景下，每步优化执行的5步Newton-Schulz迭代（NS5）以及完整动量实例化带来了大量的计算和通信开销，可能会抵消Muon在步效率方面的优势。我们提出了周期行向Muon（Periodic Row-wise Muon），该方法每K步执行一次完整的NS5谱更新，并在其余步骤中基于当前动量应用低计算和通信成本的行向约束更新。我们进一步设计了一种分布式实现方案，该方案在非刷新步骤中直接对分片动量进行操作，并通过分桶全收集和通信-计算重叠来加速谱刷新。在所有规模上，Muon相对于AdamW的最佳生成质量提升了12.9%至19.1%。与原生Muon相比，周期行向Muon在1.3B至4B模型上的最佳生成质量差距保持在0.5%以内，并在9B模型上提升了4.5%。该方法将优化器时间减少46.9%至54.3%，端到端步时间减少15.7%至24.3%，逻辑通信量减少66.7%，同时达到各自最佳生成质量所需的主动训练时间减少33.7%至64.8%。这些结果表明，周期行向Muon在保持Muon生成质量优势的同时，将其转化为大型DiTs的端到端训练效率。

Recent diffusion-based models achieve photorealism in image inpainting but require many sampling steps, limiting practical use. Few-step text-to-image models offer faster generation, but naively applying them to inpainting yields poor harmonization and artifacts between the background and inpainted region. We trace this cause to random Gaussian noise initialization, which under low function evaluations causes semantic misalignment and reduced fidelity. To overcome this, we propose InverFill, a one-step inversion method tailored for inpainting that injects semantic information from the input masked image into the initial noise, enabling high-fidelity few-step inpainting. Instead of training inpainting models, InverFill leverages few-step text-to-image models in a blended sampling pipeline with semantically aligned noise as input, significantly improving vanilla blended sampling and even matching specialized inpainting models at low NFEs. Moreover, InverFill does not require real-image supervision and only adds minimal inference overhead. Extensive experiments show that InverFill consistently boosts baseline few-step models, improving image quality and text coherence without costly retraining or heavy iterative optimization.

近期基于扩散模型的图像修复方法实现了照片级真实感，但需要大量采样步数，限制了其实际应用。少步文本到图像模型虽然能实现更快的生成，但将其直接应用于修复任务会在背景与修复区域之间产生严重的和谐化问题和伪影。我们追溯其原因，发现低函数评估次数下，随机高斯噪声初始化会导致语义错位和保真度下降。为解决这一问题，我们提出了InverFill，这是一种专为修复任务设计的一步逆推方法，能够将输入带掩码图像的语义信息注入初始噪声中，从而实现高保真的少步修复。InverFill无需训练修复模型，而是利用少步文本到图像模型构建混合采样流程，以语义对齐的噪声作为输入，显著提升了标准混合采样的效果，甚至在低NFE条件下可媲美专用修复模型。此外，InverFill不需要真实图像监督，仅增加极少的推理开销。大量实验表明，InverFill持续提升了基线少步模型的性能，在无需昂贵重训练或大量迭代优化的情况下改善了图像质量和文本一致性。

Advances in radiance fields have enabled photorealistic novel view synthesis. In several domains, large-scale real-world datasets have been developed to support comprehensive benchmarking and to facilitate progress beyond scene-specific reconstruction. However, for distractor-free radiance fields, a large-scale dataset with clean and cluttered images per scene remains lacking, limiting the development. To address this gap, we introduce DF3DV-1K, a large-scale real-world dataset comprising 1,048 scenes, each providing clean and cluttered image sets for benchmarking. In total, the dataset contains 89,924 images captured using consumer cameras to mimic casual capture, spanning 128 distractor types and 161 scene themes across indoor and outdoor environments. A curated subset of 41 scenes, DF3DV-41, is systematically designed to evaluate the robustness of distractor-free radiance field methods under challenging scenarios. Using DF3DV-1K, we benchmark nine recent distractor-free radiance field methods and 3D Gaussian Splatting, identifying the most robust methods and the most challenging scenarios. Beyond benchmarking, we demonstrate an application of DF3DV-1K by fine-tuning a diffusion-based 2D enhancer to improve radiance field methods, achieving average improvements of 0.96 dB PSNR and 0.057 LPIPS on the held-out set (e.g., DF3DV-41) and the On-the-go dataset. We hope DF3DV-1K facilitates the development of distractor-free vision and promotes progress beyond scene-specific approaches. The dataset and leaderboard are available at https://johnnylu305.github.io/df3dv1k_web/.

辐射场的进展使得逼真的新视图合成成为可能。在若干领域中，大规模真实世界数据集已被开发用于支持综合基准测试，并推动超越场景特定重建的进展。然而，对于无干扰辐射场，仍缺乏包含干净和杂乱图像的大规模数据集，这限制了相关领域的发展。为弥补这一空白，我们提出了DF3DV-1K，这是一个大规模真实世界数据集，包含1048个场景，每个场景提供干净和杂乱的图像集用于基准测试。该数据集共有89924张图像，使用消费级相机拍摄以模拟随意捕获场景，涵盖128种干扰类型和161种场景主题，涵盖室内外环境。我们精心挑选了41个场景的子集DF3DV-41，系统性地设计用于评估无干扰辐射场方法在挑战性场景下的鲁棒性。使用DF3DV-1K，我们对九种近期无干扰辐射场方法以及3D Gaussian Splatting进行了基准测试，识别出了最鲁棒的方法和最具挑战性的场景。此外，我们展示了DF3DV-1K的应用，通过微调基于扩散的2D增强器来改进辐射场方法，在保留集（如DF3DV-41）和On-the-go数据集上实现了平均0.96 dB PSNR和0.057 LPIPS的提升。我们希望DF3DV-1K能够促进无干扰视觉领域的发展，并推动超越场景特定方法的进步。数据集和排行榜可访问 https://johnnylu305.github.io/df3dv1k_web/。

We present Chatterbox-Flash, a zero-shot text-to-speech model obtained by fine-tuning a pretrained autoregressive TTS decoder into a block-diffusion decoder, enabling parallel token generation within each block while retaining block-by-block streaming. We find that naively transferring mainstream block-diffusion decoding to discrete speech tokens degrades quality, as a long-tail token distribution biases parallel position selection toward a few high-frequency tokens. To mitigate this without architectural modification, we introduce two inference-time techniques: prior-calibrated scoring, which subtracts the block-level marginal token distribution, and an early-decoding schedule, which adaptively terminates iteration based on calibrated confidence. On standard zero-shot TTS benchmarks, Chatterbox-Flash attains high-fidelity synthesis comparable to strong autoregressive and non-autoregressive baselines, while supporting streaming inference with time-to-first-packet on par with streaming AR systems and substantially lower real-time factor. Code and audio samples are available at https://github.com/resemble-ai/chatterbox-flash.

我们提出Chatterbox-Flash，这是一个零样本文本转语音模型，通过将预训练的自回归TTS解码器微调为块扩散解码器实现，能够在每个块内并行生成token，同时保持逐块流式推理。我们发现，直接将主流块扩散解码方法迁移到离散语音token会导致质量下降，因为长尾token分布会使并行位置选择偏向少数高频token。为在不做架构修改的情况下解决这一问题，我们引入两种推理时技术：先验校准评分（减去块级边缘token分布）和早期解码调度（基于校准置信度自适应终止迭代）。在标准零样本TTS基准上，Chatterbox-Flash实现了与强自回归和非自回归基线相当的高保真合成，同时支持流式推理，首包时间与流式自回归系统相当，且实时因子显著更低。代码和音频样本见 https://github.com/resemble-ai/chatterbox-flash。

Generative video compression can recover rich visual details at low bitrates, but simultaneously achieving high temporal consistency and low inference cost remains challenging. To address this issue, we propose DiffVC-ONE, a diffusion-based generative video compression framework built on a one-step Video Diffusion Transformer. First, we introduce a Unified Unidirectional Latent Compressor that uses a shared model to efficiently and uniformly compress compact latent slices. We then develop a Video DiT-based One-Step Diffusion Enhancer that uses the reconstructed latent slices as content anchors and performs single-step spatio-temporal perceptual enhancement over an entire group of pictures. Finally, a Hybrid Condition Generator extracts structural, strength, and semantic conditions from the reconstructed content and quantization information. These conditions preserve faithful regions, control the degree of generative enhancement, and supplement content-aware perceptual details during one-step diffusion enhancement. Extensive experiments on multiple standard benchmarks demonstrate that DiffVC-ONE achieves state-of-the-art perceptual quality and temporal consistency with low inference cost.

生成式视频压缩能够在低比特率下恢复丰富的视觉细节，但同时实现高时间一致性和低推理成本仍具挑战性。针对这一问题，我们提出了DiffVC-ONE，一个建立在单步视频扩散变换器上的扩散生成式视频压缩框架。首先，我们引入了统一单向潜在压缩器，使用共享模型对紧凑潜在切片进行高效统一的压缩。随后，我们开发了基于视频DiT的单步扩散增强器，以重建的潜在切片为内容锚点，对整个图像组执行单步时空感知增强。最后，混合条件生成器从重建内容和量化信息中提取结构、强度和语义条件。这些条件在单步扩散增强过程中保留忠实区域、控制生成增强程度，并补充内容感知的感知细节。在多个标准基准测试上的广泛实验表明，DiffVC-ONE以低推理成本实现了最先进的感知质量和时间一致性。

Generating egocentric video from a single exocentric video is an emerging and important topic for AR/VR and physical AI. Compared with conventional novel view synthesis, exo-to-ego generation is a significantly harder task because the standard geometric conditioning becomes highly unreliable under extreme view changes and large unobservable regions. We present Grounded-Exo2Ego, a principled framework that addresses these challenges at both the architectural and data levels. Architecturally, Grounded-Exo2Ego is a dual-branch video diffusion model that couples a geometric anchoring branch, which conditions the generation on the rendering of a 3D reconstruction, with a novel semantic grounding branch, which goes beyond the prevailing geometry-based approach and improves quality by synthesizing challenging regions based on object-level context. Additionally, we found that the overlooked issue of camera-reconstruction misalignment severely undermines exo-to-ego learning. We thus introduce a camera re-localization algorithm that resolves this issue and substantially improves quality across all metrics. We further develop a fully automated synthetic data engine that generates and renders rigged 3D characters in procedurally generated environments. Evaluation on the challenging EgoExo4D dataset shows that our method outperforms recent state-of-the-art approaches by large margins across all metrics. Detailed ablations validate improvements from each of our contributions at both the data and architectural level.

从单个外视角视频生成内视角视频是AR/VR和物理人工智能领域一个新兴且重要的课题。与传统的新视角合成相比，外视到内视生成是一个更具挑战性的任务，因为在极端视角变化和大面积不可见区域的情况下，标准几何条件变得极不可靠。我们提出了Grounded-Exo2Ego，一个在架构和数据层面解决这些挑战的 principled 框架。在架构上，Grounded-Exo2Ego是一个双分支视频扩散模型，耦合了一个几何锚定分支（该分支基于3D重建的渲染进行条件生成）和一个新颖的语义接地分支（该分支超越了现有的基于几何的方法，基于目标级上下文合成挑战性区域，从而提升生成质量）。此外，我们发现长期以来被忽视的相机-重建对齐问题严重损害了外视到内视的学习效果。因此，我们引入了一种相机重定位算法来解决这个问题，并显著提升所有指标上的质量。我们还开发了一个全自动的合成数据引擎，用于在程序化生成的环境中生成和渲染带骨骼的3D角色。在具有挑战性的EgoExo4D数据集上的评估表明，我们的方法在所有指标上都以较大优势优于最新的先进方法。详细的消融实验验证了我们在数据和架构层面各贡献所带来的改进。

Text-guided zero-shot object counters excel at spatial localization but categorize poorly on novel or fine-grained classes: natural language is too coarse to fully specify visual identity, so they fail to separate visually similar distractors. Few-shot counters sidestep this with visual exemplars, but require manual annotations on every image. To resolve this dilemma, we introduce RECOUNT, a plug-and-play framework for image-guided zero-shot counting. Rather than specify a category with a text prompt, our key insight is to specify it visually, from a single off-scene reference image. However, we find that a lone reference image provides narrow coverage of a category's appearance and is unreliable across diverse scenes. We therefore repurpose a diffusion model as an automated contrastive data engine that expands the reference into a diverse exemplar gallery, supplying the discriminative detail that text cannot. RECOUNT preserves the class-agnostic proposals of any frozen counter and offloads categorization to a separate visual module (a frozen backbone with a lightweight head trained on this synthetic data) that matches each proposal against the target and distractor galleries. Applied to a frozen counter, RECOUNT attains the best zero-shot accuracy on both benchmarks, cutting counting error (MAE) by 55% on LookAlikes and 21% on PairTally relative to the strongest prior zero-shot counter.

文本引导的零样本目标计数器在空间定位方面表现出色，但在新颖类别或细粒度类别上的分类能力较差：自然语言过于粗糙，无法完整描述视觉特征，因此无法区分视觉相似的干扰物。少样本计数器通过视觉样本规避了这一问题，但需要对每张图像进行手动标注。为解决这一难题，我们提出了RECOUNT，一个用于图像引导零样本计数的即插即用框架。我们的核心思想是：不再使用文本提示指定类别，而是通过单张离场参考图像以视觉方式指定类别。然而，我们发现单一的参考图像对类别的外观覆盖范围有限，且在多样化的场景中不可靠。因此，我们重新利用扩散模型作为自动对比数据引擎，将参考图像扩展为多样化的样本库，提供文本无法描述的区分性细节。RECOUNT保留任何冻结计数器的类别无关候选区域提议，并将分类任务卸载到一个独立的视觉模块（一个在合成数据上训练的带有轻量级头部的冻结主干网络），该模块将每个候选区域与目标和干扰样本库进行匹配。应用于冻结计数器后，RECOUNT在两个基准数据集上实现了最佳的零样本精度，将计数误差（MAE）在LookAlikes数据集上降低55%，在PairTally数据集上降低21%，相较于最强的先前零样本计数器。

Panoramic image generation is increasingly important for immersive applications such as virtual reality, augmented reality, and 3D content creation. Unlike perspective images, panoramic images represent a viewer-centered $360^\circ$ surrounding space, where directional expressions such as left, right, front, and behind play a central role in spatial understanding. However, existing text-to-panorama methods largely rely on implicit spatial reasoning and often fail to faithfully ground object-level directional descriptions in spherical panoramic scenes. A straightforward alternative is to introduce explicit layouts, but requiring manually specified spatial conditions reduces the flexibility of language-based interaction and does not directly resolve the misalignment between egocentric directional language and panoramic image space. To address this issue, we propose PanoCtrl, an object-centric framework for controllable text-to-panorama generation. Our method explicitly bridges natural language and spherical panoramic space by converting textual descriptions into structured object-level spherical conditions and integrating them into the diffusion process. Specifically, we introduce PanoParse, a text-conditioned parser that predicts object semantics and spherical bounding field-of-view (BFoV) parameters, and \textbf{PanoControl}, which injects object-level semantic and spatial guidance into the diffusion transformer through object-aware attention and spatial residual enhancement. To support this task, we construct PanoGround, a dataset with object-level spherical annotations and diverse directional descriptions for controllable panoramic generation. Extensive experiments demonstrate that PanoCtrl achieves state-of-the-art performance in both spatial alignment and image quality.

全景图像生成对于虚拟现实、增强现实和3D内容创建等沉浸式应用日益重要。与透视图像不同，全景图像呈现以观察者为中心的360°环绕空间，其中左、右、前、后等方向表达在空间理解中起着核心作用。然而，现有的文本到全景图方法在很大程度上依赖隐式空间推理，往往无法忠实地将对象级方向描述锚定在球面全景场景中。一个直接的替代方案是引入显式布局，但这需要手动指定空间条件，降低了基于语言交互的灵活性，并未直接解决自我中心方向语言与全景图像空间之间的错位问题。为解决这一问题，我们提出了PanoCtrl，一个用于可控文本到全景图生成的对象中心框架。我们的方法通过将文本描述转换为结构化对象级球面条件并将其集成到扩散过程中，显式地桥接自然语言和球面全景空间。具体而言，我们引入了PanoParse，一个文本条件解析器，可预测对象语义和球面边界视场（BFoV）参数，以及PanoControl，它通过对象感知注意力和空间残差增强将对象级语义和空间引导注入扩散Transformer。为支持此任务，我们构建了PanoGround，这是一个具有对象级球面注释和多样化方向描述的数据集，用于可控全景生成。大量实验表明，PanoCtrl在空间对齐和图像质量方面均达到了最先进的性能。

Single-image 3D human reconstruction often suffers from over-smoothed textures and geometric inconsistencies. While diffusion models improve generative quality, their reliance on multi-view synthesis prior to 3D reconstruction is computationally expensive and prone to view inconsistency. We propose DiGS-Avatar, which reformulates this task as an efficient, diffusion-based UV-latent completion task, ensuring 3D consistency by design. To capture accurate spatial structure, we introduce a teacher-student framework where a multi-view teacher provides geometrically aligned pseudo-ground-truth latents to supervise a single-view diffusion student. Treating this inferred latent as a robust structural skeleton, our method injects high-level semantic features to accurately recover fine textural details without disrupting spatial integrity. The refined representation is then decoded into 3D Gaussian primitives. Extensive experiments demonstrate that DiGS-Avatar achieves state-of-the-art or highly competitive visual fidelity and zero-shot generalization, while reconstructing a fully animatable 3D avatar in just 0.71 seconds. Code is available at https://github.com/KLMAV-CUC/DiGS-Avatar.

单图像3D人体重建常常面临纹理过度平滑和几何不一致的问题。虽然扩散模型提升了生成质量，但其在3D重建前依赖多视角合成的策略计算密集且容易出现视角不一致。本研究提出DiGS-Avatar，将该任务重新表述为一个高效的基于扩散的UV潜在完成任务，从根本上确保3D一致性。为捕捉精确的空间结构，本方法引入师生框架，其中多视角教师提供几何对齐的伪真实值潜在向量来监督单视角扩散学生。将推断的潜在向量作为稳健的结构骨架，本方法注入高级语义特征以准确恢复精细纹理细节，同时不破坏空间完整性。优化后的表示随后被解码为3D高斯基元。大量实验表明，DiGS-Avatar实现了最先进的或极具竞争力的视觉保真度和零样本泛化能力，同时仅需0.71秒即可重建一个完整的可动画化3D化身。代码可通过https://github.com/KLMAV-CUC/DiGS-Avatar获取。

AI-generated image manipulation localization identifies edited pixels, but its OOD performance lags behind image-level detection partly because pixel supervision entangles forensic evidence with dataset-specific mask geometry and semantic boundaries. Extending image-level distribution alignment to localization, we construct COCO-ControlNet with source-image Canny edges and depth maps to align semantics and geometry, improving OOD performance across multiple localizers. Yet tighter Mask-VAE Reconstruction Alignment (Mask-VAE) underperforms COCO-ControlNet, showing that VAE reconstruction artifacts transfer poorly to local diffusion-inpainting artifacts. We also identify \emph{boundary adhesion}, where fine-tuned segmentation models snap predictions to semantic object contours rather than true manipulation boundaries. These findings motivate GAP-SAM, which encodes an image and its frozen VAE reconstruction into a global artifact token and injects it into SAM3's feature pyramid via zero-gated FiLM before pixel decoding. Without prescribing a spatial region, this token modulates dense decoding to preserve localization while suppressing semantic-boundary shortcuts. Across six datasets, GAP-SAM averages 79.8 Pixel-F1, outperforming the strongest prior method by 12.6 points. It also performs best at every tested severity of JPEG compression, Gaussian blur, and resizing.

AI生成图像篡改定位旨在识别被篡改的像素，但其分布外（OOD）性能落后于图像级检测，部分原因在于像素级监督将取证证据与数据集特定的掩膜几何形状和语义边界纠缠在一起。将图像级分布对齐扩展到定位任务，我们构建了COCO-ControlNet，使用源图像的Canny边缘和深度图来对齐语义和几何信息，从而提升了多种定位器在分布外数据上的性能。然而，更紧密的Mask-VAE重建对齐（Mask-VAE）性能却不如COCO-ControlNet，表明VAE重建伪影难以迁移到局部扩散修复伪影。我们还发现了"边界粘连"现象，即微调后的分割模型将预测 snap 到语义物体轮廓而非真实篡改边界。这些发现催生了GAP-SAM方法，它将图像及其冻结的VAE重建编码为全局伪影令牌，并通过零门控FiLM注入SAM的特征金字塔，然后进行像素解码。在不预设空间区域的情况下，该令牌调节密集解码过程，以在保留定位能力的同时抑制语义边界捷径。在六个数据集上，GAP-SAM平均达到79.8的Pixel-F1分数，比最强基线方法高出12.6分。它在JPEG压缩、高斯模糊和缩放的各种测试 severity 级别下也表现最佳。

Omnimodal generation is central to a wide range of content creation and editing applications. In-context conditioning is essential to this paradigm. It allows diffusion transformers to process text instructions and visual references in a shared attention sequence. However, each reference image introduces thousands of tokens. Computation therefore grows rapidly with the number of references. Existing methods reduce computation through structured sparse attention, which limits interactions between reference and target tokens. This structure also makes the reference K and V independent of the denoising target, allowing them to be computed once and reused across steps. However, it blocks visual references from attending to the text instruction. This substantially degrades instruction following and reference fidelity in multi-reference editing. To resolve this conflict, we jointly redesign the token sequence and attention mask. Our beyond-mask design uses static text anchors to connect the instruction to the reference branch. It preserves exact K and V reuse without adding parameters. However, this direct architectural conversion degrades generation quality. We recover the lost performance through teacher-forced velocity distillation, followed by a short on-policy stage in which the teacher supervises student-visited states. To our knowledge, this is the first use of on-policy distillation for architectural recovery in diffusion models. Across three image-editing benchmarks, our method matches full-attention generation quality. With five reference images, it accelerates the complete 40-step denoising process by 3.92x, while static text anchors introduce negligible runtime overhead; the speedup reaches 5.47x at ten references in our scaling study.

多模态生成是内容创作与编辑应用的核心技术。上下文条件是多模态生成范式的关键要素，它使扩散变换器能够在统一的注意力序列中处理文本指令和视觉参考。然而，每张参考图像会引入数千个标记，计算量随参考图像数量急剧增长。现有方法通过结构化稀疏注意力来降低计算成本，但这限制了参考标记与目标标记之间的交互。这种结构还使得参考的键（K）和值（V）与去噪目标解耦，允许它们一次性计算并在多步去噪过程中复用。然而，这导致视觉参考无法关注文本指令，严重降低了多参考编辑任务中的指令遵循能力和参考保真度。为解决这一矛盾，我们联合重新设计了标记序列和注意力掩码。我们的掩码外设计使用静态文本锚点将指令连接到参考分支，在不增加参数的前提下保留了精确的键值复用。然而，这种直接的架构转换会降低生成质量。我们通过教师强制速度蒸馏来恢复丢失的性能，随后采用一个短期策略阶段，由教师对学生访问的状态进行监督。据我们所知，这是首次将策略蒸馏用于扩散模型中的架构恢复。在三个图像编辑基准测试中，我们的方法达到了全注意力生成质量的水平。在使用五张参考图像的情况下，它将完整的40步去噪过程加速了3.92倍，同时静态文本锚点引入的运行时开销可以忽略不计；在我们的扩展性研究中，使用十张参考图像时加速比达到5.47倍。

Digital 3D objects used in games and animation are often required to be compositional; that is, decomposed into semantically meaningful parts. Recent 3D generation methods can produce high-quality compositional objects conditioned on image or text prompts. Yet, such global conditioning lacks the precise part-level controllability required for professional creative workflows. To address this, we introduce MultiCube, a novel compositional 3D generation method that provides explicit, independent control over both the semantics and spatial arrangement of each part. MultiCube takes as input a global text prompt, a text schema specifying the desired parts, and a spatial layout indicating the bounding boxes of the parts in the given schema. It outputs a 3D object composed of distinct meshes, one per specified part, that adhere to the given semantic and spatial conditions. Our approach employs a two-stage diffusion process, first generating a schema- and layout-aligned monolithic mesh, then decomposing the mesh into individual parts simultaneously. A novel Part Layout Adapter is used to encode per-part conditions independently of the other parts. Experiments demonstrate that our method can generate high-quality compositional 3D objects with precise part-level control, including those with unique layouts difficult to achieve with text or image prompting alone. Project page: https://multi-cube.github.io

游戏和动画中使用的数字3D对象通常需要具有组合性，即能够分解为语义上有意义的部件。近期3D生成方法可以根据图像或文本提示生成高质量的组合式对象。然而，这种全局条件控制缺乏专业创意工作流所需的精确部件级可控性。针对这一问题，我们提出了MultiCube，这是一种新型的组合式3D生成方法，可对每个部件的语义和空间布局提供明确的独立控制。MultiCube的输入包括一个全局文本提示、一个指定所需部件的文本模式，以及一个表示该模式中各部件边界框的空间布局。它输出一个由独立网格组成的3D对象，每个网格对应一个指定部件，并符合给定的语义和空间条件。我们的方法采用两阶段扩散过程，首先生成一个符合模式和布局的整体网格，然后同时将其分解为各个部件。我们设计了一种新颖的部件布局适配器，用于独立编码每个部件的条件，而不受其他部件的影响。实验表明，我们的方法能够生成具有精确部件级控制的高质量组合式3D对象，包括那些仅通过文本或图像提示难以实现的独特布局。

Driven by the emergence of Controllable Video Diffusion, existing Sim2Real methods for autonomous driving video generation typically rely on explicit intermediate representations to bridge the domain gap. However, these modalities face a fundamental Consistency-Realism Dilemma. Low-level signals (e.g., edges, blurred images) ensure precise control but compromise realism by "baking in" synthetic artifacts, whereas high-level priors (e.g., depth, semantics, HDMaps) facilitate photorealism but lack the structural detail required for consistent guidance. In this work, we present Driving with DINO (DwD), a novel framework that leverages Vision Foundation Module (VFM) features as a unified bridge between the simulation and real-world domains. We first identify that these features encode a spectrum of information, from high-level semantics to fine-grained structure. To effectively utilize this, we employ Principal Subspace Projection to discard the high-frequency elements responsible for "texture baking," while concurrently introducing Random Channel Tail Drop to mitigate the structural loss inherent in rigid dimensionality reduction, thereby reconciling realism with control consistency. Furthermore, to fully leverage DINOv3's high-resolution capabilities for enhancing control precision, we introduce a learnable Spatial Alignment Module that adapts these high-resolution features to the diffusion backbone. Finally, we propose a Causal Temporal Aggregator employing causal convolutions to explicitly preserve historical motion context when integrating frame-wise DINO features, which effectively mitigates motion blur and guarantees temporal stability. Project page: https://albertchen98.github.io/DwD-project/

受可控视频扩散技术出现的推动，现有的自动驾驶视频生成 Sim2Real 方法通常依赖显式的中间表示来弥补领域差距。然而，这些模态面临一个根本性的“一致性-真实性困境”。低层信号（如边缘、模糊图像）虽能确保精确控制，但通过“纹理固化”引入合成伪影从而损害真实性；而高层先验（如深度语义、高精地图）虽能促进照片级真实感，却缺乏一致性引导所需的结构细节。本研究提出 Driving with DINO（DwD）框架，利用视觉基础模型特征作为仿真与真实域之间的统一桥梁。我们首先发现这些特征编码了从高层语义到细粒度结构的信息谱系。为此，我们采用主成分子空间投影来丢弃导致“纹理固化”的高频元素，同时引入随机通道尾drop来缓解刚性降维中固有的结构损失，从而协调真实性与控制一致性。此外，为充分利用 DINOv3 的高分辨率能力增强控制精度，我们引入可学习的空间对齐模块，将这些高分辨率特征适配到扩散主干网络。最后，我们提出因果时序聚合器，采用因果卷积在整合逐帧 DINO 特征时显式保留历史运动上下文，有效缓解运动模糊并确保时序稳定性。

Sparse-view 3D modeling represents a fundamental tension between reconstruction fidelity and generative plausibility. While feed-forward reconstruction excels in efficiency and input alignment, it often lacks the global priors needed for structural completeness. Conversely, diffusion-based generation provides rich geometric details but struggles with multi-view consistency. We present RecGen3D, a framework that combines these two paradigms into a cooperative system. To overcome inherent conflicts in coordinate spaces, 3D representations, and training objectives, we align both models within a shared canonical space. We employ decoupled cooperative learning, which maintains stable training while enabling seamless collaboration during inference. Specifically, the reconstruction module is adapted to provide canonical geometric anchors, while the diffusion generator leverages latent-augmented conditioning to refine and complete the geometric structure. Experimental results demonstrate that RecGen3D achieves superior fidelity and robustness, outperforming existing methods in creating complete and consistent 3D models from sparse observations.

稀疏视角三维建模在重建保真度与生成合理性之间存在根本矛盾。前馈重建虽然在效率和输入对齐方面表现出色，但往往缺乏实现结构完整性所需的全局先验知识。相比之下，基于扩散的生成方法虽能提供丰富的几何细节，却在多视角一致性方面面临挑战。我们提出了RecGen3D框架，将这两种范式整合为一个协作系统。为克服坐标空间、三维表示和训练目标之间的内在冲突，我们在共享规范空间中对齐两个模型。我们采用解耦协作学习技术，在保持训练稳定性的同时实现推理过程中的无缝协作。具体而言，重建模块被适配用于提供规范几何锚点，而扩散生成器则利用潜在增强条件化来精化和完善几何结构。实验结果表明，RecGen3D实现了卓越的保真度和鲁棒性，在从稀疏观测创建完整一致的三维模型方面优于现有方法。

While personalizing Image-to-Video (I2V) diffusion models with specific visual effects is increasingly demanded for high-end generation, current practice requires training a separate Low-Rank Adaptation (LoRA) module for each effect, incurring substantial data curation and iterative optimization costs that hinder interactive control. We present Prompt2Effect, a weight-driven hypernetwork that amortizes per-effect training by directly synthesizing effect-specific LoRA weights in a single forward pass. Unlike prior hypernetworks that regress adapter weights purely from semantics, Prompt2Effect is explicitly conditioned on the frozen base model weights, grounding prediction in the structural geometry of each layer. Furthermore, instead of predicting raw LoRA matrices, we introduce an SVD-canonicalized parameterization that resolves factorization ambiguity and stabilizes large-scale synthesis. Extensive experiments demonstrate that Prompt2Effect achieves on-par or superior video quality and effect alignment compared to conventional LoRA fine-tuning, while reducing the computational cost from 56 GPU training hours to 3.3 seconds of hypernetwork inference. When used as initialization for subsequent fine-tuning, our predicted weights further improve final performance and accelerate optimization by approximately 10x.

随着利用特定视觉特效对图像到视频（Image-to-Video, I2V）扩散模型进行个性化定制的需求日益增长，当前实践需要为每种特效单独训练一个低秩适应（Low-Rank Adaptation, LoRA）模块，这带来了大量的数据整理和迭代优化成本，阻碍了交互式控制。我们提出了Prompt2Effect，一种权重驱动的超网络（hypernetwork），通过单次前向传播直接合成特效专属的LoRA权重来分摊每种特效的训练成本。与先前纯粹从语义信息回归适配器权重的超网络不同，Prompt2Effect明确以冻结的基础模型权重为条件，将预测锚定在每层的结构几何中。此外，我们没有直接预测原始LoRA矩阵，而是引入了一种SVD规范化参数化方法，以解决因子化歧义并稳定大规模合成。大量实验表明，Prompt2Effect在视频质量和特效对齐方面达到了与常规LoRA微调相当或更优的水平，同时将计算成本从56 GPU训练小时降低到3.3秒的超网络推理时间。当用作后续微调的初始化时，我们预测的权重进一步提升了最终性能，并将优化速度加快约10倍。

Instruction-based video editing is commonly built on video-pretrained generative backbones: a video diffusion transformer is adapted, at considerable cost, to condition on a source video and an editing instruction. In this report we explore a different route and show that a strong instruction-based image editing model can edit videos by operating directly on video-VAE latents. Starting from Qwen-Image-Edit, we arrange the latent frames of a Wan~2.1 video VAE as tiles of one large virtual image, reuse the editor's image positional encoding for every tile, and bridge the two latent spaces with a pair of lightweight input/output projections warm-started from the editor&x27;s own patchify and unpatchify layers, so that at initialization a (static) video is embedded exactly as an image the model already understands. The whole system is then fine-tuned on the public Ditto-1M editing triplets, and a few denoising steps of Wan~2.2 serve as an optional temporal enhancer. We motivate the design with a chain of zero-training observations: the stock image editor already edits a video presented as a contact sheet; it is indifferent to whether the sheet's tokens come from one joint encode or from per-frame encodes stitched in latent space; and it even edits genuine video latents zero-shot to a clearly recognizable degree, leaving fine-tuning only a fidelity gap to close. Our results suggest that, despite the large investment in training video latent spaces, per-frame video latents remain close enough to the image domain that mature image editing priors transfer with minimal adaptation. Project Page: https://yunpeng1998.github.io/Qwen-Video-Edit-Page Code: https://github.com/yunpeng1998/Qwen-Video-Edit Model: https://huggingface.co/yunpeng1998/Qwen-Video-Edit

基于指令的视频编辑通常构建于视频预训练的生成骨干网络之上：视频扩散变换器需要以相当大的成本进行适配，以条件于源视频和编辑指令。本报告探索了一种不同的方法，证明了强大的基于指令的图像编辑模型可以直接在视频VAE潜空间上操作来编辑视频。从Qwen-Image-Edit出发，我们将Wan 2.1视频VAE的潜空间帧排列为一张大虚拟图像的切片，重用编辑器的图像位置编码应用于每个切片，并用一对轻量级的输入/输出投影桥接两个潜空间，这些投影从编辑器自身的patchify和unpatchify层热启动，使得在初始化时，一个（静态）视频被嵌入为模型已经理解的图像。整个系统随后在公开的Ditto-1M编辑三元组上进行微调，Wan 2.2的少量去噪步骤可作为可选的时序增强器。我们通过一系列零训练观察来解释这一设计：现成的图像编辑器已经可以编辑呈现为拼接画幅的视频；它对于该画幅的token是来自联合编码器还是来自潜空间中拼接的逐帧编码毫不在意；甚至它还能零样本编辑真实视频潜空间达到明显可识别的程度，微调只需弥补保真度差距即可。我们的结果表明，尽管在训练视频潜空间方面投入巨大，但逐帧视频潜空间仍与图像域足够接近，使得成熟的图像编辑先验能够以最小适配进行迁移。项目主页：https://yunpeng1998.github.io/Qwen-Video-Edit-Page 代码：https://github.com/yunpeng1998/Qwen-Video-Edit 模型：https://huggingface.co/yunpeng1998/Qwen-Video-Edit

While text-to-3D generation has advanced rapidly, achieving high geometric fidelity at low inference cost remains challenging. Existing text-to-3D methods either decode discrete shape tokens autoregressively or iteratively refine global 3D representations with diffusion or flow models. However, autoregressive decoding is sequential and cannot revise errors, whereas diffusion and flow-matching models repeatedly process the full representation, making high-quality generation increasingly expensive. In this paper, we propose Block3D, a block-wise diffusion framework that partitions the discrete shape-token sequence into contiguous blocks, generates the blocks autoregressively, and jointly denoises all tokens within the current block. To alleviate error accumulation, we introduce confidence-guided intra-block correction, which revises low-confidence tokens before each block is finalized. On a held-out set from TRELLIS-500K, Block3D reduces mean end-to-end generation time from 25.71 seconds to 4.99 seconds, achieving a $5.15\times$ speedup over the fine-tuned autoregressive baseline without sacrificing geometric fidelity.

尽管文本到3D生成技术发展迅速，但在低推理成本下实现高几何保真度仍然具有挑战性。现有的文本到3D方法要么自回归地解码离散形状token，要么使用扩散模型或流匹配模型迭代优化全局3D表示。然而，自回归解码是顺序执行的且无法修正错误，而扩散和流匹配模型需要重复处理完整表示，导致高质量生成的计算成本不断增加。本文提出Block3D，一个分块扩散框架，该框架将离散形状token序列划分为连续块，自回归地生成块，并联合去噪当前块内的所有token。为缓解误差累积，我们引入置信度引导的块内修正方法，在每个块最终确定前修正低置信度token。在TRELLIS-500K的保留测试集上，Block3D将平均端到端生成时间从25.71秒减少到4.99秒，相较于微调后的自回归基线实现了5.15倍的加速，同时不牺牲几何保真度。

Meshes are among the most common 3D scene representations, but directly generating meshes is challenging because the representation contains important symmetries, including permutation invariance of faces and vertices. MeshFlow learns to generate triangle meshes directly as triangle soups, avoiding the need to serialize meshes into long autoregressive sequences. We adopt equivariant optimal-transport flow matching models that respect the key symmetries of triangle soups: arbitrary permutations of faces and permutations of the vertices within each face. Toward this goal, we propose a simple yet effective modification to the Diffusion Transformer architecture, resulting in a scalable network capable of modeling a velocity field while maintaining the desired equivariance. We further introduce an optimal-transport-based training objective that improves convergence by eliminating supervision signals that violate these symmetries. MeshFlow achieves mesh quality comparable to state-of-the-art autoregressive mesh generators while providing about an 18$\times$ speedup during inference. Project page is at https://qiisun.github.io/MeshFlow/.

网格是最常见的3D场景表示形式之一，但直接生成网格具有挑战性，因为该表示包含重要的对称性，包括面和顶点的置换不变性。MeshFlow 学习将三角形网格直接生成为无序三角形集合，避免了将网格序列化为冗长的自回归序列。我们采用等变最优传输流匹配模型，该模型尊重三角形集合的关键对称性：面的任意置换以及每个面内顶点的置换。为此，我们对 Diffusion Transformer 架构提出了一种简单而有效的修改方案，构建了一个可扩展的网络，能够在保持所需等变性的同时建模速度场。我们进一步引入了一种基于最优传输的训练目标，通过消除违反这些对称性的监督信号来改善收敛性。MeshFlow 实现了与最先进的自回归网格生成器相当的网格质量，同时在推理过程中提供了约 18 倍的加速。项目页面位于 https://qiisun.github.io/MeshFlow/。

AI-generated image manipulation localization identifies edited pixels, but its OOD performance lags behind image-level detection partly because pixel supervision entangles forensic evidence with dataset-specific mask geometry and semantic boundaries. Extending image-level distribution alignment to localization, we construct COCO-ControlNet with source-image Canny edges and depth maps to align semantics and geometry, improving OOD performance across multiple localizers. Yet tighter Mask-VAE Reconstruction Alignment (Mask-VAE) underperforms COCO-ControlNet, showing that VAE reconstruction artifacts transfer poorly to local diffusion-inpainting artifacts. We also identify \emph{boundary adhesion}, where fine-tuned segmentation models snap predictions to semantic object contours rather than true manipulation boundaries. These findings motivate GAP-SAM, which encodes an image and its frozen VAE reconstruction into a global artifact token and injects it into SAM3's feature pyramid via zero-gated FiLM before pixel decoding. Without prescribing a spatial region, this token modulates dense decoding to preserve localization while suppressing semantic-boundary shortcuts. Across six datasets, GAP-SAM averages 79.8 Pixel-F1, outperforming the strongest prior method by 12.6 points. It also performs best at every tested severity of JPEG compression, Gaussian blur, and resizing.

AI生成图像篡改定位旨在识别被篡改的像素，但其分布外（OOD）性能落后于图像级检测，部分原因在于像素级监督将取证证据与数据集特定的掩膜几何形状和语义边界纠缠在一起。将图像级分布对齐扩展到定位任务，我们构建了COCO-ControlNet，使用源图像的Canny边缘和深度图来对齐语义和几何信息，从而提升了多种定位器在分布外数据上的性能。然而，更紧密的Mask-VAE重建对齐（Mask-VAE）性能却不如COCO-ControlNet，表明VAE重建伪影难以迁移到局部扩散修复伪影。我们还发现了"边界粘连"现象，即微调后的分割模型将预测 snap 到语义物体轮廓而非真实篡改边界。这些发现催生了GAP-SAM方法，它将图像及其冻结的VAE重建编码为全局伪影令牌，并通过零门控FiLM注入SAM的特征金字塔，然后进行像素解码。在不预设空间区域的情况下，该令牌调节密集解码过程，以在保留定位能力的同时抑制语义边界捷径。在六个数据集上，GAP-SAM平均达到79.8的Pixel-F1分数，比最强基线方法高出12.6分。它在JPEG压缩、高斯模糊和缩放的各种测试 severity 级别下也表现最佳。

We present Swift-Image, a compact unified model for text-to-image generation, single-image editing, and multi-image editing. Our goal is to explore how far a relatively small visual generator can be pushed through systematic training engineering under a constrained computational budget. Swift-Image adopts an efficient 6B single-stream DiT and a progressive training pipeline that evolves from broad semantic coverage to higher resolution, stronger visual quality, and unified generation-editing supervision. For post-training, we employ parallel expert reinforcement learning followed by multi-teacher on-policy distillation to alleviate interference among heterogeneous objectives. We further decouple high-level reasoning from pixel-level rendering with a Prompt Enhancer that translates user requests into generator-aligned visual specifications. For efficient deployment, structural pruning and few-step distillation produce 3B and accelerated variants. Swift-Image achieves leading aggregate performance among evaluated open-source models with only 6B parameters and 243K GPU training hours; the compressed 3B model incurs nearly no loss, while few-step distillation further improves aggregate editing performance with substantially fewer sampling steps. Our study also summarizes practical lessons for architecture, data curriculum, post-training, prompt enhancement, and model compression.

我们提出了Swift-Image，一个用于文本到图像生成、单图像编辑和多图像编辑的紧凑统一模型。我们的目标是在有限的计算预算下，通过系统的训练工程，探索一个相对较小的视觉生成器能够达到的性能极限。Swift-Image采用了一个高效的60亿参数单流DiT和一个渐进式训练管道，该管道从广泛的语义覆盖逐步发展到更高分辨率、更强视觉质量和统一的生成-编辑监督。对于后训练，我们采用并行专家强化学习，随后进行多教师在线蒸馏，以减轻异构目标之间的干扰。我们进一步将高级推理与像素级渲染解耦，通过一个提示增强器将用户请求转换为与生成器对齐的视觉规范。为了高效部署，结构化剪枝和少步蒸馏产生了30亿参数的加速变体。Swift-Image在仅用60亿参数和24.3万GPU训练小时数的条件下，在评估的开源模型中取得了领先的综合性能；压缩后的30亿参数模型几乎没有性能损失，而少步蒸馏则在大幅减少采样步数的同时进一步提升了综合编辑性能。我们的研究还总结了架构、数据课程、后训练、提示增强和模型压缩方面的实践经验。