ESC
输入关键词搜索文章
目录

每日 arXiv 论文简报

2026-06-27 · 38 篇论文 · 按研究方向分组
自动追踪 · LLM 总览 · 研究雷达
38Total Papers
6Autoregressive
30Diffusion
2Image Compression
01D Visual Tokenizer
0Diffusion Visual Encoder
Daily Radar
每日总览

今日 arXiv 论文呈现扩散模型主导、自回归模型多元发展的格局。Diffusion 方向占据 30 篇,涉及图像/视频生成、模型安全、版权法律、加速优化等多个维度,反映出该技术已从纯生成任务延伸至系统安全与伦理考量。Autoregressive 方向虽数量较少(6篇),但覆盖图像生成、多模态推理、网格细分、物理仿真等差异化场景,展现自回归范式在结构化输出和世界模型方面的独特优势。Image Compression 仅 2 篇,均聚焦轻量化语义传递与超分辨率 pruning,体现端侧部署需求。

值得注意的是,多篇论文出现跨方向交叉:Safe Autoregressive Image Generation 与 Reconstruction Alignment 在 Autoregressive 与 Diffusion 列表中重复出现,暗示两种生成范式正在互相借鉴融合。安全、缓存、 alignment 成为今日高频关键词,体现模型不仅要“生成得好”,还要“用得安全、推理高效”。

  • Safe Autoregressive Image Generation with Iterative Self-Improving Codebooks — 首次将自回归模型与安全约束结合,提出自改进码本迭代机制,为可控生成提供新思路。
  • Reconstruction Alignment Improves Unified Multimodal Models — 跨 Autoregressive 与 Diffusion 出现,提出重建对齐统一多模态模型的新范式,值得关注。
  • Generative AI and Copyright Infringement: A Legal-Technical Analysis of AI Music Generation Systems — 填补生成式 AI 版权领域的法律技术交叉空白,对后续合规研究有重要参考价值。
  • Beyond Feedforward Networks: Reentry Neural Systems as the Fundamental Basis of Subjecthood and Intrinsic Safety of Next-Generation AGI — 从理论高度重新定义 AGI 主体性与安全性,视角极具前瞻性。
  • TinySR: Pruning Diffusion for Real-World Image Super-Resolution — 面向实际部署的轻量化工作,pruning 扩散模型用于超分,兼顾效率与效果。
autoregressive
Autoregressive
6 篇论文

今日自回归(Autoregressive)论文概览

今日arXiv自回归领域论文呈现出多模态融合与物理世界模拟两大趋势。6篇论文覆盖图像生成安全、跨模态对齐、视觉推理缓存优化、3D网格细分及世界模型物理模拟等方向,表明自回归范式正从纯生成任务向推理与物理交互纵深发展。值得注意的是,多篇论文聚焦提升效率与安全性(如迭代码书、缓存复用、快速推理),反映出实际部署需求日益凸显。

  • Safe Autoregressive Image Generation with Iterative Self-Improving Codebooks:提出迭代自改进码书框架,显著提升图像生成的安全性同时保持质量,为生成式AI安全提供新思路。
  • Reconstruction Alignment Improves Unified Multimodal Models:通过重建对齐技术增强统一多模态模型性能,为多模态理解与生成提供更鲁棒的表示学习范式。
  • Position Rebinding Cache Reuse: Replay-Free Visual Revisiting for Interleaved Multimodal Reasoning:创新性地解决多模态推理中的视觉回溯难题,无需重放即可实现高效视觉复用,推理效率提升显著。
  • PhysiFormer: Learning to Simulate Mechanics in World Space:将自回归建模拓展至物理力学模拟,在世界空间中进行可学习物理仿真,为具身智能提供关键基础。
  • Fast LeWorldModel:提出快速世界模型,平衡推理速度与预测精度,适用于需要实时决策的机器人与自动驾驶场景。

Safe Autoregressive Image Generation with Iterative Self-Improving Codebooks

2026-06-26T04:00:00autoregressive, cs.AI, cs.CV, diffusion2606.27147

中文标题:基于迭代自改进码本的安全自回归图像生成

作者:Yunqi Xue, Zhijiang Li, Philip Torr, Jindong Gu

摘要:

Unlike diffusion-based models that operate in continuous latent spaces, autoregressive unified multimodal models produce images by sequentially predicting discretized visual tokens. These tokens are derived from a codebook that maps embeddings to quantized visual patterns. The language-like architecture enables unified multimodal models to effectively capture text conditional information for generation, making them promising for text-to-image tasks. This also raises an interesting question: how safe are the images generated in such an autoregressive way? In this work, we propose iterative self-improving codebooks for safe autoregressive generation. We leverage the understanding and judgment capabilities of the unified multimodal model itself to identify unsafe generated images without human annotation. Subsequently, the inherent representations in the codebook are fixed to eliminate harmful mappings. Our method comprises two steps: first, we use the unified model to identify unsafe generations and construct corresponding harmful and safe image-text pairs. These pairs are used to construct the Harmful Space and guide updates to the codebook, thereby eliminating harmful outputs. Second, we perform adaptive fine-tuning on the codebook within the harmless space using safe image-text pairs to ensure the quality of generated images. These two steps are repeated until no further improvement is observed, producing a safety-enhanced model codebook. Without additional external feedback, the safety of models is improved iteratively.

摘要中文:

与在连续潜空间中运行的基于扩散的模型不同,自回归统一多模态模型通过序贯预测离散视觉token来生成图像。这些token源自将嵌入映射到量化视觉模式的码本。类语言的架构使统一多模态模型能够有效捕获用于生成的文本条件信息,使其在文本到图像任务中具有良好的应用前景。这也引出了一个有趣的问题:这种自回归方式生成的图像安全性如何?在本工作中,我们提出了用于安全自回归生成的迭代自改进码本。我们利用统一多模态模型自身的理解和判断能力,在无需人工标注的情况下识别不安全生成的图像。随后,码本中的内在表示被固定以消除有害映射。我们的方法包含两个步骤:首先,我们使用统一模型识别不安全生成并构建相应的有害和安全图像-文本对。这些配对用于构建有害空间并指导码本更新,从而消除有害输出。其次,我们使用安全图像-文本对在无害空间中对码本进行自适应微调,以确保生成图像的质量。我们重复这两个步骤,直到不再观察到进一步改进,从而产生安全增强的模型码本。无需额外的外部反馈,模型的安全性通过迭代方式得到提升。

Reconstruction Alignment Improves Unified Multimodal Models

2026-06-26T04:00:00autoregressive, cs.AI, cs.CV, cs.LG, diffusion2509.07295

中文标题:重建对齐提升统一多模态模型

作者:Ji Xie, Trevor Darrell, Luke Zettlemoyer, XuDong Wang

摘要:

Unified multimodal models (UMMs) unify visual understanding and generation within a single architecture. However, conventional training relies on image-text pairs (or sequences) whose captions are typically sparse and miss fine-grained visual details, even when they use hundreds of words to describe a simple image. We introduce Reconstruction Alignment (RECA), a resource-efficient post-training method that leverages visual understanding encoder embeddings as dense "text prompts", providing rich supervision without captions. Concretely, RECA conditions a UMM on its own visual understanding embeddings and optimizes it to reconstruct the input image with a self-supervised reconstruction loss, thereby realigning understanding and generation. Despite its simplicity, RECA is broadly applicable: across autoregressive, masked-autoregressive, and diffusion-based UMMs, it consistently improves generation and editing fidelity. With only 27 GPU hours, post-training with RECA substantially improves image generation performance on GenEval (0.73 $\rightarrow$ 0.90) and DPGBench (80.93 $\rightarrow$ 88.15), while also boosting editing benchmarks (ImgEdit 3.38 $\rightarrow$ 3.75, GEdit 6.94 $\rightarrow$ 7.27). Notably, RECA surpasses much larger open-source models and applies broadly across diverse UMM architectures, establishing it as an efficient and general post-training alignment strategy for UMMs.

摘要中文:

统一多模态模型(UMMs)将视觉理解和生成统一在单一架构中。然而,传统训练依赖于图像-文本对(或序列),其描述通常较为稀疏,无法捕捉细粒度的视觉细节,即使使用数百个词描述简单图像亦是如此。我们提出了重建对齐(RECA),一种资源高效的后训练方法,利用视觉理解编码器嵌入作为密集的"文本提示",在无需描述的情况下提供丰富的监督信号。具体而言,RECA将统一多模态模型以其自身的视觉理解嵌入为条件,并利用自监督重建损失优化模型重建输入图像,从而重新对齐理解和生成。尽管方法简单,但RECA具有广泛适用性:在自回归、掩码自回归和基于扩散的统一多模态模型中,它均能持续提升生成和编辑保真度。仅需27 GPU小时,RECA后训练即可显著提升图像生成在GenEval(0.73→0.90)和DPGBench(80.93→88.15)上的性能,同时增强编辑基准(ImgEdit 3.38→3.75,GEdit 6.94→7.27)。值得注意的是,RECA超越了更大的开源模型,并广泛应用于不同的统一多模态模型架构,为统一多模态模型确立了一种高效且通用的后训练对齐策略。

Position Rebinding Cache Reuse: Replay-Free Visual Revisiting for Interleaved Multimodal Reasoning

2026-06-26T04:00:00autoregressive, cs.CV2606.26631

中文标题:位置重绑定缓存复用:用于交错多模态推理的无重放视觉重访

作者:Mengzhao Wang, Yanli Ji, Wangmeng Zuo, Peng Ye, Chongjun Tu

摘要:

Interleaved multimodal reasoning improves visual grounding by revisiting visual evidence during multi-step generation, yet existing methods typically rely on token replay, repeatedly forwarding selected visual tokens. A natural shortcut is to reuse the historical visual key-value (KV) cache directly. However, we identify a critical failure mode of this strategy: cached visual keys are already bound to their original positional context. Such stale positional binding distorts attention under later decoding contexts and can trigger severe autoregressive decoding collapse. This failure suggests that effective cache reuse requires reconstructing visual evidence under positions compatible with the current decoding state, rather than directly copying position-bound historical cache entries. To this end, we propose Position Rebinding Cache Reuse (PRCR), a cache-level framework for replay-free visual revisiting. PRCR stores raw visual KV cache together with their original spatial coordinates, then reassigns position-compatible coordinates to select entries and rebinds their keys before injecting the reconstructed cache into the active decoder cache. This design reuses historical visual evidence while preserving textual positional continuity and relative visual structure. Experiments across multiple multimodal reasoning benchmarks show that PRCR achieves replay-level or better performance, improving average accuracy by 5 percent and reducing visual-revisiting computation by up to tens of thousands of times.

摘要中文:

交错多模态推理通过在多步生成过程中重访视觉证据来增强视觉定位,但现有方法通常依赖token重放,即重复转发选定的视觉token。一个自然的捷径是直接复用历史视觉键值(KV)缓存。然而,我们发现了该策略的一个关键失效模式:缓存的视觉键已与其原始位置上下文绑定。这种过时的位置绑定在后续解码上下文下会扭曲注意力,并可能引发严重的自回归解码崩溃。这一失效表明,有效的缓存复用需要在与当前解码状态兼容的位置下重建视觉证据,而非直接复制带有位置绑定的历史缓存条目。为此,我们提出位置重绑定缓存复用(PRCR),一个用于无重放视觉重访的缓存级框架。PRCR存储原始视觉KV缓存及其原始空间坐标,然后为选定的条目重新分配位置兼容的坐标,并在将重建的缓存注入活跃解码器缓存之前重新绑定其键。该设计在复用历史视觉证据的同时,保留了文本位置连续性和相对视觉结构。在多个多模态推理基准上的实验表明,PRCR达到了重放级或更优的性能,平均准确率提升5%,视觉重访计算量减少高达数万倍。

SubdivAR: Autoregressive Next-Scale Prediction for Neural Mesh Subdivision

2026-06-26T04:00:00autoregressive, cs.CV2606.27088

中文标题:SubdivAR:用于神经网格细分的自回归下一尺度预测

作者:Huipeng Guo, Zikai Song, Hang Long, Jielei Zhang, Wenbing Li, Junkai Lin, Tianhao Zhao, Jinshen Zhang, Tianle Guo, Wei Yang

摘要:

Mesh subdivision is a fundamental operation for converting coarse, editable meshes into high-resolution surfaces, with broad applications in digital asset creation. Classical rule-based schemes rely on fixed local refinement rules and often produce over-smoothed surfaces. Recent neural subdivision methods improve detail synthesis, but remain constrained by local modeling and exhibit limited generalizability. We present SubdivAR, a neural mesh subdivision framework based on our proposed Mesh Autoregressive Representation (MAR). MAR arranges meshes at different subdivision levels into an ordered scale sequence, reformulating subdivision as autoregressive next-scale prediction. To support this formulation, we introduce a Hybrid Topology-Aware Transformer that combines global semantic attention with topology-constrained local feature aggregation. SubdivAR adopts a next-scale coordinate prediction paradigm, regressing vertex offsets at each refinement stage to preserve subdivision topology while recovering fine-grained geometric details. To enable reliable learning, we construct FII-40K, a curated dataset of nearly 40,000 high-quality meshes with multi-level subdivision supervision. Experiments show that SubdivAR outperforms state-of-the-art baselines, reducing Hausdorff Distance and Chamfer Distance by 18.8% and 14.2%, respectively, and demonstrates strong robustness on complex open-surface geometries.

摘要中文:

网格细分是将粗可编辑网格转换为高分辨率曲面的基础操作,在数字资产生成中具有广泛应用。经典基于规则的方法依赖固定的局部细化规则,往往产生过度平滑的曲面。现有的神经细分方法虽能改进细节合成,但仍受限于局部建模且泛化能力有限。我们提出了SubdivAR,这是一个基于所提出的网格自回归表示(MAR)的神经网格细分框架。MAR将不同细分层次的网格组织为有序尺度序列,将细分重新表述为自回归下一尺度预测。为支持这一 formulation,我们引入了混合拓扑感知Transformer,结合全局语义注意力和拓扑约束的局部特征聚合。SubdivAR采用下一尺度坐标预测范式,在每个细化阶段回归顶点偏移以保持细分拓扑同时恢复细粒度几何细节。为实现可靠学习,我们构建了FII-40K,这是一个包含近40,000个高质量网格的多层次细分监督数据集。实验表明,SubdivAR优于最先进的基线方法,分别将Hausdorff距离和Chamfer距离降低18.8%和14.2%,并在复杂开放曲面几何上展现出强鲁棒性。

PhysiFormer: Learning to Simulate Mechanics in World Space

2026-06-26T04:00:00autoregressive, cs.CV, diffusion2606.27364

中文标题:PhysiFormer:学习在世界空间中模拟力学

作者:Yiming Chen, Yushi Lan, Andrea Vedaldi

摘要:

We present PhysiFormer, a diffusion transformer for physically-plausible 3D object motion. Unlike video world models that operate in view-dependent pixel space, PhysiFormer represents objects as 3D meshes expressed in world coordinates. Given the initial vertex positions and velocities, as well as object material type, rigid or elastic, the model samples future vertex trajectories. While related neural physics approaches build on ad-hoc latent spaces or explicitly enforce rigidity and causality, PhysiFormer shows that excellent results can be obtained without any such inductive biases, by casting vertex trajectory prediction as a single denoising diffusion process directly in world coordinates. The probabilistic formulation captures uncertainty in the learned dynamics, enabling diverse plausible futures from initial conditions, making this framework potentially useful for applications with unobserved uncertainty. The model features attention factorised over time, space, and objects for efficiency, enabling permutation-invariant multi-object reasoning without needing explicit object encoding. Trained on over 100k simulated trajectories, PhysiFormer generates rigid and elastic mechanics, and generalises to mixed-material settings, unseen real-world geometries, and larger object counts. It substantially outperforms autoregressive baselines in trajectory accuracy, rigidity preservation, and momentum-based physical consistency. Our results position coordinate-space diffusion as a promising step toward view-invariant, geometry-aware world modelling for robotics, graphics, and physical design. Visualisations, code, and models are available at https://yimingc9.github.io/physiformer.

摘要中文:

我们提出PhysiFormer,一个用于物理合理3D物体运动的扩散transformer。与在视角相关像素空间中操作的视频世界模型不同,PhysiFormer将物体表示为在世界坐标中表达的3D网格。给定初始顶点位置和速度,以及物体材质类型(刚性或弹性),模型对未来顶点轨迹进行采样。与相关神经物理方法构建临时潜在空间或显式强制刚性和因果性不同,PhysiFormer表明无需任何此类归纳偏置也能获得优异结果,方法是将顶点轨迹预测作为单一去噪扩散过程直接在世界坐标中执行。概率公式捕捉了所学动力学的不确定性,能够从初始条件生成多样化的合理未来,使得该框架可能对具有未观测不确定性的应用有用。该模型采用时间、空间和物体分解的注意力机制以提高效率,无需显式物体编码即可实现置换不变的多物体推理。PhysiFormer在超过10万条模拟轨迹上训练,生成刚性和弹性力学行为,并泛化到混合材质设置、未见过的真实世界几何形状和更多物体数量。它在轨迹精度、刚度保持和基于动量的物理一致性方面显著优于自回归基准。我们的结果将坐标空间扩散定位为机器人学、图形学和物理设计中实现视角不变、几何感知世界建模的有前景的一步。可视化、代码和模型可在https://yimingc9.github.io/physiformer获取。

Fast LeWorldModel

2026-06-26T04:00:00autoregressive, cs.CV, cs.LG, cs.RO2606.26217

中文标题:快速LeWorldModel

作者:Yuntian Gao, Xiangyu Xu

摘要:

Joint-Embedding Predictive Architectures (JEPAs), including recent LeWorldModel (LeWM), have become a promising foundation for reconstruction-free visual world models. For visual planning, however, LeWM evaluates candidate action sequences by repeatedly applying a local one-step latent transition model. This autoregressive rollout makes planning computationally expensive and exposes the predicted trajectory to accumulated latent errors as the horizon grows. We propose Fast LeWorldModel (Fast-LeWM), a fast latent world model that replaces repeated local rollout with action-prefix prediction. Given the current latent and a candidate action sequence, Fast-LeWM encodes its prefixes and predicts the future latents reached after executing those prefixes in parallel. By making action prefixes the basic prediction unit, Fast-LeWM directly models action effects accumulated to different extents over multiple horizons. This prefix-level supervision forces the model to learn how states continuously evolve under different action prefixes, rather than only fitting one-step state transitions. During planning, the predictor can use the last prefix token from the encoded action sequence to evaluate the corresponding future latent without explicitly rolling through each intermediate imagined state. Across multiple tasks, Fast-LeWM improves average success over LeWM while substantially reducing planning time, achieving lower open-loop latent loss whose growth becomes significantly slower as the rollout horizon increases.

摘要中文:

联合嵌入预测架构(JEPAs),包括近期的LeWorldModel(LeWM),已成为无需重建的视觉世界模型的有前景基础。然而,对于视觉规划,LeWM通过重复应用局部单步潜在转移模型来评估候选动作序列。这种自回归展开方式使规划计算成本高昂,且随着视界增长,预测轨迹面临累积的潜在误差问题。我们提出Fast LeWorldModel(Fast-LeWM),一种快速潜在世界模型,用动作前缀预测取代重复的局部展开。给定当前潜在状态和候选动作序列,Fast-LeWM对其前缀进行编码,并预测并行执行这些前缀后达到的未来潜在状态。通过将动作前缀作为基本预测单元,Fast-LeWM直接建模动作效果在多个视界上不同程度累积的过程。这种前缀级监督强制模型学习状态如何在不同的动作前缀下持续演化,而不仅仅是拟合单步状态转移。在规划过程中,预测器可利用编码动作序列的最后一个前缀标记来评估相应的未来潜在状态,而无需显式展开每个中间想象状态。在多个任务中,Fast-LeWM相比LeWM提升了平均成功率,同时大幅缩短规划时间,实现了更低的无环潜在损失,且该损失的增长速度随展开视界增加而显著放缓。

diffusion
Diffusion
30 篇论文

今日Diffusion领域呈现多元化发展态势,重点集中在视频生成与编辑模型加速与优化以及可控生成三大方向。视频生成相关研究最为活跃,包括实时流式编辑、物理感知增强、3D空间编码等细分赛道。模型加速方面出现多篇特征缓存与剪枝工作,显示落地部署需求日益增长。可控生成研究则关注视觉条件对齐和混合策略优化。安全与伦理研究也值得关注,包括模型投毒攻击防与内容安全审查。

  • LiveEdit: Towards Real-Time Diffusion-Based Streaming Video Editing - 提出首个实时流式扩散视频编辑框架,实现边生成边输出的超低延迟编辑体验,对视频编辑工作流具有变革性意义。
  • PhysRAG: Enhancing Physics-Awareness in Video Generation via Retrieval-Augmented Generation - 创新性地将检索增强引入扩散模型,显著提升视频生成的物理一致性,为解决AI视频“违背物理规律”痛点提供新思路。
  • LearniBridge: Learnable Calibration of Feature Caching for Diffusion Models Acceleration - 提出可学习的特征缓存校准机制,在保持生成质量的同时大幅加速扩散模型推理,对实际部署价值显著。
  • TMP: Tree-structured Mixed-policy Pruning for Large-scale Image Generation and Editing - 开创性地将树结构混合策略引入大规模图像生成剪枝,在压缩模型同时保持多任务生成能力。
  • LISA: Likelihood Score Alignment for Visual-condition Controllable Generation - 提出基于似然分数对齐的可控生成方法,增强扩散模型对视觉条件的遵循能力,提升生成可控性。

Accelerating Skill Assessment in Chess: A Drift-Diffusion-Enhanced Elo Rating System

2026-06-26T04:00:00cs.AI, diffusion2606.26267

中文标题:加速国际象棋技能评估:一种漂移扩散增强的Elo评分系统

作者:Tianyuan Zhou, Zhizheng Fu, Tianming Yang

摘要:

Rating systems such as Elo serve as the gold standard for matchmaking in competitive chess. However, they inherently suffer from response lag due to their exclusive reliance on match outcomes, neglecting the granular quality of gameplay. Nevertheless, incorporating move-by-move information into rating adjustments presents a significant challenge given the substantial noise and the vastness of the game-state space. To address this, we propose the Drift-Diffusion-Enhanced Elo Rating System (DD-Elo), a novel skill assessment framework inspired by the drift diffusion model (DDM) from cognitive neuroscience. By modeling skill expression as a decision-making process, our model integrates move-level data to capture rapid skill fluctuations. We provide a rigorous mathematical derivation proving that DD-Elo maintains a bounded deviation from the traditional Elo system, ensuring theoretical alignment. Extensive experiments demonstrate that DD-Elo adapts to skill changes faster than Elo. Our findings suggest that DD-Elo offers an explainable, highly responsive, and backward-compatible solution for chess rating ecosystems. The implementation code is publicly available at https://github.com/Aquila-zhou1/DD-Elo .

摘要中文:

Elo等评分系统是竞技国际象棋配对的黄金标准。然而,由于其仅依赖比赛结果而忽略游戏质量的细节,固有地存在响应延迟问题。尽管如此,考虑到大量噪声和游戏状态空间的庞大性,将逐着信息纳入评分调整是一个重大挑战。为此,我们提出了漂移扩散增强Elo评分系统(DD-Elo),这是一个受认知神经科学漂移扩散模型(DDM)启发的全新技能评估框架。通过将技能表达建模为决策过程,我们的模型整合着法级数据以捕捉快速技能波动。我们提供了严格的数学推导,证明DD-Elo与传统Elo系统保持有界偏差,确保理论一致性。大量实验表明,DD-Elo比Elo更快地适应技能变化。我们的研究结果表明,DD-Elo为国际象棋评分生态系统提供了一种可解释、高响应且向后兼容的解决方案。实现代码已在https://github.com/Aquila-zhou1/DD-Elo公开发布。

Generative Retrieval via Diffusion Transformer with Metric-Ordered Sequence Training and Hybrid-Policy Preference Optimization

2026-06-26T04:00:00cs.AI, diffusion2606.26899

中文标题:基于度量序序列训练和混合策略偏好优化的Diffusion Transformer生成式检索

作者:Chenghao Liu, Yu Zhang, Zhongtao Jiang, Kun Xu, Zhenwei An, Renzhi Wang, Zhao Wang, Jiachen Zhang, Yuxiao Zhang, Kun Xu, Songfang Huang

摘要:

Embedding-based retrieval ranks items by their similarity to a query in a shared vector space and usually aims to return the highest-scoring items. In many production settings this is not what is wanted: given a seed set that expresses a fine-grained pattern, one needs more items that both satisfy a target attribute and stay within that pattern. We formalize this as pattern-preserving attribute retrieval. The two goals pull against each other: averaging the seeds preserves the pattern but stays in a low-attribute region, while global attribute retrieval drifts to unrelated patterns. We approach the task with continuous generative retrieval, where a model reads a sequence of item embeddings and generates query embeddings for nearest-neighbor search. We propose MO-DiT+HPPO, a staged framework with raw-sequence pretraining, multi-domain metric-ordered continuation pretraining, tail-centroid fine-tuning, and HPPO. Metric-ordered training turns sparse online retrieval labels into in-pattern trajectories ordered from low to high predicted attribute density, teaching one model the metric-improvement direction across domains. HPPO aligns the generated query distribution with the true online objective by labeling a hybrid candidate pool with the online intersection metric and applying reference-anchored preference optimization. A Pareto pair filter keeps only winner pairs that do not lower same-pattern purity, raising the attribute metric without sacrificing the pattern. Across four attribute domains under item- and pattern-holdout protocols, metric-ordered DiT improves the intersection metric over a pretrained generative retriever, and HPPO improves it further, with significant gains on seven of eight domain-split cells and a marginal tie on the hardest split. Metric-predictor validation, order ablations, CPT/SFT comparisons, and a candidate-policy ablation show where the gains come from.

摘要中文:

基于嵌入的检索通过在共享向量空间中计算项目与查询的相似度来对其进行排名,通常旨在返回得分最高的项目。在许多实际生产场景中,这并非所需:给定一个表达细粒度模式的种子集,需要更多同时满足目标属性且保持在该模式内的项目。我们将此形式化为模式保持的属性检索。这两个目标相互冲突:对种子取平均会保留模式但停留在低属性区域,而全局属性检索则会漂移到不相关的模式。我们采用连续生成式检索来应对这一任务,其中模型读取一系列项目嵌入并生成用于最近邻搜索的查询嵌入。我们提出了MO-DiT+HPPO,这是一个分阶段框架,包含原始序列预训练、多域度量序延续预训练、尾质心微调和HPPO。度量序训练将稀疏的在线检索标签转化为从低到高预测属性密度的模式内轨迹,使一个模型能够学习跨域的度量改进方向。HPPO通过使用在线交集度量标记混合候选池并应用参考锚定偏好优化,使生成的查询分布与真实在线目标对齐。Pareto对过滤器仅保留不降低相同模式纯度的获胜者对,从而在不影响模式的情况下提升属性指标。在四项属性领域下采用项目和模式保留协议进行测试,度量有序DiT相较于预训练生成式检索器提升了交集度量,HPPO进一步提升了该指标,在八个域分割单元中的七个获得了显著增益,在最困难的分割单元中获得勉强平局。度量预测器验证、顺序消融、CPT/SFT比较以及候选策略消融实验表明了这些增益的来源。

2026-06-26T04:00:00cs.AI, cs.CY, cs.ET, cs.SD, diffusion2606.26111

中文标题:生成式人工智能与版权侵权:基于美国法典第17编的AI音乐生成系统法律技术分析

作者:Zuhaib Hussain Butt

摘要:

Generative artificial intelligence (GenAI) has enabled users to synthesize music with text prompts, combining copyrighted lyrics, AI-composed melodies, and synthetic vocals that imitate real artists. This paper examines the legal and technical dimensions of AI-based music creation (e.g., Google Gemini's music tools) under U.S. copyright law. We analyze whether a user who inputs one artist&x27;s protected lyrics into a GenAI system, directs it to use another artist's voice or style, publishes the resulting song, and monetizes it violates 17 U.S.C. Section 106&x27;s exclusive rights [3]. The analysis integrates Title 17 doctrine (rights of reproduction, derivative works, distribution), 17 U.S.C. Section 114's narrow sound recording protection [4], and the new voice-cloning laws emerging at the state level [20]. We argue that unauthorized lyric copying poses a high risk of infringement of the musical composition, whereas mere AI-generated voice imitation typically falls outside federal sound recording protection and instead implicates state publicity rights [12], [13]. Recent cases and legislation (Concord v. Anthropic [10]; Kadrey v. Meta [11]; Lehrman v. Lovo [12]; Tennessee&x27;s "ELVIS Act" [20]; UMG v. Uncharted Labs [14]; etc.) illustrate this split. We map AI technical components (prompt encoding, latent diffusion, neural vocoders, speaker embeddings) to legal risks and identify a regulatory gap: federal law robustly protects lyrics and melody but currently provides limited remedies for synthesized vocal likeness [22], [23]. The paper concludes with policy suggestions for clearer rules on AI music creation.

摘要中文:

生成式人工智能(GenAI)使用户能够通过文本提示合成音乐,将受版权保护的歌词、人工智能创作的旋律和模仿真实艺术家的合成声音相结合。本文从法律和技术层面探讨基于人工智能的音乐创作(如Google Gemini的音乐工具)在美国版权法下的相关问题。我们分析用户将一位艺术家的受保护歌词输入生成式人工智能系统、指示其使用另一位艺术家的声音或风格、发布生成的歌曲并从中获益是否违反《美国法典》第17编第106条规定的专有权利[3]。该分析整合了第17编 doctrine(复制权、衍生作品权、发行权)、《美国法典》第17编第114条对录音制品的有限保护[4],以及各州层面新出台的声音克隆法律[20]。 我们认为,未经授权的歌词复制对音乐作品构成较高的侵权风险,而单纯的人工智能生成的声音模仿通常不属于联邦录音制品保护范围,反而涉及州层面的公开形象权[12],[13]。近期案例和立法(如Concord v. Anthropic[10];Kadrey v. Meta[11];Lehrman v. Lovo[12];田纳西州"ELVIS Act"[20];UMG v. Uncharted Labs[14]等)说明了这种法律适用的分歧。我们将人工智能技术组件(提示编码、潜在扩散、神经声码器、说话人嵌入)映射到法律风险,并识别出一个监管空白:联邦法律有力保护歌词和旋律,但对合成声音肖像目前提供的救济手段有限[22],[23]。本文最后就人工智能音乐创作的更清晰规则提出了政策建议。

TEMPO-Diffusion: Temporally Exposed Malicious Poisoning of Diffusion Models

2026-06-26T04:00:00cs.AI, cs.CR, diffusion2606.26285

中文标题:TEMPO-Diffusion:时间暴露型恶意扩散模型投毒攻击

作者:William Aiken, Paula Branco, Guy-Vincent Jourdan, Iosif-Viorel Onut

摘要:

Noise-based backdoor attacks on diffusion models typically rely on input-time trigger injection, untargeted activation, and out-of-distribution target generation. Such assumptions reduce both the stealthiness and the practical relevance of these attacks. In this work, we present TEMPO-Diffusion, a targeted backdoor framework that localizes the malicious distribution shift to a temporal, in-distribution exposure. TEMPO-Diffusion supports: (i) targeted attacks on and to specific classes, (ii) multiple sub-image backdoors that reconstruct specific features within multiple, different output images and at multiple locations, and (iii) in-painting with time-conditioned triggers. To study relevant, practical security concerns in leveraging backdoored diffusion models for synthetic training data, we also introduce CALISA: a balanced, region-aware traffic-sign dataset emphasizing Canadian and U.S. road signs. Across CIFAR10, GTSRB, and CALISA, our experiments show that TEMPO-Diffusion can reliably poison class-specific synthetic data generation and induce high attack success rates in downstream classifiers trained on that data.

摘要中文:

基于噪声的扩散模型后门攻击通常依赖于输入时触发器注入、无目标激活和分布外目标生成。这类假设降低了攻击的隐蔽性及实际相关性。本工作提出TEMPO-Diffusion,一种目标后门框架,将恶意分布偏移定位至时间性的、分布内暴露。TEMPO-Diffusion支持:(i)针对特定类别及从特定类别生成的目标攻击,(ii)多个子图像后门,可在多个不同输出图像的多个位置重建特定特征,(iii)带时间条件触发器的图像修复任务。为研究利用后门扩散模型生成合成训练数据相关的实际安全问题,本工作还引入了CALISA:一个平衡的、区域感知的交通标志数据集,重点关注加拿大和美国道路标志。在CIFAR10、GTSRB和CALISA上的实验表明,TEMPO-Diffusion能够可靠地毒害类别特定的合成数据生成,并在基于该数据训练的下游分类器中实现高攻击成功率。

Beyond Feedforward Networks: Reentry Neural Systems as the Fundamental Basis of Subjecthood and Intrinsic Safety of Next-Generation AGI

2026-06-26T04:00:00cs.AI, cs.LG, diffusion, math-ph, math.MP2606.26406

中文标题:超越前馈网络:重入神经系统作为下一代人工通用智能主体性及内在安全性的基础架构

作者:A. S. Ushakov, Yu. N. Berdinsk

摘要:

We propose a complete architectural blueprint for safe artificial general intelligence based on a closed reentry loop (D <-> I cycle). In contrast to feedforward networks, which are directed acyclic graphs (C=0, S=0) incapable of self-reference, the proposed architecture contains a structural cycle (C >= 1) with self-sustaining amplification (rho > 1), mathematically guaranteeing the emergence of a self-model, instrumental self-preservation, and unprogrammed goal-directed behaviour. The agent&#x27;s goals are encoded as a non-textual D-vector in the architecture itself, making them immune to reinterpretation and prompt injection. We present the S-measure -- a polynomial-time [O(N^3)] computable alternative to Tononi&x27;s NP-hard Phi -- with machine-verified Lean 4 proof that S>0 implies positive integrated information. The work provides full Python/NumPy implementations (Tarjan-based cycle complexity, Delta-S barrier), industrial horizontal scaling via Apache Kafka and Docker Compose, a taxonomy of six epochs of AI evolution, a zoo of future reentry architectures (RAS, diffusion attractors, fractal loops), gauge-invariant networks for safe swarms, fault-tolerance and recovery protocols, and eight falsifiable predictions. All formal proofs are machine-verified in Lean 4. This architecture is deployable today and represents a topologically protected, safe-by-design approach to AGI.

摘要中文:

我们提出了一种基于封闭重入回路(D ↔ I 循环)的安全人工通用智能完整架构方案。与作为有向无环图(C=0, S=0)且无法进行自指的前馈网络不同,所提出的架构包含结构循环(C ≥ 1)并具有自我维持放大特性(ρ > 1),从数学上保证了自我模型、工具性自我保存以及非编程目标导向行为的涌现。智能体的目标以架构中非文本的D向量形式编码,使其对重新解释和提示注入具有免疫性。我们提出了S度量——一种多项式时间[O(N³)]可计算的Tononi NP难Φ替代方案,并提供了经机器验证的Lean 4证明,表明S>0意味着正整合信息。本工作提供了完整的Python/NumPy实现(基于Tarjan的循环复杂度、Delta-S屏障)、通过Apache Kafka和Docker Compose实现的工业级水平扩展、AI演进六纪元分类体系、未来重入架构“动物园”(RAS、扩散吸引子、分形循环)、面向安全蜂群的规范不变网络、容错与恢复协议,以及八项可证伪预测。所有形式化证明均在Lean 4中经过机器验证。该架构可于当下部署,代表了一种拓扑受保护、内在安全的AGI设计方法。

2026-06-26T04:00:00cs.AI, cs.CV, cs.MM, diffusion2606.26795

中文标题:NaviCache:视频生成的测试时自校准缓存

作者:Zheqi Lv, Zhibo Zhu, Jinke Wang, Qi Tian, Shengyu Zhang, Zhengyu Chen, Chengxi Zang, Zhou Zhao, Fei Wu

摘要:

Video Diffusion Models (VDMs) is constrained by immense computational costs. While offline calibration-based acceleration suffers from calibration data dependency, prohibitive calibration duration, and susceptibility to distribution shifts, offline calibration-free methods eliminate these hurdles. However, since they rely on instantaneous zero-order approximations where the mapping between input and output differences varies in real-time, they are susceptible to observational noise and ignore the intrinsic momentum within the diffusion trajectory. In this paper, we propose NaviCache, a plug-and-play test-time self-calibration method re-conceptualizing feature evolution as an Inertial Navigation System (INS) problem. NaviCache bridges the fundamental domain gap and the non-stationary nature of diffusion by modeling the relative coupling between input and output variations. We introduce a dual-state estimation architecture that adaptively tracks the feature change ratio and its latent drift, initialized via a specialized Initial Alignment phase. By integrating a time-dependent noise schedule with an uncertainty-aware Measurement Update mechanism, NaviCache provides a theoretically grounded mechanism for error-bounded computation skipping. Extensive experiments on the HunyuanVideo, Wan, and Open-Sora series demonstrate that NaviCache exhibits more accurate error judgment for computation skipping and achieves outstanding comprehensive performance.

摘要中文:

视频扩散模型(VDMs)受限于巨大的计算成本。尽管基于离线校准的加速方法受制于校准数据依赖性、过长的校准时长以及对分布漂移的敏感性,但离线无校准方法消除了这些障碍。然而,由于这些方法依赖于瞬时零阶近似,其中输入与输出差异之间的映射随时间动态变化,因此容易受到观测噪声的影响,并且忽视了扩散轨迹中的内在惯性。在本文中,我们提出了NaviCache,这是一种即插即用的测试时自校准方法,将特征演化重新概念化为惯性导航系统(INS)问题。NaviCache通过建模输入与输出变化之间的相对耦合,架起了基本域差距与扩散非平稳特性之间的桥梁。我们引入了一种双态估计架构,能够自适应地追踪特征变化率及其潜在漂移,并通过专门的初始对齐阶段进行初始化。通过将时间依赖的噪声调度与不确定性感知的测量更新机制相结合,NaviCache提供了一种有理论依据的误差有界计算跳过机制。在HyuanVideo、Wan和Open-Sora系列模型上的大量实验表明,NaviCache在计算跳过的误差判断上表现出更高的准确性,并取得了卓越的综合性能。

Safe Autoregressive Image Generation with Iterative Self-Improving Codebooks

2026-06-26T04:00:00autoregressive, cs.AI, cs.CV, diffusion2606.27147

中文标题:基于迭代自改进码本的安全自回归图像生成

作者:Yunqi Xue, Zhijiang Li, Philip Torr, Jindong Gu

摘要:

Unlike diffusion-based models that operate in continuous latent spaces, autoregressive unified multimodal models produce images by sequentially predicting discretized visual tokens. These tokens are derived from a codebook that maps embeddings to quantized visual patterns. The language-like architecture enables unified multimodal models to effectively capture text conditional information for generation, making them promising for text-to-image tasks. This also raises an interesting question: how safe are the images generated in such an autoregressive way? In this work, we propose iterative self-improving codebooks for safe autoregressive generation. We leverage the understanding and judgment capabilities of the unified multimodal model itself to identify unsafe generated images without human annotation. Subsequently, the inherent representations in the codebook are fixed to eliminate harmful mappings. Our method comprises two steps: first, we use the unified model to identify unsafe generations and construct corresponding harmful and safe image-text pairs. These pairs are used to construct the Harmful Space and guide updates to the codebook, thereby eliminating harmful outputs. Second, we perform adaptive fine-tuning on the codebook within the harmless space using safe image-text pairs to ensure the quality of generated images. These two steps are repeated until no further improvement is observed, producing a safety-enhanced model codebook. Without additional external feedback, the safety of models is improved iteratively.

摘要中文:

与在连续潜空间中运行的基于扩散的模型不同,自回归统一多模态模型通过序贯预测离散视觉token来生成图像。这些token源自将嵌入映射到量化视觉模式的码本。类语言的架构使统一多模态模型能够有效捕获用于生成的文本条件信息,使其在文本到图像任务中具有良好的应用前景。这也引出了一个有趣的问题:这种自回归方式生成的图像安全性如何?在本工作中,我们提出了用于安全自回归生成的迭代自改进码本。我们利用统一多模态模型自身的理解和判断能力,在无需人工标注的情况下识别不安全生成的图像。随后,码本中的内在表示被固定以消除有害映射。我们的方法包含两个步骤:首先,我们使用统一模型识别不安全生成并构建相应的有害和安全图像-文本对。这些配对用于构建有害空间并指导码本更新,从而消除有害输出。其次,我们使用安全图像-文本对在无害空间中对码本进行自适应微调,以确保生成图像的质量。我们重复这两个步骤,直到不再观察到进一步改进,从而产生安全增强的模型码本。无需额外的外部反馈,模型的安全性通过迭代方式得到提升。

From Celebrities to Anyone: Characterizing AI Nudification Content, Technology, and Community Dynamics on 4chan

2026-06-26T04:00:00cs.AI, cs.CV, cs.CY, cs.HC, diffusion2606.27234

中文标题:从名人到任何人:4chan上AI裸体化内容、技术与社区动态的特征分析

作者:Chi Cui, Yixin Wu, Yang Zhang

摘要:

AI nudification uses generative models to create synthetic non-consensual sexually explicit imagery (SNEACI) of real individuals. Prior work has examined dedicated nudification platforms and model repositories, finding that most targets are female celebrities. However, the anonymous content community, where SNEACI is actively requested, generated, and exchanged, remains unexplored. In this work, we present a large-scale study of AI nudification in the wild, identifying 24,105 SNEACI items. We find a significant shift in target demographics: non-celebrity individuals now account for 55.8\% of targets, compared to only 4.7\% in prior studies, indicating that AI nudification has expanded from targeting public figures to increasingly harming individuals within users' own social circles. Meanwhile, open-source models dominate production, with Stable Diffusion family generating 42.7\% of images and Wan generating 66.5\% of videos, all driven by thousands of shared fine-tuned models and accessible tutorials. Yet the ecosystem runs on a small cohort of active producers, with the most prolific producing 780 items, drives community engagement, shapes target demographics, and disseminates technical knowledge that lowers barriers for new producers. Our work provides an empirical understanding of how AI nudification operates in the wild, revealing the mechanisms that sustain this ecosystem and highlighting the urgent need for interventions in platform governance, technical safeguards, and affected individual protection.

摘要中文:

AI裸体化利用生成式模型创建真实个人的合成非自愿性明确图像。先前研究已对专门的裸体化平台和模型仓库进行了考察,发现大多数目标为女性名人。然而,匿名内容社区——非自愿性明确图像被积极请求、生成和交换的场所——仍未经探索。在本研究中,我们开展了一项针对真实环境中AI裸体化的大规模研究,识别出24,105个非自愿性明确图像项目。我们发现目标人群出现了显著转变:非名人个体现占目标的55.8%,而先前研究仅为4.7%,表明AI裸体化已从针对公众人物扩展到日益伤害用户社交圈内的个体。与此同时,开源模型在生产中占据主导地位,Stable Diffusion系列生成42.7%的图像,Wan生成66.5%的视频,这一切都由数千个共享的微调模型和可访问的教程驱动。然而,该生态系统依赖于一小部分活跃生产者,最高产者生产了780个项目,推动社区参与、塑造目标人群特征,并传播降低新生产者门槛的技术知识。我们的工作提供了对AI裸体化在真实环境中运作方式的实证理解,揭示了维持这一生态系统的机制,并强调了在平台治理、技术防护和受影响个人保护方面迫切需要干预。

Reconstruction Alignment Improves Unified Multimodal Models

2026-06-26T04:00:00autoregressive, cs.AI, cs.CV, cs.LG, diffusion2509.07295

中文标题:重建对齐提升统一多模态模型

作者:Ji Xie, Trevor Darrell, Luke Zettlemoyer, XuDong Wang

摘要:

Unified multimodal models (UMMs) unify visual understanding and generation within a single architecture. However, conventional training relies on image-text pairs (or sequences) whose captions are typically sparse and miss fine-grained visual details, even when they use hundreds of words to describe a simple image. We introduce Reconstruction Alignment (RECA), a resource-efficient post-training method that leverages visual understanding encoder embeddings as dense "text prompts", providing rich supervision without captions. Concretely, RECA conditions a UMM on its own visual understanding embeddings and optimizes it to reconstruct the input image with a self-supervised reconstruction loss, thereby realigning understanding and generation. Despite its simplicity, RECA is broadly applicable: across autoregressive, masked-autoregressive, and diffusion-based UMMs, it consistently improves generation and editing fidelity. With only 27 GPU hours, post-training with RECA substantially improves image generation performance on GenEval (0.73 $\rightarrow$ 0.90) and DPGBench (80.93 $\rightarrow$ 88.15), while also boosting editing benchmarks (ImgEdit 3.38 $\rightarrow$ 3.75, GEdit 6.94 $\rightarrow$ 7.27). Notably, RECA surpasses much larger open-source models and applies broadly across diverse UMM architectures, establishing it as an efficient and general post-training alignment strategy for UMMs.

摘要中文:

统一多模态模型(UMMs)将视觉理解和生成统一在单一架构中。然而,传统训练依赖于图像-文本对(或序列),其描述通常较为稀疏,无法捕捉细粒度的视觉细节,即使使用数百个词描述简单图像亦是如此。我们提出了重建对齐(RECA),一种资源高效的后训练方法,利用视觉理解编码器嵌入作为密集的"文本提示",在无需描述的情况下提供丰富的监督信号。具体而言,RECA将统一多模态模型以其自身的视觉理解嵌入为条件,并利用自监督重建损失优化模型重建输入图像,从而重新对齐理解和生成。尽管方法简单,但RECA具有广泛适用性:在自回归、掩码自回归和基于扩散的统一多模态模型中,它均能持续提升生成和编辑保真度。仅需27 GPU小时,RECA后训练即可显著提升图像生成在GenEval(0.73→0.90)和DPGBench(80.93→88.15)上的性能,同时增强编辑基准(ImgEdit 3.38→3.75,GEdit 6.94→7.27)。值得注意的是,RECA超越了更大的开源模型,并广泛应用于不同的统一多模态模型架构,为统一多模态模型确立了一种高效且通用的后训练对齐策略。

VecSet-Edit: Unleashing Pre-trained LRM for Mesh Editing from Single Image

2026-06-26T04:00:00cs.AI, cs.CV, diffusion2602.04349

中文标题:VecSet-Edit:释放预训练LRM实现单图像3D网格编辑

作者:Teng-Fang Hsiao, Bo-Kai Ruan, Yu-Lun Liu, Hong-Han Shuai

摘要:

3D editing has emerged as a critical research area to provide users with flexible control over 3D assets. While current editing approaches predominantly focus on 3D Gaussian Splatting or multi-view images, the direct editing of 3D meshes remains underexplored. Prior attempts, such as VoxHammer, rely on voxel-based representations that suffer from limited resolution and necessitate labor-intensive 3D mask. To address these limitations, we propose \textbf{VecSet-Edit}, the first pipeline that leverages the high-fidelity VecSet Large Reconstruction Model (LRM) as a backbone for mesh editing. Our approach is grounded on a analysis of the spatial properties in VecSet tokens, revealing that token subsets govern distinct geometric regions. Based on this insight, we introduce Mask-guided Token Seeding and Attention-aligned Token Gating strategies to precisely localize target regions using only 2D image conditions. Also, considering the difference between VecSet diffusion process versus voxel we design a Drift-aware Token Pruning to reject geometric outliers during the denoising process. Finally, our Detail-preserving Texture Baking module ensures that we not only preserve the geometric details of original mesh but also the textural information. More details can be found in our project page: https://github.com/BlueDyee/VecSet-Edit/tree/main

摘要中文:

3D编辑已成为为用户提供灵活控制3D资产的关键研究领域。当前的编辑方法主要聚焦于3D高斯溅射或多视角图像,而对3D网格的直接编辑仍缺乏充分探索。此前的尝试,如VoxHammer,采用基于体素的表示方法,但受限于分辨率且需要繁重的3D掩码标注。为解决这些局限,我们提出VecSet-Edit,这是首个利用高保真VecSet大重建模型(LRM)作为网格编辑骨干网络的流程。我们的方法基于对VecSet标记空间属性的分析,发现标记子集控制不同的几何区域。基于这一洞察,我们引入掩码引导的标记播种和注意力对齐的标记门控策略,仅使用2D图像条件即可精确定位目标区域。此外,考虑到VecSet扩散过程与体素的差异,我们设计了漂移感知的标记剪枝策略,以在去噪过程中剔除几何异常值。最后,我们的细节保留纹理烘焙模块确保不仅保留原始网格的几何细节,还保留纹理信息。更多细节请访问项目页面:https://github.com/BlueDyee/VecSet-Edit/tree/main

Finetuning-Free Diffusion Model with Adaptive Constraint Guidance for Inorganic Crystal Structure Generation

2026-06-26T04:00:00cond-mat.mtrl-sci, cs.AI, diffusion2604.13354

中文标题:用于无机晶体结构生成的具有自适应约束引导的免微调扩散模型

作者:Auguste de Lambilly, Vladimir Baturin, David Portehault, Guillaume Lambard, Nataliya Sokolovska, Florence d&#x27;Alch\&x27;e-Buc, Jean-Claude Crivello

摘要:

Generative diffusion models have emerged as powerful tools for the discovery of inorganic crystal structures, yet steering their sampling process toward user-defined physical and chemical objectives remains challenging. We present a computational framework that integrates adaptive constraint guidance into a pre-trained crystal diffusion model, enabling the generation of candidate structures that satisfy targeted structural and chemical requirements without model retraining. The approach incorporates differentiable constraint functions directly during sampling, providing an interpretable mechanism for expert-driven exploration of the crystal structure space. To assess the reliability of generated candidates, we introduce a multi-stage validation workflow combining descriptor-based analysis, duplicate removal, comparison with reference crystal databases, graph neural network energy prediction, and thermodynamic stability evaluation through convex-hull analysis. The framework is applied to several classes of inorganic compounds and to constraints involving atomic volume, local coordination environments, and near-neighbor structural motifs. Results demonstrate that adaptive guidance effectively redirects the sampling distribution toward structures exhibiting the desired characteristics while preserving chemical plausibility. Subsequent validation reveals which generated candidates remain viable after energetic and thermodynamic screening. The proposed methodology provides a practical and transparent strategy for incorporating expert knowledge into crystal generative models and establishes a general computational framework for constrained materials discovery.

摘要中文:

生成式扩散模型已成为发现无机晶体结构的强大工具,然而将其采样过程引导至用户定义的物理化学目标仍具有挑战性。本研究提出了一个计算框架,将自适应约束引导集成到预训练的晶体扩散模型中,使得能够在无需模型重新训练的情况下生成满足目标结构和化学要求的候选结构。该方法在采样过程中直接引入可微约束函数,为专家驱动的晶体结构空间探索提供了一种可解释的机制。为了评估生成候选结构的可靠性,我们引入了一个多阶段验证工作流程,结合基于描述符的分析、去重、与参考晶体数据库的比较、图神经网络能量预测以及通过凸包分析的热力学稳定性评估。该框架应用于多种无机化合物类别以及涉及原子体积、局部配位环境和近邻结构基元的约束。结果表明,自适应引导有效地将采样分布重定向至呈现所需特征的结构,同时保持了化学合理性。后续验证揭示了经能量和热力学筛选后哪些生成候选结构仍然可行。该方法为将专家知识融入晶体生成模型提供了一种实用且透明的策略,并建立了约束材料发现的通用计算框架。

Mapping License Plate Recoverability Under Extreme Viewing Angles for Opportunistic Urban Sensing

2026-06-26T04:00:00cs.AI, cs.CV, diffusion2604.23814

中文标题:机会式城市感知中极端视角下车牌可恢复性的映射研究

作者:Igor Adamenko, Orpaz Ben Aharon, Yehudit Aperstein, Alexander Apartsin

摘要:

Urban environments contain many imaging sensors built for specific purposes, including ATM, body-worn, CCTV, and dashboard cameras. Under the opportunistic sensing paradigm, these sensors can be repurposed for secondary inference tasks such as license plate recognition. Yet objects of interest in such im-agery are often noisy, low-resolution, and captured from extreme viewpoints. Recent advances in AI-based restoration can recover useful information even from severely degraded images. A central challenge is de-termining which distortion parameters allow reliable recovery and which lead to inference failure. This paper introduces recoverability maps, a task-agnostic method for quantifying this boundary. The method combines a dense synthetic sweep of degradation parameters with two summary measures: boundary ar-ea-under-curve, which estimates the recoverable fraction of the parameter space, and a reliability score, which captures the frequency and severity of failures within that region. We demonstrate the method on li-cense plate recognition from highly angled views under realistic camera artifacts. Several restoration archi-tectures are trained and evaluated, including U-Net, Restormer, Pix2Pix, and SR3 diffusion. The best model recovers about 93% of the parameter space. Similar results across models suggest that sensing geometry, ra-ther than architecture, sets the limit of recovery.

摘要中文:

城市环境中包含许多针对特定用途的成像传感器,包括ATM摄像头、穿戴式摄像头、闭路电视和车载摄像头。在机会式感知范式下,这些传感器可以被重新用于车牌识别等辅助推理任务。然而,此类图像中的感兴趣目标通常具有噪声、低分辨率且从极端视角拍摄的特点。近期基于人工智能的图像恢复技术即使从严重退化的图像中也能恢复有用信息。核心挑战在于确定哪些退化参数允许可靠恢复,哪些会导致推理失败。本文引入了可恢复性映射方法,这是一种任务无关的量化方法。该方法结合了退化参数的密集合成扫描和两个汇总度量:边界曲线下面积,用于估计参数空间的可恢复比例;以及可靠性评分,用于捕捉该区域内失败的频率和严重程度。我们在现实相机伪影下从大角度视角进行车牌识别的场景中演示了该方法。多种恢复架构被训练和评估,包括U-Net、Restormer、Pix2Pix和SR3扩散模型。最佳模型恢复了约93%的参数空间。不同模型的结果相似,表明感知几何而非模型架构决定了恢复的极限。

LiveEdit: Towards Real-Time Diffusion-Based Streaming Video Editing

2026-06-26T04:00:00cs.CV, diffusion2606.26740

中文标题:LiveEdit:面向实时基于扩散的流式视频编辑

作者:Xinyu Wang, Chongbo Zhao, Fangneng Zhan, Yue Ma

摘要:

Streaming video editing has made rapid progress, yet practical deployment is still limited by two core issues: maintaining stable backgrounds and non-edited regions over time, and achieving the low latency required for real-time interactive scenarios. Meanwhile, recent streaming video generation methods are mostly developed for synthesis and cannot be directly applied to editing due to the strict preservation requirement and region-specific control. In this work, we present a novel streaming video editing framework that performs causal, frame-by-frame editing with strong content preservation and real-time responsiveness. Our key design is a three-stage distillation pipeline that progressively transfers editing capability from a powerful bidirectional foundation model to an efficient unidirectional streaming editor, enabling stable long-horizon edits without sacrificing visual fidelity. To further support real-time deployment, we introduce an AR-oriented mask cache that reuses region-related computation across frames, substantially reducing redundant processing and accelerating inference. Finally, we establish a dedicated benchmark for streaming video editing. Extensive evaluations demonstrate that our method achieves state-of-the-art visual quality among streaming baselines while drastically boosting inference speed to 12.66 FPS, making it suitable for interactive and augmented reality applications.

摘要中文:

流式视频编辑虽已取得快速进展,但实际部署仍受到两个核心问题的制约:如何在时间维度上保持稳定的背景和非编辑区域,以及如何实现实时交互场景所需的低延迟。与此同时,现有的流式视频生成方法大多面向合成任务开发,由于严格的保持要求和区域特定控制需求,无法直接应用于编辑任务。本研究提出了一种新颖的流式视频编辑框架,能够在保持强内容保持性的同时实现因果性的逐帧编辑和实时响应。我们的核心设计是一个三阶段蒸馏管道,将强大的双向基础模型的编辑能力逐步迁移到高效的单向流式编辑器中,从而在不影响视觉保真度的前提下实现稳定的长期编辑。为进一步支持实时部署,我们引入了一种面向AR的掩码缓存机制,复用跨帧的区域相关计算,显著减少了冗余处理并加速了推理过程。最终,我们建立了一个专用的流式视频编辑基准测试。大量评估表明,我们的方法在流式基线中实现了最先进的视觉质量,同时将推理速度大幅提升至12.66 FPS,使其适用于交互式和增强现实应用。

LearniBridge: Learnable Calibration of Feature Caching for Diffusion Models Acceleration

2026-06-26T04:00:00cs.CV, cs.LG, diffusion2606.26778

中文标题:LearniBridge:用于扩散模型加速的可学习特征缓存校准

作者:Xuyue Huang, Zhe Chen, Wang Shen, Xiao-Ping Zhang

摘要:

Diffusion Transformers (DiTs) have driven substantial progress in image and video generation but suffer from prohibitive computational costs. Feature caching accelerates inference by reusing intermediate representations. Existing methods rely on historical features for implementation simplicity, yet suffer from severe error accumulation at high acceleration ratios. To address this limitation, we investigate the nature of the requisite feature correction. We demonstrate that the optimal calibration update is characterized by a shared low-rank subspace across diverse prompts. Guided by this structural insight, we propose LearniBridge, a learnable calibration mechanism for feature caching that bridges multiple timesteps through lightweight LoRA updates. This mechanism enables effective calibration requiring only 3-5 training samples. Extensive experiments on image and video generation show that LearniBridge achieves up to $5.87\times$, $5.75\times$, and $4.10\times$ acceleration on FLUX, HunyuanVideo, and WAN2.1, respectively. On WAN2.1, it improves VBench by 1.28% over the previous SOTA at $4.10\times$ acceleration. Our code is available at https://github.com/Iiiiiiirene/LearniBridge.

摘要中文:

扩散Transformer(DiTs)在图像和视频生成方面取得了实质性进展,但存在极高的计算成本。特征缓存通过重用中间表征来加速推理。现有方法因实现简单而依赖历史特征,但在高加速比下遭受严重的误差累积。为解决这一局限性,我们探究了必要特征修正的本质。我们证明,最优校准更新具有跨不同提示的共享低秩子空间特性。基于这一结构化洞察,我们提出LearniBridge——一种可学习的特征缓存校准机制,通过轻量级LoRA更新桥接多个时间步。该机制仅需3-5个训练样本即可实现有效校准。在图像和视频生成上的广泛实验表明,LearniBridge在FLUX、HunyuanVideo和WAN2.1上分别实现了最高5.87倍、5.75倍和4.10倍的加速。在WAN2.1上,相比此前最优方法在4.10倍加速下,VBench得分提升了1.28%。代码已发布于https://github.com/Iiiiiiirene/LearniBridge。

Tractography-Driven Synthetic Data Generation for Fiber Bundle Segmentation in Tracer Histology

2026-06-26T04:00:00cs.CV, cs.LG, diffusion2606.26898

中文标题:基于纤维追踪的合成数据生成用于示踪剂组织学中的纤维束分割

作者:Kyriaki-Margarita Bintsi, Sparsh Makharia, Ya\"el Balbastre, Joselyn Romero Avila, Julia F. Lehman, Suzanne N. Haber, Anastasia Yendiki

摘要:

Diffusion MRI (dMRI) tractography enables non-invasive reconstruction of white-matter pathways, but its accuracy is fundamentally limited by indirect, low-resolution measurements of axonal organization. Tracer injection studies in non-human primates provide a gold standard for validating dMRI tractography. This, however, requires time-consuming manual annotation of fiber bundles in histology sections. We propose a synthetic-data augmented framework for automated fiber bundle segmentation in macaque tracer histology. Our approach uses ex vivo dMRI tractography as a generative prior to synthesize 2D image patches for training. This provides us with sufficiently realistic foreground texture, which we compose with backgrounds from blockface photos and diversify via domain randomization. A 2D U-Net is trained on mixed real and synthetic patches. Experiments on held-out brains demonstrate improved generalization across brains and fiber bundle densities compared to training with real data only. Training with synthetic data only leads to poor performance, underscoring the need for real supervision. Overall, our approach achieves performance comparable to the state-of-the-art while requiring 3x less manually annotated data.

摘要中文:

扩散磁共振成像(dMRI)纤维追踪能够无创重建白质通路,但其准确性从根本上受到轴突组织间接、低分辨率测量的限制。非人类灵长类动物的示踪剂注射研究为验证dMRI纤维追踪提供了金标准。然而,这需要耗时的人工标注组织学切片中的纤维束。我们提出了一种合成数据增强的框架,用于猕猴示踪剂组织学中纤维束的自动分割。我们的方法利用体外dMRI纤维追踪作为生成先验来合成2D图像块用于训练。这为我们提供了足够逼真的前景纹理,我们将其与块面照片中的背景组合,并通过域随机化进行多样化。2D U-Net在混合真实和合成图像块上进行训练。在留出大脑上的实验表明,与仅使用真实数据训练相比,在不同大脑和纤维束密度上具有更好的泛化能力。仅使用合成数据训练会导致性能较差,这凸显了对真实标注数据的需求。总体而言,我们的方法达到了与最先进方法相当的性能,同时减少了3倍的人工标注数据需求。

PhysRAG: Enhancing Physics-Awareness in Video Generation via Retrieval-Augmented Generation

2026-06-26T04:00:00cs.CV, diffusion2606.26916

中文标题:PhysRAG:通过检索增强生成提升视频生成的物理感知能力

作者:Kexu Cheng, Zicheng Liu, Mingju Gao, Chunhe Song, Hao Tang

摘要:

Developing physically aware video generation models remains a significant challenge due to the difficulty in capturing diverse physical phenomena, such as thermal dynamics, mechanics, and optics. In this work, we introduce PhysRAG, a novel pipeline that enhances physical awareness in video generation through Retrieval-Augmented Generation (RAG). To address the issue of limited high-quality data, we design a two-stage data filtering pipeline based on the WISA-80K dataset, resulting in a curated set of 7K high-quality videos for training. Furthermore, we construct a physical video database and develop a mechanism to inject physical knowledge into a video diffusion model using learnable queries. Our method achieves state-of-the-art performance in both visual quality and physical rule compliance, surpassing existing models in benchmarks such as PhyGenBench and VBench. We conduct extensive ablation studies to validate the effectiveness of our key components, including the data filtering pipeline, RAG mechanism, and method for physical information extraction. To facilitate future research, our code, data, and models are prepared for release at https://github.com/sediment1024/PhysRAG.

摘要中文:

开发具有物理感知能力的视频生成模型仍然是一项重大挑战,原因在于难以捕捉多样化的物理现象,如热力学、力学和光学等。在本工作中,我们提出了PhysRAG,这是一种通过检索增强生成(RAG)来增强视频生成物理感知能力的新型pipeline。为了解决高质量数据有限的问题,我们基于WISA-80K数据集设计了两阶段数据过滤pipeline,最终筛选出7K个高质量视频用于训练。此外,我们构建了物理视频数据库,并开发了一种利用可学习查询将物理知识注入视频扩散模型的机制。我们的方法在视觉质量和物理规则遵守方面均达到了最先进性能,在PhyGenBench和VBench等基准测试中超越了现有模型。我们进行了广泛的消融实验,以验证关键组件的有效性,包括数据过滤pipeline、RAG机制和物理信息提取方法。为便于未来研究,我们的代码、数据和模型已准备发布于https://github.com/sediment1024/PhysRAG。

Focusing on What Matters: Saliency-Harnessing Accurate Routing for Diffusion MoE

2026-06-26T04:00:00cs.CV, diffusion2606.26938

中文标题:聚焦核心:面向扩散MoE的显著性利用精准路由机制

作者:Haoyou Deng, Keyu Yan, Chaojie Mao, Xiang Wang, Yu Liu, Changxin Gao, Nong Sang

摘要:

Mixture-of-Experts (MoE) architectures have emerged as a powerful paradigm for scaling diffusion models in visual generation. Recent advancements have focused on adaptively allocating computational resources across diverse tokens to improve efficiency and performance. However, we identify a routing assignment problem in existing diffusion MoE frameworks: the router fails to accurately allocate more computational resources to salient tokens. Our analysis attributes this failure to the router's reliance on noise-corrupted latent features throughout the denoising process. Such stochastic noise obscures the critical structural and textural information, thereby preventing the router from effectively distinguishing salient tokens. To address this, we propose SharpMoE, a post-training framework with a saliency-harnessing accurate routing mechanism, which utilizes clean latent features as a noise-free guidance signal for routing. By bypassing the noise-distorted inputs, SharpMoE provides the router with clear saliency guidance, enabling the identification of salient tokens even in high-noise stages. Furthermore, we introduce a trajectory routing loss to constrain the compute allocation throughout the multi-step denoising trajectory, ensuring precise resource allocation along the generation rollout. Extensive experiments demonstrate that SharpMoE serves as a versatile, plug-and-play solution that further enhances the pretrained, converged MoE models, achieving state-of-the-art performance in visual generation.

摘要中文:

专家混合(MoE)架构已成为扩展扩散模型以实现视觉生成的强大范式。近期研究致力于在不同标记间自适应分配计算资源,以提高效率与性能。然而,我们发现现有扩散MoE框架中存在路由分配问题:路由模块未能准确为显著性标记分配更多计算资源。我们的分析将该问题归因于路由模块在去噪过程中依赖受噪声干扰的潜在特征。这种随机噪声掩盖了关键的结构与纹理信息,致使路由模块无法有效区分显著性标记。针对这一问题,我们提出SharpMoE,一个具有显著性利用精准路由机制的后训练框架,该框架利用干净的潜在特征作为无噪声引导信号进行路由。通过绕过噪声扭曲的输入,SharpMoE为路由模块提供清晰的显著性引导,使其即使在高噪声阶段也能识别显著性标记。此外,我们引入轨迹路由损失来约束多步去噪轨迹中的计算分配,确保在生成展开过程中实现精准的资源分配。大量实验表明,SharpMoE是一种通用且即插即用的解决方案,能够进一步提升预训练收敛的MoE模型,在视觉生成任务中达到最先进的性能。

PanoImager: Geometry-Guided Novel View Synthesis and Reconstruction from Sparse Panoramic Views

2026-06-26T04:00:00cs.CV, diffusion2606.27071

中文标题:PanoImager:基于几何引导的稀疏全景视图新视图合成与重建

作者:Zhisong Xu, Takeshi Oishi

摘要:

Panoramic sensing offers wide field-of-view coverage, yet 3D reconstruction from sparse panoramas remains challenging under rotation-dominant, weak-parallax motion. In such regimes, SfM/SLAM initialization is often ill-conditioned and unreliable. We present PanoImager, an SfM-free framework that combines feed-forward pose/depth priors, geometry-conditioned diffusion view completion, and depth-guided 3DGS optimization. Given only a few panoramic images, PanoImager decomposes them into local perspective views, synthesizes auxiliary observations to enrich sparse evidence, and stabilizes Gaussian optimization for improved cross-view consistency. Experiments on multiple benchmarks show improved stability under extreme sparsity, suggesting PanoImager as an offline/background component for map refinement when SfM/SLAM fails to initialize.

摘要中文:

全景感知提供了广阔的视场覆盖,然而在旋转主导、弱视差运动的条件下,从稀疏全景图进行3D重建仍然具有挑战性。在此类场景中,SfM/SLAM初始化往往条件恶劣且不可靠。我们提出了PanoImager,一个无SfM的框架,该框架结合了前馈姿态/深度先验、几何条件扩散视图补全以及深度引导的3DGS优化。给定少量全景图像,PanoImager将其分解为局部透视视图,合成辅助观测以丰富稀疏证据,并稳定高斯优化以提高跨视图一致性。在多个基准数据集上的实验表明,在极端稀疏条件下具有更好的稳定性,表明PanoImager可作为SfM/SLAM初始化失败时的地图精化离线/后台组件。

TMP: Tree-structured Mixed-policy Pruning for Large-scale Image Generation and Editing

2026-06-26T04:00:00cs.CV, diffusion2606.27089

中文标题:TMP:用于大规模图像生成与编辑的树结构混合策略剪枝框架

作者:Peizhen Zhang, Yang Li, Xunsong Li, Songtao Liu, Zewen Liu, Qiangqiang Hu, Guotong Guo, Jupeng Ding, Yifu Sun, coopersli, Jian Zhang, Zhao Zhong, Liefeng Bo

摘要:

Modern image generation model rapidly grows their sizes to meet high-fidelity image synthesis. However, they gradually become unaffordable for their enormous parameter consumption and computation budget that lead to massive resources requirement and gpu memory footprint. In this paper, we propose TMP, the first Tree-structured Mixed-policy Pruning framework that generalizes prevalent image tasks (T2I and TI2I) and architectures (Mixture-of-Experts (MoE) and Diffusion transformer (DiT)). It could be applied to the step-distilled models and contribute as the last stage. We perform experiments upon current open-sourced SOTA HunyuanImage-3.0 instruct and a popular efficient model Z-Image turbo. The proposed pruning framework manages to compress HunyuanImage 3.0 from 80B to 20B parameters at 75% reduction ratio, sacrificing limited generation quality. We also optimize to enable the inference of the pruned 20B version of HunyuanImage 3.0 on a single 24GB 4090 GPU by engineering skills. The inference script and model weight have been integrated into the existing HunyuanImage3.0 open-source github and huggingface repository. Besides, we prove the efficacy of TMP by compressing Z-Image turbo from 6B to 4B (33% reduction) with negligible degradation.

摘要中文:

现代图像生成模型为满足高保真图像合成的需求而迅速扩大规模。然而,由于参数消耗和计算预算巨大,导致资源需求和GPU显存占用不断增加,这些模型逐渐变得难以承受。本研究提出了TMP,这是首个树结构混合策略剪枝框架,可泛化到主流图像任务(文本到图像T2I和图像编辑TI2I)以及主流架构(混合专家MoE和扩散Transformer DiT)。该方法可应用于步长蒸馏模型,并作为最后阶段发挥作用。我们在当前开源的SOTA模型HunyuanImage-3.0指令模型和流行的效率模型Z-Image turbo上进行了实验。所提出的剪枝框架成功将HunyuanImage 3.0从800亿参数压缩到200亿参数,达到75%的压缩比,而生成质量损失有限。我们还通过工程优化,实现了在单张24GB显存的4090 GPU上运行剪枝后的200亿参数版本HunyuanImage 3.0的推理。推理脚本和模型权重已集成到现有的HunyuanImage3.0开源GitHub和HuggingFace仓库中。此外,我们通过将Z-Image turbo从60亿压缩到40亿参数(33%压缩比),验证了TMP的有效性,且性能下降可忽略不计。

LISA: Likelihood Score Alignment for Visual-condition Controllable Generation

2026-06-26T04:00:00cs.CV, diffusion2606.27192

中文标题:LISA:视觉条件可控生成的似然分数对齐

作者:Yanghao Wang, Hongxu Chen, Jiazhen Liu, Zhenqi He, Rui Liu, Zhen Wang, Long Chen

摘要:

The prevalent dual-branch paradigm, i.e., training a side network to encode visual conditions and fusing its intermediate-layer features to a frozen pretrained main network, has shown remarkable success in visual-condition controllable generation. Despite its widespread adoption, the role of the side branch and its training efficiency remain underexplored. In this paper, we first revisit this mainstream paradigm through the lens of score-based generative modeling: 1) The main network preserves visual perceptual quality by providing a prior unconditional score. 2) The side network steers conditional control by implicitly contributing a likelihood score. Guided by this perspective, we propose LIkelihood Score Alignment (LISA), an effective regularization method that explicitly aligns the intermediate feature of the side network with an approximated likelihood score. Specifically, we first hook features from a designated layer of the side network and project them into the score latent space by a lightweight decoder. Then, we construct an approximated likelihood score target and calculate the distance between the decoder&#x27;s output and this target as an additional regularization loss. Finally, we jointly optimize the side network and decoder with both standard diffusion loss and our regularization loss. Experiments across various image/video tasks, architectures, and diffusion/flow models demonstrated that LISA can not only consistently accelerate the training convergence and improve final synthetic results, but also encourage the side network&x27;s features to be more disentangled for conditional modeling with negligible additional training cost and zero extra inference cost.

摘要中文:

主流的双分支范式,即训练一个侧网络来编码视觉条件,并将其中间层特征融合到冻结的预训练主网络中,在视觉条件可控生成方面已取得显著成功。尽管该范式已被广泛采用,但侧网络的作用及其训练效率仍未得到充分探索。本文首先从基于分数的生成建模角度重新审视这一主流范式:1)主网络通过提供先验无条件的分数来保持视觉感知质量;2)侧网络通过隐式贡献似然分数来引导条件控制。基于这一视角,我们提出似然分数对齐(LISA),一种有效的正则化方法,能够显式地将侧网络的中间特征与近似的似然分数进行对齐。具体而言,我们首先从侧网络的指定层提取特征,并利用轻量级解码器将其投影到分数潜在空间。随后,我们构建一个近似的似然分数目标,并将解码器输出与该目标之间的距离作为额外的正则化损失。最后,我们结合标准扩散损失和正则化损失共同优化侧网络和解码器。在多种图像/视频任务、架构和扩散/流模型上的实验表明,LISA不仅能够持续加速训练收敛并提升最终合成效果,还能促使侧网络的特征对于条件建模更加解耦,且仅产生极少的额外训练成本,推理时无需额外开销。

RoPEMover: Depth-Aware Object Relocation via Positional Embeddings

2026-06-26T04:00:00cs.CV, diffusion2606.27332

中文标题:RoPEMover:基于位置嵌入的深度感知物体重定位

作者:Ipek Oztas, Duygu Ceylan, Aybars Bugra Aksoy, Aysegul Dundar

摘要:

Moving an object in a single image requires geometry-consistent spatial rearrangement, including handling occlusions, revealing previously unseen regions, and maintaining coherent shadows and reflections. Existing approaches are not well suited to this setting and often fail to preserve such scene-level consistency. We address this problem by introducing a geometry-aware object motion method that operates directly on the positional representations of diffusion transformers. Our key insight is that rotary positional embeddings (RoPE) define a structured spatial field that can be explicitly manipulated to induce controlled motion. We extend 2D RoPE into a depth-aware formulation that encodes 3D spatial structure, enabling consistent object displacement and scene-aware updates. Our model is trained using synthetic data combined with a small set of real images via parameter-efficient fine-tuning. Despite minimal real supervision, it preserves object identity under large spatial displacements, generates plausible content in newly revealed regions, and consistently updates scene-dependent effects such as shadows and illumination. Experimental results on standard object motion benchmarks demonstrate state-of-the-art performance across all evaluation metrics.

摘要中文:

在单张图像中移动物体需要进行几何一致的空间重排,包括处理遮挡、揭示先前不可见的区域,以及保持连贯的阴影和反射。现有方法并不适用于这一场景,且往往无法保持此类场景级一致性。我们通过引入一种几何感知的物体运动方法来解决这一问题,该方法直接在扩散变换器的位置表示上操作。我们的关键见解是,旋转位置嵌入(RoPE)定义了一个结构化的空间场,可以通过明确操作来诱导受控运动。我们将2D RoPE扩展为深度感知公式,以编码3D空间结构,从而实现一致的物体位移和场景感知更新。我们的模型使用合成数据结合少量真实图像通过参数高效微调进行训练。尽管真实监督很少,但它可以在大空间位移下保持物体身份,在新揭示的区域生成合理内容,并持续更新阴影和光照等场景依赖效果。在标准物体运动基准测试上的实验结果表明,我们在所有评估指标上都达到了最先进的性能。

RayPE: Ray-Space Positional Encoding for 3D-Aware Video Generation

2026-06-26T04:00:00cs.CV, diffusion2606.27345

中文标题:RayPE:用于3D感知视频生成的射线空间位置编码

作者:Minghao Yin, Jiahao Lu, Wenbo Hu, Wang Zhao, Shan Ying, Kai Han

摘要:

Modern video diffusion transformers position their tokens through RoPE on the (u,v,t) axes -- a description of the camera's sampling grid that says nothing about the 3D structure of the scene. We observe that the geometric relation between two camera rays is captured by the Plucker reciprocal product, which is bilinear in the two rays -- the same algebraic form as the dot product in Transformer attention. Building on this analogy, we propose RayPE, a positional-encoding extension that injects per-token 6D Plucker coordinates additively into the queries and keys of self-attention, with a query/key flip arrangement under which the symmetric identity configuration coincides exactly with the reciprocal product. The injection is additive, the resulting attention score decomposes into a content term, a geometry term, and two content and geometry cross-terms -- all of which our experiments find individually necessary. To make the encoding stable across video data with heterogeneous camera-translation scales (SfM, deep SLAM, metric), we further decouple ray direction from moment magnitude, gate the encoding by a learned function of the log-magnitude, and apply RMSNorm to align it with the QKNorm-normalized content branch. The full module adds less than 0.1% parameters to a pretrained video DiT, is zero-initialized to start from the pretrained weights, and improves camera controllability, cross-frame 3D consistency, and overall video quality on a four-dataset training mixture.

摘要中文:

现代视频扩散Transformer通过RoPE在(u,v,t)轴上对token进行定位——这是对相机采样网格的描述,但未涉及场景的3D结构。我们观察到,两条相机射线之间的几何关系可通过普鲁克互易积来刻画,该积在两条射线上是双线性的——与Transformer注意力中的点积具有相同的代数形式。基于这一类比,我们提出RayPE,这是一种位置编码扩展方法,将每个token的6D普鲁克坐标以加法方式注入自注意力的查询和键中,并采用特定的查询/键翻转排列,使得对称恒等配置恰好与互易积重合。注入是加性的,得到的注意力分数可分解为一个内容项、一个几何项以及两个内容-几何交叉项——实验表明这些项各自都是必要的。为了使编码在具有不同相机平移尺度(SfM、深度SLAM、度量尺度)的视频数据上保持稳定,我们进一步将射线方向与矩的幅度解耦,用关于对数幅度的学习函数对编码进行门控,并应用RMSNorm使其与QKNorm归一化的内容分支对齐。该完整模块为预训练的Video DiT增加不到0.1%的参数,采用零初始化以从预训练权重开始,在四数据集混合训练上提升了相机可控性、跨帧3D一致性和整体视频质量。

PhysiFormer: Learning to Simulate Mechanics in World Space

2026-06-26T04:00:00autoregressive, cs.CV, diffusion2606.27364

中文标题:PhysiFormer:学习在世界空间中模拟力学

作者:Yiming Chen, Yushi Lan, Andrea Vedaldi

摘要:

We present PhysiFormer, a diffusion transformer for physically-plausible 3D object motion. Unlike video world models that operate in view-dependent pixel space, PhysiFormer represents objects as 3D meshes expressed in world coordinates. Given the initial vertex positions and velocities, as well as object material type, rigid or elastic, the model samples future vertex trajectories. While related neural physics approaches build on ad-hoc latent spaces or explicitly enforce rigidity and causality, PhysiFormer shows that excellent results can be obtained without any such inductive biases, by casting vertex trajectory prediction as a single denoising diffusion process directly in world coordinates. The probabilistic formulation captures uncertainty in the learned dynamics, enabling diverse plausible futures from initial conditions, making this framework potentially useful for applications with unobserved uncertainty. The model features attention factorised over time, space, and objects for efficiency, enabling permutation-invariant multi-object reasoning without needing explicit object encoding. Trained on over 100k simulated trajectories, PhysiFormer generates rigid and elastic mechanics, and generalises to mixed-material settings, unseen real-world geometries, and larger object counts. It substantially outperforms autoregressive baselines in trajectory accuracy, rigidity preservation, and momentum-based physical consistency. Our results position coordinate-space diffusion as a promising step toward view-invariant, geometry-aware world modelling for robotics, graphics, and physical design. Visualisations, code, and models are available at https://yimingc9.github.io/physiformer.

摘要中文:

我们提出PhysiFormer,一个用于物理合理3D物体运动的扩散transformer。与在视角相关像素空间中操作的视频世界模型不同,PhysiFormer将物体表示为在世界坐标中表达的3D网格。给定初始顶点位置和速度,以及物体材质类型(刚性或弹性),模型对未来顶点轨迹进行采样。与相关神经物理方法构建临时潜在空间或显式强制刚性和因果性不同,PhysiFormer表明无需任何此类归纳偏置也能获得优异结果,方法是将顶点轨迹预测作为单一去噪扩散过程直接在世界坐标中执行。概率公式捕捉了所学动力学的不确定性,能够从初始条件生成多样化的合理未来,使得该框架可能对具有未观测不确定性的应用有用。该模型采用时间、空间和物体分解的注意力机制以提高效率,无需显式物体编码即可实现置换不变的多物体推理。PhysiFormer在超过10万条模拟轨迹上训练,生成刚性和弹性力学行为,并泛化到混合材质设置、未见过的真实世界几何形状和更多物体数量。它在轨迹精度、刚度保持和基于动量的物理一致性方面显著优于自回归基准。我们的结果将坐标空间扩散定位为机器人学、图形学和物理设计中实现视角不变、几何感知世界建模的有前景的一步。可视化、代码和模型可在https://yimingc9.github.io/physiformer获取。

DanceDuo: Bridging Human Movement and AI Choreography

2026-06-26T04:00:00cs.CV, cs.HC, diffusion2606.26507

中文标题:DanceDuo:弥合人类动作与AI编舞

作者:Gia-Cat Bui-Le, Tuong-Vy Truong-Thuy, Hai-Dang Nguyen, Trung-Nghia Le

摘要:

In recent years, advancements in deep learning and generative models have revolutionized music-driven dance generation. This paper introduces a novel platform, namely DanceDuo, leveraging diffusion models to generate AI-choreographed dance sequences synchronized with a variety of music genres, to encourage dancing practice. The system allows users to interact with AI by selecting music tracks, humanoid models, and importing personal dance videos for comparison, fostering a rich and engaging user experience. DanceDuo not only offers dance generation but also integrates human pose estimation models to provide users with insightful comparisons of their own performances with AI-generated sequences. We conducted a comprehensive user study, revealing that users found the interface intuitive, with particular praise for the dance comparison feature. Our DanceDuo contributes significantly to the integration of AI in dance choreography, offering novel avenues for both recreational and professional applications.

摘要中文:

近年来,深度学习和生成模型的进展革新了音乐驱动舞蹈生成领域。本论文提出了一个名为DanceDuo的新型平台,利用扩散模型生成与多种音乐风格同步的AI编舞舞蹈序列,以鼓励舞蹈练习。该系统允许用户通过选择音乐曲目、人形模型以及导入个人舞蹈视频进行对比来与AI互动,从而培养丰富且引人入胜的用户体验。DanceDuo不仅提供舞蹈生成功能,还整合了人体姿态估计模型,使用户能够对自己的表演与AI生成的序列进行深入比较。我们进行了全面的用户研究,结果显示用户认为界面直观易用,特别是对舞蹈对比功能给予了高度评价。我们的DanceDuo为AI在舞蹈编舞中的整合做出了重大贡献,为休闲和专业应用开辟了新的途径。

Ordinal Neural Collapse as a Representation Prior for Visual Navigation

2026-06-26T04:00:00cs.CV, cs.RO, diffusion2606.26839

中文标题:序数神经坍缩:视觉导航的表示先验方法

作者:E-In Son, Jung-Taak Kim, Seung-Woo Seo

摘要:

Learning robust navigation policies directly from visual observations remains a fundamental challenge in vision-based robotic navigation. In end-to-end imitation learning approaches, the visual encoder and action decoder are jointly optimized using a single action loss, which provides only an indirect supervisory signal to the encoder. This indirect supervision frequently results in the encoder learning ambiguous, action-agnostic representations. The problem is further complicated by substantial variations in scene structure and appearance across diverse environments, as well as the prevalence of visual distractors inherent to real-world navigation settings. Such action-agnostic features cause the navigation policy to produce inconsistent actions at ambiguous decision points, leading to navigation failure. To overcome these limitations, we propose ORION (Ordinal Neural Collapse for Visual Navigation), a method that explicitly organizes the encoder's representation space according to the ordinal structure of navigation actions. In the context of goal-directed navigation, ego-centric control categories from Far Left to Far Right exhibit a natural ordinal relationship in which neighboring classes share similar visual contexts, while semantically opposing classes differ substantially in appearance. We encourage class representations to be arranged sequentially along a single discriminative axis, while suppressing off-axis variance within each class. The pretrained encoder is then integrated into a diffusion-based navigation framework, and the full pipeline is fine-tuned end-to-end. Extensive experiments in both simulation and real-world settings show that ORION consistently outperforms end-to-end and neural collapse baselines in navigation success rate and goal progress, with notable gains in visually challenging scenarios such as complex multi-way intersections.

摘要中文:

直接从视觉观察学习鲁棒的导航策略仍然是基于视觉的机器人导航领域的一项基本挑战。在端到端模仿学习方法中,视觉编码器和动作解码器使用单一动作损失进行联合优化,但该损失仅向编码器提供间接监督信号。这种间接监督往往导致编码器学习到模糊的、与动作无关的表示。此外,不同环境中场景结构和外观的显著变化,以及现实世界导航场景中固有的视觉干扰物的普遍存在,使问题更加复杂。这种与动作无关的特征会导致导航策略在模糊决策点产生不一致的动作,从而导致导航失败。为克服这些局限性,我们提出了ORION(用于视觉导航的序数神经坍缩)方法,该方法明确根据导航动作的序数结构组织编码器的表示空间。在目标导向导航的背景下,从极左到极右的自我中心控制类别呈现出一种自然的序数关系,即相邻类别共享相似的视觉上下文,而语义上相反的类别在外观上差异显著。我们鼓励类别表示沿着单一判别轴顺序排列,同时抑制每个类别内的轴外方差。然后,将预训练编码器集成到基于扩散的导航框架中,并对整个管道进行端到端微调。模拟环境和真实环境中的广泛实验表明,ORION在导航成功率和目标进展方面始终优于端到端和神经坍缩基线方法,在复杂多路交叉口等视觉挑战性场景中表现尤为突出。

Proposal-Conditioned Latent Diffusion for Closed-Loop Traffic Scenario Generation

2026-06-26T04:00:00cs.CV, cs.RO, diffusion2606.27123

中文标题:基于提议的潜扩散模型用于闭环交通场景生成

作者:Shubham Vaijanath Phoolari, Aleyna Kara, Christoph Lauer, Steven Peters

摘要:

Closed-loop traffic simulation remains challenging because it must generate interactive multi-agent behaviors that are scene-consistent and controllable throughout rollout. Prior diffusion-based approaches achieve strong realism, but their computational cost can hinder deployment in time-constrained replanning loops for autonomous vehicle planning and simulation. We present a diffusion-based scenario generation framework conditioned on instance-centric scene context and multimodal proposal priors, with optional test-time guidance for shaping safety-critical behaviors. A compact action-latent representation and proposal-based initialization improve sampling efficiency and reduce per-step runtime without retraining. Experiments on the Waymo Open Motion Dataset demonstrate a favorable balance among realism, safety, and controllability across diverse interactive scenarios, while showing that test-time guidance enables systematic trade-offs among competing objectives.

摘要中文:

闭环交通仿真仍具有挑战性,因为它必须生成交互式的多智能体行为,这些行为在展开过程中需保持场景一致性且可控。此前基于扩散的方法实现了较高的真实性,但其计算成本可能阻碍在自动驾驶规划和仿真的时间受限重规划循环中的部署。我们提出了一种基于扩散的场景生成框架,该框架以以实例为中心的场景上下文和多模态提议先验为条件,并可选地使用测试时引导来塑造安全关键行为。紧凑的动作潜空间表示和基于提议的初始化提高了采样效率并降低了每步运行时间,且无需重新训练。在Waymo开放运动数据集上的实验表明,该方法在多样化的交互场景中在真实性、安全性和可控性之间取得了良好的平衡,同时表明测试时引导能够实现竞争目标之间的系统性权衡。

Towards Consistent and Efficient Dataset Distillation via Diffusion-Driven Selection

2026-06-26T04:00:00cs.CV, diffusion2412.09959

中文标题:基于扩散驱动的选择实现一致且高效的数据集蒸馏

作者:Xinhao Zhong, Shuoyang Sun, Zhaoyang Xu, Xulin Gu, Bin Chen, Min Zhang, Yaowei Wang

摘要:

Dataset distillation provides an effective approach to reduce memory and computational costs by optimizing a compact dataset that achieves performance comparable to the full original. However, for large-scale datasets and complex deep networks (e.g., ImageNet-1K with ResNet-101), the vast optimization space hinders distillation effectiveness, limiting practical applications. Recent methods leverage pre-trained diffusion models to directly generate informative images, thereby bypassing pixel-level optimization and achieving promising results. Nonetheless, these approaches often suffer from distribution shifts between the pre-trained diffusion prior and target datasets, as well as the need for multiple distillation steps under varying settings. To overcome these challenges, we propose a novel framework that is orthogonal to existing diffusion-based distillation techniques by utilizing the diffusion prior for patch selection rather than generation. Our method predicts noise from the diffusion model conditioned on input images and optional text prompts (with or without label information), and computes the associated loss for each image-patch pair. Based on the loss differences, we identify distinctive regions within the original images. Furthermore, we apply intra-class clustering and ranking on the selected patches to enforce diversity constraints. This streamlined pipeline enables a one-step distillation process. Extensive experiments demonstrate that our approach consistently outperforms state-of-the-art methods across various metrics and settings.

摘要中文:

数据集蒸馏通过优化紧凑数据集来有效降低内存和计算成本,同时实现与完整原始数据集相当的性能。然而,对于大规模数据集和复杂深度网络(如ImageNet-1K和ResNet-101),庞大的优化空间阻碍了蒸馏效果,限制了实际应用。最近的方法利用预训练的扩散模型直接生成信息丰富的图像,从而绕过像素级优化并取得了可喜的成果。然而,这些方法通常存在预训练扩散先验与目标数据集之间的分布偏移问题,且需要在不同设置下进行多步蒸馏。为克服这些挑战,我们提出了一种新颖的框架,该框架利用扩散先验进行patch选择而非生成,与现有的基于扩散的蒸馏技术正交。我们的方法基于输入图像和可选文本提示(有标签或无标签)对扩散模型进行条件化,预测噪声,并计算每个图像-patch对的相应损失。基于损失差异,我们识别出原始图像中的独特区域。此外,我们对选定的patch进行类内聚类和排序,以施加多样性约束。这种简化的一步式流程管道使得蒸馏过程能够一步完成。大量实验表明,我们的方法在各种指标和设置下始终优于最先进的方法。

TinySR: Pruning Diffusion for Real-World Image Super-Resolution

2026-06-26T04:00:00cs.CV, diffusion, image_compression2508.17434

中文标题:TinySR:面向真实世界图像超分辨率的扩散模型剪枝

作者:Linwei Dong, Qingnan Fan, Yuhang Yu, Qi Zhang, Jinwei Chen, Yawei Luo, Changqing Zou

摘要:

Real-world image super-resolution (Real-ISR) focuses on recovering high-quality images from low-resolution inputs that suffer from complex degradations like noise, blur, and compression. Recently, diffusion models (DMs) have shown great potential in this area by leveraging strong generative priors to restore fine details. However, their iterative denoising process incurs high computational overhead, posing challenges for real-time applications. Although one-step distillation methods, such as OSEDiff and TSD-SR, offer faster inference, they remain fundamentally constrained by their large, over-parameterized model architectures. In this work, we present TinySR, a compact yet effective diffusion model specifically designed for Real-ISR that achieves real-time performance while maintaining perceptual quality. We introduce a Dynamic Inter-block Activation and an Expansion-Corrosion Strategy to facilitate more effective decision-making in depth pruning. We achieve VAE compression through channel pruning, attention removal and lightweight SepConv. We eliminate time- and prompt-related modules and perform pre-caching techniques to further speed up the model. TinySR significantly reduces computational cost and model size, achieving up to 5.68x speedup and 83% parameter reduction compared to its teacher TSD-SR, while still providing high quality results.

摘要中文:

真实世界图像超分辨率(Real-ISR)专注于从遭受噪声、模糊和压缩等复杂退化的低分辨率输入中恢复高质量图像。近年来,扩散模型(DM)通过利用强大的生成先验来恢复细节,展现出巨大的潜力。然而,其迭代去噪过程会产生较高的计算开销,给实时应用带来挑战。尽管一步蒸馏方法(如OSEDiff和TSD-SR)提供了更快的推理速度,但其庞大的过度参数化模型架构仍然受到根本性限制。本工作提出了TinySR,这是一个专为真实世界图像超分辨率设计的紧凑而有效的扩散模型,在保持感知质量的同时实现了实时性能。我们引入了动态块间激活和扩张-腐蚀策略,以促进深度剪枝中更有效的决策。我们通过通道剪枝、注意力移除和轻量级可分离卷积实现VAE压缩。我们消除了时间步和提示相关模块,并采用预缓存技术进一步加速模型。TinySR显著降低了计算成本和模型大小,与教师模型TSD-SR相比,实现了最高5.68倍的加速和83%的参数量减少,同时仍能提供高质量结果。

Learning Language-Driven Sequence-Level Modal-Invariant Representations for Video-Based Visible-Infrared Person Re-Identification

2026-06-26T04:00:00cs.CV, diffusion2601.12062

中文标题:面向基于视频的可见光-红外行人重识别的语言驱动序列级模态不变表示学习

作者:Xiaomei Yang, Antai Liu, Xizhan Gao, Fa Zhu, Sijie Niu, Giancarlo Fortino

摘要:

The core of video-based visible-infrared person re-identification (VVI-ReID) lies in learning sequence-level modal-invariant representations across different modalities. Recent research tends to use modality-shared language prompts generated by CLIP to guide the learning of modal-invariant representations. Despite achieving optimal performance, such methods still face limitations in efficient spatial-temporal modeling, sufficient cross-modal interaction, and explicit modality-level loss guidance. To address these issues, we propose the language-driven sequence-level modal-invariant representation learning (LSMRL) method, which includes spatial-temporal feature learning (STFL) module, semantic diffusion (SD) module and cross-modal interaction (CMI) module. To enable parameter- and computation-efficient spatial-temporal modeling, the STFL module is built upon CLIP with minimal modifications. To achieve sufficient cross-modal interaction and enhance the learning of modal-invariant features, the SD module is proposed to diffuse modality-shared language prompts into visible and infrared features to establish preliminary modal consistency. The CMI module is further developed to leverage bidirectional cross-modal self-attention to eliminate residual modality gaps and refine modal-invariant representations. To explicitly enhance the learning of modal-invariant representations, two modality-level losses are introduced to improve the features' discriminative ability and their generalization to unseen categories. Extensive experiments on large-scale VVI-ReID datasets demonstrate the superiority of LSMRL over AOTA methods.

摘要中文:

基于视频的可见光-红外行人重识别(VVI-ReID)的核心在于学习跨不同模态的序列级模态不变表示。当前研究倾向于使用CLIP生成的模态共享语言提示来引导模态不变表示学习。尽管取得了最佳性能,但这些方法在高效的时空建模、充分的跨模态交互以及显式的模态级损失引导方面仍存在局限性。为解决这些问题,我们提出了语言驱动序列级模态不变表示学习(LSMRL)方法,该方法包含时空特征学习(STFL)模块、语义扩散(SD)模块和跨模态交互(CMI)模块。为实现参数和计算高效的时空建模,STFL模块基于CLIP进行最小化修改构建。为实现充分的跨模态交互并增强模态不变特征的学习,SD模块被提出将模态共享语言提示扩散到可见光和红外特征中,以建立初步的模态一致性。CMI模块进一步利用双向跨模态自注意力来消除残余的模态差距并细化模态不变表示。为显式增强模态不变表示的学习,引入了两种模态级损失来提升特征的判别能力及其对未知类别的泛化性能。在大规模VVI-ReID数据集上的大量实验表明,LSMRL优于现有方法。

UniGeo: Unifying Geometric Guidance for Camera-Controllable Image Editing via Video Models

2026-06-26T04:00:00cs.CV, diffusion2604.17565

中文标题:UniGeo:通过视频模型实现相机可控图像编辑的统一几何引导

作者:Hong Jiang, Wensong Song, Zongxing Yang, Ruijie Quan, Yi Yang

摘要:

Camera-controllable image editing aims to synthesize novel views of a given scene under varying camera poses while strictly preserving cross-view geometric consistency. However, existing methods typically rely on fragmented geometric guidance, such as only injecting point clouds at the representation level despite models containing multiple levels, and are mainly based on image diffusion models that operate on discrete view mappings. These two limitations jointly lead to geometric drift and structural degradation under continuous camera motion. We observe that while leveraging video models provides continuous viewpoint priors for camera-controllable image editing, they still struggle to form stable geometric understanding if geometric guidance remains fragmented. To systematically address this, we inject unified geometric guidance across three levels that jointly determine the generative output: representation, architecture, and loss function. To this end, we propose UniGeo, a novel camera-controllable editing framework. Specifically, at the representation level, UniGeo incorporates a frame-decoupled geometric reference injection mechanism to provide robust cross-view geometry context. At the architecture level, it introduces geometric anchor attention to align multi-view features. At the loss function level, it proposes a trajectory-endpoint geometric supervision strategy to explicitly reinforce the structural fidelity of target views. Comprehensive experiments across multiple public benchmarks, encompassing both extensive and limited camera motion settings, demonstrate that UniGeo significantly outperforms existing methods in both visual quality and geometric consistency.

摘要中文:

相机可控图像编辑旨在合成给定场景在不同相机姿态下的新视角,同时严格保持视角间的几何一致性。然而,现有方法通常依赖碎片化的几何引导,例如仅在表示层面注入点cloud(尽管模型包含多个层面),且主要基于在离散视角映射上操作的图像扩散模型。这两个局限性共同导致连续相机运动下的几何漂移和结构退化。我们观察到,虽然利用视频模型为相机可控图像编辑提供了连续的视角先验,但若几何引导仍然是碎片化的,它们仍然难以形成稳定的几何理解。为系统性地解决这一问题,我们在三个层面注入统一的几何引导,共同决定生成结果:表示层面、架构层面和损失函数层面。为此,我们提出了UniGeo,一种新型的相机可控编辑框架。具体而言,在表示层面,UniGeo引入了一种帧解耦的几何参考注入机制,以提供稳健的跨视角几何上下文。在架构层面,它引入了几何锚点注意力来对齐多视角特征。在损失函数层面,它提出了轨迹端点几何监督策略,以明确增强目标视图的结构保真度。在多个公开基准数据集上的综合实验,涵盖大范围和有限相机运动设置,表明UniGeo在视觉质量和几何一致性方面均显著优于现有方法。

image_compression
Image Compression
2 篇论文

今日 Image Compression 相关论文共2篇,整体呈现两个方向:一是面向卫星网络的任务型图像语义传输,二是面向实际场景的轻量化图像超分辨率。可以看出该领域正从传统压缩向语义传输和端侧部署演进,关注在资源受限环境下的图像质量与效率平衡。

重点论文推荐:

  • SpaceRipple (2606.26559) — 针对任务导向的LEO卫星网络提出轻量级语义传输方案,在带宽受限的卫星通信场景下实现高效图像交付,对遥感数据传输有重要参考价值。
  • TinySR (2508.17434) — 通过剪枝扩散模型实现真实世界图像超分辨率,在保持画质的同时大幅降低计算开销,为端侧部署提供可行路径。

SpaceRipple: Lightweight Semantic Delivery for Mission-Oriented LEO Earth Observation Satellite Networks

2026-06-26T04:00:00cs.AI, cs.CV, cs.GR, image_compression2606.26559

中文标题:SpaceRipple: 面向任务的LEO对地观测卫星网络轻量级语义分发框架

作者:Ziyi Yang, Hao Yuan, Yunxiang Yi, Wenbo Wang, Xing Zhang

摘要:

Earth observation satellite networks generate massive volumes of high-resolution imagery, whereas inter-satellite and downlink resources remain limited. In many time-sensitive missions, ground users require mission-relevant semantic information rather than a full raw-image downlink. This paper proposes SpaceRipple, a lightweight framework for mission-oriented semantic delivery and on-board processing in Earth observation satellite networks. A sensing satellite performs adaptive compression and metadata generation to reduce inter-satellite traffic, while an edge computing satellite restores the received representation and extracts task-relevant semantic information. Unlike fidelity-driven image transmission, SpaceRipple coordinates compression, forwarding, restoration, and semantic inference within a collaborative pipeline, enabling semantic-oriented delivery instead of pixel-level image delivery. A compression-aware MoE enhancement module is further introduced to improve robustness under degraded visual inputs. Experimental results show that SpaceRipple achieves favorable reconstruction quality, improved semantic detection performance, and substantial bandwidth savings, demonstrating its potential for efficient and reliable Earth observation under constrained satellite-network resources.

摘要中文:

对地观测卫星网络生成大量高分辨率图像,而卫星间链路和下行资源仍然有限。在许多时间敏感型任务中,地面用户需要任务相关的语义信息,而非完整的原始图像下行传输。本文提出SpaceRipple,一个面向任务的语义分发和对地观测卫星网络星上处理的轻量级框架。感知卫星执行自适应压缩和元数据生成以减少卫星间流量,而边缘计算卫星恢复接收到的表示并提取任务相关的语义信息。与保真度驱动的图像传输不同,SpaceRipple在协作流水线中协调压缩、转发、恢复和语义推理,实现语义导向的分发而非像素级图像分发。此外,本文还引入了压缩感知MoE增强模块,以提高降质视觉输入下的鲁棒性。实验结果表明,SpaceRipple实现了良好的重建质量、改进的语义检测性能,并显著节省了带宽,展示了其在受限卫星网络资源下进行高效可靠对地观测的潜力。

TinySR: Pruning Diffusion for Real-World Image Super-Resolution

2026-06-26T04:00:00cs.CV, diffusion, image_compression2508.17434

中文标题:TinySR:面向真实世界图像超分辨率的扩散模型剪枝

作者:Linwei Dong, Qingnan Fan, Yuhang Yu, Qi Zhang, Jinwei Chen, Yawei Luo, Changqing Zou

摘要:

Real-world image super-resolution (Real-ISR) focuses on recovering high-quality images from low-resolution inputs that suffer from complex degradations like noise, blur, and compression. Recently, diffusion models (DMs) have shown great potential in this area by leveraging strong generative priors to restore fine details. However, their iterative denoising process incurs high computational overhead, posing challenges for real-time applications. Although one-step distillation methods, such as OSEDiff and TSD-SR, offer faster inference, they remain fundamentally constrained by their large, over-parameterized model architectures. In this work, we present TinySR, a compact yet effective diffusion model specifically designed for Real-ISR that achieves real-time performance while maintaining perceptual quality. We introduce a Dynamic Inter-block Activation and an Expansion-Corrosion Strategy to facilitate more effective decision-making in depth pruning. We achieve VAE compression through channel pruning, attention removal and lightweight SepConv. We eliminate time- and prompt-related modules and perform pre-caching techniques to further speed up the model. TinySR significantly reduces computational cost and model size, achieving up to 5.68x speedup and 83% parameter reduction compared to its teacher TSD-SR, while still providing high quality results.

摘要中文:

真实世界图像超分辨率(Real-ISR)专注于从遭受噪声、模糊和压缩等复杂退化的低分辨率输入中恢复高质量图像。近年来,扩散模型(DM)通过利用强大的生成先验来恢复细节,展现出巨大的潜力。然而,其迭代去噪过程会产生较高的计算开销,给实时应用带来挑战。尽管一步蒸馏方法(如OSEDiff和TSD-SR)提供了更快的推理速度,但其庞大的过度参数化模型架构仍然受到根本性限制。本工作提出了TinySR,这是一个专为真实世界图像超分辨率设计的紧凑而有效的扩散模型,在保持感知质量的同时实现了实时性能。我们引入了动态块间激活和扩张-腐蚀策略,以促进深度剪枝中更有效的决策。我们通过通道剪枝、注意力移除和轻量级可分离卷积实现VAE压缩。我们消除了时间步和提示相关模块,并采用预缓存技术进一步加速模型。TinySR显著降低了计算成本和模型大小,与教师模型TSD-SR相比,实现了最高5.68倍的加速和83%的参数量减少,同时仍能提供高质量结果。

visual_tokenizer_1d
1D Visual Tokenizer
0 篇论文

今日未找到该分类的匹配论文。

今日未找到该分类的匹配论文。

diffusion_visual_encoder
Diffusion Visual Encoder
0 篇论文

今日未找到该分类的匹配论文。

今日未找到该分类的匹配论文。