每日 arXiv 论文简报

Key-value (KV) caching is essential for efficient autoregressive inference in transformer based dialog systems, yet existing strategies treat all cached entries uniformly or apply coarse eviction heuristics that fail to adapt as dialog topics evolve. We propose Fractional Decay KV-Cache (FD-KVC), a novel algorithm that maintains a dual-channel scoring mechanism for each cached KV pair: a cumulative attention channel that tracks aggregate importance (akin to H2O), and a recency-weighted relevance channel governed by temporal decay and reinforcement-inspired updates. The combination enables FD-KVC to both preserve historically important tokens and rapidly adapt when dialog topics shift. An adaptive learning rate driven by an ownership loss function ensures convergence without oscillation. FD-KVC operates entirely on CPU with negligible overhead. Across five diverse multi-turn dialog scenarios with 600 dialogs each, FD-KVC outperforms H2O, the state-of-the-art heavy-hitter baseline, by +6.7% on composite late-turn alignment, with improvements of +127% on topic-shift, +87% on gradual evolution, and +30% on mixed-topic dialogs. FD-KVC adapts to new topics 3.6X faster than H2O and achieves the highest topic diversity (80.6%) across all methods. Ablation studies confirm the contribution of each component.

键值（KV）缓存对于基于Transformer的对话系统中的高效自回归推理至关重要，但现有策略对所有缓存条目采用统一处理或使用粗粒度淘汰策略，无法适应对话主题的演变。我们提出分数衰减KV缓存（FD-KVC），这是一种新颖的算法，为每个缓存的KV对维护双通道评分机制：累积注意力通道用于追踪整体重要性（类似于H2O），以及由时间衰减和强化学习启发更新控制的新近度加权相关性通道。该组合使FD-KVC既能保留历史上重要的标记，又能在对话主题转换时快速适应。由所有权损失函数驱动的自适应学习率确保收敛且无振荡。FD-KVC完全在CPU上运行，开销可忽略不计。在五个不同的多轮对话场景（每个场景600个对话）中，FD-KVC在复合后期轮次对齐上优于当前最先进的热门基线H2O达+6.7%，主题转换提升+127%，渐进演化提升+87%，混合主题对话提升+30%。FD-KVC适应新主题的速度比H2O快3.6倍，并在所有方法中实现最高的主题多样性（80.6%）。消融实验证实了各组件的贡献。

Speculative decoding accelerates autoregressive generation by verifying a draft block with a target model in parallel. Under standard greedy verification, decoding stops at the first draft token that differs from the target argmax, discarding the remaining target-scored suffix. Although accepting such a mismatch changes the decoding trajectory, it can make a contiguous suffix reusable when its tokens remain target-greedy under the realized prefix. In this paper, we introduce \textbf{Approximate Speculative Decoding (ASD)}, a training-free verifier that replaces binary first-mismatch truncation with budgeted longest-prefix selection. ASD accepts selected mismatches subject to a local target-logit regret gate, a per-block exception cap, and a persistent request-level regret budget, then reuses the contiguous target-greedy suffix without additional approximate decisions or target-model forward passes. ASD requires neither a new draft model nor fine-tuning, and exactly reduces to standard greedy verification when the budget is zero. Experiments show that ASD improves fixed-workload throughput by $3.05\%$--$15.26\%$ over matched strict verification and averages a $7.78\%$ gain across seven Qwen3-14B + DSpark-14B tasks. On DeepSeek-V4-Flash (284B) with DSpark it also raises verifier-side acceptance by roughly $10\%$--$16\%$ on GSM8K and MATH-500 in an FP4-to-FP8 compatibility setting. The source code is publicly available at: https://github.com/Kissmetothemoon/ASD

投机解码通过使用目标模型并行验证草稿块来加速自回归生成。在标准贪婪验证下，解码在第一个与目标模型argmax不同的草稿token处停止，并丢弃剩余的目标评分后缀。尽管接受此类不匹配会改变解码轨迹，但当其token在实现的prefix下保持目标贪婪时，可以使连续后缀被复用。本文中，我们引入了一种无需训练的验证器——近似投机解码（Approximate Speplicative Decoding, ASD），它用预算化的最长前缀选择替代二元首不匹配截断。ASD在局部目标logit遗憾门控、每块异常上限和持续的请求级遗憾预算约束下接受选中的不匹配，然后在无需额外近似决策或目标模型前向传播的情况下复用连续的目标贪婪后缀。ASD既不需要新的草稿模型，也不需要微调，且当预算为零时恰好退化为标准贪婪验证。实验表明，ASD在固定工作量吞吐量上比严格验证提升了3.05%至15.26%，并在七个Qwen3-14B + DSpark-14B任务上平均获得7.78%的收益。在DeepSeek-V4-Flash（284B）配合DSpark使用时，在FP4到FP8兼容设置下，GSM8K和MATH-500上的验证器端接受率也提高了约10%至16%。源代码已公开于：https://github.com/Kissmetothemoon/ASD

Structured documents such as invoices, forms, reports, and scientific articles derive meaning from the interplay between spatial layout, textual content, and logical structure. Generative models operating at the pixel or token level often struggle to capture these dependencies effectively. We explore FRAGMENT, a generative framework that represents a document as a typed relational graph and factorizes its distribution as p(structure, content) = p(structure) * p(content | structure). The framework consists of two stages. The first stage, the Architect, is a causally masked Transformer conditioned on document category that autoregressively generates the graph topology and typed spatial relations. The second stage, the Builder, is a GATv2-based graph attention network that enriches the graph with normalized bounding boxes, text, and visual style attributes. Both stages define explicit likelihood models, yielding a tractable document-level likelihood that serves as an anomaly score for forgery detection. For controlled editing, a prompt-conditioned extension injects instruction embeddings into the Builder through cross-attention, enabling semantic and entity-aware modifications. We describe training on DocLayNet and fine-tuning on FUNSD and SROIE. Experiments on DocLayNet, FUNSD, and SROIE evaluate FRAGMENT alongside representative autoregressive, layout-only, and graph-based baselines, providing an empirical analysis of the characteristics and trade-offs of the proposed factorized graph generation framework.

发票、表格、报告和科学论文等结构化文档的含义来源于空间布局、文本内容与逻辑结构之间的相互作用。以像素或 token 级别运行的生成模型往往难以有效捕捉这些依赖关系。我们探索了 FRAGMENT——一种将文档表示为类型化关系图并将分布分解为 p(structure, content) = p(structure) * p(content | structure) 的生成框架。该框架包含两个阶段。第一阶段 Architect 是一个基于文档类别条件约束的因果掩码 Transformer，自回归地生成图拓扑结构和类型化空间关系。第二阶段 Builder 是一个基于 GATv2 的图注意力网络，用于为图添加归一化边界框、文本和视觉风格属性。两个阶段均定义了显式的似然模型，可产生可处理的文档级似然值，作为异常评分用于伪造检测。为了实现可控编辑，我们通过跨注意力机制将提示条件嵌入注入 Builder，提出了一个指令条件扩展版本，实现语义化和实体感知的修改。我们描述了在 DocLayNet 上的训练以及在 FUNSD 和 SROIE 上的微调过程。在 DocLayNet、FUNSD 和 SROIE 上进行的实验对 FRAGMENT 进行了评估，并与具有代表性的自回归、仅布局和基于图的基线方法进行了对比，对所提出的因子化图生成框架的特性与权衡进行了实证分析。

Autoregressive perception models trained to localize visual entities under the open-vocabulary setting are mostly trained using Supervised fine-tuning (SFT) with maximum likelihood, yet it optimizes a proxy objective (per-token cross-entropy) that is fundamentally misaligned with perception metrics such as precision and recall. In this paper, we explore post-training reinforcement learning (RL), specifically GRPO, to directly align these models with their evaluation metrics. Building up on the recently introduced Falcon Perception, we design an RL framework that addresses perception-specific challenges: reward design for set-structured outputs and multi-head sampling control. We discover multiple benefits from RL for perception: first, RL unlocks state-of-the-art performance in very dense scenes (up to 500 objects per scene), a regime where most existing systems degrade sharply or collapse; furthermore it fixes common issues in autoregressive perception models like mask repetitions and removes almost entirely the need for NMS and coordinate deduplication, which improve both performance and efficiency and remove the need for hyperparameters tuning; overall, we notice improvements on all levels of difficulties in referring expression segmentation (on PBench and SACO-Gold), and we find an elegant way to preserve the knowledge of whether an object exists or not (as evaluated by MCC) without training on negative samples. We show that a simple reward that penalizes false negatives and positives is sufficient. We develop two hybrid self-annotation pipelines, respectively tailored for difficult referring expressions and very dense scenes, and show their benefits on RL-training. Model weights are released as a Falcon Perception revision~\footnote{https://huggingface.co/tiiuae/Falcon-Perception}. Datasets will be published.

在开放词汇设置下训练用于定位视觉实体的自回归感知模型通常采用最大似然监督微调（SFT）进行训练，然而其优化的是一个代理目标（每token交叉熵），该目标与精确率、召回率等感知指标存在根本性的不对齐。本文探索采用训练后强化学习（RL），具体为GRPO算法，直接将这些模型与评估指标对齐。基于近期引入的Falcon Perception，我们设计了一个解决感知特定挑战的RL框架：针对集合结构化输出的奖励设计以及多头采样控制。我们发现了RL在感知任务中的多项优势：首先，RL在非常密集的场景（每场景最多500个物体）中实现了最先进的性能，而现有大多数系统在此类场景下性能会急剧下降或崩溃；此外，它修复了自回归感知模型中的常见问题如掩码重复，并几乎完全消除了对NMS和坐标去重的需求，这既提升了性能又提高了效率，并省去了超参数调优的麻烦；总体而言，我们在指代表达分割任务的所有难度级别上均观察到改进（在PBench和SACO-Gold上），并且我们找到了一种优雅的方法来保留物体是否存在的知识（在MCC评估中），而无需对负样本进行训练。我们表明，一个简单的惩罚假阴性和假阳性的奖励就足够了。我们开发了两种混合自标注流程，分别针对困难的指代表达和非常密集的场景，并展示了它们在RL训练中的优势。模型权重已作为Falcon Perception修订版发布。数据集将予以发布。

Time series imputation is a crucial area for reliable time series analysis, yet it remains challenging due to the complex temporal dynamics and noise of real-world data. Existing approaches, however, exhibit two limitations: missing and observed values are embedded within the same representation space without explicit structural separation, and continuous diffusion-based methods are trained to predict added noise rather than the original signal. To address these, we propose the Masked Diffusion Time-series Imputation Model (MDTIM), which leverages the training paradigm of masked diffusion model for imputation tasks. The MASK token is structurally orthogonal to valid observations, and the model directly predicts the original values, naturally aligning both the representation and the learning objective with the imputation task. To bridge the gap between discrete masked diffusion and the continuous, ordinal nature of time series, we further introduce Stochastic Discretization, which maps continuous values to ordinal-aware tokens while preserving continuous dynamics. Our experiments on diverse benchmarks confirm that MDTIM achieves superior robustness and scalability, consistently outperforming state-of-the-art deterministic and generative baselines across various missing scenarios.

时间序列插补是可靠时间序列分析的关键领域，但由于现实世界数据的复杂时间动态和噪声，该任务仍具有挑战性。然而，现有方法存在两个局限性：缺失值和观测值在相同的表示空间中嵌入，没有明确的结构分离；且基于连续扩散的方法被训练为预测添加的噪声而非原始信号。针对这些问题，我们提出了掩码扩散时间序列插补模型（MDTIM），它利用掩码扩散模型的训练范式来完成插补任务。MASK token在结构上与有效观测正交，模型直接预测原始值，自然地使表示和学习目标与插补任务保持一致。为弥合离散掩码扩散与时间序列连续、有序特性之间的差距，我们进一步引入了随机离散化方法，在保留连续动态的同时将连续值映射到有序感知token。我们在多样化基准测试上的实验证实，MDTIM实现了卓越的鲁棒性和可扩展性，在各种缺失场景下始终优于最先进的确定性和生成式基线模型。

Action-conditioned world models allow robots to predict the future consequences of candidate actions without additional physical interaction, supporting policy evaluation, planning, and data augmentation. We present Mask2Real-WM, a two-stage action-conditioned world model for dexterous manipulation that decouples pixel prediction into a dynamics model and a rendering model. The dynamics model predicts future segmentation masks from past masks and 23-DoF action sequences. The rendering model maps the predicted masks to photorealistic RGB using a ControlNet-augmented Stable Video Diffusion backbone. The smaller sim-to-real gap in segmentation space enables the dynamics model to benefit from large-scale pretraining on over 50 h of synthetic simulation data, followed by fine-tuning on fewer than 2.5 h of real demonstrations. Experiments on a dexterous pick-and-place benchmark show that mask conditioning and simulation pretraining are both required for per-DoF action controllability across all 23 degrees of freedom. In contrast, monolithic baselines capture broad hand and end-effector trajectories but do not reliably reflect fine-grained, per-joint action effects.

动作条件世界模型使机器人能够预测候选动作的未来后果，无需额外的物理交互，从而支持策略评估、规划和数据增强。我们提出 Mask2Real-WM，这是一种用于灵巧操作的两阶段动作条件世界模型，将像素预测解耦为动力学模型和渲染模型。动力学模型根据历史分割掩码和23-DoF动作序列预测未来的分割掩码。渲染模型利用ControlNet增强的Stable Video Diffusion主干网络，将预测的掩码映射到逼真的RGB图像。分割空间的较小sim-to-real差距使动力学模型能够从超过50小时的合成模拟数据的大规模预训练中受益，随后在少于2.5小时的真实演示数据上进行微调。在灵巧拾取放置基准测试上的实验表明，掩码条件和模拟预训练对于实现所有23个自由度的每自由度动作可控性都是必需的。相比之下，整体式基线方法能够捕捉到广泛的手部和末端执行器轨迹，但无法可靠地反映细粒度的每关节动作效果。

Synthetic tabular data are valued for preserving not just column-wise marginals but inter-column dependency. Yet the most commonly reported certification score, a linear (logistic-regression) classifier two-sample test (C2ST), is largely blind to it: a fully-factorized baseline that destroys all inter-column dependency still appears nearly real, a known weakness we confirm on four benchmarks, while pairwise Trend penalizes the same baseline only mildly. We therefore apply a stronger, gradient-boosted C2ST and decompose its score into marginal, dependency, and numerical-categorical cross terms, each read against a zero-dependency reference and a real-data oracle. Applied to flow-matching (TabbyFlow) and diffusion (TabDiff) generators, it exposes a persistent dependency gap of the same order in both. Destroying dependency outright with every marginal intact collapses minority-class F1 by 0.38-0.61, though the generators' much smaller residual gaps do not track the shortfalls that remain. The gap is neither a structural limitation of mean-field objectives nor an artifact of sampling discretization, and a 16x capacity increase does not close it. Shrinking capacity eightfold, however, doubles it, so the measurement does respond to capacity; what remains points to the absence of direct dependency supervision.

合成表格数据的价值不仅在于保持列边缘分布，更在于保留列间依赖性。然而，最常报告的认证分数，即线性（逻辑回归）分类器双样本检验（C2ST），在很大程度上无法检测到这一点：一个完全因式分解的基线模型破坏了所有列间依赖性，却仍然看起来几乎逼真，我们在四个基准上确认了这一已知弱点，而成对趋势（pairwise Trend）对同一基线的惩罚却非常轻微。因此，我们应用了一种更强的梯度提升C2ST，并将其分数分解为边缘项、依赖项和数值-类别交叉项，每一项都相对于零依赖参考和真实数据 oracle 进行评估。应用于流匹配（TabbyFlow）和扩散（TabDiff）生成器，揭示了两者之间存在相同量级的持续依赖差距。彻底破坏依赖性同时保持所有边缘分布完整，会使少数类F1分数下降0.38-0.61，尽管生成器残留的差距并未追踪到剩余的不足。该差距既不是平均场目标的结构限制，也不是采样离散化的artifact，16倍的容量增加也无法缩小它。然而，将容量减少八倍会使差距翻倍，因此该测量确实对容量有响应；剩余的差距指向缺乏直接依赖监督。

Existing 3D relighting methods operate through either explicit material decomposition, diffusion-based view-space generation, or a combination of both, requiring full recomputation for each new lighting condition. We observe that recent latent scene representations, which encode multi-view images into a set of compact tokens with no fixed physical semantics, open up a novel design space for relighting. We present LumiTokens, a framework that formulates 3D relighting as a direct transformation on latent scene tokens, without explicit 3D representations, rendering equations, or physics-based decomposition. Our model introduces a Scene Token Editor that processes scene tokens jointly with light-ray tokens through self-attention, producing updated tokens that can be decoded into multi-view-consistent relit images. To support diverse lighting types through a unified interface, all lighting signals, including environment maps, point lights, and area lights, are parameterized as Plucker ray tokens, enabling native 3D user interaction with a representation that carries no explicit spatial structure. Crucially, this design supports progressive relighting: because the editor's output remains in the same latent space as its input, a user can incrementally build up illumination one light source at a time, with each edit composing in token space. Experiments demonstrate that LumiTokens achieves comparable or superior relighting quality to other methods and supports progressive, composable lighting edits. Project page: https://neu-vi.github.io/LumiTokens/

现有3D重光照方法通常通过显式材质分解、基于扩散的视图空间生成或两者结合实现，每种新光照条件都需完整重算。研究表明，近期将多视角图像编码为紧凑tokens集合的潜在场景表示，为重光照开辟了全新的设计空间。本文提出LumiTokens框架，将3D重光照表述为潜在场景tokens的直接变换，无需显式3D表示、渲染方程或物理分解。该模型引入场景Token编辑器，通过自注意力机制联合处理场景tokens和光线tokens，生成可解码为多视角一致重光照图像的更新tokens。为通过统一接口支持多样化光照类型，所有光照信号（包括环境贴图、点光源和面光源）均参数化为Plucker光线tokens，实现与不带显式空间结构的表示进行原生3D用户交互。关键在于，该设计支持渐进式重光照：由于编辑器输出与输入处于同一潜在空间，用户可逐个光源逐步构建照明，每次编辑在token空间中组合。实验表明，LumiTokens达到与其他方法相当或更优的重光照质量，并支持渐进式、可组合的光照编辑。项目主页：https://neu-vi.github.io/LumiTokens/

Audio-driven lip synchronization manipulates the mouth region of a talking-face video to match the driving audio while preserving head pose, identity, and background. Although the task is inherently local editing, prevailing approaches reconstruct the entire lower face with heavy GAN- or diffusion-based decoders, incurring substantial latency and, more critically, hallucinating intra-oral details such as teeth and lip wrinkles instead of preserving authentic textures. We contend that the bottleneck in identity preservation is not the scarcity of reference frames, but the lack of a mechanism that faithfully transfers the genuine textures they already contain. We therefore present EfficientSync, a real-time deformation-based framework that retains reference textures rather than resynthesizing them. First, the Dynamic Texture Mixer reformulates multi-reference fusion as channel-wise selection, evaluating each spatially aligned reference in a global context and aggregating them by channel-wise weighted summation, preserving textural integrity at low cost. Second, Spatio-Temporal Shifted Adaptive Masking decomposes the source frame into lip-generation conditions and an independent background prior, suppressing lower-face leakage while blending the synthesized mouth seamlessly into the background. Third, STAR Sampling, a zero-overhead pre-processing step, retrieves the sharpest and most topologically diverse reference frames. Experiments on HDTF and VFHQ show state-of-the-art visual quality and identity preservation at 166 FPS on a single GPU. Video demos: https://alunaticat.github.io/EfficientSync/index.html.

音频驱动的唇同步技术可操作说话人视频的嘴部区域以匹配驱动音频，同时保留头部姿态、身份特征和背景。尽管该任务本质上属于局部编辑，但现有方法通常使用繁重的GAN或扩散解码器重建整个下人脸，导致高延迟，更关键的是会产生口腔内部细节（如牙齿和唇纹）的幻觉，而非保留真实纹理。我们认为，身份特征保留的瓶颈并非参考帧的缺乏，而是缺乏一种能够忠实转移已有参考帧中真实纹理的机制。因此，我们提出了EfficientSync，一种保留参考纹理而非重新合成纹理的实时变形框架。首先，动态纹理混合器将多参考融合重新表述为通道级选择，在全局上下文中评估每个空间对齐的参考帧，并通过通道级加权求和进行聚合，以低成本保留纹理完整性。其次，时空移位自适应掩码将源帧分解为唇部生成条件和独立的背景先验，抑制下人脸信息泄露，同时将合成的嘴部无缝融合到背景中。第三，STAR采样作为一种零开销的预处理步骤，检索出最清晰且拓扑多样性最高的参考帧。在HDTF和VFHQ数据集上的实验表明，该方法在单GPU上以166 FPS的速度达到了最先进的视觉质量和身份保持效果。视频演示：https://alunaticat.github.io/EfficientSync/index.html

Spatial representation learning for autonomous driving aims to map raw visual signals into structured 3D scene representations, where object-centric bounding boxes and rendering-oriented 3D primitives (\eg, 3D Gaussians) serve as two distinct yet highly complementary levels for scene understanding. Existing methods typically treat dynamic reconstruction and instance-level perception as separate tasks, despite their shared goal of estimating the underlying 3D world state. As a result, dynamic reconstruction is under-constrained while 3D detection lacks geometric grounding. To address this gap, we propose USR-Drive, a unified conditional generative framework that, given only posed multi-view driving videos, jointly recovers dense dynamic geometry and instance-level object layouts within a shared scene representation. Specifically, USR-Drive represents dense Gaussian primitives and sparse 3D bounding boxes as two aligned latent token streams and jointly denoises them with a unified multi-modal diffusion Transformer. Unlike prior paradigms that use boxes as external conditions or predict them with detached modules, USR-Drive treats them as mutually constrained state variables with a Unified Positional Encoding (UPE) that aligns heterogeneous tokens within a shared metric spatiotemporal coordinate. Via such unified representation and generative framework, the two modalities reinforce each other: geometry supplies dense metric evidence for box prediction, while boxes provide instance-level structural priors that help preserve spatial consistency and reduce ambiguity in sequential 3D geometric representation. Our approach successfully delivers state-of-the-art results for both dynamic reconstruction and 3D detection on the nuScenes and VKitti datasets.

自动驾驶空间表征学习旨在将原始视觉信号映射为结构化的3D场景表征，其中以物体为中心的边界框和面向渲染的3D基元（如3D高斯）是两种截然不同但高度互补的场景理解层次。现有的方法通常将动态重建和实例级感知视为独立任务，尽管它们的共同目标是估计底层3D世界状态。这导致动态重建缺乏约束，而3D检测缺乏几何锚定。为解决这一差距，我们提出了USR-Drive，一个统一的条件生成框架，仅给定带姿态的多视角驾驶视频，即可在共享场景表征中联合恢复密集动态几何和实例级物体布局。具体而言，USR-Drive将密集高斯基元和稀疏3D边界框表示为两个对齐的潜在token流，并使用统一的多模态扩散Transformer对它们进行联合去噪。与先前使用边界框作为外部条件或使用解耦模块预测边界框的范式不同，USR-Drive将它们视为相互约束的状态变量，并采用统一位置编码（UPE）在共享度量时空坐标中对异构token进行对齐。通过这种统一表征和生成框架，两种模态相互增强：几何为边界框预测提供密集度量证据，而边界框则提供实例级结构先验，有助于保持空间一致性并减少顺序3D几何表征中的歧义。我们的方法在nuScenes和VKitti数据集上的动态重建和3D检测任务均取得了最先进的成果。

Music-driven character animation enables and enhances transformative applications in entertainment and interactive education. However, synthesizing realistic drumming motion from audio remains challenging due to the inherent tension between high-acceleration dynamics and the need for extreme spatial-temporal precision. Existing approaches, often reliant on motion matching or MIDI input, struggle with generalizing to diverse real-world audio. Moreover, the field lacks standardized evaluation metrics capable of distinguishing precise drumming from noisy motion. In this paper, we introduce a generative diffusion framework featuring a dual-objective loss function that decouples skeletal integrity from drumstick precision, thus enabling centimeter-level stick precision without sacrificing natural body dynamics. Additionally, leveraging our own dataset and data augmentation strategy, the model generalizes to non-curated, in-the-wild audio. To rigorously evaluate performance, we propose two novel metrics: an impact-to-target distance to quantify spatial precision and an audio-motion correlation score to assess temporal alignment. Our quantitative analysis and user studies demonstrate that our system generates high-quality motion that is often indistinguishable from ground-truth performances.

音乐驱动角色动画能够推动娱乐和互动教育领域的变革性应用。然而，由于高加速度动态与极端时空精度之间的内在矛盾，从音频合成逼真的鼓手动作仍然具有挑战性。现有方法通常依赖于动作匹配或MIDI输入，难以推广到多样的真实世界音频。此外，该领域缺乏能够区分精确鼓手动作与噪声动作的标准化评估指标。本论文提出了一种生成式扩散框架，采用双目标损失函数将骨骼完整性与鼓棒精度解耦，从而在不影响自然身体动态的前提下实现厘米级鼓棒精度。此外，利用我们自己的数据集和数据增强策略，该模型能够泛化到非精选的真实世界音频。为严格评估性能，我们提出了两个新指标：击打目标距离用于量化空间精度，音频动作相关性评分用于评估时间对齐。我们的定量分析和用户研究表明，我们系统生成的动作质量很高，通常难以与真实表演区分开来。

With the rapid development of image synthesis techniques, AI-generated images have become increasingly realistic, which heightens the potential risk associated with their misuse and creates a growing need for reliable detection. However, the growing diversity of generative models makes it increasingly difficult for detectors to generalize to images produced by unseen generators. Most existing methods rely on artifacts associated with specific generators, which limits their generalization to images produced by unseen models. To address this problem, we investigate structural characteristics arising from the image generation process itself. Image generation fundamentally involves constructing spatially rich content from more compact representations, while preserving the semantic identity of structures across different spatial locations. We formalize these properties through dimension-increasing shift-equivariant transformations and show that such transformations induce a self-similar structure in the Fourier spectrum. Across successive generation stages, this structure can propagate recursively and form a hierarchical fractal self-similar pattern. Consequently, different spectral sub-regions exhibit consistent structural correspondences inherited from the generation process, providing a generator-agnostic cue for detection. Based on this observation, we propose Fractal-CNN, which captures spectral self-similarity rather than generator-specific spectral values. Extensive experiments across diverse GAN- and diffusion-based generators demonstrate that Fractal-CNN achieves strong cross-generator generalization, with an average detection accuracy of 93.93% across 16 test generators.

随着图像合成技术的快速发展，AI生成图像日益逼真，这加剧了滥用带来的潜在风险，并对可靠检测提出了日益增长的需求。然而，生成模型的多样性不断增加，使得检测器越来越难以泛化到未见生成器产生的图像。大多数现有方法依赖于与特定生成器相关的伪影，这限制了它们对未知模型生成图像的泛化能力。为解决这一问题，我们研究了图像生成过程本身产生的结构特征。图像生成从根本上涉及从更紧凑的表示构建空间丰富的内容，同时保持不同空间位置结构的语义一致性。我们通过维度增加的移位等变变换来形式化这些性质，并证明这种变换在傅里叶频谱中诱导出自相似结构。在连续的生成阶段中，这种结构可以递归传播并形成层次化的分形自相似模式。因此，不同的频谱子区域呈现出由生成过程继承的一致结构对应，这为检测提供了一种与生成器无关的线索。基于这一观察，我们提出了Fractal-CNN，它捕捉的是频谱自相似性而非生成器特定的频谱值。在多种GAN和扩散模型生成器上的广泛实验表明，Fractal-CNN实现了强大的跨生成器泛化能力，在16个测试生成器上的平均检测准确率达到93.93%。

Diffusion models excel at generating high-quality images but can memorize and reproduce harmful concepts when prompted. Although fine-tuning methods have been proposed to unlearn a target concept, they struggle to fully erase it while maintaining generation quality on other concepts, leaving models vulnerable to jailbreak attacks. Existing jailbreak methods demonstrate this vulnerability but offer limited insight into how unlearned models retain harmful concepts, limiting progress on effective defenses. In this work, we show that the erased concept persists as a coherent, interpretable linear subspace of the token embedding space, and that both an attack and a defense follow directly from this structure. We introduce SubAttack, a novel jailbreaking attack that reads out this subspace by learning an orthogonal set of attack token embeddings, each being a linear combination of human-interpretable textual elements, revealing that unlearned models still retain the target concept through related textual components. Furthermore, our attack is also more powerful and transferable across text prompts, initial noises, and unlearned models than prior attacks. Conversely, projecting out the same subspace yields SubDefense, a lightweight plug-and-play defense mechanism that suppresses the residual concept in unlearned models. SubDefense provides stronger robustness than existing defenses while better preserving safe generation quality. Extensive experiments across multiple unlearning methods, concepts, and attack types demonstrate that our approach advances both understanding and mitigation of vulnerabilities in diffusion unlearning.

扩散模型在生成高质量图像方面表现优异，但在受到提示时可能会记忆并复现有害概念。尽管已有研究提出通过微调方法来反学习目标概念，但这些方法难以在完全擦除目标概念的同时保持对其他概念的生成质量，导致模型仍易受到越狱攻击。现有的越狱攻击方法虽然展示了这一漏洞，但对反学习模型如何保留有害概念的机制缺乏深入理解，限制了有效防御的发展。本工作表明，被擦除的概念以连贯的、可解释的线性子空间形式存在于标记嵌入空间中，攻击和防御均可直接从该结构中构建。我们提出了SubAttack，这是一种新型越狱攻击，通过学习一组正交的攻击标记嵌入来读取该子空间，每个攻击标记嵌入都是人类可解释的文本元素的线性组合，揭示了反学习模型仍通过相关文本组件保留目标概念。此外，我们的攻击在跨文本提示、初始噪声和反学习模型方面比现有攻击更强大且更具可转移性。相反，通过对同一子空间进行投影消除，我们提出了SubDefense，这是一种轻量级的即插即用防御机制，用于抑制反学习模型中的残留概念。SubDefense在保持安全生成质量的同时，提供了比现有防御更强的鲁棒性。在多种反学习方法、概念和攻击类型上进行的广泛实验表明，我们的方法推进了对扩散反学习模型漏洞的理解和缓解。

Video world models have attracted significant attention for their ability to produce high-fidelity future visual observations conditioned on past observations and navigation actions. However, achieving temporally and spatially consistent generation over long horizons remains an open challenge: existing approaches either compress past frames without explicitly accounting for 3D viewpoint geometry or retrieve only a handful of spatially relevant frames without increasing the total amount of retained history. In this paper, we propose WorldPack, a video world model that introduces spatially-aware compressed memory to address both limitations simultaneously. The key insight is that compression rates should not be uniform or temporally determined, but should instead be dynamically allocated based on 3D spatial relevance to the current viewpoint. WorldPack achieves this through two tightly coupled mechanisms: trajectory packing, which fits substantially more historical frames into a fixed-length context through hierarchical frame compression, and geometric selection, which leverages camera pose information and field-of-view overlap to assign lower compression to spatially important frames and higher compression to less relevant ones. Together, these mechanisms expand the effective context from 4 to 22 frames with moderate computational overhead: trajectory packing increases diffusion-model inference time by 16%, while FoV-based geometric selection introduces an additional cost. We evaluate WorldPack on LoopNav, a Minecraft benchmark for long-horizon spatial consistency, and conduct comprehensive experiments on the RECON, real-world navigation dataset, across multiple metrics. WorldPack outperforms strong baselines, including Oasis, Mineworld, DIAMOND, and NWM, with pronounced gains on spatial reasoning tasks that require recall of distant observations.

视频世界模型因能够基于历史观察和导航动作生成高保真未来视觉观测而受到广泛关注。然而，在长时序中实现时空一致性生成仍然是一个开放性挑战：现有方法要么对历史帧进行压缩而未明确考虑3D视角几何，要么仅检索少量空间相关帧而不增加保留的历史总量。本文中，我们提出WorldPack，一个引入空间感知压缩记忆的视频世界模型，以同时解决这两个局限性。核心观点是压缩率不应该是均匀的或由时间决定的，而应该根据当前视角的3D空间相关性动态分配。WorldPack通过两个紧密耦合的机制实现这一点：轨迹打包，通过分层帧压缩将更多历史帧装入固定长度的上下文；几何选择，利用相机位姿信息和视野重叠为空间重要帧分配较低压缩率，为不相关帧分配较高压缩率。这些机制将有效上下文从4帧扩展到22帧，计算开销适中：轨迹打包将扩散模型推理时间增加16%，而基于FoV的几何选择引入了额外成本。我们在LoopNav上评估WorldPack，这是Minecraft上用于长时序空间一致性的基准，并在RECON上对真实世界导航数据集进行了全面实验。WorldPack优于强大的基线模型，包括Oasis、Mineworld、DIAMOND和NWM，在需要回忆远距离观察的空间推理任务中表现尤为突出。

Modern vehicle platforms are equipped with a rich sensor suite, including LiDAR, calibrated multi-camera rigs, and accurate ego-motion, that in principle offers strong signal for re-rendering a driving scene from novel viewpoints. A growing line of recent work leverages video diffusion models for this task, using their generative priors to synthesize plausible novel views from sparse vehicle observations. In practice, however, existing methods exploit only a fragment of this signal, and their quality tends to degrade as the target trajectory departs from the recorded driving path. We argue that this is fundamentally a multi-sensor fusion problem: sparse LiDAR reprojections supply accurate but incomplete metric geometry, surround-view reference imagery supplies dense appearance but no metric depth, and camera poses tie the two together across views. We introduce StreetNVS, a video diffusion framework that jointly conditions on all three signals through a Reference-Enhanced Camera Attention module based on a relative ray-level positional encoding. We develop a two-stage curriculum training strategy that gradually exposes the model to increasingly sparse LiDAR. On the Waymo Open Dataset, StreetNVS substantially outperforms state-of-the-art baselines under sparse LiDAR conditioning, matches methods that rely on 10-100 times denser point clouds. We further show capabilities of synthesizing coherent videos along extreme out-of-trajectory paths such as elevation, lane-shift, pullback, and rotation. Our website: https://streetnvs.github.io

现代车辆平台配备了丰富的传感器套件，包括LiDAR、经过校准的多摄像头阵列以及精确的自运动信息，原则上这些为从新视角重新渲染驾驶场景提供了强信号。越来越多的近期工作利用视频扩散模型来完成这一任务，使用其生成先验从稀疏的车辆观测中合成合理的新视图。然而，在实践中，现有方法仅利用了该信号的一小部分，且其质量往往随着目标轨迹偏离记录行驶路径而下降。我们认为这本质上是一个多传感器融合问题：稀疏LiDAR重投影提供准确但不完整的度量几何信息，环绕视图参考图像提供密集外观但缺乏度量深度，而相机姿态则跨越视图将两者联系在一起。我们提出了StreetNVS，一种视频扩散框架，通过基于相对射线级位置编码的参考增强相机注意力模块对所有三个信号进行联合条件处理。我们开发了两阶段课程训练策略，逐步使模型适应越来越稀疏的LiDAR。在Waymo开放数据集上，StreetNVS在稀疏LiDAR条件下显著优于现有最先进基线方法，性能可媲美依赖10-100倍更密集点云的方法。我们进一步展示了沿极端非轨迹路径（如elevation、lane-shift、pullback和rotation）合成连贯视频的能力。

Hyperspectral pansharpening aims to reconstruct a high resolution hyperspectral (HRHS) image from a panchromatic (PAN) image and a low resolution hyperspectral (LRHS) image while preserving both spatial details and spectral fidelity. Recent diffusion based methods exploit pretrained image priors by generating a low dimensional representation and subsequently mapping it to the hyperspectral domain. However, the observed panchromatic and hyperspectral images are typically imposed only through external reconstruction objectives, limiting their direct interaction with the diffusion prior. To address this issue, we propose dual-modality image-prompted diffusion model (DIDM) for zero shot hyperspectral pansharpening. DIDM encodes the low resolution hyperspectral and panchromatic observations into spectral and spatial prompt tokens, respectively, and injects them into intermediate features of a frozen remote sensing diffusion model through cross attention, allowing complementary spectral and spatial information to directly guide diffusion feature evolution. In addition, we introduce a panchromatic guided weighted pixel aware total variation regularizer that combines low resolution hyperspectral degradation fidelity and panchromatic response fidelity with gradient adaptive structural regularization, thereby preserving structural discontinuities while suppressing spurious variations in homogeneous regions. Extensive experiments on Pavia, Chikusei, and Houston under reduced resolution protocols show that DIDM achieves the best performance across all evaluated metrics, while full resolution evaluation on FR1 yields the highest HQNR among the compared methods. These results demonstrate that internal dual modality prompting and panchromatic guided structural regularization provide an effective balance between spatial detail enhancement and spectral preservation.

高光谱全色锐化的目标是从全色（PAN）图像和低分辨率高光谱（LRHS）图像重建高分辨率高光谱（HRHS）图像，同时保留空间细节和光谱保真度。近期基于扩散的方法通过生成低维表示并将其映射到高光谱域来利用预训练图像先验。然而，观测到的全色图像和高光谱图像通常仅通过外部重建目标施加限制，限制了它们与扩散先验的直接交互。为解决这一问题，我们提出了用于零样本高光谱全色锐化的双模态图像提示扩散模型（DIDM）。DIDM将低分辨率高光谱和全色观测分别编码为光谱提示令牌和空间提示令牌，并通过交叉注意力将它们注入冻结遥感扩散模型的中间特征中，使互补的光谱和空间信息能够直接引导扩散特征演化。此外，我们引入了一种全色引导的加权像素感知全变分正则化器，该正则化器将低分辨率高光谱降质保真度和全色响应保真度与梯度自适应结构正则化相结合，从而在抑制均匀区域虚假变化的同时保留结构不连续性。在Pavia、Chikusei和Houston数据集上进行的降分辨率协议实验表明，DIDM在所有评估指标上均取得了最佳性能，而在FR1全分辨率评估中则获得了最高的HQNR。这些结果表明，内部双模态提示和全色引导结构正则化在空间细节增强和光谱保真之间提供了有效的平衡。

Large antenna arrays allow wireless systems to serve more users and achieve higher data rates, but they also make channel feedback expensive: the receiving device must repeatedly report a large complex-valued channel matrix to the base station. Most neural compressors treat this matrix like an image and replace it with a fixed-length code that only a matched neural decoder can interpret. The message therefore does not adapt to channel complexity, and changing the antenna count typically requires retraining. We ask whether a device can instead report only the few dominant propagation paths underlying each channel. We introduce the Gramian Chebyshev Neural Operator (GCNO), a physics-based, variable-rate compressor that identifies a sample-dependent set of path directions. GCNO uses receive-transmit channel structure to locate paths, a first-order Taylor correction to refine directions that fall between grid points, and least squares to recover their complex strengths. It is trained without path labels, and the base station reconstructs the channel analytically from the transmitted path tuples rather than through a learned decoder. Across three ray-traced environments, GCNO achieves better reconstruction accuracy at the same payload - or lower payload at the same accuracy - than neural feedback baselines, and transfers to unseen antenna counts without retraining.

大规模天线阵列使无线系统能够服务更多用户并实现更高的数据速率，但也导致信道反馈代价高昂：接收设备必须向基站重复报告大型复值信道矩阵。大多数神经压缩器将此矩阵视为图像，并用固定长度的代码替换，只有匹配的神经解码器才能解释。因此，消息无法适应信道复杂性，且改变天线数量通常需要重新训练。本文探讨设备是否可以仅报告每个信道背后的少数主导传播路径。提出了Gramian Chebyshev神经算子（GCNO），这是一种基于物理的变率压缩器，能够识别样本相关的路径方向集合。GCNO利用接收-发射信道结构定位路径，采用一阶泰勒校正细化落在网格点之间的方向，并使用最小二乘法恢复复数幅度。该方法在无路径标签的情况下进行训练，基站从传输的路径元组分析重建信道，而非通过学习解码器。在三个射线追踪环境中的研究表明，GCNO在相同有效载荷下实现了更好的重建精度，或在相同精度下实现了更低的有效载荷，且能够迁移到未见过的新天线数量而无需重新训练。

Instance-level artwork recognition requires matching a handheld visitor photograph to a specific work in a large museum collection. This is challenging because painting datasets typically provide clean catalog images for training, while test queries are captured under oblique viewpoints, gallery lighting, reflections, frames, and other scene-level variations. We present SynGallery, a synthetic gallery dataset for artwork retrieval that addresses this gap without collecting additional real photographs. Starting from catalog images of real paintings, we place each artwork into a procedurally generated 3D gallery scene and render it from multiple viewpoints under varied geometric and appearance conditions, while preserving the exact identity of the original work. The resulting dataset contains 24,490 rendered views of 4,898 paintings from the Met benchmark. We show that these synthetic views provide a stronger training signal than the corresponding studio photographs. At the same number of training data points, training only on SynGallery improves art painting recognition from 67.18 to 73.47 GAP$^-$. When added to the full Met training set, SynGallery improves the published benchmark protocol from 35.97 to 38.48 GAP. Ablation experiments show that the gain comes from scene-level view variation rather than photographic realism: reducing the five rendered viewpoints to a single frontal view removes most of the improvement, while simulating capture artifacts such as blur, sensor noise, and image compression consistently reduces performance.

实例级艺术品识别需要将手持访客照片与大型博物馆收藏中的特定作品进行匹配。这项任务具有挑战性，因为绘画数据集通常提供用于训练的干净目录图像，而测试查询是在倾斜视角、画廊照明、反光、画框及其他场景级变化条件下拍摄的。我们提出SynGallery，这是一个用于艺术品检索的合成画廊数据集，无需收集额外的真实照片。从真实绘画的目录图像出发，我们将每件作品置于程序化生成的3D画廊场景中，在不同的几何和外观条件下从多个视角进行渲染，同时保留原始作品的精确身份。结果数据集包含来自Met基准的4,898幅绘画的24,490个渲染视图。我们表明这些合成视图提供了比相应工作室照片更强的训练信号。在相同数量的训练数据点下，仅在SynGallery上训练可将艺术绘画识别从67.18提升至73.47 GAP⁻。当添加到完整Met训练集时，SynGallery将已发布的基准协议从35.97提升至38.48 GAP。消融实验表明，收益来自场景级视角变化而非照片级真实性：将五个渲染视角减少到单个正面视角会消除大部分改进，而模拟拍摄伪影（如模糊、传感器噪声和图像压缩）则会持续降低性能。