Spec-AUF
面向 block drafter speculative decoding,提出 Acceptance-Until-Failure (AUF):
在不引入辅助目标、verifier rollout 或推理流程改动的前提下,将 cross-entropy
支持截断到 drafter 的第一次预测失败处,使训练监督更集中于推理中实际可提交的
accepted prefix。在 Qwen3-8B 固定 drafter backbone 与 serving 设置下,AUF
使 DFlash 平均 emitted length τ 提升 8.8%。arXiv preprint。
Spec-AUF studies block-drafter speculative decoding and proposes
Acceptance-Until-Failure (AUF), a detached change to the cross-entropy
support that truncates supervision at the drafter's first predicted
failure, without auxiliary objectives, verifier rollouts, or changes
to the inference pipeline. On Qwen3-8B under fixed drafter backbones
and serving settings, AUF improves DFlash average emitted length τ
by 8.8% across six benchmarks. arXiv preprint.
BridgeMoE
面向大规模 MoE 模型的专家卸载运行时。该工作分析不同 GPU 专家缓存比例下的
瓶颈转移:低缓存场景主要受传输限制,较高缓存比例下则暴露 kernel launch
和 micro-kernel 开销。BridgeMoE 结合 EAGLE3 草稿路径上的置信度门控树扩展、
共享卸载流水线和融合算子,在 Qwen3-30B-A3B 与 GPT-OSS-20B 上提升稳态解码吞吐;
固定缓存场景下相对自回归卸载基线平均达到 1.44x。
An offloaded MoE runtime for large MoE models. BridgeMoE studies the
regime shift across GPU expert cache ratios: low-cache execution is
transfer-bound, while higher cache ratios expose launch overhead and
micro-kernel cost. It combines confidence-gated tree expansion on an
EAGLE3 draft path with a shared offload pipeline and fused kernels,
improving steady-state decode throughput on Qwen3-30B-A3B and
GPT-OSS-20B. In the fixed-cache regime, it reaches 1.44x average
speedup over the autoregressive offloading baseline.
FlashRT
VLA 实时推理加速库 FlashRT core contributor,参与 Qwen3-VL、GROOT、Pi0.5
等路径的内核、构建和部署支持:
[PR #125],
[PR #120],
[PR #117],
[PR #116],
[PR #101],
[PR #100],
[PR #99],
[PR #98],
[PR #96],
[PR #93].
Core contributor to FlashRT, a real-time VLA inference acceleration
library. Contributed kernel, build, and deployment support for paths
including Qwen3-VL, GROOT, and Pi0.5:
[PR #125],
[PR #120],
[PR #117],
[PR #116],
[PR #101],
[PR #100],
[PR #99],
[PR #98],
[PR #96],
[PR #93].