Efficient LLM Adversarial Training via Low-Rank Defense and Circuit-Guided Surrogates
Weiyi He, Yuping Lin, Jiliang Tang, Yue Xing
- Digest date
- 2026-08-03
- Submitted
- 2026-07-31
- arXiv ID
- 2607.28959
Summary
The paper speeds up latent adversarial training (LAT) for LLMs from two angles: representation fine-tuning (ReFT) on the defense side (while flagging a token-mismatch failure mode between ReFT application and attack), and circuit-extraction to build lightweight surrogate models for attack generation, avoiding full forward-backward passes through the base model.
Abstract
Adversarial training is one of the most effective defenses against adversarial attacks, yet the computational cost remains prohibitive at modern scales, especially for large language models (LLMs). While existing mitigation strategies, e.g., latent adversarial training (LAT), have been developed, they still incur a high computational cost. In this work, we comprehensively investigate computation-efficient strategies to speed up LAT from two complementary perspectives: (1) Defense-side optimization: We explore the representation fine-tuning (ReFT) within LAT, and reveal a potential issue if there is a mismatch on which tokens to apply ReFT and the attack. (2) Attack-side optimization: When computing adversarial attacks in each LAT iteration, we extract only the relevant circuits from the LLM to construct a lightweight surrogate model, avoiding the computation in the forward-backward passes through the full model during the attack generation. For both perspectives, we provide theoretical justifications and numerical evidence to illustrate the effectiveness of the proposed strategies. Ultimately, compared to standard LAT with full fine-tuning, our method on average reduces per-step adversarial-training FLOPs by 48.1% while requiring only 0.0118% trainable parameters.