Notes
Research notes
Bilingual notes for paper readings and technical reflections around LLM research and engineering.
Browse
Paper Readings
The DINO Series: Architecture, Training, and Task Adaptation
A practical account of DINO, DINOv2, and DINOv3: self-distillation, local features, training steps, and losses, with a DINOv2 + SimCLR retrieval case study.
Self-Evolving Coding Agents: Taxonomy, Feedback Loops, and Trustworthy Evolution
A structured reading of how coding agents evolve their frameworks, memory, skills and tools, models, and collaboration structures, organized by evolution object, timing, and software evidence, with an extension on validation, versioning, and rollback for trustworthy evolution.
SkillRL: How a Failure Becomes a Reusable Skill
A reading of SkillRL through ALFWorld's Skipping State Changes failure, tracing candidate-skill extraction, SkillBank organization, skill-aware policy training, and the boundary between paper evidence, repository behavior, and production admission controls.
Kimi Linear and KDA: From Channel-Wise Forgetting to Hardware-Efficient Linear Attention
A technical reading of Kimi Linear through its recurrence, WY/UT parallel algorithm, 3:1 KDA/MLA hybrid, experiments, and public implementations.
Kimi K3: Architecture, Training, and Million-Token Agentic RL
A plain-language explanation of Kimi K3's KDA, Block AttnRes, LatentMoE, 1M context, and nine-policy RL, with clear boundaries between paper claims and external tests.
Data Agents: Levels, State of the Art, and Open Problems
A lifecycle-based reading of the L0–L5 autonomy framework, tracing data agents from responsive assistants and procedural executors to supervised orchestrators while examining gaps in responsibility, governance, and evaluation.
From Qwen to Qwen3.6: The Architecture and Training Evolution of Seven Generations
A seven-generation account of how Qwen evolved across attention, MoE, data, post-training, native multimodality, and agent environments—and why Qwen3-Next is an architectural bridge rather than a separate generation.
SAPO: Replacing Hard Ratio Clipping with a Continuous Soft Gate
An analysis of SAPO through its surrogate objective, gradient weights, asymmetric temperatures, and Qwen3-VL experiments, with attention to its token-level behavior and sequence-level approximation.
Entropy Collapse: Policy Entropy Consumption in LLM Reinforcement Learning
A note on entropy collapse in LLM reinforcement learning, covering policy entropy, the difference between SFT and RL, DAPO's Clip-Higher strategy, and covariance regularization.
From Qwen-VL to Qwen3-VL: Four Generations of Architecture and Training
A technical review of how four Qwen-VL generations evolved across vision-language alignment, dynamic resolution, spatiotemporal position encoding, video modeling, and deep visual fusion.
CapRL: Stimulating Vision-Language Captioning Capabilities with Reinforcement Learning
CapRL evaluates captions through the MCQ accuracy of a vision-free LLM, turning subjective caption-quality scoring into a verifiable reward for training image-captioning models.
Browse
Technical Reflections
A Comprehensive Guide to RSI (Self-Evolution, Self-Evolving Agents)
A taxonomy of Recursive Self-Improvement (RSI) across improvement targets, evolution structures, update roles, timing, and feedback.
Query Understanding and Rewriting: Training, Retrieval, and Agent Architecture Boundaries
Intent recognition, clarification, behavior-driven rewriting, distillation, and retrieval evaluation, with evidence-dependent multi-hop training data and a discussion of Agent state management versus AIGC recaptioning.
From Policy Lag to Advantage Collapse: Effective Update Signals in LLM Reinforcement Learning
An analysis of ineffective LLM RL updates through rollout policies, PPO clipping, homogeneous GRPO rewards, and interventions across data, sampling, supervision, and optimization.
Building a Cold-Start Evaluation Dataset for AI Agents Without Production Data
A practical method for building auditable pre-launch agent eval datasets from capability maps, human-authored gold seeds, controlled mutations, state oracles, resettable environments, and release gates.
KL Divergence: From Mathematical Derivations to LLM Reinforcement Learning
Understand the mathematical structure of KL divergence and what it controls in PPO, DPO, GRPO, and On-Policy Distillation.
Agentic RL Trajectory Data Contracts: From Harness State to Trainable Samples
A practical account of how Agentic RL preserves auditable token-level training data from model calls, context reconstruction, rollout grouping, and credit assignment.
Local Credit Assignment and the Boundary of Global Correctness in OPD: Starting from a Distilled-RL Intuition
Starting from a teacher-defined dense reward intuition, this note derives OPD's reverse-KL signal, examines local agreement after a corrupted prefix, and separates the roles of KAT, BRTS, TrOPD, local support matching, and verifiers.
Recursive Self-Improvement: From Artifacts and Harnesses to Model–Scaffold Co-evolution
A two-axis review of 2026 recursive self-improvement work, with an auditable standard for admitting self-generated updates.
What RL Actually Changes: Reasoning Patterns, Distillation, and Controllable Post-Training
A technical reflection on SFT, RLVR, distillation, and second-stage RL through the existence, selection, transfer, and control of reasoning patterns.
How to Evaluate AI Agents Scientifically: Metrics, Environments, Judges, and the Regression Loop
A practical guide to AI agent evaluation covering outcome and process metrics, environment and task design, LLM-as-a-Judge, failure attribution, and dual regression.
From Clio to an Executable Skill: Engineering LLM-Assisted Embedding Clustering
An engineering reconstruction of Clio's semantic clustering subsystem, evaluated on 20 Newsgroups to expose the gains, costs, reproducibility limits, and missing privacy controls of LLM-assisted hierarchy building.
The Benchmark Behind the Benchmark: Judge Noise and Trustworthy Agent Comparisons
An analysis of agent noise, judge variance, evidence contracts, continuous scoring, ensembles, and model comparison through a 106-task multi-judge experiment, with a clear distinction between consistency and correctness.
Treating Agents as Algorithms: Engineering Skills, CLIs, and Workflows
An engineering reflection on constraining execution variability and making multi-step agent tasks recoverable through responsibility boundaries, progressive disclosure, Gate validation, persistent state, and Workflow orchestration.
Long Trajectories, Learned Values, and Adaptive Verification: The Changing Constraints of Agentic RL
A synthesis of GLM-5.2, Qwen, GenAC, OPID, and two related studies on how long-horizon agents reshape trajectory sampling, credit assignment, reward verification, and dense supervision.
Keeping an SFT Gradient in DPO: From Relative Preference to a Chosen-Likelihood Anchor
A code-centered analysis of how chosen-response SFT changes DPO gradients, data and memory constraints, and the experiments needed to retain or reject the hypothesis.
Agent Experience Learning: From Textual Reflection and Programmatic Skills to Policy Internalization
A technical synthesis of how LLM agents turn task trajectories into retrievable experience, executable skills, and parametric behavior, with an emphasis on library maintenance, evaluation, and engineering boundaries.
Dissecting GRPO Loss in Code: Components, Negative Values, and Objective Ascent
A line-by-line analysis of importance ratios, clipping, KL penalties, and aggregation in a minimal GRPO implementation, explaining negative loss values, objective ascent, and near-zero experiments.
From 1D-RoPE to Qwen's MRoPE: Frequency Allocation in Rotary Position Embeddings
Starting from a six-dimensional vector, this note derives the relative-position property of 1D-RoPE and compares blockwise MRoPE in Qwen2.5-VL with Interleaved-MRoPE in Qwen3-VL.
PPO, DPO, and GRPO: Objectives and Training Loops for LLM Alignment
A comparison of PPO, DPO, and GRPO through their objectives, advantage estimators, training loops, engineering tradeoffs, and practical boundaries.
OPD: Capability Integration Interface in Post-training
A technical reflection on how OPD becomes a capability integration interface in post-training through Qwen3, GLM-5, MiMo-V2, and DeepSeek-V4.