NTU Part II, pages 51–55. Seven worked examples trace token-by-token TTT updates, memory in Titans, Nested Learning update rates and a change of instructions in context.
Topic: #self-improvement
A collection of 24 posts about self-improvement.
NTU Part II, pages 46–50. Seven worked examples connect SEAL self-edit rewards, two timescales of learning, learned optimizers and MAML. Follow the inner and outer updates and check each reported result against its evaluation unit.
NTU Part II, pages 41–45. Seven worked examples connect gradient descent, PostTrainBench, autoresearch and program evolution. Calculate what changes when an agent edits a training algorithm.
NTU Part II, pages 36–40. Follow an edit across generations, compare three parent-selection rules in Hyperagents, and calculate how Learning to Self-Evolve rewards an editor. Seven worked examples separate task performance from editing skill.
Pages 31–35 of NTU’s second lecture separate task success from skill retention. Six worked examples trace harness edits, capability-preserving evolution, and the editor in the Darwin Gödel Machine.
Pages 26–30 of NTU’s second lecture compare resetting a model for each input with carrying its updates forward. Rotation prediction, noisy speech recognition, and experience replay show how to measure both adaptation and forgetting.
Pages 21–25 connect ExpRAG, BetterTogether and E-SPL. Papers and Omar Khattab’s lecture explain experience retrieval, alternating optimization and reevaluation when goals change.
Pages 16–20 of the NTU lecture explore DGM’s search archive, declarative DSPy programs, harness and model parameters, experience retrieval and fine-tuning.
Pages 11–15 examine OPRO, GEPA, ALMA and the Darwin Gödel Machine. Slides and author commentary illuminate prompt search, memory design and code self-modification.
Pages 6–10 connect prompt, tool and workflow changes to SWE-agent, DSPy, AFlow and τ-bench. The discussion covers candidate selection, memory, cost and repeatability.
Pages 1–5 of the second lecture distinguish training loss from evaluation and compare how Absolute Zero, R-Zero, SPICE and R-Few construct self-training loops.
ReAct, Generative Agents, Reflexion and SWE-agent show how weights, harness design and runtime state affect behavior and what makes an evaluation fair.
Weak-to-strong generalization, performance gap recovery, OpenAI’s auxiliary confidence loss and Anthropic’s automated alignment researcher illustrate both gains and transfer limits.
PostTrainBench and FT-Dojo distinguish the training agent from the target model and examine time budgets, weighted scores and data contamination.
Proposer–solver self-play connects R-Zero’s plateau, execution checks in Absolute Zero and external information in SPICE and R-Few.
Policy gradients select lower-entropy paths. Speech-recognition word error rates, AI evaluation and generated learning tasks show how the signal is used.
Token-wise estimates of sequence entropy connect to the policy gradient. A probability tree and automatic differentiation show which term disappears when samples are held fixed.
Self-evaluation, majority votes and entropy provide different signals. Tent, SUTA and LLM entropy minimization illustrate when each signal is useful.
Input perplexity, early gains and collapse in unsupervised RLVR introduce test-time training. A worked example compares episodic and cumulative adaptation.
Which part of a system changes, and what feedback drives it? Evaluation and generalization evidence put claims about recursive self-improvement in context.
Loss functions, knowledge distillation and self-generated pseudo-labels illustrate when a model’s own outputs provide a new learning signal.
Worked examples contrast revising an answer with updating parameters through supervised learning on corrections or reinforcement learning on revision actions.
A counterexample shows how intermediate rewards can change the objective. Potential-based shaping, generated reward code and policy evaluation offer ways to check the effect.
Eureka, REvolve and RF-Agent introduce reward search. The lesson connects dopamine prediction errors, preference-based reward models, RLHF and RLAIF.