This, IMO is the biggest insight into where we're at and where we're going:
> Because both evaluation and self-modification are coding tasks, gains in coding ability can translate into gains in self-improvement ability.
There's a thing that I've noticed early into LLMs: once they unlock one capability, you can use that capability to compose stuff and improve on other, related or not, capabilities. For example "reflexion" goes into coding - hey, this didn't work, let me try ... Then "tools". Then "reflxion" + "tools". And so on.
You can get workflows that have individual parts that aren't so precise become better by composing them, and letting one component influence the other. Like e2e coding gets better by checking with "gof" tools (linters, compilers, etc). Then it gets even better by adding a coding review stage. Then it gets even better by adding a static analysis phase.
Now we're seeing this all converge on "self improving" by combining "improving" components. And so on. This is really cool.
I disagree that evaluation is always a coding task. Evaluation is scrutiny for the person who wants the thing. It’s subjective. So, unless you’re evaluating something purely objective, such as an algorithm, I don’t see how a self contained, self “improving “ agent accomplishes the subjectivity constraint - as by design you are leaving out the subject.
Sure. There will always be subjective tasks where the person who asks for something needs to give feedback. But even there we could come up with ways to make it easier / faster / better ux. (one example I saw my frontend colleagues do is use a fast model to create 9 versions of a component, in a grid. And they "at a glance" decide which one is "better", and use that going forwards).
OTOH, there's loads you can do for evaluation before a human even sees the artifact. Things like does the site load, does it behave the same, did anything major change on the happy path, etc etc. There's a recent-ish paper where instead of classic "LLM as a judge" they used LLMs to come up with rubrics, and other instances check original prompt + rubrics on a binary scale. Saw improvements in a lot of evaluations.
Then there's "evaluate by having an agent do it" for any documentation tracking. Say you have a project, you implement a feature, and document the changes. Then you can have an agent take that documentation and "try it out". Should give you much faster feedback loops.
In science there are ways to surface subjectivity (cannot be counted) into observable quantized phenomena. Take opinion polls for instance: "approval" of a political figure can mean many things and is subjective, but experts in the field make "approval" into a number through scientific methods. These methods are just an approximation and have many IFs, they're not perfect (and for presidential campaign analysis in particular they've been failing for reasons I won't clarify here), but they're useful nonetheless.
Another thing that get quantized is video preferences to maximize engagement.
>You can get workflows that have individual parts that aren't so precise become better by composing them, and letting one component influence the other. Like e2e coding gets better by checking with "gof" tools (linters, compilers, etc). Then it gets even better by adding a coding review stage. Then it gets even better by adding a static analysis phase.
This is the exact point I make whenever people say LLMs aren't deterministic and therefore not useful.
Yes, they are "stochastic". But you can use them to write deterministic tools that create machine readable output that the LLM can use. As you mention, you keep building more of these tools and tying them together and then you have a deterministic "network" of "lego blocks" that you can run repeatably.
Agree. It's code all the way down. The key is to give agents a substrate where they can code up new capabilities and then compose them meaningfully and safely.
Larger composition, though, starts to run into typical software design problems, like dependency graphs, shared state, how to upgrade, etc.
So what are software packages now a days other than precomputed subsets of capabilities. Like a mesh that data gets pushed through to produce what? What are the optimal subset of prebuilt programs to accomplish any task?
Maybe unlock means "recognize and solve a problem with an order of magnitude fewer tokens than the first time you did it". The same way humans might spend a lot of time thinking about a certain problem and various ways to solve it, but once they go through that process, and then recognize it again, they don't need to go to the same process and jump right to the solution.
I found that running an agent in ralph loop, showing it the agent text and saying "run this, if it fails - identify the reason, and modify the agent instructions to avoid this, acceptance criteria are this and that" worked surprisingly well. Not sure if it qualifies as a self-referential self improving, but it was something.
No matter how far we go, we end up with generation / discrimination architecture.
Its is the core of any and all learning/exellency; exposure to chaotic perturbations allow selection of solutions that are then generalized to further, ever more straining problems; producing increasingly applicable solutions.
This is the core of evolution, and is actually derivable from just a single rule.
I don't think generation/discrimination is fundamental. A more general framing is evolutionary epistemology (Donald T. Campbell, 1974, essay found in "The Philosophy of Karl Popper"), which holds that knowledge emerges through variation and selective retention. As Karl Popper put it, "We choose the theory which best holds its own in competition with other theories; the one which, by natural selection, proves itself the fittest to survive."
On this view, learning in general operates via selection under uncertainty. This is less visible in individual cognition, where we tend to over-attribute agency, but it is explicit in science: hypotheses are proposed, subjected to tests, and selectively retained, precisely because the future cannot be deduced from the present.
In that sense, generation/discrimination is a particular implementation of this broader principle (a way of instantiating variation and selection) not the primitive itself.
I agree, I meant to be explicit that the one rule was "gravity";
Variation (chaos) comes from the tidal push/pull of all cumulative processes - all processes are nearly periodic (2nd law) and get slower - guaranteeing oscillator harmonics at intervals.
These intervals are astronomically convulted, but still promise a Fourier distribution of frequency: tidal effects ensure synchronization eventually, as all periods resonate eventually.
As systems are increasingly exposed to pendulums of positive and negative coherence, they will generalize for variance, and eventually for increasingly (fourier) selective filters of increasingly resiliente traits, that will generalize.
The system would eventually be increasingly resilient and eventually an awareness would develop.
Awareness of past periodic cycles would improve fitness (with or without consciousness) and eventually the mechanistic processes would be in the systems nature.
This is why we have pointless traditions, folk lore, collective unconscious artifacts, cyclical cataclysmic religions, the Fermi Paradox, the great filters...
Variation and selection are woven, but understanding how it all stems from gravity by means of nearly perioidic oscillators (spinning planets, tidal pools, celestial bodies) due to the conservation of angular momentum, due to the 3body problem.....that is what took a genius to reconcile
Awareness would be any form of agency, goal seeking, or loss minimizing.
As Briggs–Rauscher reactions can eventually lead to Belousov–Zhabotinsky reactions, the system can maintain homeostasis with its environment (and continuing to oscillate) by varying reactants in a loss minimizing fashion.
This loss minimizing would be done during scarcity to limp towards an abundance phase.
This is the mechanism that hypothetical tidal pools batteries would had exhibited to continue between periods of sunlight/darkness/acidity that eventually gets stratified as a resilency trait.
I'm not sure what your argument is here, except stating an opinion that loss minimization is equivalent to agency. But even if that was accepted, which is a huge stretch, it doesn't stretch all the way to awareness.
It is, in context of its place in the cosmic scale.
Loss minimizing to a few problems will generalize into abstraction, and a few solutions will develop.
These systems with more generalizable resilency traits will encounter increasingly varied selective sieves.
Systems that survive this seive will exhibit increasingly sophisticated, generalizable solutions to prevent loss of needed dependent reactions/resources.
These solutions must exert influence to be effective; influencing the environment for its own benefit.
As systems influence their environment, delineation of "self" and "environment" becomes a fundamental barrier.
The system would prefer itself, or be outcompeted by a similar system that does.
This layer of semi-life like material would form between sunlight and the oscillating reaction, and eventually envelope it, minimizing surface tension by means of a spherical cell like structure.
Small stuff runs off of loss minimizing at a force level for its mechanistic affect; from covalent bonds to cellular ion transport, the path of lesser resistance is the fundamental forces.
As systems become more complex, the minimizing is less directly attributable to the fundamental forces and becomes more of a Byzantine dependency/feedback network.
This byzantine labyrinth of interactions is called biology.
The delineation of self, the ego.
At the highest levels, geopolitics. At the human level, mate suppression.
Lowest level, energy conservation.
A cool illusion, just another emergent property of our geometrical solution: higher dimensional aperiodic tilings of a 10^80 faceted complex polyhedra "walking" on another large aperioidic Penrose plane, that is getting smaller in a dimension we observe as "energy".
Basically a dice with a bajillion sides is getting rolled along an increasingly slim poker table, house winning eventually.
Time only goes one way, protons dont decay, energy is radiated unto the cosmic background hiss, until homogeneity is reached as CMB, and entrophy reaches 1.
I dont know where it comes from, but I know the shape it makes as it rolls by.
The most important part of engineering is problem-solving, which feedback loops don't necessarily do. The reason we are here as engineers is: 2.5 billion years ago, the earth made cyanobacteria, which flourished, then flooded the earth with toxic oxygen, killing almost all life on the planet. The initial feedback loop didn't solve a problem, it destroyed a use case. That's not a solution to a problem that an engineer would choose, even if those organisms that came after were pretty happy about it...
Systems emerge in times of abundance, and are whittled in times of scarcity.
The great oxygenation was a time of near catyclismsic scarcity for most complex organisms, as resources scale to food/energy requirements imply the most complex organisms were the most dependent on the environment, and were most impacted by changes.
Inversely, oxygenation was our most crucial abundancy pre cursor, as it provides a large substrate chemically for life to exhibit
I've been experimenting with similar concept myself. The linter loop is the only thing that can keep the agent sane in my opinion, and if anyone can generalize bun+tsc loop to other tasks, this would finally be a way to trust LLMs output.
I was annoyed at how Claude Code ignores my CLAUDE.md and skills, so I was looking for ways to expand type checking to them. So I wrote a wrapper on top of claude-agents-sdk that reads my CLAUDE.md and skills, and compiles them into rules - could be linter rules or custom checking scripts.
Then it hooks up to all tools and runs the checks. The self improving part comes if some rule doesn't work: I run the tool with the session id in review mode, it proposes the fixes and improves the rule checkers. (not the md files) So it's kinda like vibe coding rules, definitely lowers the bar for me to maintain them. Repo: https://github.com/chebykinn/agent-ruler
It's surprising that this works so well considering that AI-generated AGENTS.md files have been shown to be not very useful. I think the key difference here is that the real-world experience helps the agent reach regions of its latent space that wouldn't occur naturally through autoregression.
I wonder how much of the improvement is due to the agent actually learning new things vs. reaching parts of its latent space that enable it to recall things it already knows. Did the agent come up with novel RL reward design protocols based on trial and error? Or did the tokens in the environment cause it to "act smarter"?
Self-improving AI systems aim to reduce reliance on human engineering by learning to improve their own learning and problem-solving processes. Existing approaches to self-improvement rely on fixed, handcrafted meta-level mechanisms, fundamentally limiting how fast such systems can improve. The Darwin Gödel Machine (DGM) demonstrates open-ended self-improvement in coding by repeatedly generating and evaluating self-modified variants. Because both evaluation and self-modification are coding tasks, gains in coding ability can translate into gains in self-improvement ability. However, this alignment does not generally hold beyond coding domains. We introduce \textbf{hyperagents}, self-referential agents that integrate a task agent (which solves the target task) and a meta agent (which modifies itself and the task agent) into a single editable program. Crucially, the meta-level modification procedure is itself editable, enabling metacognitive self-modification, improving not only the task-solving behavior, but also the mechanism that generates future improvements. We instantiate this framework by extending DGM to create DGM-Hyperagents (DGM-H), eliminating the assumption of domain-specific alignment between task performance and self-modification skill to potentially support self-accelerating progress on any computable task. Across diverse domains, the DGM-H improves performance over time and outperforms baselines without self-improvement or open-ended exploration, as well as prior self-improving systems. Furthermore, the DGM-H improves the process by which it generates new agents (e.g., persistent memory, performance tracking), and these meta-level improvements transfer across domains and accumulate across runs. DGM-Hyperagents offer a glimpse of open-ended AI systems that do not merely search for better solutions, but continually improve their search for how to improve.
This 'self vs non-self' logic is very similar to how plants prevent self-pollination. They have a biological 'discrimination' system to recognize and reject their own genetic code.
I asked Pi to implement a skill. It was written in TS. Then I ask it to use the skill in two different sessions, none of them can get it working. One has to wrap it with JS and call it. The other has to take the curl commands out of the skill and call them directly. Which is quite smart BTW. But what is the point of making a convoluted TS skill at all?
Completely unrelated. Recursive Language Models are just "what if we replaced putting all the long text into the context window with a REPL which lets you read parts of the context through tool calls and launch partitioned subagents", ie divide-and-conquer applied to attention space.
They also tend to imply symbolic recursion which seems to be the biggest deal out of everything by a wide margin.
When you can nest 10+ agents deep and guarantee you will get back home without losing any data in any of the stack frames, the ability to chunk through complex problems goes up dramatically.
I think even code bases will have self improving agents. Software is moving from just the product code, to the agent code that maintains the product. Engineering teams/companies that move in this direction will vastly out produce others.
I've had to really shift how I think about building code bases, alot of logic can go into claude skills and sub agents. Requires essentially relearning software engineering
This, IMO is the biggest insight into where we're at and where we're going:
> Because both evaluation and self-modification are coding tasks, gains in coding ability can translate into gains in self-improvement ability.
There's a thing that I've noticed early into LLMs: once they unlock one capability, you can use that capability to compose stuff and improve on other, related or not, capabilities. For example "reflexion" goes into coding - hey, this didn't work, let me try ... Then "tools". Then "reflxion" + "tools". And so on.
You can get workflows that have individual parts that aren't so precise become better by composing them, and letting one component influence the other. Like e2e coding gets better by checking with "gof" tools (linters, compilers, etc). Then it gets even better by adding a coding review stage. Then it gets even better by adding a static analysis phase.
Now we're seeing this all converge on "self improving" by combining "improving" components. And so on. This is really cool.
OTOH, there's loads you can do for evaluation before a human even sees the artifact. Things like does the site load, does it behave the same, did anything major change on the happy path, etc etc. There's a recent-ish paper where instead of classic "LLM as a judge" they used LLMs to come up with rubrics, and other instances check original prompt + rubrics on a binary scale. Saw improvements in a lot of evaluations.
Then there's "evaluate by having an agent do it" for any documentation tracking. Say you have a project, you implement a feature, and document the changes. Then you can have an agent take that documentation and "try it out". Should give you much faster feedback loops.
Another thing that get quantized is video preferences to maximize engagement.
This is the exact point I make whenever people say LLMs aren't deterministic and therefore not useful.
Yes, they are "stochastic". But you can use them to write deterministic tools that create machine readable output that the LLM can use. As you mention, you keep building more of these tools and tying them together and then you have a deterministic "network" of "lego blocks" that you can run repeatably.
Larger composition, though, starts to run into typical software design problems, like dependency graphs, shared state, how to upgrade, etc.
I've been working on this front for over two years now too: https://github.com/smartcomputer-ai/agent-os/
> once they unlock one capability,
What does it mean to unlock? Its an llm nothing is locked. The output is a as good as the context, model and environment. Nothing is hidden or locked.
Its is the core of any and all learning/exellency; exposure to chaotic perturbations allow selection of solutions that are then generalized to further, ever more straining problems; producing increasingly applicable solutions.
This is the core of evolution, and is actually derivable from just a single rule.
On this view, learning in general operates via selection under uncertainty. This is less visible in individual cognition, where we tend to over-attribute agency, but it is explicit in science: hypotheses are proposed, subjected to tests, and selectively retained, precisely because the future cannot be deduced from the present.
In that sense, generation/discrimination is a particular implementation of this broader principle (a way of instantiating variation and selection) not the primitive itself.
Variation (chaos) comes from the tidal push/pull of all cumulative processes - all processes are nearly periodic (2nd law) and get slower - guaranteeing oscillator harmonics at intervals.
These intervals are astronomically convulted, but still promise a Fourier distribution of frequency: tidal effects ensure synchronization eventually, as all periods resonate eventually.
As systems are increasingly exposed to pendulums of positive and negative coherence, they will generalize for variance, and eventually for increasingly (fourier) selective filters of increasingly resiliente traits, that will generalize.
The system would eventually be increasingly resilient and eventually an awareness would develop.
Awareness of past periodic cycles would improve fitness (with or without consciousness) and eventually the mechanistic processes would be in the systems nature.
This is why we have pointless traditions, folk lore, collective unconscious artifacts, cyclical cataclysmic religions, the Fermi Paradox, the great filters...
Variation and selection are woven, but understanding how it all stems from gravity by means of nearly perioidic oscillators (spinning planets, tidal pools, celestial bodies) due to the conservation of angular momentum, due to the 3body problem.....that is what took a genius to reconcile
I am not sure how this is a necessary conclusion to the premises you provide.
As Briggs–Rauscher reactions can eventually lead to Belousov–Zhabotinsky reactions, the system can maintain homeostasis with its environment (and continuing to oscillate) by varying reactants in a loss minimizing fashion.
This loss minimizing would be done during scarcity to limp towards an abundance phase.
This is the mechanism that hypothetical tidal pools batteries would had exhibited to continue between periods of sunlight/darkness/acidity that eventually gets stratified as a resilency trait.
Loss minimizing to a few problems will generalize into abstraction, and a few solutions will develop.
These systems with more generalizable resilency traits will encounter increasingly varied selective sieves.
Systems that survive this seive will exhibit increasingly sophisticated, generalizable solutions to prevent loss of needed dependent reactions/resources.
These solutions must exert influence to be effective; influencing the environment for its own benefit.
As systems influence their environment, delineation of "self" and "environment" becomes a fundamental barrier.
The system would prefer itself, or be outcompeted by a similar system that does.
This layer of semi-life like material would form between sunlight and the oscillating reaction, and eventually envelope it, minimizing surface tension by means of a spherical cell like structure.
Small stuff runs off of loss minimizing at a force level for its mechanistic affect; from covalent bonds to cellular ion transport, the path of lesser resistance is the fundamental forces.
As systems become more complex, the minimizing is less directly attributable to the fundamental forces and becomes more of a Byzantine dependency/feedback network.
This byzantine labyrinth of interactions is called biology.
The delineation of self, the ego.
At the highest levels, geopolitics. At the human level, mate suppression. Lowest level, energy conservation.
All loss minimizing
Basically a dice with a bajillion sides is getting rolled along an increasingly slim poker table, house winning eventually.
Time only goes one way, protons dont decay, energy is radiated unto the cosmic background hiss, until homogeneity is reached as CMB, and entrophy reaches 1.
I dont know where it comes from, but I know the shape it makes as it rolls by.
I've always felt that the most important part of engineering was feedback loops.
Maybe nature is the greatest engineer ever?
The great oxygenation was a time of near catyclismsic scarcity for most complex organisms, as resources scale to food/energy requirements imply the most complex organisms were the most dependent on the environment, and were most impacted by changes.
Inversely, oxygenation was our most crucial abundancy pre cursor, as it provides a large substrate chemically for life to exhibit
I was annoyed at how Claude Code ignores my CLAUDE.md and skills, so I was looking for ways to expand type checking to them. So I wrote a wrapper on top of claude-agents-sdk that reads my CLAUDE.md and skills, and compiles them into rules - could be linter rules or custom checking scripts. Then it hooks up to all tools and runs the checks. The self improving part comes if some rule doesn't work: I run the tool with the session id in review mode, it proposes the fixes and improves the rule checkers. (not the md files) So it's kinda like vibe coding rules, definitely lowers the bar for me to maintain them. Repo: https://github.com/chebykinn/agent-ruler
I wonder how much of the improvement is due to the agent actually learning new things vs. reaching parts of its latent space that enable it to recall things it already knows. Did the agent come up with novel RL reward design protocols based on trial and error? Or did the tokens in the environment cause it to "act smarter"?
Abstract:
Self-improving AI systems aim to reduce reliance on human engineering by learning to improve their own learning and problem-solving processes. Existing approaches to self-improvement rely on fixed, handcrafted meta-level mechanisms, fundamentally limiting how fast such systems can improve. The Darwin Gödel Machine (DGM) demonstrates open-ended self-improvement in coding by repeatedly generating and evaluating self-modified variants. Because both evaluation and self-modification are coding tasks, gains in coding ability can translate into gains in self-improvement ability. However, this alignment does not generally hold beyond coding domains. We introduce \textbf{hyperagents}, self-referential agents that integrate a task agent (which solves the target task) and a meta agent (which modifies itself and the task agent) into a single editable program. Crucially, the meta-level modification procedure is itself editable, enabling metacognitive self-modification, improving not only the task-solving behavior, but also the mechanism that generates future improvements. We instantiate this framework by extending DGM to create DGM-Hyperagents (DGM-H), eliminating the assumption of domain-specific alignment between task performance and self-modification skill to potentially support self-accelerating progress on any computable task. Across diverse domains, the DGM-H improves performance over time and outperforms baselines without self-improvement or open-ended exploration, as well as prior self-improving systems. Furthermore, the DGM-H improves the process by which it generates new agents (e.g., persistent memory, performance tracking), and these meta-level improvements transfer across domains and accumulate across runs. DGM-Hyperagents offer a glimpse of open-ended AI systems that do not merely search for better solutions, but continually improve their search for how to improve.
Here is a breakdown - https://vectree.io/c/plant-self-incompatibility-logic
https://github.com/NousResearch/hermes-agent
But this idea of having a task agent & meta agent maybe has wings. Neat submission.
https://gist.github.com/ontouchstart/40db1244c0ff523f830df1c...
When you can nest 10+ agents deep and guarantee you will get back home without losing any data in any of the stack frames, the ability to chunk through complex problems goes up dramatically.
I've had to really shift how I think about building code bases, alot of logic can go into claude skills and sub agents. Requires essentially relearning software engineering