the scan catches surface stuff. funnier signal would be tracking when the agent reads the same file 3 times in a row, or deletes what it just wrote. you can hear the frustration in the access pattern.
I need a version of this which swears loudly when an assumption it made turns out to be wrong, with the volume/passion/verbosity correlated with how many tokens it's burned on the incorrect approach.
i didnt realize i needed the volume scaling with tokens burned as much as i do now xD
imagine the screaming when it confidently refactors something for 40k tokens and then finds out the thing it deleted was load bearing
I have general reviewer named Feynman with his personality that shits on anything other agents do and sends it back before it hits me and it sounds perfect to include some sound bites from YouTube clips. Great idea!!
Does this actually relate to the code quality being observed by the agent? The readme isn't very clear on that IMO. I have some projects I'd love to try this out on, but only if I am to get an accurate representation of the LLMs suffering.
So it is left up to agent to decide.
So looks like it's mainly looking for FIXME/TODO etc comments, deep nesting, large files, broad catches, stuff like that.