> Their ability to refactor a codebase goes in the toilet pretty quickly.
Very much this. I tried to get Claude to move some code from one file to another. Some of the code went missing. Some of it was modified along the way.
Humans have strategies for refactoring, e.g. "I'm going to start from the top of the file and Cut code that needs to be moved and Paste it in the new location". LLM don't have a clipboard (yet!) so they can't do this.
Claude can only reliably do this refactoring if it can keep the start and end files in context. This was a large file, so it got lost. Even then it needs direct supervision.
> Humans have strategies for refactoring, e.g. "I'm going to start from the top of the file and Cut code that needs to be moved and Paste it in the new location". LLM don't have a clipboard (yet!) so they can't do this.
For my own agent I have a `move_file` and `copy_file` tool with two args each, that at least GPT-OSS seems to be able to use whenever it suits, like for moving stuff around. I've seen it use it as part of refactoring as well, moving a file to one location, copying that to another, the trim both of them but different trims, seems to have worked OK.
If the agent has access to `exec_shell` or similar, I'm sure you could add `Use mv and cp if you need to move or copy files` to the system prompt to get it to use that instead, probably would work in Claude Code as well.
Remember 20 years ago when Eclipse could move a function by manipulating the AST and following references to adjust imports and callers, and it it didn't lose any code?
I have a suite of agent tools that is just waiting on my search service for a release, it includes `srefactor` and `spatch` commands that have fuzzy semantic alignment with strong error guards, they use LSP and tree sitter to enable refactoring/patching without line numbers or anything and ensure the patch is correct.
Nice. This sounds like the right approach. As an aside, it’s crazy that a mature LSP server is not a first class requirement for language choice in 2025. I used to write mini LSP servers before working on a project starting when LSP came out a few years ago. Now that there is wider adoption, I don’t find myself reaching for this quite as often, but it’s still a really nice way to ease development on mature codebases that have grown their own design patterns.
I think it's likely that these agent-based development will inevitably add more imperative tools to their arsenal to lower cost, improve speed and accuracy.
Claude’s utility really drops when any task requires a working set larger than the context window.
On the one hand, it’s kind or irritating when it goes great-great-great-fail.
On the other hand, it really enforces the best practices of small classes, small files, separation of concerns. If each unit is small enough it does great.
Unfortunately, it’s also fairly verbose and not great at recognizing that it is writing the same code over and over again, so I often find some basic file has exploded to 3000 lines, and a simple “identity repeated logic and move to functions” prompt shrinks it to 500 lines.
Very much this. I tried to get Claude to move some code from one file to another. Some of the code went missing. Some of it was modified along the way.
Humans have strategies for refactoring, e.g. "I'm going to start from the top of the file and Cut code that needs to be moved and Paste it in the new location". LLM don't have a clipboard (yet!) so they can't do this.
Claude can only reliably do this refactoring if it can keep the start and end files in context. This was a large file, so it got lost. Even then it needs direct supervision.