> But I think a better option is often fiddling and seeing what happens. The trick is to make it so that fiddling is safe. E.g., on a project where I have good test coverage and find mystery code, I can just delete it and see what fails. Or I set up a big smoke test that runs a bunch of input and compares outputs to see what changes.
The problem “fiddle and see if Rome burns” is that you may have broken something that only runs once a year/decade and you won’t know if that process isn’t in the list of tests.
And by the time it breaks, will anyone remember the probable cause?
If people are building things that only run once a year or once a decade and haven't included sufficient tests to make sure those things work and keep working, then the failure lies with them, not with the people who work on the system later. The real "probable cause" isn't some later change. It's the initial negligent construction.
There was a good discussion of this in the comments, starting at https://www.jefftk.com/p/accidentally-load-bearing#fb-109168...
In addition to tests, you can also add logging to your running system, or make the change behind a flag that you A/B test in production.