I Forgot to Attack the Premise
I'm setting up my agent workflow again on a clean laptop. I couldn't bring any of my old skills or instructions with me. But, in a way, I was kinda looking forward to rebuilding my setup.
I now wish I backed everything up. I really underestimated how tailored my old setup was.
Recently I was asked to work on a ticket. It was quite prescriptive, detailed, and described what needed built. So... I built it.
But my changes were rejected. And correctly so. The solution on the ticket wasn't quite what was needed.
It turned out the ticket was written by AI, it described a solution, and an AI built it. And me, being a meat proxy in the middle, never questioned if it was right.
Could I have spotted it? Well, I don't think so.
I'm still getting to know this system. It's big, and complex, and I don't know it well enough to look at a ticket and see the solution is wrong. So I trusted it.
But it's still my fault. I have no problem owning it because I care more about what I can change in the future to prevent it. I could have done something different. I've written about treating failure as data in the past. This is no different.
As I look back, what struck me was how my old setup would have caught this. When I planned work with my agent, it treated any solution on a ticket as a suggestion. It went looking for evidence and data to back the solution, and it never assumed the solution was correct. My triage skill called it Attack the Premise.
We could talk about how tickets should describe problems not solutions and all that, but there is a super simple reason that triggered why I added it to my old instructions: tickets sit in backlogs.
A ticket written 6 months ago might be out of date, or might have been wrong from the start. So I never assumed a ticket was accurate. It's the same problem as implicit freshness: treating something as current because nobody said otherwise.
I don't have the instructions anymore. It just worked. I never refined it. I think that's why I forgot it was there. I rarely hit the problem because, for me, it was solved and it just became "how I work".
I've added a new version, but now I can't help but wonder what else my old setup was doing for me that I ended up taking for granted.