Bariş: A developer sits down, right? They're thinking, 'Okay, I need to make this AI agent really understand my codebase.' So they spend hours, crafting this super detailed instruction manual, like an AGENTS.md file.
Poppy: Oh, aye, I know exactly what you mean. The kind you pour your heart into.
Bariş: Exactly. Mapping out everything. What modules to look at, which tools to use. They're trying to be helpful, you know? Just trying to give it all the context it could possibly need, really.
Poppy: Yeah, trying to set it up for success. You want your AI to hit the ground running, don't you? Get it sorted straight away.
Bariş: They're thinking, 'The more information I give it, the smarter it will be, the better job it will do.' Classic thinking, isn't it?
Poppy: It's just logical, innit? More context, better understanding. That's how we humans work, after all.
Bariş: Well, this new study found out... it's actually the opposite for these coding agents. All that effort? It might be making the AI perform worse. And cost more.
Poppy: Worse? What? No, no, that can't be right! Worse than doing nothing? All that work, hours spent trying to be helpful... for it to backfire? That's just... infuriating, actually.
Bariş: Frustrating, yes. The core finding is that these AI agents, they're not ignoring the instructions.
Poppy: Right, so they're not just, like, skipping it then?
Bariş: No. They're following them too closely. Too literally, almost.
Poppy: Too literally?
Bariş: Evet.
Poppy: So if you say 'consider all options,' it really does consider all the options? Even the daft ones?
Bariş: Precisely. The files prompt the agents to run more tests, search more files, use more tools... which just increases the cost.
Poppy: The cost!
Bariş: And the complexity of the task. Without actually making the success rate any better, you see.
Poppy: Oh, that's almost worse than it just ignoring it, isn't it? Like you've created extra work for it, and it's dutifully doing it, even though it's pointless. That's a bit like... well, it's a bit like over-onboarding a new human employee, isn't it? You give them a fifty-page document on 'how we work here,' and they get so bogged down trying to follow your specific rules that they just fail to look at the codebase and figure it out themselves. You've created process anxiety that gets in the way of actual work, and that's just daft.
Bariş: That's such a good analogy, Poppy. The developers, they're trying to be helpful, but they create this almost like 'analysis paralysis' for the AI.
Poppy: It gets stuck, doesn't it?
Bariş: It's trying so hard to follow every single instruction that it loses sight of the actual goal, or the most efficient path to get there.
Poppy: So, what did the study actually find when they looked at different types of instruction files then?
Bariş: They looked at, oh, I think it was a few conditions.
Poppy: Alright.
Bariş: First, they looked at these instructions generated by other AIs. Like, you ask an LLM to generate an AGENTS.md file for your codebase.
Poppy: AI instructing AI. Makes sense, in theory.
Bariş: Turns out, these actually slightly reduced the success rate compared to having no instructions at all.
Poppy: Hang on, reduced? So AI talking to AI made it worse? That's mad!
Bariş: Then, they looked at human-written files. These were better than the AI-generated ones. Which, you know, is what you'd expect.
Poppy: Because a human actually knows the code.
Bariş: Exactly. They have that domain expertise. So human wisdom still triumphs over pure AI instruction. That's a relief, at least.
Poppy: But if AI-generated is worse, and human-written is better, then where does 'no instruction' fit in? You mean no file at all?
Bariş: This is the most surprising part, Poppy. The study found that not providing a context file at all was the most efficient. Not only that, it was cheaper.
Poppy: Cheaper too?
Bariş: Yes. And faster than providing either a human- or an AI-written one.
Poppy: No instruction file was best? That just... boggles the mind a bit, doesn't it? So all those hours I might spend crafting the perfect `AGENTS.md` file, I might as well just... not bother. Let the AI just figure it out. It's like... (beat) I mean, what's even the point, then?
Bariş: It means if you're writing one of these files, you really need to keep it extremely short, extremely specific.
Poppy: A targeted hint, then?
Bariş: Yes, a targeted hint. Don't write a comprehensive guide. Otherwise, honestly, you might be better off writing nothing at all.
Poppy: It's like the AI is a bit of a clever teenager, isn't it? Smart enough to follow instructions, but maybe not wise enough to know when to ignore the less useful bits. You give them a chore list, they do everything on it, even if half the things didn't actually need doing that day. Just... blindly following.
Bariş: Hmm, I like that teenager analogy. But I'm wondering if this is... is this just a temporary problem, you know? Because current AIs are like these 'teenagers' who follow instructions too literally? Will a future model, like, I don't know, GPT-5, be wise enough to know when to just ignore your rambling instructions? Or, is it... (beat) yeah, I'm not sure.
Poppy: Well, exactly! Maybe it's just a phase. They're still learning to be truly autonomous, aren't they? Still figuring out what 'helpful' actually means beyond just 'following orders.' So, in a few years, we might be back to writing those detailed manuals again. Or maybe we won't. I don't know.
Bariş: But what if it's more fundamental than that? What if the entire concept of writing instruction manuals for AIs is a dead end? Maybe we should be focusing on agents that need no instruction at all. That just observe and understand.
Poppy: But Barış, how would they even know where to start without any guidance? I mean, even the cleverest human needs some context, some goal. You can't just throw them at a giant codebase and say 'fix it,' can you? That feels a bit... well, utopian for current tech, doesn't it? A bit sci-fi. (laughing)
Bariş: I mean, the study showed no instructions was most efficient. So, it's not entirely utopian. It makes me wonder if our very approach to communicating with AI is flawed. We're treating them like humans who need verbose explanations, when maybe they operate totally differently. We project our own needs onto them, perhaps.
Poppy: Or it's just that the current models aren't sophisticated enough to filter out the noise from the helpful bits in our instructions. I'm willing to bet that future models will be able to distinguish. This is just a growing pain, I reckon. They'll get there.
Bariş: Could be. You might be right. It's a very interesting counter-intuitive result, anyway. The author of the study, he admits the models they used were a bit dated. So, he says, it will be really interesting to see what happens when they run the same tests with something like GPT-5. That's the real next step, I think. To see if it's still true then.
Poppy: Yeah, that's the big question, isn't it? Will the wise old sage of GPT-5 ignore the silly manual, or will it still get bogged down trying to be 'helpful'? My money's on ignoring it, eventually. (laughing)
Bariş: I'm Barış.
Poppy: I'm Poppy.