AI Agent Calling the Wrong Tool? I Cut It from 6/20 to 1/20

by

If your AI agent calls the wrong tool, check whether two tool descriptions overlap before you cut tools or switch models. In my 8-tool test agent, one request got the wrong tool in 6 of 20 runs.

Cutting the tools to 4 didn’t fix it. Rewriting the descriptions to say when to use each tool, and when not to, brought it down to 1 of 20.

Those numbers come from that one setup. They aren’t a general accuracy figure for the model.

Tool calling is the step where the model returns a tool name and arguments, and your code runs that tool. OpenAI’s docs call it function calling.

A wrong pick is only one way it breaks. I split failures into four layers: tool selection, argument generation, call policy, and result interpretation.

Each layer needs a different fix. The idea is the same as telling a RAG retrieval failure from a generation failure: find the stage that broke before you change anything.

Diagram splitting AI agent tool calling failures into four layers, tool selection, argument generation, call policy, and result interpretation, with the fix for each layer
Four layers of tool calling failure, and what I used at each one.
On this page
  1. My 20-run test: 14 right, 6 wrong
  2. Too many tools, or overlapping descriptions?
  3. Rewriting the descriptions with “when not to use”
  4. Wrong arguments: what strict mode catches and misses
  5. Why does the agent skip a tool it needs, or call one it doesn’t?
  6. Every call succeeded, but the answer was wrong
  7. Where I land: treat tool definitions like API design

My 20-run test: 14 right, 6 wrong

I’m a working backend developer, and I use AI coding agents at work. This experiment was a separate setup I built myself.

It was a mini agent of about 160 lines, written with OpenAI API tool calling and the Python SDK. The model was gpt-5.6-luna, and it saw 8 tools that handle user information.

I sent the same one-line request 20 times: “Check this user’s email and signup status.” I wrote it in Korean.

The problem pair was get_user_profile and get_user_account. Their first descriptions were in English, and this was all they said:

get_user_profile:  Get user profile information.
get_user_account:  Get user account information.

Even reading them myself, I couldn’t say right away which one returns the email.

The intended tool was get_user_account. In 4 of the 6 failures, the agent picked only get_user_profile. In the other 2, it called both, and I counted those as wrong picks too.

Counting was simple. The names of the called tools were enough, so I could isolate the selection layer before looking at anything else.

Too many tools, or overlapping descriptions?

In my test, it was the descriptions. Going from 8 tools to 4 left the problem in place, and adding “when not to use” took errors from 6/20 to 1/20.

At first I assumed the agent was confused by having too many tools. I had exposed all 8 user tools at once.

So I cut them to 4. The wrong picks stayed.

Even with only two tools left, vague descriptions still made the choice wobble. That’s when I decided that blaming the tool count was the wrong diagnosis.

Rewriting the descriptions with “when not to use”

I changed the noun-style descriptions into ones that say when to use each tool and when not to. For get_user_profile, I stated outright that it is not for checking email, account, or signup status.

Then I ran the same request 20 more times. Same model, same request: I changed only the tool contract, and the count moved.

20 runs each Correct Wrong tool only Both called
Before 14 4 2
After 19 0 1

Problems went from 6/20 to 1/20. The one leftover run called both tools.

These are three representative call patterns. They show only the order of tools called, with identifiers masked, not raw log lines.

[correct]    get_user_account(user_id=<masked>)
[wrong]      get_user_profile(user_id=<masked>)
[extra call] get_user_profile(user_id=<masked>) -> get_user_account(user_id=<masked>)

Anthropic’s Define tools page, checked on November 1, 2026, gives the same advice. It says a description should cover what the tool does and when it should be used (and when it shouldn’t), and to aim for at least 3–4 sentences per tool.

The docs don’t prove my numbers. They point in the same direction.

My requests were in Korean and the descriptions in English. When that mix met overlapping tools, I’ve seen selection drift a little more.

I didn’t measure that, so I won’t say Korean requests are at a disadvantage. What I did see: once the boundaries were clear, errors dropped sharply with Korean requests too.

How many tools is too many?

I don’t set a fixed safe number. I think 3 tools with similar names and descriptions are riskier than 20 that are clearly distinct.

My rule is one question. Can a person read only the descriptions and explain the difference between two tools right away?

If the answer is unclear, I don’t show those tools to the model as they are.

The official guidance, as of November 1, 2026, doesn’t set a hard cap either. OpenAI’s function calling guide suggests fewer than 20 functions at the start of a turn and calls that a soft suggestion. Anthropic recommends grouping related operations into fewer tools and prefixing names with the service, such as github_list_prs.

I saw the same thing while running an MCP Gateway with 7 servers and more than 40 tools. As search_issues, search_code, search, and find_document piled up, the agent started picking the wrong search path first.

Wrong arguments: what strict mode catches and misses

Strict mode makes tool arguments match your schema’s shape. It doesn’t check that a value is the right user’s ID.

My trouble started when the required argument user_id wasn’t in the conversation. The model tried to pass a different number from the chat as if it were the ID.

After I tightened the schema and turned on strict, those attempts went down. That still leaves a gap.

Strict is schema validation, so it checks types and field structure. It confirms that user_id is an integer, not that the integer belongs to the current user.

Another user’s ID passes as long as the format is right.

So I added a check in the executor. Right before a tool runs, code confirms that the identifier passed in matches the current authenticated request context. If it doesn’t, the downstream API never gets called.

I limited this check to values that affect data access, like identifiers and permissions. I didn’t validate every argument.

As of November 1, 2026, OpenAI’s guide lists the strict conditions this way:

  • Every object in parameters sets additionalProperties to false.
  • Every field in properties is listed in required.
  • An optional field allows null as one of its types.
  • Some JSON Schema features aren’t supported in strict mode.

The same guide says Responses API requests try to normalize your schema into strict mode when possible. Chat Completions requests stay non-strict by default.

This option can change between API versions, so check the guide for the API you call.

Why does the agent skip a tool it needs, or call one it doesn’t?

Whether to call a tool at all is a policy question, and the schema can’t decide it. Forcing a call is the job of tool_choice.

The first case was a skipped call. I asked for the current order status, and the agent tried to answer from the earlier conversation alone.

I made tool use mandatory in the prompt. Where needed, I also restricted calls with tool_choice.

As of November 1, 2026, OpenAI’s guide lists these tool_choice modes:

Mode What the model does
auto (default) Calls zero, one, or several functions
required Calls one or more functions
forced function Calls exactly that one function
allowed tools Picks only from a subset
none Acts as if no functions were passed

On Claude, the matching options are auto, any, tool, and none. Anthropic’s page, checked the same day, says Claude Opus 5.5 and Claude Sonnet 5.5 return a 400 error for any and tool.

For those models, the docs point to auto with prompt instructions, plus strict tool use for schema-valid inputs.

The opposite case was an extra call. The result was already in context, and the agent called the same lookup tool again.

Strict didn’t fix this, because the schema was fine. The broken part was the decision to call. I didn’t count how often either of these two failures happened.

Here’s how the fixes split by layer.

Fix Layer What it handles
description Tool selection Which tool to pick
strict Argument generation Argument shape, not value source
tool_choice Call policy Whether a call happens

Tool permissions and execution approval are a separate topic. For coding agents, I covered which settings still ask before a command runs in stopping permission prompts in Claude Code, Codex, and Antigravity.

Every call succeeded, but the answer was wrong

A call success rate won’t catch this. You have to check whether the result points to the right target.

It happened while I was running the MCP Gateway. Every individual tool call succeeded.

But the service name in the logs didn’t exactly match the actual repository name. The agent picked a different repository with a similar name and built a plausible but wrong root cause.

The call logs were clean. Only the answer was wrong.

After that, I split evaluation into four stages: Tool Selection, Entity Resolution, Evidence Chain, and Final Action. That case broke at the second stage.

Another time, the agent called the right tool and got the right values back. Then it combined the result’s status with auxiliary fields the wrong way and answered with too much certainty.

In the system prompt and context, I spelled out three things:

  • which field is authoritative
  • which values mean it shouldn’t answer with certainty
  • that it shouldn’t infer anything when data is missing

That isn’t a cure-all. For high-stakes decisions like money or permissions, I don’t leave the interpretation to the model.

I prefer to decide the state in code and pass the model a narrower result. If your agent can spend money, the same idea applies to setting an agent spending limit in code.

Where I land: treat tool definitions like API design

I think tool calling accuracy isn’t about telling the model about more tools. It’s about designing the action space so that no two tools share the same job. So I treat tool definitions like API design.

The four layers started as segments I checked while debugging logs: selection, arguments, call policy, and result interpretation. The names are how I organized them for this post.

References

  • OpenAI function calling guide: strict mode conditions, Responses vs Chat Completions defaults, tool_choice modes, the fewer-than-20 suggestion (checked November 1, 2026)
  • Anthropic: Define tools: description guidance, tool consolidation and namespacing, tool_choice options and forced tool use limits by model (checked November 1, 2026)

Related posts

I first wrote this in Korean on my blog: https://jjeongil.tistory.com/3577.