What the model does—and does not do
The model emits a tool name and JSON arguments. Your application validates those arguments, performs the real operation, and sends the result back. The inference API never executes your function.
Complete round trip
import json
import os
from openai import OpenAI
client = OpenAI(
base_url="https://prefillsystems.com/v1",
api_key=os.environ["PREFILL_API_KEY"],
)
messages = [
{"role": "user", "content": "What is the weather in Denver?"},
]
tools = [{
"type": "function",
"function": {
"name": "get_weather",
"description": "Get the current weather for a city.",
"parameters": {
"type": "object",
"properties": {
"city": {"type": "string"},
"unit": {"type": "string", "enum": ["celsius", "fahrenheit"]},
},
"required": ["city", "unit"],
"additionalProperties": False,
},
},
}]
first = client.chat.completions.create(
model="qwen/qwen3.8-27b",
messages=messages,
tools=tools,
tool_choice="auto",
)
assistant_message = first.choices[0].message
messages.append(assistant_message)
for call in assistant_message.tool_calls or []:
arguments = json.loads(call.function.arguments)
if call.function.name != "get_weather":
raise ValueError("Unexpected tool")
# Replace this object with your real, validated function result.
result = {
"city": arguments["city"],
"temperature": 72,
"unit": "fahrenheit",
"conditions": "clear",
}
messages.append({
"role": "tool",
"tool_call_id": call.id,
"content": json.dumps(result),
})
final = client.chat.completions.create(
model="qwen/qwen3.8-27b",
messages=messages,
tools=tools,
)
print(final.choices[0].message.content)Tool-choice behavior
| Value | Behavior |
|---|---|
| auto | The model may answer directly or call one or more tools. |
| required | The response must contain a tool call. |
| none | The request is handled without tool definitions, so the model answers normally. |
| Named function | Constrain the response to one specific function. |
Production checks
- Validate every argument against your own schema before execution.
- Authorize the requested operation independently of the model's choice.
- Set timeouts and make side-effecting tools idempotent where practical.
- Return errors as tool results when the model can recover; do not expose secrets.
- Preserve every tool_call_id when returning results.
Prefill Systems' public conformance run covered automatic, required, none, named, parallel, streaming, and result-round-trip tool calls. See the measured benchmark and conformance report.
Need a key? Request direct API access, or use
the basic Python setup guide first.