LLM vendors

Search2o ships with six adapters: OpenAI, Anthropic and Google Gemini for the three vendors, and Ollama, vLLM and SGLang for models you host. A new account has a profile for each of them. One API key, or one model running on your own machine, is enough to start.

The key

Each vendor profile reads its key from the agent server's environment. Set the key for the vendor you use before starting the agent server:

$ export OPENAI_API_KEY=...        # OpenAI
$ export ANTHROPIC_API_KEY=...     # Anthropic
$ export GEMINI_API_KEY=...        # Google Gemini

Where a key is read from is configurable; see Secret vault. The hosted-model profiles do not need a key.

The seeded profiles

A new account has these LLM profiles. An agent names one of them in its llm command.

Profile	Vendor	What it is for
`gpt5_mini`	OpenAI	Text
`gpt_image`	OpenAI	Image generation
`claude_haiku`	Anthropic	Text
`gemini_flash`	Google Gemini	Text
`gemini_image`	Google Gemini	Image generation
`ollama_qwen25`	Ollama on the agent server's machine	Text, Qwen 2.5 7B
`vllm_qwen25`	vLLM on the agent server's machine	Text, Qwen 2.5 7B
`sglang_qwen25`	SGLang on the agent server's machine	Text, Qwen 2.5 7B

The prices on a vendor profile are the list prices when the account was created. Edit them to match your contract, so that the cost report is accurate. The hosted-model profiles are priced at zero.

A profile for another model

A profile is one model at one vendor, so a second model needs a second profile. Create one under Profiles › LLM in the UI and set:

Field	What to set
`adapter`	`openai`, `anthropic` or `gemini`, matching the protocol the endpoint speaks, whoever serves it: a private deployment, a gateway, or another vendor's compatible API. `ollama`, `vllm` or `sglang` for a model served by one of those. For a protocol none of them speaks, see Connecting to other LLMs.
`url`	The endpoint, for example `https://api.openai.com`.
`headers`	Authorization and any other headers the vendor requires. The values are dynamic strings, so the key is read at call time: `{ f"Bearer {sys.secret['OPENAI_API_KEY']}" }`.
`model`	The model name as the vendor writes it.
`pricing`	The prices per million tokens.

Every field is described in LLM profiles. An agent changes model by naming a different profile. Nothing in the agent changes.

Models you host: Ollama, vLLM and SGLang

The ollama, vllm and sglang adapters connect to open models on your own hardware, served by Ollama, vLLM or SGLang. All three call the model server's OpenAI-compatible Responses API at /v1/responses. The seeded profiles expect the model server on the agent server's machine at its default port; for any other setup, create a profile under Profiles › LLM and set:

Field	Ollama	vLLM	SGLang
`adapter`	`ollama`	`vllm`	`sglang`
`url`	The agent server's route to the model server, ending in `/v1/responses`. With the default ports on the same machine: `http://localhost:11434/v1/responses` for Ollama, `http://localhost:8000/v1/responses` for vLLM, `http://localhost:30000/v1/responses` for SGLang.
`headers`	`Content-Type` alone when the server does not require a key.
`model`	The model as Ollama names it, for example `qwen2.5:7b`.	The name the model is served under, for example `Qwen/Qwen2.5-7B-Instruct`.

Ollama does not act on a forced tool choice. When an llm command sets outputFormat: "structured", the ollama adapter asks Ollama for JSON that fits the schemas of the offered functions and turns the reply into function calls, so the command returns the same dict as with any other adapter.

vLLM makes tool calls only when started with tool calling on: --enable-auto-tool-choice and the --tool-call-parser that fits the model. SGLang makes tool calls only when started with the --tool-call-parser that fits the model.