> ## Documentation Index > Fetch the complete documentation index at: https://allhandsai-docs-conversation-event-stream.mintlify.site/llms.txt > Use this file to discover all available pages before exploring further. # OpenAI-Compatible Endpoint > Call an OpenHands agent-server through the OpenAI Chat Completions or Responses protocol. export const path_to_script_0 = "examples/02_remote_agent_server/15_openai_compatible_gateway.py" The agent-server exposes an OpenAI-compatible API surface under `/v1` with two endpoints: * `POST /v1/chat/completions` — the OpenAI Chat Completions protocol. * `POST /v1/responses` — the OpenAI Responses protocol. Use this when you want an existing chat UI, IDE integration, evaluation harness, voice platform, or another agent to treat OpenHands as an OpenAI-style backend while still getting the full agent runtime behind the request. ## How It Works Both endpoints are protocol adapters over the **same full OpenHands agent**, not a thin wrapper around a raw LLM call. Each request: 1. Loads the OpenHands agent configured by the named profile. 2. Starts (or, for Chat Completions, optionally reuses) an OpenHands conversation. 3. Runs the agent's complete internal tool loop to completion. 4. Returns only the final assistant text as an OpenAI-shaped response. The mental model is **"access an agent conversation as if it were a model"**: you send a prompt, the agent does its work (including executing tools in its workspace), and you receive one response when it finishes. Internal tool activity is not exposed as OpenAI tool calls. ## What to Configure Most OpenAI-compatible clients ask for the same three fields: | Client Field | Value | | ------------ | --------------------------------- | | Base URL | `https://YOUR_AGENT_SERVER/v1` | | API key | Your agent-server session API key | | Model | `openhands_` | For example, a saved LLM profile named `gateway_demo` appears as the OpenAI model `openhands_gateway_demo`. Authentication maps OpenAI-style bearer tokens onto the agent-server's existing session key mechanism. The gateway accepts the same session key in either form: * `X-Session-API-Key: ` * `Authorization: Bearer ` Both are validated against the configured session API keys — there is no second credential system. When the server is configured without session keys, it remains unauthenticated just like the native agent-server API. ## Prepare a Profile OpenAI-compatible traffic is backed by an agent-server LLM profile. Create one with the native profile API first: ```bash theme={null} export AGENT_SERVER_URL="http://localhost:8000" export SESSION_API_KEY="your-session-api-key" export PROFILE_NAME="gateway_demo" export OPENHANDS_MODEL="openhands_${PROFILE_NAME}" curl -X POST "$AGENT_SERVER_URL/api/profiles/$PROFILE_NAME" \ -H "X-Session-API-Key: $SESSION_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "llm": { "model": "gpt-5-nano", "api_key": "YOUR_LLM_API_KEY" }, "include_secrets": true }' ``` Then confirm the profile is visible to OpenAI clients: ```bash theme={null} curl "$AGENT_SERVER_URL/v1/models" \ -H "Authorization: Bearer $SESSION_API_KEY" ``` ## Chat Completions (`POST /v1/chat/completions`) Each request runs a full OpenHands agent to completion and returns the final assistant text in a standard Chat Completions shape. Supported request fields: * `model` — required; must be an `openhands_` exposed via `GET /v1/models`. * `messages` — a standard list. The last `user` message becomes the agent's task; `system` and `developer` messages are folded into the agent's system context. * `stream` — `true` returns a server-sent events stream; `false` (default) returns a single response. The response includes a `X-OpenHands-ServerConversation-ID` header. Send that header on a follow-up request to continue the same server-side OpenHands conversation instead of starting a new one. ### Client Recipes ```bash theme={null} curl -i "$AGENT_SERVER_URL/v1/chat/completions" \ -H "Authorization: Bearer $SESSION_API_KEY" \ -H "Content-Type: application/json" \ -d "{ \"model\": \"$OPENHANDS_MODEL\", \"messages\": [ { \"role\": \"system\", \"content\": \"Answer directly unless you need to inspect files.\" }, { \"role\": \"user\", \"content\": \"Explain what this OpenHands endpoint does in one sentence.\" } ] }" ``` The response includes `X-OpenHands-ServerConversation-ID`. Save that header if you want a later request to continue the same agent conversation. ```python theme={null} import os from openai import OpenAI client = OpenAI( api_key=os.environ["SESSION_API_KEY"], base_url=f"{os.environ['AGENT_SERVER_URL']}/v1", ) response = client.chat.completions.with_raw_response.create( model=os.environ["OPENHANDS_MODEL"], messages=[ {"role": "user", "content": "Summarize this repository."}, ], ) completion = response.parse() conversation_id = response.headers["X-OpenHands-ServerConversation-ID"] print(completion.choices[0].message.content) follow_up = client.chat.completions.create( model=os.environ["OPENHANDS_MODEL"], messages=[{"role": "user", "content": "Now list the main packages."}], extra_headers={"X-OpenHands-ServerConversation-ID": conversation_id}, ) print(follow_up.choices[0].message.content) ``` ```javascript theme={null} import OpenAI from "openai"; const client = new OpenAI({ apiKey: process.env.SESSION_API_KEY, baseURL: `${process.env.AGENT_SERVER_URL}/v1`, }); const first = await client.chat.completions .create({ model: process.env.OPENHANDS_MODEL, messages: [ { role: "user", content: "Summarize this repository." }, ], }) .withResponse(); const conversationId = first.response.headers.get( "x-openhands-serverconversation-id", ); console.log(first.data.choices[0].message.content); const followUp = await client.chat.completions.create( { model: process.env.OPENHANDS_MODEL, messages: [{ role: "user", content: "Now list the main packages." }], }, { headers: { "X-OpenHands-ServerConversation-ID": conversationId }, }, ); console.log(followUp.choices[0].message.content); ``` For Open WebUI, LibreChat, Chatbot UI, and similar OpenAI-compatible frontends, configure a custom OpenAI provider with: * **Base URL**: `https://YOUR_AGENT_SERVER/v1` * **API key**: your agent-server session API key * **Model**: `openhands_` If the UI can store a response header and send a custom request header, persist `X-OpenHands-ServerConversation-ID` per chat thread and send it on follow-up turns. If it cannot, each request starts a new OpenHands conversation and works best for one-shot tasks. Voice platforms and webhook integrations usually have their own session or call ID. Store a mapping from that external ID to the OpenHands conversation ID: ```python theme={null} import os # Initialize this once at app startup, or replace it with durable session storage. conversation_ids: dict[str, str] = {} conversation_id = conversation_ids.get(platform_session_id) headers = {} if conversation_id: headers["X-OpenHands-ServerConversation-ID"] = conversation_id response = client.chat.completions.with_raw_response.create( model=os.environ.get("OPENHANDS_MODEL", "openhands_gateway_demo"), messages=[{"role": "user", "content": transcript_text}], extra_headers=headers, ) conversation_ids[platform_session_id] = response.headers[ "X-OpenHands-ServerConversation-ID" ] reply_text = response.parse().choices[0].message.content ``` Return `reply_text` to the voice or webhook platform. Keep the mapping for as long as that external session should continue. ### Conversation State The Chat Completions protocol usually sends full message history on every request, but the gateway does **not** reconstruct agent history from prior assistant messages. Instead: * Omit `X-OpenHands-ServerConversation-ID` to start a new OpenHands conversation. * Read `X-OpenHands-ServerConversation-ID` from the response. * Send that header on follow-up requests to continue the same OpenHands conversation. When reusing a conversation, send the newest user turn in `messages`. The server-side OpenHands conversation owns the previous agent state, tool activity, and workspace context. ## Responses (`POST /v1/responses`) The Responses endpoint targets the OpenAI Responses API — a better fit for agent-shaped traffic, with typed input/output items. It is **stateless-first** by design. ### Mental Model Every request starts a **fresh** OpenHands conversation and runs the full agent to completion. There is no server-side continuation handle: to carry context forward, clients replay prior input and output items into the next request's `input`. ```bash theme={null} curl "$AGENT_SERVER_URL/v1/responses" \ -H "Authorization: Bearer $SESSION_API_KEY" \ -H "Content-Type: application/json" \ -d "{ \"model\": \"$OPENHANDS_MODEL\", \"instructions\": \"Answer briefly.\", \"input\": \"Summarize this repository in one sentence.\", \"store\": false }" ``` Response: ```json theme={null} { "id": "resp_…", "object": "response", "created_at": 1726000000.0, "completed_at": 1726000120.0, "model": "openhands_gateway_demo", "instructions": "Answer briefly.", "output": [ { "id": "msg_…", "type": "message", "role": "assistant", "status": "completed", "content": [ { "type": "output_text", "text": "This repository is …", "annotations": [] } ] } ], "parallel_tool_calls": false, "previous_response_id": null, "status": "completed", "tool_choice": "none", "tools": [], "usage": { "input_tokens": 123, "input_tokens_details": { "cached_tokens": 0 }, "output_tokens": 45, "output_tokens_details": { "reasoning_tokens": 0 }, "total_tokens": 168 } } ``` The response also carries the `X-OpenHands-ServerConversation-ID` header, but unlike Chat Completions you **cannot** pass it back to continue that conversation — the Responses surface ignores it. Use the header only to correlate the response with the underlying OpenHands conversation through the native agent-server API. ### Replaying Context To maintain context across Responses calls, replay the previous assistant output items (and any system/developer context) into the next request's `input`: ```python theme={null} import os from typing import cast from openai import OpenAI from openai.types.responses import ResponseInputItemParam client = OpenAI( api_key=os.environ["SESSION_API_KEY"], base_url=f"{os.environ['AGENT_SERVER_URL']}/v1", ) first = client.responses.create( model=os.environ["OPENHANDS_MODEL"], instructions="You are reviewing this repository.", input="Summarize the project structure.", store=False, ) second = client.responses.create( model=os.environ["OPENHANDS_MODEL"], input=[ {"role": "developer", "content": "You are reviewing this repository."}, {"role": "user", "content": "Summarize the project structure."}, *[ item.model_dump(mode="json", exclude_none=True) for item in cast(list[ResponseInputItemParam], first.output) ], {"role": "user", "content": "Now list the main packages."}, ], store=False, ) ``` ### How Input Is Interpreted * Top-level `instructions` and any `system`/`developer` input items become the agent's **system context**. * The remaining input items become the agent's user prompt. A single `user` item is sent as-is; multiple non-system items are wrapped in `` tags so their roles are preserved. * `model` must be an `openhands_` exposed via `GET /v1/models`. ### Not Supported Yet The following OpenAI Responses features are intentionally rejected or ignored. Status codes and wording are exact. | Feature | Behavior | | ------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `previous_response_id` | Rejected with `400` — `"previous_response_id is not supported; replay input items instead"`. | | `store: true` | Rejected with `400` — `"Persistent response storage (store=True) is not supported yet"`. There is no retrievable Responses object and no `GET /v1/responses/{id}`. | | `stream: true` | Rejected with `400` — `"Streaming responses are not supported yet"`. | | `tools`, `tool_choice`, `parallel_tool_calls` | Accepted but **ignored** — the caller's declared tools do not replace OpenHands' internal tool loop. | | `temperature` and other generation-tuning fields | Accepted but **ignored**. | Setting `store: false` (the default) is **not** a data-retention control. It only signals that no Responses object is retained. The backing OpenHands conversation still follows the agent-server's normal persistence policy. ## Current Limitations (Both Endpoints) * The response contains the final assistant text only. Internal OpenHands tool activity is not exposed as OpenAI tool calls or Responses output items. * OpenAI request fields the gateway does not need are either ignored or rejected intentionally by the server implementation. Declared tools and generation-tuning fields do not change agent behavior. ## Ready-to-run example This example is available on GitHub: [examples/02\_remote\_agent\_server/15\_openai\_compatible\_gateway.py](https://github.com/OpenHands/software-agent-sdk/blob/main/examples/02_remote_agent_server/15_openai_compatible_gateway.py) ```python icon="python" expandable examples/02_remote_agent_server/15_openai_compatible_gateway.py theme={null} """Use the agent-server through an OpenAI-compatible Chat Completions client. This example starts a local agent-server, stores an LLM profile, lists it through ``GET /v1/models``, then calls ``POST /v1/chat/completions`` with the OpenAI Python SDK. The returned ``X-OpenHands-ServerConversation-ID`` header is passed back on a second call to continue the same OpenHands conversation. """ import os from uuid import UUID import httpx from openai import OpenAI from scripts.utils import ManagedAPIServer # The gateway runs a full OpenHands agent, but OpenAI clients still need a # normal model-like name. We create an LLM profile below and expose it as # `openhands_` through `/v1/models`. api_key = os.getenv("LLM_API_KEY") or os.getenv("OPENAI_API_KEY") assert api_key is not None, "Set LLM_API_KEY or OPENAI_API_KEY." llm_model = os.getenv("LLM_MODEL", "gpt-5-nano") llm_base_url = os.getenv("LLM_BASE_URL") profile_name = "gateway_demo" gateway_model = f"openhands_{profile_name}" # Start a local agent-server for the demo. `use_session_api_key=True` turns on # authentication; the same key works as both `X-Session-API-Key` for native # agent-server routes and `Authorization: Bearer ...` for OpenAI SDK calls. with ManagedAPIServer( port=8770, use_session_api_key=True, extra_env={ "OH_ENABLE_VNC": "0", "OH_ENABLE_VSCODE": "0", "OH_PRELOAD_TOOLS": "0", "OH_SECRET_KEY": "example-secret-key-for-demo-only-32b", "OH_WEBHOOKS": "[]", }, health_request_timeout=2.0, ) as server: session_api_key = ( os.getenv("SESSION_API_KEY") or os.getenv("OH_SESSION_API_KEYS_0") or server.session_api_key ) assert session_api_key is not None # Use the native REST API once to create the profile that backs the gateway # model. After that, normal OpenAI SDK calls are enough for chat traffic. api_client = httpx.Client( base_url=server.base_url, headers={"X-Session-API-Key": session_api_key}, timeout=120.0, ) openai_client = OpenAI( api_key=session_api_key, base_url=f"{server.base_url}/v1", timeout=120.0, ) llm_config = {"model": llm_model, "api_key": api_key} if llm_base_url: llm_config["base_url"] = llm_base_url # `gateway_demo` becomes visible to OpenAI clients as `openhands_gateway_demo`. profile_response = api_client.post( f"/api/profiles/{profile_name}", json={"llm": llm_config, "include_secrets": True}, ) assert profile_response.status_code == 201, profile_response.text models = openai_client.models.list() model_ids = [model.id for model in models.data] assert gateway_model in model_ids print(f"Gateway models include: {gateway_model}") # Ask through the OpenAI SDK. `with_raw_response` lets us read the custom # response header that identifies the OpenHands conversation created behind # this otherwise OpenAI-shaped request. first_response = openai_client.chat.completions.with_raw_response.create( model=gateway_model, messages=[ { "role": "system", "content": "Answer directly and do not use tools.", }, { "role": "user", "content": ( "In one sentence, explain what an OpenAI-compatible " "agent-server gateway does." ), }, ], ) first_completion = first_response.parse() conversation_id = first_response.headers.get("X-OpenHands-ServerConversation-ID") assert conversation_id is not None UUID(conversation_id) first_answer = first_completion.choices[0].message.content print(f"First answer: {first_answer}") print(f"OpenHands conversation ID: {conversation_id}") persisted_response = api_client.get(f"/api/conversations/{conversation_id}") assert persisted_response.status_code == 200, persisted_response.text # The gateway keeps conversations by default. Passing the header back lets # another OpenAI-compatible request continue the same server-side agent # conversation instead of starting over. second_completion = openai_client.chat.completions.create( model=gateway_model, messages=[ { "role": "user", "content": "Now answer in five words or fewer: what did I ask about?", } ], extra_headers={"X-OpenHands-ServerConversation-ID": conversation_id}, ) second_answer = second_completion.choices[0].message.content print(f"Second answer using same conversation: {second_answer}") conversation_response = api_client.get(f"/api/conversations/{conversation_id}") assert conversation_response.status_code == 200, conversation_response.text stats = conversation_response.json().get("stats") or {} usage_to_metrics = stats.get("usage_to_metrics") or {} accumulated_cost = sum( metrics.get("accumulated_cost", 0.0) for metrics in usage_to_metrics.values() ) # Clean up the demo resources. Real applications can keep the conversation # ID and inspect it later through the native agent-server API. api_client.delete(f"/api/conversations/{conversation_id}") api_client.delete(f"/api/profiles/{profile_name}") api_client.close() print(f"EXAMPLE_COST: {accumulated_cost}") ``` You can run the example code as-is. The model name should follow the [LiteLLM convention](https://models.litellm.ai/): `provider/model_name` (e.g., `anthropic/claude-sonnet-4-5-20250929`, `openai/gpt-4o`). The `LLM_API_KEY` should be the API key for your chosen provider. {`export LLM_API_KEY="your-api-key"\nexport LLM_MODEL="anthropic/claude-sonnet-4-5-20250929" # or openai/gpt-4o, etc.\ncd software-agent-sdk\nuv run python ${path_to_script_0}`} {`# https://app.all-hands.dev/settings/api-keys\nexport LLM_API_KEY="your-openhands-api-key"\nexport LLM_MODEL="openhands/claude-sonnet-4-5-20250929"\ncd software-agent-sdk\nuv run python ${path_to_script_0}`} **ChatGPT Plus/Pro subscribers**: You can use `LLM.subscription_login()` to authenticate with your ChatGPT account and access Codex models without consuming API credits. See the [LLM Subscriptions guide](/sdk/guides/llm-subscriptions) for details.