AI Model Handlers

When agent_framework is set to "none", HyperSaaS falls back to basic chat using provider-specific handlers. These handle direct LLM calls without tool support.

BaseChatModelHandler

Defined in chat/ai_models.py, this base class handles:

• Message history within the token budget, with a summary of older turns (chat.handlers.history)
• Image input for models that accept it
• Token estimation with tiktoken
• Usage extraction from LLM responses
• invoke(), stream() (server-sent events) and async astream()
• Structured output via invoke_structured() with Pydantic schemas

Provider Handlers

Handler Resolution

def get_ai_model_handler(session):
    # 1. Check if agent framework is set
    if session.agent_framework != "none":
        handler = get_agent_handler(session)
        if handler:
            return handler

    # 2. Fall back to provider-specific basic chat handler
    provider = session.ai_provider
    HANDLER_MAP = {
        "openai": OpenAIModelHandler,
        "anthropic": AnthropicModelHandler,
        "google": GoogleModelHandler,
    }
    return HANDLER_MAP[provider](session)

Provider Configuration

Each handler reads its API key from settings:

Session Parameters

Every chat path (the LangGraph agent and the basic handlers) reads the chat's settings through one function, generation_settings(session) in chat/model_catalog.py:

The Model Catalog

chat/model_catalog.py lists the models on offer, with each one's label, provider, tier (from the plans) and output limit, and the settings it takes. It is the one source for the model picker and its controls on the frontend (GET /api/workspaces/{id}/ai-models/) and for what each path sends, so a control the frontend shows always has an effect, and switching a chat's model never sends a setting the new model refuses.

• Reasoning models get no sampling settings. Their output limit covers their reasoning as well as the reply, so they get 16,000 tokens (REASONING_ALLOWANCE) on top of the reply's length.
• Claude gets Top P only when the chat sets it and no temperature. With extended thinking on ({"type": "enabled", "budget_tokens": N}) it gets no temperature, Top P or Top K, and the budget comes on top of the reply's length. The budget shrinks to leave at least 1,024 tokens for the answer within the model's limit, and thinking is skipped if there isn't room for Anthropic's 1,024-token minimum.

max_tokens is capped at the model's max_output_tokens from chat/pricing_config.py, on every path and in the check that a message fits the model.

Model Parameters

model_parameters is free-form JSON on the chat. Only generation settings each provider accepts get through; everything else is dropped and logged. The allow-list is in chat/model_parameters.py, and the catalog then keeps only what the chat's model takes:

Settings that have their own field on the chat (temperature, max_tokens, top_p, the penalties) come from that field only, so each reaches the provider once. Keys that change where a request goes, what it sends or how it's billed (base_url, api_key, extra_headers, extra_body, service_tier, n) are never passed on.

{
  "model_parameters": {
    "seed": 7,
    "reasoning_effort": "low"
  }
}

Models the server can call

A model is available when the server has an API key for its provider: OPENAI_API_KEY, ANTHROPIC_API_KEY or GOOGLE_API_KEY. The model list marks the others unavailable, the picker shows them as "Not set up", and creating or switching a chat to one is refused.

A new chat gets its plan's default model. If the server has no key for that model's provider, it gets the plan's first model, in catalog order, from the cheapest tier the plan has a usable model in. So a server with only an OpenAI key starts Free chats on GPT-5 mini rather than on a Gemini model it can't call.

A key that is set but can't be billed still counts as available. python manage.py check_keys --generate finds those (see Monitoring).

To offer another setting, add it to ALLOWED_MODEL_PARAMETERS and to the models' controls in the catalog. To offer another model, add it to the catalog, PRICING_CONFIG and a plan's tier.