BackendAI Chat
AI Model Handlers
Provider-specific LLM handlers for basic chat (no tools).
When agent_framework is set to "none", HyperSaaS falls back to basic chat using provider-specific handlers. These handle direct LLM calls without tool support.
BaseChatModelHandler
Defined in chat/ai_models.py, this base class handles:
- Message history preparation (last 20 messages)
- Token estimation with tiktoken
- Usage extraction from LLM responses
- Synchronous
invoke()and asyncastream()methods - Structured output via
invoke_structured()with Pydantic schemas
Provider Handlers
| Provider | Handler Class | LLM Library |
|---|---|---|
| OpenAI | OpenAIModelHandler | langchain-openai (ChatOpenAI) |
| Anthropic | AnthropicModelHandler | langchain-anthropic (ChatAnthropic) |
GoogleModelHandler | langchain-google-genai (ChatGoogleGenerativeAI) | |
| Groq | GroqModelHandler | langchain-openai with Groq endpoint |
| DeepSeek | DeepSeekModelHandler | langchain-openai with DeepSeek endpoint |
Handler Resolution
def get_ai_model_handler(session):
# 1. Check if agent framework is set
if session.agent_framework != "none":
handler = get_agent_handler(session)
if handler:
return handler
# 2. Fall back to provider-specific basic chat handler
provider = session.ai_provider
HANDLER_MAP = {
"openai": OpenAIModelHandler,
"anthropic": AnthropicModelHandler,
"google": GoogleModelHandler,
"groq": GroqModelHandler,
"deepseek": DeepSeekModelHandler,
}
return HANDLER_MAP[provider](session)Provider Configuration
Each handler reads its API key from settings:
| Provider | Setting | Env Variable |
|---|---|---|
| OpenAI | OPENAI_API_KEY | OPENAI_API_KEY |
| Anthropic | ANTHROPIC_API_KEY | ANTHROPIC_API_KEY |
GOOGLE_API_KEY | GOOGLE_API_KEY | |
| Groq | GROQ_API_KEY | GROQ_API_KEY |
| DeepSeek | DEEPSEEK_API_KEY | DEEPSEEK_API_KEY |
Session Parameters
All handlers respect the session's AI configuration fields:
| Field | Description | Default |
|---|---|---|
temperature | Randomness (0.0-2.0) | Provider default |
max_tokens | Response length limit | Provider default |
top_p | Nucleus sampling (0.0-1.0) | Provider default |
frequency_penalty | Repetition penalty (-2.0-2.0) | 0 |
presence_penalty | Topic diversity (-2.0-2.0) | 0 |
system_prompt | System message prepended to context | Empty |
model_parameters | Additional provider-specific JSON params | {} |
Some models (e.g., OpenAI o1/o3) don't support temperature or penalty parameters — the handlers automatically exclude unsupported fields.