LLM Providers

Configure which Large Language Model providers your agents use.

Supported Providers

Provider	Models	API Key Source
OpenAI	GPT-4o, GPT-4o-mini, o1, o3-mini	platform.openai.com
Google	Gemini 2.0 Flash, Gemini 2.5 Pro	aistudio.google.com
Z.Ai	Grok	x.ai

Configuration

During Setup

Run the onboarding wizard:

moxxy init

You'll be prompted to:

1. Select your provider
2. Enter your API key
3. Choose a default model

Via Web Dashboard

1. Open the web dashboard
2. Go to Config → LLM Settings
3. Select provider and enter API key
4. Click Save

Via Vault

Store credentials directly in the vault:

# OpenAI
moxxy run --agent default --prompt "Store 'sk-xxx' in vault as openai_api_key"

# Google
moxxy run --agent default --prompt "Store 'xxx' in vault as google_api_key"

# Z.Ai
moxxy run --agent default --prompt "Store 'xxx' in vault as xai_api_key"

Via API

curl -X POST http://localhost:17890/api/agents/default/vault \
  -H "Content-Type: application/json" \
  -d '{"key": "openai_api_key", "value": "sk-xxx"}'

OpenAI

Available Models

Model	Best For	Context	Speed
gpt-4o	General use, complex tasks	128K	Fast
gpt-4o-mini	Quick tasks, cost-effective	128K	Very Fast
o1	Deep reasoning	200K	Slow
o3-mini	Balanced reasoning	200K	Medium

Configuration

# Set API key
moxxy run --agent default --prompt "Store 'sk-xxx' in vault as openai_api_key"

# Set model
moxxy run --agent default --prompt "Store 'gpt-4o' in vault as llm_model"

# Optional: Set organization
moxxy run --agent default --prompt "Store 'org-xxx' in vault as openai_org_id"

Environment Variables

export OPENAI_API_KEY="sk-xxx"
export OPENAI_ORG_ID="org-xxx"  # optional

Google Gemini

Available Models

Model	Best For	Context
gemini-2.0-flash	Fast responses	1M
gemini-2.5-pro	Complex reasoning	2M

Configuration

# Set API key
moxxy run --agent default --prompt "Store 'xxx' in vault as google_api_key"

# Set model
moxxy run --agent default --prompt "Store 'gemini-2.0-flash' in vault as llm_model"

Environment Variables

export GOOGLE_API_KEY="xxx"

Z.Ai (Grok)

Available Models

Model	Best For
grok-beta	General use

Configuration

# Set API key
moxxy run --agent default --prompt "Store 'xxx' in vault as xai_api_key"

# Set model
moxxy run --agent default --prompt "Store 'grok-beta' in vault as llm_model"

Environment Variables

export XAI_API_KEY="xxx"

Model Selection

Default Model

Set the default model for an agent:

moxxy run --agent default --prompt "Store 'gpt-4o' in vault as llm_model"

Per-Task Model

For specific tasks, you can suggest:

Use a faster model to quickly summarize this text

The agent may switch to a lighter model for the task.

LLM Parameters

Temperature

Controls response creativity (0.0 - 2.0):

# More deterministic
moxxy run --agent default --prompt "Store '0.3' in vault as llm_temperature"

# More creative
moxxy run --agent default --prompt "Store '0.9' in vault as llm_temperature"

Max Tokens

Limit response length:

moxxy run --agent default --prompt "Store '4096' in vault as llm_max_tokens"

Top P

Nucleus sampling parameter:

moxxy run --agent default --prompt "Store '0.9' in vault as llm_top_p"

Multiple Providers

Different Providers per Agent

Each agent can use a different provider:

# Agent 1 uses OpenAI
moxxy run --agent assistant --prompt "Store 'openai' in vault as llm_provider"
moxxy run --agent assistant --prompt "Store 'sk-xxx' in vault as openai_api_key"

# Agent 2 uses Google
moxxy run --agent researcher --prompt "Store 'google' in vault as llm_provider"
moxxy run --agent researcher --prompt "Store 'xxx' in vault as google_api_key"

Fallback Providers

Configure fallback for reliability:

moxxy run --agent default --prompt "Store 'google' in vault as llm_fallback_provider"

Cost Management

Track Usage

Monitor API usage in provider dashboards:

• OpenAI Usage
• Google AI Studio

Cost-Saving Tips

1. Use gpt-4o-mini for simple tasks
2. Set max_tokens to avoid long responses
3. Use caching for repeated queries
4. Monitor usage regularly

Budget Alerts

Set up alerts in provider dashboards to avoid surprise bills.

Troubleshooting

Invalid API Key

Error: Authentication failed - check your API key

Solution:

1. Verify the key is correct
2. Check if the key has expired
3. Ensure the key has proper permissions

Rate Limiting

Error: Rate limit exceeded

Solution:

1. Wait and retry
2. Upgrade your plan
3. Implement exponential backoff

Model Not Available

Error: Model 'xxx' not found

Solution:

1. Check model name spelling
2. Verify model is available for your account
3. Try a different model

Timeout

Error: Request timeout

Solution:

1. Check network connection
2. Increase timeout setting
3. Try a faster model

Security Best Practices

API Key Storage

• ✅ Store in vault (encrypted)
• ✅ Use environment variables
• ❌ Don't hardcode in persona
• ❌ Don't log or print

Key Rotation

Regularly rotate API keys:

1. Generate new key in provider dashboard
2. Update vault:

   moxxy run --agent default --prompt "Update openai_api_key to 'new-key'"

3. Restart gateway:

   moxxy gateway restart

4. Revoke old key

Access Control

Limit API key permissions:

• Only grant needed scopes
• Set usage limits
• Use organization accounts for teams