Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.orq.ai/llms.txt

Use this file to discover all available pages before exploring further.

​
Overview

Codex is an AI coding assistant that supports Model Context Protocol integrations. With Orq MCP, you can manage your AI workflows directly from Codex while writing code.

​
Prerequisites

​
Installation

​
Add MCP Server via Terminal

Set your ORQ_API_KEY environment variable and add the Orq MCP server directly from the terminal: 
export ORQ_API_KEY="your-api-key-here"
codex mcp add orq --url https://my.orq.ai/v2/mcp --bearer-token-env-var ORQ_API_KEY
Replace your-api-key-here with your actual API key from Workspace Settings → API Keys.

​
Add MCP Server via UI

  1. Open Codex Settings by clicking Codex → Settings in the top-left menu
  2. Click MCP Servers in the sidebar
  3. Click Connect to a custom MCP to open the configuration form
  4. Fill in the MCP server details:
    • Name: Orq.ai
    • Connection Type: Select Streamable HTTP tab
    • URL: https://my.orq.ai/v2/mcp
  5. Add authentication in the Environment variables section:
    • Click + Add environment variable
    • Key: AUTHORIZATION
    • Value: Bearer YOUR_ORQ_API_KEY
  6. Replace YOUR_ORQ_API_KEY with your actual API key from Workspace Settings → API Keys
  7. Click Save
The MCP server should connect automatically and all Orq.ai tools will be available immediately.

​
Verify Installation

In Codex chat, ask: 
Can you list the available models from Orq?
If configured correctly, Codex will display your AI models from your Orq.ai workspace.
Codex MCP Success

​
Available Commands

Use natural language to ask Codex to perform these operations:
  • Create an agent with custom instructions and tools
  • Get agent configuration for [agent-key]
  • Update agent [agent-key] with new instructions or model
  • Configure agent with evaluators and guardrails
  • Get analytics overview for my workspace
  • Show me workspace metrics for the last 7 days
  • Query analytics filtered by deployment ID
  • Create a dataset called "customer-queries"
  • List all datapoints in dataset [dataset-key]
  • Add datapoints to dataset [dataset-key]
  • Update datapoint [datapoint-id]
  • Delete specific datapoints in dataset [dataset-key]
  • Delete dataset [dataset-key]
  • Create an experiment from dataset [dataset-key]
  • List all experiment runs
  • Export experiment run [run-id] as CSV
  • Run experiment and auto-evaluate results
  • Get evaluator configuration for [evaluator-key]
  • Create an LLM-as-a-Judge evaluator for tone
  • Create a Python evaluator to check response length
  • Add evaluator to experiment [experiment-key]
  • Update evaluator [evaluator-key] with a new prompt
  • Update Python evaluator [evaluator-key] with revised code
  • List traces from the last 24 hours
  • Show me traces with errors
  • Get span details for trace [trace-id]
  • Find the slowest traces from today
  • Show all traces for thread [thread-id]
  • List all available chat models
  • List all available embedding models
  • List registry keys for filtering traces
  • List top values for [attribute-key]
  • Search the Orq.ai docs for [topic]
  • Delete agent [agent-key]
  • Delete experiment [experiment-key]
  • Delete evaluator [evaluator-key]
  • Delete prompt [prompt-key]
  • Delete knowledge base [knowledge-base-key]
Use delete_dataset to delete a dataset along with all its datapoints.

​
Usage Examples

​
Chat Commands

Use natural language to interact with Orq: 
Create a dataset called "API Integration Tests" with 30 test cases
The assistant will:
  1. Generate 30 synthetic test case examples
  2. Use create_dataset to create a new dataset named “API Integration Tests”
  3. Use create_datapoints to add all test cases to the dataset
  4. Confirm creation with the dataset ID and summary
Show me errors from my traces in the last 24 hours
The assistant will:
  1. Calculate the time range for the last 24 hours
  2. Use list_traces with error status filter
  3. Display trace IDs, error messages, and timestamps
  4. Provide a summary of error types and frequency
Create an experiment comparing GPT-5.2 and Claude Sonnet 4.6 using the "user-feedback" dataset
The assistant will:
  1. Search for the “user-feedback” dataset using search_entities
  2. Use create_experiment with two configurations (one for GPT-5.2, one for Claude Sonnet 4.6)
  3. Run the experiment against all datapoints in the dataset
  4. Display the experiment ID and status

​
Code Context Integration

Codex can use Orq data while you’re coding: 
# Your code
def get_recommendations(user_query):
    response = orq.deployments.invoke(
        key="recommendation-engine",
        inputs={"query": user_query}
    )
    return response
Ask Codex: 
How has this recommendation engine performed over the last week? Check Orq analytics.
The assistant will:
  1. Use query_analytics with deployment key filter for “recommendation-engine”
  2. Set time range to the last 7 days
  3. Analyze metrics like request count, error rate, latency, and token usage
  4. Provide a summary report with trends and insights

​
Troubleshooting

  1. Verify the MCP endpoint URL
  2. Check your API key is valid
  3. Ensure network connectivity
  4. Review Codex logs for errors
  1. Confirm API key is valid
  2. Check API key permissions
  3. Try regenerating the API key
  4. Verify the Authorization header format
  1. Check the tool name is correct
  2. Verify required parameters are provided
  3. Review error messages in Codex
  4. Consult MCP tools list