Setup for Eval Protocol Development

Modern AI coding agents like Claude Code, Cursor, and GitHub Copilot can dramatically accelerate evaluation development, but they need the right context to be effective. This guide shows you how to supercharge your AI coding assistant with Model Context Protocol (MCP) servers that provide real-time access to Eval Protocol documentation and examples.

Recommended MCP Servers

Documentation Server: https://evalprotocol.io/mcp - Complete EP documentation, tutorials, and API references
Deep Wiki Server: https://mcp.deepwiki.com/mcp - GitHub repository analysis and code search across EP projects

Claude Code Integration

Command Line Setup
Configuration File

claude mcp add --transport http eval-protocol-docs https://evalprotocol.io/mcp

claude mcp add --transport http eval-protocol-deep-wiki https://mcp.deepwiki.com/mcp

Create an mcp.json file in your project root:

{
  "mcpServers": {
    "eval-protocol-docs": {
      "url": "https://evalprotocol.io/mcp"
    },
    "eval-protocol-deep-wiki": {
      "url": "https://mcp.deepwiki.com/mcp"
    }
  }
}

Enable Web Access: Since the prompt references GitHub URLs, enable web access in Claude Code’s settings by adding the WebFetch tool permission in .claude/settings.json.

Cursor Integration

One-Click Install
Manual Configuration

Install both MCP servers with one click each:

Documentation Server

Access complete EP documentation, tutorials, and API references

Deep Wiki Server

GitHub repository analysis and code search across EP projects

After clicking the install button above, you’ll need to press the “Install” button in Cursor to complete the setup:

Create an mcp.json file in your workspace root:

{
  "mcpServers": {
    "eval-protocol-docs": {
      "url": "https://evalprotocol.io/mcp"
    },
    "eval-protocol-deep-wiki": {
      "url": "https://mcp.deepwiki.com/mcp"
    }
  }
}

Example Prompt to Develop With

With your MCP environment configured, the next step is to give your AI coding agent its mission. We start with a general prompt that defines its role and how to use Eval Protocol. Below that, you append the specific instructions for your project. This “meta-prompting” approach is crucial for guiding the agent effectively. Here is a prompt template to use:

You are an applied AI engineer whose job is to write tests called "evals" in the form of code. An "eval" helps determines whether an AI application is working as expected by programmatically assessing the output of the model. To do this, you will use a library called "eval-protocol" (aka EP) that helps you easily author, run, and review evals. Evals accept whats called an EvaluationRow and outputs an EvaluationRow (or multiple EvaluationRows depending on the mode of the @evaluation_test decorator). In the eval, a score from 0 to 1 is generated based on the output of the model. Use the provided tools to help you understand more about eval-protocol so that you can generate evals for the given task. The tools can help provide examples of evals, guide you with tutorials on how to use eval-protocol, retrieve reference documentation for the eval-protocol API, and ask questions about the source code of eval-protocol.

GitHub source repos to query with the eval-protocol-deep-wiki tool:
- for the docs, see eval-protocol/eval-protocol
- for the Python SDK, see eval-protocol/python-sdk

Please follow the below instructions to write our evals:
{insert your prompt here}

Next Steps

With MCP-enhanced AI coding agents, you can:

Build faster: Leverage real-time EP knowledge for accurate code generation
Reduce errors: Get context-aware suggestions that follow EP best practices
Learn efficiently: Ask specific questions and get authoritative answers
Scale confidently: Use proven patterns from the EP codebase and documentation

The combination of Eval Protocol’s robust framework with MCP-enhanced AI agents creates a powerful development experience. Your coding assistant becomes an expert in EP, dramatically accelerating evaluation development while maintaining code quality and best practices.

Getting Started

Integrations

Using the Logs UI

Reference

Setup for Eval Protocol Development

Recommended MCP Servers

Claude Code Integration

Cursor Integration

Documentation Server

Deep Wiki Server

Example Prompt to Develop With

Next Steps

​Recommended MCP Servers

​Claude Code Integration

​Cursor Integration

Documentation Server

Deep Wiki Server

​Example Prompt to Develop With

​Next Steps

Recommended MCP Servers

Claude Code Integration

Cursor Integration

Example Prompt to Develop With

Next Steps