CLI
Ensure the
eval-protocol package is installed so the ep CLI is available.- uv
- pip

First look at the UI
@evaluation_test—whether from the VSCode Test
Explorer/Debugger or from the CLI via pytest—the UI automatically shows
running tests and you can watch rollouts live in the chat interface. When a
test finishes, detailed evaluation results appear to the right of the chat.
Tests are stored under a SQLite database on your local device at
.eval_protocol/logs.db in the root of your Python project.
To run your tests in CLI, you use the pytest command directly.
- Run with uv virtual environment
- Run with your current Python environment
Example of a test running in VSCode and the UI showing live rollouts
Left: VSCode Test using @evaluation_test
Right: Log Viewer UI at http://localhost:8000

