CLI
Ensure the
eval-protocol
package is installed so the ep
CLI is available.- uv
- pip

First look at the UI
@evaluation_test
—whether from the VSCode Test
Explorer/Debugger or from the CLI via pytest
—the UI automatically shows
running
tests and you can watch rollouts live in the chat interface. When a
test finishes, detailed evaluation results appear to the right of the chat.
Tests are stored under a SQLite database on your local device at
.eval_protocol/logs.db
in the root of your Python project.
To run your tests in CLI, you use the pytest
command directly.
- Run with uv virtual environment
- Run with your current Python environment
Example of a test running in VSCode and the UI showing live rollouts
Left: VSCode Test using @evaluation_test
Right: Log Viewer UI at http://localhost:8000