> ## Documentation Index
> Fetch the complete documentation index at: https://evalprotocol.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Running Rollouts with GitHub Actions

If you already have an agent, you can integrate it with Eval Protocol by
using the [GithubActionRolloutProcessor](https://github.com/eval-protocol/python-sdk/blob/main/eval_protocol/pytest/github_action_rollout_processor.py).

`GithubActionRolloutProcessor` delegates rollout execution to GitHub Actions workflows
that you control. It's useful for implementing rollouts with your existing agent
codebase by wrapping it in a GitHub Actions workflow.

## High Level Flow

<Frame>
  <img src="https://mintcdn.com/fireworksai-staging/iPG4-Wrt4Hi0QDhF/assets/github-action-rollout-processor.png?fit=max&auto=format&n=iPG4-Wrt4Hi0QDhF&q=85&s=f3aa3871aca403266e196eb55d848f76" alt="Github Action Rollout Processor flow" width="1738" height="1298" data-path="assets/github-action-rollout-processor.png" />
</Frame>

1. **/init triggers one rollout**: Eval Protocol dispatches a GitHub Actions workflow with `completion_params`, `metadata` (incl. `rollout_id`), and `model_base_url`.
2. **Polling to check rollout status**: The processor finds the `rollout:<rollout_id>` run and polls GitHub Actions until it completes.
3. **Send chat completions and store as trace**: The workflow executes your agent and sends completions/logs to Fireworks with the rollout’s correlation tags.
4. **Once rollout finished, pull full trace and evaluate**: Eval Protocol fetches the Fireworks trace by `rollout_id` and scores the result.

<Note>Everything inside the dotted box is handled by Eval Protocol — you only need to implement the GitHub Actions workflow, more on this below.</Note>

## Setup

For the GitHub token, create a [Personal Access Token (classic)](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens#personal-access-tokens-classic) with permissions for `repo` and `workflow`.

```bash theme={null}
GITHUB_TOKEN="ghp_..."
```

## GitHub Actions Contract

We expect the GitHub Actions workflow to accept the following inputs:

<ParamField body="completion_params" type="string" required>
  JSON completion parameters (model, temperature, etc.)
</ParamField>

<ParamField body="metadata" type="string" required>
  JSON string containing rollout execution metadata
</ParamField>

<ParamField body="model_base_url" type="string" required>
  Base URL for the model API (e.g., "[https://tracing.fireworks.ai](https://tracing.fireworks.ai)")
</ParamField>

<ParamField body="api_key" type="string" required>
  API key for the model API
</ParamField>

<Note>You must add `run-name: rollout:${{ fromJSON(inputs.metadata).rollout_id }}` to your workflow. The GitHub API doesn't return the run ID when dispatching, so this allows us to find and monitor the correct workflow run.</Note>

```yaml theme={null}
name: Eval Protocol Rollout
run-name: rollout:${{ fromJSON(inputs.metadata).rollout_id }}

on:
  workflow_dispatch:
    inputs:
      completion_params:
        description: 'JSON completion params'
        required: true
        type: string
      metadata:
        description: 'JSON serialized metadata object'
        required: true
        type: string
      model_base_url:
        description: 'Base URL for the model API'
        required: true
        type: string
      api_key:
        description: 'API key for the model API'
        required: true
        type: string
```

## Metadata Correlation

When making model calls in your GitHub Actions workflow, include the following metadata
in your traces and logs so that `eval-protocol` can correlate them with the
corresponding `EvaluationRow`s during result collection. `GithubActionRolloutProcessor`
automatically generates this and sends it to the server, so you don't need to worry
about wrangling metadata.

* `invocation_id`
* `experiment_id`
* `rollout_id`
* `run_id`
* `row_id`

## Example

See the following repo for a simple end to end example:

* [Github Action Rollout Processor Hello World](https://github.com/eval-protocol/github-action-rollout-processor-hello-world)