SDR-Arena

🎯 What is SDR-Arena?

SDR-Arena evaluates how well LLM-based agents can perform generative personalization — the task of researching a prospect and articulating why a specific product addresses that prospect's needs. Each agent acts as an SDR:

• Receives a seller–buyer pair and a time boundary
• Researches the prospect through time-restricted web search (only information available before the original interaction date)
• Generates personalized value propositions (pitch points)
• Is scored against ground-truth pitch points extracted from real-world customer success stories

The framework is rooted in Bayesian Persuasion: the SDR agent must select and present information that maximally shifts the prospect's beliefs toward the value of the product.

📑 Dataset: SDR-Bench Corpus

The benchmark draws from the SDR-Bench corpus — 6,279 customer success stories spanning 22 industries and 200 enterprises. Each success story captures a real seller–buyer engagement with documented value propositions.

Note: The underlying paper also studies emails and call transcripts, but this public leaderboard benchmarks exclusively on success stories. This ensures a consistent ground truth derived from published customer outcomes.

For each prompt the agent receives:

• The seller (product/company) and buyer (prospect company)
• A temporal boundary (W_t) — the date before which web information is considered available
• Instructions to generate personalized pitch points

🧪 Evaluation: Coverage Judge

Agent outputs are evaluated using an LLM-as-Judge Coverage Scoring pipeline. Ground-truth pitch points are extracted from each success story, and a judge LLM evaluates how well the agent's output covers each ground-truth point on a 0–5 Likert scale:

The Weighted Coverage Score (WCS) is computed as:

WCS = ∑ scores / (5 × N)

where N is the number of ground-truth points. The final quality score is expressed as a percentage (0–100%).

📏 Metrics

📄 Submission Format

Upload a JSON file with your agent's outputs. The expected structure:

{
  "agent_name": "my-agent-v1",
  "agent_author": "Your Name",
  "agent_description": "Brief description...",
  "results": {
    "0": {
      "prompt_id": 0,
      "status": "completed",
      "output": "Agent's personalized pitch points...",
      "duration_seconds": 45.2,
      "tokens": {
        "prompt_tokens": 12000,
        "completion_tokens": 3000,
        "total_tokens": 15000
      },
      "searches": [
        { "queries": ["query"], "num_results": 10 }
      ]
    }
  }
}

Each result entry should contain the agent's personalized pitch points for the corresponding benchmark prompt. Include timing, token, and search metrics if available.

📤 How to Submit

• Run your agent on the SDR-Bench prompt dataset
• Collect the outputs into a JSON file following the format above
• Go to the Upload Results tab
• Upload your JSON file — it will be validated automatically
• The SDR-Arena team will evaluate your results and update the leaderboard