# Job Eval Measurement Readout - 2026-05-28

## Current Numbers

- Completed Spark job-eval reports: `34`
- Failed/incomplete Spark scenarios: `7`
- Candidate answers submitted in completed reports: `170`
- Companies compared: `11`
- Jobs compared: `12`
- Average completed-report score: `4.34`

## Metabase

- Dashboard: `https://metabasembapp1.sellsystems.agency/public/dashboard/9f8b9428-1310-42c5-b8fd-4a1f14210dd6`
- Added saved questions:
  - `InterviewCoach Job Eval KPI - Completed Reports`
  - `InterviewCoach Job Eval KPI - Avg Score`
  - `InterviewCoach Job Eval Company Ranking`
  - `InterviewCoach Job Eval Job Ranking`
  - `InterviewCoach Job Eval Runtime Failures`
- Source views:
  - `analytics_job_eval_overview`
  - `analytics_job_eval_company_ranking`
  - `analytics_job_eval_job_ranking`
  - `analytics_job_eval_failure_overview`

## Grafana

- Dashboard: `https://grafanambapp1.sellsystems.agency/d/interviewcoach-overview/interviewcoach-overview`
- Added Prometheus-backed panels:
  - `Job Eval Reports`
  - `Job Eval Answers`
  - `Job Eval Companies`
  - `Job Eval Avg Score`
- Prometheus gauges:
  - `interviewcoach_job_eval_completed_reports_total`
  - `interviewcoach_job_eval_failed_reports_total`
  - `interviewcoach_job_eval_answers_submitted_total`
  - `interviewcoach_job_eval_companies_total`
  - `interviewcoach_job_eval_jobs_total`
  - `interviewcoach_job_eval_avg_score`

## Privacy Boundary

Raw private CV text and raw candidate answers are not loaded into Metabase or Grafana. The measurement layer stores job URLs, company/title, situation IDs, report scores, report summaries/gaps/next steps, and sanitized runtime failure causes.

## Rebuild Commands

```bash
backend/.venv/bin/python scripts/load-job-eval-analytics.py
backend/.venv/bin/python ops/metabase/provision_interviewcoach_metabase.py
systemctl restart interviewcoach-backend.service
systemctl restart grafana-server.service
```