Agent activity metrics
Six live metrics on the AI configuration page that show how the interviewer is performing.
The Agent activity panel on the study's AI configuration page tracks the interviewer's behavior across all real (non-test) conversations:
| Metric | What it measures |
|---|---|
| Follow-ups Used | Average follow-ups asked per conversation, against the study's follow-up depth budget. |
| Generation Latency | Median response time, in milliseconds, for generating a follow-up. |
| Quality Control | Validations failed versus validations run: how often the guardrails rejected a draft question. |
| Interventions | Conversations the agent chose to end early. |
| Probe Yield | How many probe answers landed in a synthesized theme: whether follow-ups are producing substance. |
| Live Failures | Of live generation calls, how many failed (timed out or errored), all-time, with a last-24-hours breakdown. |
The panel also shows Cache Served: how many follow-ups were answered instantly from the pre-generated cache instead of a live call. Full reasoning for any single conversation is visible in its transcript's agent events.
Draft Hit Rate covers the typing-pause speculation specifically: of the follow-up drafts generated while a respondent was still typing, how many were actually served when they submitted their answer. A low rate means answers usually changed after the pause, so those speculative generations were spent without payoff. The rate can read low while interviews are still in progress, because a draft is counted as spent the moment it is generated but only counts as served once the respondent submits.