🔴 ai-report-gen quality collapsed on day 8 — 54% of interactions ending in abandonment. Running undetected for 6 weeks. AgentLens would have fired this alert within 2 hours of the threshold breach.
Quality dropped below threshold (50) at day 8, hour 3. Abandon rate crossed 40% by day 8, hour 7. Your uptime monitor showed green the entire time.
AgentLens
AI feature performance across all accounts · last 30 days
AI Interactions
1,256
last 30 days · all accounts
Avg Quality Score
75 / 100
across all 4 features
Most-used Feature
AI Insight Summary
383 interactions
Below Threshold
1 of 4
quality score < 50
| Feature | 30-day Volume | Quality | Retry Rate | Abandon Rate | Trend | |
|---|---|---|---|---|---|---|
| AI Report Genai-report-gen | 177 | 31 | 60.5% | 54.2% | ↓ Down | |
| AI Insight Summaryai-insight-summary | 383 | 89 | 9.7% | 8.1% | ↑ Up | |
| AI Anomaly Detectai-anomaly-detect | 342 | 93 | 6.1% | 6.1% | ↑ Up | |
| AI Forecastai-forecast | 354 | 88 | 9.3% | 10.7% | ↑ Up |
A prompt update shipped. Quality score dropped 84 → 29 within 48 hours. Uptime remained 100%. No alert fired.
Uptime tools: all green
Support tickets: 2 in 6 weeks (normal)
Login rates: holding steady
AgentLens: 54% abandon rate · 3 accounts in silent struggle · £61k ARR at renewal risk
Real-time
Engineering alerts
Daily digest
CS team
incident.io tells you something broke. AgentLens tells you which customers felt it and what revenue is at risk.
Volume
177
Retry
60.5%
Abandon
54.2%
Action: Audit the prompt diff from day 8 — roll back or A/B test the original against current output. Prioritise the three accounts with active silent-struggle flags before the next renewal cycle.
Volume
354
Retry
9.3%
Abandon
10.7%
Action: Validate training data recency. Stale inputs degrade forecast accuracy and correlate directly with abandonment in accounts that rely heavily on this feature.