AI Systems Observability
-
Alert When Context Window Usage Exceeds 90%
Prevent silent LLM failures by catching prompt truncation early - a single alert can save hours of debugging broken AI behavior.quick win -
Hallucination Rate Tracking to Cut False Facts and Protect User Trust
Hallucinations undermine trust faster than outages. You can’t stop LLMs from making things up, but you can count it, log it, and cut it down. This post shows how to track hallucinations in prod and feed the signals back into your system.guide -
LLM Metrics That Actually Matter in Prod (Not BLEU or Accuracy)
Benchmarks like BLEU and accuracy stop being useful once your model hits prod. What matters are signals like user edits, off-topic drift, long-winded answers, and drop-offs. These are the metrics you should be tracking and charting.guide