Tag: llm
All the articles with the tag "llm".
-
What an eval suite is, and how to build one
Your first eval suite doesn't need to be huge. Start with checks that catch real regressions: deterministic asserts, a small golden set, and replays from production logs.
-
Why Streaming LLMs Need Attention Sinks
A walkthrough of attention sinks: what they are, why softmax produces them by accident, why naive sliding-window inference collapses without them, and how a four-token reservation lets streaming inference run to four million tokens with no quality loss.