$ vishctl get notes -l topic=api-gateway

#api-gateway

1 note

Pixel art of a token bucket metering small requests through a shield-shaped gateway while blocking an oversized request

Rate Limiting LLM APIs: Count Tokens, Not Requests

A diagram-first guide to token budgets, fair admission, and load shedding for shared LLM APIs, with a runnable Python simulation.