Inference Cost
Category: Infrastructure“IN-fur-ence COST”
What you pay each time an AI produces an answer, usually measured by how many tokens go in and out.
Lesson #5: Trust, Risk & Running AI Responsibly →A taxi meter: the ride is cheap, but it runs every single trip.
A free app limiting you to a few AI answers a day.
Summarizing a 300-page spec costs far more than answering one short question.
Cutting support costs by using a small model first and a big one only when needed.
AI pilots look cheap; the bill arrives when you multiply by every customer, every day.
Inference cost is the per-answer price of running AI.
Estimate how many AI answers your team would need daily. Multiply by 30 — is that sustainable?
Hover or focus a choice to preview the new review date.
Exercises by category
- Agents0/21 · 0%
- Models0/10 · 0%
- Data0/11 · 0%
- Infrastructure0/22 · 0%
- Fundamentals0/15 · 0%
- NobleHub0/20 · 0%
Next badge: First Nail — 1 more exercise
Related terms
Picked automatically from the same category and the ideas this term shares.
Hand-picked pair
A small chunk of text — roughly part of a word — that an AI reads and writes one piece at a time.
Hand-picked pair
Inference is when an AI uses what it has already learned to answer a question, make a prediction, or complete a task.
Hand-picked pair
The delay between asking an AI something and getting the answer back.
Same category: Infrastructure
The written rules for who may use AI at your organization, for what, and who is accountable when it goes wrong.
Same category: Infrastructure
A trick where hidden text in a document or web page tells the AI to ignore your instructions and do something else.
Same category: Infrastructure
Running code or an AI in a walled-off space where it can work freely but cannot reach anything important if it goes wrong.