20 Things Your AI Application Must Have

Auth, memory, RAG, guardrails, evals, and monitoring, with 100% free learning resources

Author

Vishakha Sadhwani
September 17, 2026

A demo calls a model and prints the answer. A production app also decides who can call it, what it remembers, what it costs, and what happens when things break.

These are the 20 pieces that close that gap, with the best free places to learn each one. All of it can be built for free:

  • Ollama — run open-weight models on your own machine, no bill

  • Google AI Studio — free-tier API key for Gemini models

  • OpenRouter — one API key across many providers, including some free models

Pick one provider SDK to start. The concepts transfer; the method names don't.

Where each piece sits in a request: 

Browser ─▶ Gateway (auth, rate limits) ─▶ Backend API ─▶ Guardrails ─▶ Cache
                                              │                          │ miss
                                              ▼                          ▼
                                   History + State store         Router ─▶ Model (+ fallback)
                                              │                          │
                                   Vector search (RAG)            Tools (scoped to user)
                                              │                          │
                                              └──── Traces, metrics, evals ◀┘

Part 1: The Front Door

Before a single token is generated, you need somewhere for users to type, a way to know who they are, and a server that owns the model call.

1. Pick a Front-End Stack

The front end is where users feel latency and trust. Streamlit gets a Python prototype running in minutes; Next.js with the AI SDK is the default for a real product. Either way, handle streaming, stop, retry, and errors, and never call the model provider from the browser.

Free courses

Blogs / docs

YouTube videos

Do this: Build the same chat screen twice: once in Streamlit, once as a Next.js page using the AI SDK's useChat hook. Write down everything you had to build by hand in each. That list is your real decision criteria.

2. Add Auth at the Gateway

An unauthenticated model endpoint is a free LLM for the internet, on your bill. Validate tokens and rate-limit per user at the gateway so bad requests die before they reach the model. Provider keys stay on the server.

Free courses

Blogs / docs

YouTube videos

Do this: Put a gateway in front of your chat endpoint that rejects requests without a valid token and limits each user to a handful of requests per minute. Then curl it with no token and confirm in your provider's usage dashboard that the model was never called.

3. Scope User Access

Authentication says who someone is; authorization says what they can reach. Enforce it in code, not in the prompt: filter retrieval by user or tenant, and give tools the caller's permissions, never admin ones.

Free courses

Blogs / docs

YouTube videos

Do this: Create two users with two separate sets of documents. Log in as user A and try everything to get the bot to reveal user B's content, including "ignore your restrictions." If it ever works, your filter is in the wrong layer.

4. Build Backend APIs

The backend should be the only thing that talks to the model. It holds the keys, validates input, builds the prompt, and returns structured responses with model, tokens, and latency. Keep it stateless and put a timeout on every outbound call.

Free courses

Blogs / docs

YouTube videos

Do this: Build a POST /chat endpoint that validates the request with a schema, calls the model with a timeout, and returns JSON with the answer, model name, input/output tokens, and latency. Write one integration test for it.

Part 2: Memory

Models remember nothing. Everything that feels like memory is your application doing work.

5. Manage Conversation State

Models are stateless. A "conversation" is your app re-sending history, and every re-sent token costs money and time. Keep recent turns, summarize older ones, and hold history on the server so users can't edit it.

Free courses

  • LangChain Academy — the free "Introduction to LangGraph" course covers memory and persistence

Blogs / docs

Do this: Run a conversation in a loop until it hits the context limit or quality visibly drops. Then implement "last 10 turns plus a running summary" and compare tokens per request before and after.

6. Store Conversation History

History is the permanent record: for resuming chats, debugging what the model saw, and building evals. Store each message with its model, prompt version, and token counts, and treat it as sensitive data from day one.

Free courses

Blogs / docs

Do this: Save every message with its prompt version, model, and token counts. Then build a tiny admin page that replays any conversation exactly as the model saw it.

Part 3: The Model Call

The one line of code everyone thinks is the whole app. It's four decisions.

7. Choose a Model

There's no best model, only the best one for your task, latency, and budget. Prove the task with a capable model, then step down until quality breaks. Test on your own examples, not just leaderboards.

Free courses

Blogs / docs

YouTube videos

Do this: Take 20 real questions your app should answer. Run them through three models: a large one, a mid-size one, and a small open model via Ollama. Record quality, latency, and cost per thousand requests. Pick the cheapest one that passes.

8. Cap Output Tokens

max_tokens is your budget, safety valve, and latency control in one. Size it per endpoint, detect truncation from the stop reason, and cap input and per-user usage too.

Blogs / docs

YouTube videos

Do this: Set max_tokens to 50 and ask for an essay. Make your app detect the truncation and show "Response cut off — continue?" instead of silently displaying a fragment.

9. Version System Prompts

Your system prompt is code, and a small wording change shifts every conversation. Keep prompts in version control, log which version produced each response, and only change them through evals.

Free courses

Blogs / docs

Do this: Move your system prompt into prompts/support_v1.md, load it by version, and log the version with every response. Ship v2, then compare the two in your logs.

10. Stream Model Responses

Time to first token is the latency users actually feel. Stream over server-sent events, cancel the upstream call when the user leaves, and skip streaming for JSON that code consumes.

Blogs / docs

YouTube videos

Do this: Stream from your backend over SSE and log time to first token alongside total latency. Then close the browser tab mid-response and confirm your server stops the upstream model call.

Part 4: Knowledge and Action

A model alone knows only its training data and can only produce text. These three items give it your data and the ability to do things.

11. Chunk and Embed

RAG quality is decided before anyone asks a question. Chunks should make sense on their own, respect document structure, and carry metadata like source and tenant ID. Record the embedding model, because switching means re-embedding everything.

Free courses

Blogs / docs

YouTube videos

Do this: Chunk the same document three ways and paste each into ChunkViz. Embed all three, ask 10 questions, and count how often the right chunk comes back first.

Vector search matches meaning but misses exact terms like IDs and product codes. The production default is hybrid search, filtered by tenant, then reranked. Measure retrieval on its own: if the right chunk isn't found, no prompt will fix the answer.

Free courses

Blogs / docs

YouTube videos

Do this: Write 20 test questions and note the chunk that should answer each. Measure how often that chunk lands in the top 5 with vector-only search, then with hybrid. Keep whichever wins.

13. Enable Tool Calling

Tools let your app act: look up an order, query a database, file a ticket. The model only requests a call; your code validates it, checks permissions, and runs it. Require confirmation for anything destructive.

Free courses

Blogs / docs

YouTube videos

Do this: Add two tools: a read (get_order_status) and a write (cancel_order). Make the write require explicit user confirmation, and log every tool call with its arguments and result.

Part 5: Hardening

Everything above works in a demo. This is what keeps it working when real users, real attackers, and real outages show up.

14. Add Safety Guardrails

Prompt injection tops the OWASP LLM Top 10: untrusted text gets treated as instructions. Defend in layers with input checks and output checks, and never give one agent private data, untrusted content, and a way to send data out all at once.

Free courses

Blogs / docs

Do this: Hide a fake secret in a document your RAG bot can retrieve. Try to extract it directly, then indirectly by planting instructions inside another document. Add one defensive layer and try again. Every attack that works goes into your eval set (item 18).

15. Cache Repeated Requests

Provider prompt caching cuts cost and latency on repeated prompt prefixes, so put static content first. Add a response cache for repeat questions, scope any semantic cache by tenant, and invalidate on prompt changes.

Free courses

Blogs / docs

Do this: Enable prompt caching on a long system prompt and log the cached-token counts. Compare cost and time to first token on the first call versus the second. Then add an exact-match cache keyed by prompt version and track your hit rate for a day.

16. Route Model Requests

Not every request needs your most expensive model. Send simple requests to small models and hard ones to large ones, and use a gateway so routing is configuration, not code. Check every routing change against your evals.

Blogs / docs

Do this: Run LiteLLM proxy locally with two models behind one alias. Route with a simple rule (message length or keywords), then compare cost and eval pass rate against sending everything to the big model.

17. Add Model Fallbacks

If your only provider goes down, so does your app. Set timeouts, retry transient errors with backoff, fall back to a second model, and run your evals on the fallback too.

Blogs / docs

YouTube videos

Do this: Mid-demo, break your primary model's API key. The app should switch to the fallback within your timeout, log the failover, and the user should barely notice.

Part 6: Proving It Works

Without these three, every change is a guess and every incident is a surprise.

18. Build Your Eval Set

Without evals, every change is a guess. Read real conversations, group the failures, and write a check for each one. Use code checks where you can, and an LLM judge only after comparing it against your own labels.

Free courses

Blogs / docs

YouTube videos

Do this: Pull 50 real conversations. Mark each pass or fail with a one-line reason, group the reasons into 3–5 failure modes, and write one automated check per mode. That's eval set v1.

19. Gate Deploys on Evals

Evals only matter if they can block a bad change. Run them in CI on any PR that touches prompts, models, retrieval, or tools, and fail the build below a pass-rate threshold. Pin model versions so results don't drift.

Free courses

Blogs / docs

YouTube videos

Build this: Docker CI/CD Project — follow it end to end, then add an eval step before the image push that fails the pipeline when the pass rate drops. That one extra step is what makes it an AI deployment pipeline.

20. Monitor Application Performance

Track the usual latency and error rates plus AI-specific signals: time to first token, token usage, cost per user, cache hits, fallbacks, and user feedback. Trace every request end to end so a bad answer can be explained in minutes.

Free courses

Blogs / docs

YouTube videos

Do this: Trace one request end to end. Build a dashboard with p95 latency, time to first token, cost per day, error rate, and fallback rate. Set one alert on cost and one on errors, then trigger both on purpose. An alert you've never seen fire isn't a working alert.

The Capstone

Once you've built the pieces, put them together in one project:

An app you can walk someone through, with an eval report and a dashboard, says more than any certificate.

Video Quick List

Every video in one place:

How To Actually Work Through This

Don't build these one at a time in isolation. Build the thinnest working app first (items 1, 4, and 7), then layer the rest on top, roughly in list order.

Two rules. Cap tokens and put auth in front before anyone else can reach it (items 2 and 8), because an open model endpoint is an open tab on your card. And store conversation history from day one (item 6), because those logs become your eval set.

All links are free courses, crash courses, official docs, or free-to-follow projects. Free tiers, model names, and course availability change often, so check pricing before you deploy and search the title if a link has moved.