ExponenLabsExponenLabs
All of Pulse

The 5-layer AI stack for a 20–200 person company

ExponenLabsUpdated 5 min read
Five stacked layers rising from a dark base, with an upward arrow beside them.

Most companies with 20 to 200 people already use AI. None of it connects, and nobody knows what it costs or what it may touch. That is a pile of subscriptions, not an intelligence stack.

A stack has five layers on shared foundations. None needs a platform team. All need an owner.

Key points in 30 seconds

  • Five layers: knowledge, models, agents, guardrails and people.
  • Start with knowledge. Every other layer depends on it.
  • Exception: if anything already runs by itself, add spend caps and a kill switch first.
  • Name one senior owner for the whole stack, not five owners for five layers.

What are the five layers?

The five layers. Tap one.

Knowledge

Your own information, made easy for software to search.

What to do first

Pick one body of knowledge people ask about every day, such as support answers. Make it searchable well. One source done properly beats ten done badly.

What is the knowledge layer?

Useful AI answers from your own information: contracts, specs, past tickets, pricing rules. If the model cannot find the right fact, it makes up a believable one. That is a knowledge problem, not a model problem.

The fix is retrieval-augmented generation (RAG): search your data first, then give the model what was found. Search must work two ways. It must understand meaning, so "how do I cancel" finds "ending your subscription". It must also match exact terms, so a part number returns that record.

Our open-source Hybrid Search RAG runs both searches over your own database and merges the results. It uses a recipe recommender to stay easy to follow, but the pattern is the one we build into real products.

What is the model layer?

Most small companies never choose a model. They use the one that came with their first tool. That stops working for two reasons:

  • Different jobs need different models. Sorting emails does not need frontier prices.
  • Some data must stay in your control. It may need a model you host, or contract terms you have read.

The answer is routing. Put a thin gateway between your apps and the model providers. Then you can send each request to the right model, switch providers without rewriting code, and see cost in one place. Open-source gateways do this well. For the open versus closed choice, read Open-source or closed-source AI tools?

What is the agent layer?

Everyone wants to start with agents. They go wrong most often when the lower layers are missing. An agent has three parts, and each can break alone. The harness is what it can touch: tools, files, memory, permissions. The loop is how it decides it is finished. The graph is a fixed route through steps you already know. We explain all three in Harness, loop and graph.

A demo agent will meet untested inputs in real use, so change it using evidence. Our open-source Recursive Agentic Improvements scaffolds an agent, tests it on repeatable scenarios and improves it step by step. It works across the main agent frameworks.

What are guardrails and evals?

Guardrails limit what an agent can do and spend. Evals are tests that show whether output is still correct after a change. This layer decides whether AI saves money or quietly spends it. Three guardrails matter most:

  • Spend caps per run and per hour, not only per month. A monthly budget trips after the money is gone. See What a sane agent spend cap looks like.
  • A kill switch that one person can use in under a minute, without a deploy.
  • An agent register. One list of every agent and automation that runs by itself: what it does, what it can access, what it may spend, and its named owner. It is the most boring document in the stack, and the one we check first.

Evals can be simple: a few dozen real examples with known good answers, run after every change, catch most regressions before a customer does.

What are decision rights?

This layer is not software. Some points always need a person: a refund above a limit, sending a contract, changing a price, telling a customer something new. If these are not written down, people either avoid the AI or let it do too much. Both cost money.

For each workflow, write three things: what the AI does alone, what it drafts for a person to approve, and what stays human. Our free AI-Native Business Deployment planner turns eight short answers about your business into that split, with a week-one plan.

Also train the people whose work changes, or they will ignore the AI or trust it blindly.

Where should you start?

Our usual order is knowledge, then guardrails on anything already running, then models, then the first agent, then decision rights. Each layer makes the next one safer. Try the helper.

Which layer should you build first?

Tick what you already have.

Build next: Knowledge

Every other layer depends on knowledge. An agent that cannot find the right answer will guess.

Who should own it?

In most companies this size, IT owns the subscriptions, a team lead owns one agent, and finance notices the bill. Nobody owns the stack.

Each piece has a different person, or nobody.

  • SubscriptionsIT
  • One agentA team lead
  • The billFinance notices it
  • The whole stackNobody
What matters is one person owning the whole stack, not five people owning one layer each.

That is the job of one senior person. If the work is mostly product and platform, that is a CTO. If it is mostly operations across sales, support, finance and people, that is a Chief AI Officer. Most companies this size cannot hire either quickly, or need one full time. That is why we built the Forward Deployed CTO and Fractional CAIO roles: one senior person inside your team who builds the stack with your people and hands it over.

Working with us

You can build all five layers yourself with the open-source parts above. If you want a senior person to lead it, see how a Forward Deployed CTO works. If the change spans the whole business, see the Fractional CAIO.

Frequently asked questions

What is an intelligence stack?

An intelligence stack is the set of layers a company needs to run AI as part of how it works, rather than as a handful of separate tools. We describe it as five layers: knowledge (your own information, made searchable), models (which AI models you use and how requests are routed between them), agents (the software that uses models to do real work), guardrails and evals (the limits and tests that keep that work safe and correct), and people and decision rights (who owns each piece and what humans still decide).

Does a 50-person company need an intelligence stack?

It needs the thinking behind one, not a platform. A company of 20 to 200 people usually already has the raw material — documents, a CRM, support tickets, a few AI subscriptions. What is missing is the connective tissue: one searchable knowledge layer, a deliberate choice of models, a register of what runs by itself, and a named owner for each part. That can be built in small steps with open-source parts and the tools you already pay for.

Which layer of the intelligence stack should we build first?

Usually knowledge, because every other layer depends on it. An agent that cannot find the right answer in your own information will guess, and no model upgrade fixes that. The exception is guardrails: if you already have anything running unattended, put spend caps and a kill switch on it before you build anything new.

Who should own the intelligence stack in a small company?

One senior person, with the authority to make decisions across teams. In a company without a CTO or Chief AI Officer that is often a fractional or embedded leader — a Forward Deployed CTO when the work is mostly product and platform, a fractional CAIO when it is mostly operations and business-wide adoption. What matters is that one person owns the whole stack, not five people owning one layer each.

Make your business AI-native, starting this month.

You speak to the senior lead who would do the work. If we are not the right fit, we will say so.