ExponenLabsExponenLabs
All of Pulse

Open-source or closed-source AI? How an SME decides

ExponenLabsUpdated 4 min read
A closed padlock and an open padlock with five toggle switches between them, each set to one side.

"Which AI tools do you use, open source or closed?" We get this question often. Our answer: whichever fits the job and your budget. We publish our own tooling openly. We recommend closed-source tools where they are clearly better. Here is the method we use. You can use it too.

Key points in 30 seconds

  • It is not one choice. Decide for each layer: the model, search over your data, agents, gateway and monitoring.
  • Five questions decide it: data, cost, capability, lock-in and team skills.
  • A free licence is not a free system. Someone must host, patch, secure and watch it.
  • Test, do not guess. Run thirty to fifty of your own examples through each option.
  • Lock-in is about where your knowledge lives, not about open or closed.

What do the words mean?

Closed-source

You use it through someone else’s service. You send your data to their API or use their app. You cannot see or change how it works.

Example: The major model providers, and most AI features inside business software.

Open-weight is the middle case that causes most confusion.

The practical point: "open or closed" is a separate decision for each layer of your stack.

What are the five questions?

1. How sensitive is the data?

This question comes first because it can end the talk. Think of health records, some financial data, or anything a contract says must stay in a named country. A closed-source API may be ruled out, or allowed only under special terms.

Check two things before you rule it out:

  • What the provider's business terms say about keeping your data and training on it. Business plans often differ from consumer plans.
  • Whether the provider offers the model inside a cloud region you already use.

If neither works, a self-hosted open-weight model is the fallback. For most everyday business data, a closed-source API under business terms you have read is fine. The mistake is not reading the terms.

2. How much does it cost at your volume?

A free licence is not a free system. Someone has to host, patch, secure and watch it. For a small team, that time is expensive.

  • Low or unpredictable volume favours closed-source APIs. You pay per use. Someone else runs the servers.
  • High, steady volume on a simple task can favour a smaller open-weight model you host. API costs add up, and a small model may be good enough.
  • Simple plumbing (a vector database, a gateway, a tracing tool) often favours open source at any volume. It is cheap to run and you keep control.

Do the sum with your own numbers. Measure what a normal request costs today. Multiply by your expected volume. Compare it with the hours needed to run the alternative.

3. Is there a capability gap?

For some jobs, the best closed-source models are clearly ahead. For others, such as sorting emails, extracting fields from a form or tagging tickets, a smaller open model does the job well.

Do not decide this from benchmarks or opinions, including ours. Run a small test. Take thirty to fifty real examples with answers you know are right. Run them through each option. Choose the one that gets them right at a cost you accept. You can reuse the same test set when you change models later.

4. How hard would it be to leave?

Lock-in is not about open or closed. It is about where your knowledge and logic live, and how much you would have to rewrite to move.

Three habits that keep you free to leave

0 of 3 in place

  • LiteLLM documentation

Without these, even an open-source stack can lock you in, to the one engineer who understands it.

5. Can your team run it?

The best tool is the one your people can fix at 2am when it breaks. An open-source system that nobody understands is a risk, not a saving. A closed-source product your team cannot see inside is also a risk, in a different way.

Be honest about who will own each part after launch. If the answer is "nobody yet", choose the option that needs less running. Make hiring or training part of the plan.

Which option fits your job?

Pick one answer for each question. The result is a starting point, not a rule.

Answer the five questions for one AI job

1. How sensitive is the data?
2. What is your volume?
3. Is there a capability gap?
4. What do you want to own?
5. Can your team run it?

Answer all five questions

Pick one answer for each question to see a plain recommendation.

Rows rarely all point the same way. That is normal. Decide each layer on its own: the model, search, agents, gateway and monitoring.

How do we usually choose?

For a company of 20 to 200 people, we start from this pattern:

  • Models: closed-source APIs for most work at first, behind a gateway, with business terms checked. Open-weight models where data or volume makes the case, proven with a test set first.
  • Search over your own data: open-source parts in your own accounts, because your knowledge lives here. Our Hybrid Search RAG example is built this way, on Qdrant and Postgres.
  • Agents: open-source frameworks, so the logic is readable and yours. Our Recursive Agentic Improvements tooling works across several of the main ones.
  • Guardrails and monitoring: open-source gateways and tracing where the team can run them. A managed version where it cannot. We covered the options in What a sane agent spend cap looks like.
  • How we build software with agents: written in the open in our AI Engineering Cookbook.

We publish these because the pieces are commodity and getting cheaper. The part that is not commodity is the judgment: which to use for which job, in which order, and who owns it.

Working with us

The tools we use and publish are on our Toolkit page. If you would rather have someone make these calls inside your team, that is part of what the AI-Native CTO OS installs.

Frequently asked questions

Should a small business use open-source or closed-source AI?

Usually both, for different jobs. Closed-source models are often the fastest way to strong capability, with no servers to run. Open-source parts are often better for the layers you want to own and change: search over your own data, the gateway that routes requests to models, the agent code and the logging. Decide per job, on five questions: how sensitive the data is, what it costs at your volume, whether the open option is good enough, how hard it would be to leave, and whether your team can run it.

Is open-source AI cheaper than closed-source AI?

Not automatically. The licence may be free, but someone has to host, update, secure and monitor it, and that time is a real cost for a small team. Open source tends to win on cost when volume is high and steady, or when the part is simple to run. Closed-source APIs tend to win when volume is low or unpredictable, because you only pay for what you use and someone else runs the infrastructure.

What is the difference between open-source and open-weight AI models?

An open-weight model publishes the trained model so you can download and run it yourself, but its licence may put conditions on how it is used, and the training data and code are often not published. Open source in the traditional software sense means the code is published under a licence that lets you use, change and share it. Many popular 'open' models are open-weight. Read the licence before building a product on one.

How do we avoid AI vendor lock-in?

Keep the parts that hold your knowledge and your logic in your own accounts, in formats you control, and put a thin gateway between your applications and any model provider so switching models is a configuration change rather than a rewrite. Keep a small set of test cases with known good answers so you can check a replacement model before you switch. You can use closed-source models heavily and still not be locked in, if those three things are true.

Make your business AI-native, starting this month.

You speak to the senior lead who would do the work. If we are not the right fit, we will say so.