Open-source or closed-source AI? How an SME decides

"Which AI tools do you use, open source or closed?" We get this question often. Our answer: whichever fits the job and your budget. We publish our own tooling openly. We recommend closed-source tools where they are clearly better. Here is the method we use. You can use it too.
Key points in 30 seconds
- It is not one choice. Decide for each layer: the model, search over your data, agents, gateway and monitoring.
- Five questions decide it: data, cost, capability, lock-in and team skills.
- A free licence is not a free system. Someone must host, patch, secure and watch it.
- Test, do not guess. Run thirty to fifty of your own examples through each option.
- Lock-in is about where your knowledge lives, not about open or closed.
What do the words mean?
Closed-source
You use it through someone else’s service. You send your data to their API or use their app. You cannot see or change how it works.
Example: The major model providers, and most AI features inside business software.
Open-weight
The trained model is published, so you can run it on your own hardware or cloud. The licence may have conditions, and the training data is usually not published.
Example: Several well-known model families. Read the licence before you build a product on one.
Open-source
The code is published under a licence that lets you run it, change it and keep it.
Example: Much of the plumbing around AI: vector databases, agent frameworks, model gateways, logging and tracing tools.
The practical point: "open or closed" is a separate decision for each layer of your stack.
What are the five questions?
1. How sensitive is the data?
This question comes first because it can end the talk. Think of health records, some financial data, or anything a contract says must stay in a named country. A closed-source API may be ruled out, or allowed only under special terms.
Check two things before you rule it out:
- What the provider's business terms say about keeping your data and training on it. Business plans often differ from consumer plans.
- Whether the provider offers the model inside a cloud region you already use.
If neither works, a self-hosted open-weight model is the fallback. For most everyday business data, a closed-source API under business terms you have read is fine. The mistake is not reading the terms.
2. How much does it cost at your volume?
A free licence is not a free system. Someone has to host, patch, secure and watch it. For a small team, that time is expensive.
- Low or unpredictable volume favours closed-source APIs. You pay per use. Someone else runs the servers.
- High, steady volume on a simple task can favour a smaller open-weight model you host. API costs add up, and a small model may be good enough.
- Simple plumbing (a vector database, a gateway, a tracing tool) often favours open source at any volume. It is cheap to run and you keep control.
Do the sum with your own numbers. Measure what a normal request costs today. Multiply by your expected volume. Compare it with the hours needed to run the alternative.
3. Is there a capability gap?
For some jobs, the best closed-source models are clearly ahead. For others, such as sorting emails, extracting fields from a form or tagging tickets, a smaller open model does the job well.
Do not decide this from benchmarks or opinions, including ours. Run a small test. Take thirty to fifty real examples with answers you know are right. Run them through each option. Choose the one that gets them right at a cost you accept. You can reuse the same test set when you change models later.
4. How hard would it be to leave?
Lock-in is not about open or closed. It is about where your knowledge and logic live, and how much you would have to rewrite to move.
Three habits that keep you free to leave
0 of 3 in place
5. Can your team run it?
The best tool is the one your people can fix at 2am when it breaks. An open-source system that nobody understands is a risk, not a saving. A closed-source product your team cannot see inside is also a risk, in a different way.
Be honest about who will own each part after launch. If the answer is "nobody yet", choose the option that needs less running. Make hiring or training part of the plan.
Which option fits your job?
Pick one answer for each question. The result is a starting point, not a rule.
Answer the five questions for one AI job
Lean closed-source
Start with closed-source models behind a gateway, and check the business terms. Revisit open parts when data or volume makes the case.
Use a mix, layer by layer
Most answers look like this. Use closed-source models for hard work. Choose open source for the layers you want to own, such as search over your data. Prove any switch with a test set first.
Lean open-source or open-weight
Self-hosting looks like a good fit. Prove it first: test thirty to fifty real examples against the closed option. Then make sure someone owns it after launch.
Choose the option that needs less running
Your answers favour open parts, but your team cannot run them yet. Choose a closed or managed option now, and make hiring or training part of the plan.
Answer all five questions
Pick one answer for each question to see a plain recommendation.
How do we usually choose?
For a company of 20 to 200 people, we start from this pattern:
- Models: closed-source APIs for most work at first, behind a gateway, with business terms checked. Open-weight models where data or volume makes the case, proven with a test set first.
- Search over your own data: open-source parts in your own accounts, because your knowledge lives here. Our Hybrid Search RAG example is built this way, on Qdrant and Postgres.
- Agents: open-source frameworks, so the logic is readable and yours. Our Recursive Agentic Improvements tooling works across several of the main ones.
- Guardrails and monitoring: open-source gateways and tracing where the team can run them. A managed version where it cannot. We covered the options in What a sane agent spend cap looks like.
- How we build software with agents: written in the open in our AI Engineering Cookbook.
We publish these because the pieces are commodity and getting cheaper. The part that is not commodity is the judgment: which to use for which job, in which order, and who owns it.
Working with us
The tools we use and publish are on our Toolkit page. If you would rather have someone make these calls inside your team, that is part of what the AI-Native CTO OS installs.
Frequently asked questions
Should a small business use open-source or closed-source AI?
Usually both, for different jobs. Closed-source models are often the fastest way to strong capability, with no servers to run. Open-source parts are often better for the layers you want to own and change: search over your own data, the gateway that routes requests to models, the agent code and the logging. Decide per job, on five questions: how sensitive the data is, what it costs at your volume, whether the open option is good enough, how hard it would be to leave, and whether your team can run it.
Is open-source AI cheaper than closed-source AI?
Not automatically. The licence may be free, but someone has to host, update, secure and monitor it, and that time is a real cost for a small team. Open source tends to win on cost when volume is high and steady, or when the part is simple to run. Closed-source APIs tend to win when volume is low or unpredictable, because you only pay for what you use and someone else runs the infrastructure.
What is the difference between open-source and open-weight AI models?
An open-weight model publishes the trained model so you can download and run it yourself, but its licence may put conditions on how it is used, and the training data and code are often not published. Open source in the traditional software sense means the code is published under a licence that lets you use, change and share it. Many popular 'open' models are open-weight. Read the licence before building a product on one.
How do we avoid AI vendor lock-in?
Keep the parts that hold your knowledge and your logic in your own accounts, in formats you control, and put a thin gateway between your applications and any model provider so switching models is a configuration change rather than a rewrite. Keep a small set of test cases with known good answers so you can check a replacement model before you switch. You can use closed-source models heavily and still not be locked in, if those three things are true.