Short answer
Organisations that cannot send code to external AI providers can combine open-source coding agents with open-weight models served on their own infrastructure or in a sovereign cloud. This gives full control over data, at the cost of running GPU infrastructure, accepting a quality gap to the strongest hosted models on hard tasks, and owning security and updates yourself.
On this page
Who needs this
For most companies, an enterprise contract with a major AI provider, possibly routed through an EU cloud region, is sufficient. For some it is not: defence and security-related industries, parts of the public sector, critical infrastructure operators, organisations with strict contractual obligations to clients, and companies whose source code is their core asset and whose risk appetite is very low. For them the question is not whether to use coding agents but how to use them without any code leaving their control.
The good news: the open ecosystem has matured. Capable open-source agents exist, open-weight models have improved substantially for code, and serving software for running them on your own hardware is widely used.
The three building blocks
1. The agent
The agent is the program that reads files, plans, edits code and runs commands. Several open-source agents exist, ranging from terminal tools (for example Aider, and the open-source CLIs some major vendors publish) to IDE extensions (such as Cline and Continue) and autonomous agent platforms (such as OpenHands). Many of them can talk to any model endpoint that follows a common API format, which is what makes self-hosting possible.
Selection criteria for the agent:
- Can it use a custom, self-hosted model endpoint?
- Does it work without any external network calls, including telemetry?
- How does it handle permissions for commands and file access?
- Does it support project context files and MCP for internal tools?
- Is the project actively maintained, and can you audit and pin versions?
2. The model
Open-weight models from several vendors are available in sizes from small enough for a workstation to large models that need multiple data-centre GPUs. Some are specialised for code. Agentic work, where the model must plan, call tools and recover from errors over many steps, is noticeably more demanding than code completion, and smaller models struggle with it.
Practical guidance:
- Use small, fast models for completion and simple edits
- Use the largest model you can serve reliably for agentic tasks
- Check the licence: open-weight licences differ, and some have usage restrictions
- Evaluate on your own repositories, not only public benchmarks
3. The serving infrastructure
Models run on inference servers such as vLLM or similar, exposing an API the agent can call. Options for where this runs:
| Option | Data control | Effort | Typical fit |
|---|---|---|---|
| Developer workstation | Very high | Low per user, but limited model size | Experiments, completion |
| On-premise GPU servers | Very high | High: procurement, operations, capacity planning | Organisations with existing data centres |
| Sovereign or EU cloud with dedicated GPUs | High, depending on contract | Medium | Teams without own hardware who need EU jurisdiction |
| Managed open-model endpoints in an EU region | Medium to high | Low | Compromise between control and simplicity |
Honest trade-offs
Quality. On complex, multi-step tasks, the strongest hosted models are generally still ahead of what most organisations can self-host. The gap varies by task and narrows over time, but plan for it: self-hosted agents may need smaller task scopes, more explicit briefs and stronger tests.
Cost structure. Hosted tools cost per seat or usage. Self-hosting shifts cost to GPUs, operations staff and capacity that sits idle at night. For a small team, self-hosting is often more expensive; for large organisations with steady usage, the picture can change.
Operations. You own uptime, scaling, model updates, security patches and monitoring. This is a real platform team responsibility.
Security is not automatic. Data no longer leaves your infrastructure, but agents still execute commands, read secrets if allowed and can be manipulated by prompt injection. All controls from security and data privacy of AI coding tools still apply.
A hybrid model many organisations choose
Pure self-hosting for everything is rarely necessary. A common pattern is to classify repositories:
- Restricted repositories use only self-hosted agents and models
- Confidential repositories use enterprise contracts with EU processing
- Internal and public code uses the approved hosted tools
This puts the expensive sovereign setup where it is truly needed. Document the classification in your AI coding policy; our guide to writing one includes a data-class table you can extend.
A pilot plan
- Pick one restricted repository and a team of five to ten engineers.
- Shortlist two open-source agents and two or three open-weight models.
- Stand up an inference server with access control and logging.
- Run a set of real tickets, measuring completion rate, review rounds and latency.
- Compare against a hosted tool on a non-restricted repository with similar tickets to quantify the gap.
- Decide on scale-out, hardware sizing and operating model.
Expect the pilot to take several weeks including hardware or cloud provisioning.
Where kalAIDoCreate fits
Self-hosted coding agents are one part of a larger sovereignty question. Once organisations run their own models, they want other departments to benefit too, with the same controls. kalAIDoCreate is an enterprise and public-sector AI orchestration platform with governed agents, audit trails, SSO, integrations and EU data residency, and an expert team that helps design and operate such setups. It provides the governance and integration layer so self-hosted models can serve multiple use cases under one policy rather than as isolated installations. For the procurement and sovereignty context in public institutions, see AI development for the public sector.
Skills for teams working with self-hosted agents
Working effectively with less capable models requires sharper habits: precise briefs, smaller tasks, good context files and test-driven verification. neooutput runs hands-on agentic coding workshops that build exactly these habits and can adapt exercises to an organisation's approved, self-hosted toolchain. The agentic coding workshop for public sector IT covers the sovereignty angle specifically.
FAQ
Can open-source coding agents match Claude Code or Codex?
The agents themselves can be capable; the difference usually comes from the model. With the strongest hosted models, results on complex tasks are generally better. With self-hosted open-weight models, expect good results on well-scoped tasks and more variance on hard ones.
What hardware do we need?
It depends on the model size and number of concurrent users. Small models run on a single workstation GPU, while large models suitable for agentic work need data-centre GPUs with substantial memory. Size it during a pilot with real usage data.
Is self-hosting required for GDPR compliance?
No. GDPR can be met with hosted tools under appropriate contracts and safeguards. Self-hosting is usually driven by stricter requirements such as confidentiality obligations, security classifications or sovereignty policies.
Can we combine self-hosted and hosted tools?
Yes, and most organisations do. Classify repositories by sensitivity and allow each class only the tools that meet its requirements.