"We'll just self-host LangChain — it's open source." Famous last words. Three months later your team has a half-built agent platform: prompt versioning, evaluation harness, secrets management, observability, cost dashboards, on-call rotations. None of which is the actual product you set out to build.
The DIY pitch (and why it's seductive)
Open-source AI agent frameworks are genuinely powerful. The sales pitch writes itself:
- No per-seat licensing
- Total control over models, prompts, and tooling
- Data stays in your VPC
- "How hard can it be? It's just Python and a vector store."
For about three weeks, this works. Then the cracks start showing.
The five hidden costs
1. The orchestration tax
Agent loops fail in creative ways: infinite loops, hallucinated tool names, context window blowouts. Handling these gracefully requires retry logic, circuit breakers, max-step limits, and timeout management. None of it is in the framework's quick-start.
2. Observability you didn't think you needed
An agent's behaviour is non-deterministic. When something goes wrong in production, "check the logs" is useless without traces, replay, and the prompt+response history. Building decent agent observability is a multi-week project on its own.
3. Cost runaway
An agent loop with a context window blowout can spend $50 in 90 seconds. Without per-agent budgets, alerting, and rate limits, your first runaway will land before your monitoring does. We've seen single bugs cost teams £2,000 overnight.
4. Security and credential hygiene
Agents talk to dozens of APIs. Secrets management, credential rotation, scoped access, audit logs — all on you. One leaked OpenAI key in a Git repo costs more than a year of any hosted platform.
5. The on-call burden
Once your agent is in a critical path (customer support, internal data, sales workflows), it needs uptime guarantees. Someone has to wake up at 3am when the orchestrator OOMs. That someone gets paid more than £100K.
When DIY is genuinely the right call
- You have highly specialised model needs — fine-tuned in-house, exotic embeddings, custom inference
- You have strict data residency or compliance requirements that hosted platforms can't meet
- The agent is your product, not an internal tool — at which case the engineering investment is core, not overhead
- You have a ML platform team with capacity to maintain it for years
When hosted is the right call
- The agent is internal automation, not a product
- You want to start delivering value within weeks, not quarters
- You don't have an ML platform team (or you do, but they have higher-leverage things to build)
- You'd rather pay a known monthly cost than build and maintain infrastructure that will need rebuilding in 12 months as the field shifts
A 4-question test
- Is this agent customer-facing or internal? Internal → strongly favour hosted. Customer-facing → consider whether the agent is core differentiation.
- Will we have an ML platform engineer maintaining this in 18 months? If "no" or "unclear", hosted.
- Is our use case 80%+ aligned with what hosted platforms support out of the box? If yes, hosted. If we need bespoke models or unusual tools, lean DIY.
- What's the cost of being wrong? Hosted lets you switch direction in days. DIY lets you switch direction in quarters.
The pragmatic answer
For most teams: start hosted, migrate selectively if and when you outgrow it. The reverse path (DIY now, "we'll switch later when it's painful") almost never happens — by the time it's painful, you've sunk too much engineering into the bespoke pieces to walk away.
Tools like BetterClaw are designed for exactly this starting point: visual no-code builder, 200+ pre-vetted skills, per-agent cost controls, AES-256 encryption, no Docker or YAML to manage. Deploy your first agent in two minutes; migrate to a custom stack later if your needs justify it.
Weighing the build-vs-buy decision for AI agents? We've helped a dozen teams pick a side — happy to be a sounding board.