People keep asking me this. Here is the actual tradeoff matrix I use with clients instead of a generic answer.
External API Self-hosted Time to first working demo Hours Days to weeks Time to production-ready Weeks Months (or days with a platform like PrivOS at https://privos.ai/) Inference quality (frontier tasks) Higher Slightly lower on complex reasoning Data leaves your network Yes No GDPR / data residency Depends on DPA Fully controlled Cost at low volume Cheaper More expensive Cost at high volume Gets expensive fast Predictable infra cost Vendor lock-in High Low Maintenance overhead Almost none Real and ongoing Access control granularity Platform-dependent You control it entirely Audit log completeness Vendor-defined You define it Works behind firewall/VPN No Yes Model upgrade control Vendor decides timing You decide timing Fine-tuning on your data Data leaves your network Stays internal
When external API wins:
- You are moving fast and data sensitivity is low
- You need frontier reasoning quality right now
- You do not have engineering capacity to maintain infrastructure
- Your compliance requirement is "enterprise agreement" not "data residency"
When self-hosted wins:
- Any regulated data (health, financial, legal, HR)
- GDPR special category data
- Clients contractually require data not leave your infrastructure
- High query volume where API cost compounds
- You need full audit control for compliance evidence
- The word "subprocessor chain" makes your legal team uncomfortable
The case that's genuinely unclear:Mid-market companies with moderate sensitivity data and limited DevOps capacity. External API with strong enterprise terms is defensible. Self-hosted with a deployment platform (not DIY) is also defensible. Run the 36-month cost model and the compliance scenario and see which one you can actually sleep next to.
The right answer depends on your threat model, your compliance requirements, and your team's capacity. Anyone who gives you a definitive answer without knowing those three things is selling something.