Why We Run Local Models Daily — And the One Cloud Query That Still Beats Them

TL;DR: Local models (Ollama + qwen2.5-coder) handle 70% of our coding tasks at zero marginal cost; we still route architectural decisions to Claude 3.5 Sonnet. The break-even is ~500 cloud queries/mo — full comparison.

The Context

4 devs, M2/M3 MacBooks (24-48GB RAM). Tried going fully local for 2 weeks. 80% of tasks worked; the 20% that failed (complex refactors, unknown library usage, architectural tradeoffs) cost hours of debugging. Hybrid approach: local by default, cloud on demand.

What We Tested

Tool	Use Case	Verdict	Why
Ollama + qwen2.5-coder:7b	Boilerplate, syntax, simple refactors, tests	✅	4GB RAM; 50 tok/s on M3 Max; quality matches GPT-4o on HumanEval
Ollama + qwen2.5-coder:14b	Medium complexity, multi-file context	✅	9GB RAM; handles 8K context; better at “add feature across 5 files”
Ollama + codellama:34b	Heavy reasoning, architecture	❌	20GB RAM; 8 tok/s; still hallucinates on unfamiliar libs
LM Studio	GUI for local models, easy model swap	⚠️	Good for eval; not for daily driver (no IDE integration)
Continue.dev (local)	IDE plugin routing to Ollama	✅	@codebase context works with local; /edit /comment commands

The Pivot Point

A dev asked local 14B: “Migrate this Express middleware to FastAPI with proper async patterns.” Output used deprecated request.state pattern. Same prompt to Claude 3.5 Sonnet: correct FastAPI 0.110+ patterns, async lifespan, dependency injection. Cloud won on unfamiliar framework version knowledge.

What We Use Now

Continue.dev config with routed models:

• Default: qwen2.5-coder:7b (Ollama) — inline, tests, docs, simple edits
• @complex tag: qwen2.5-coder:14b (Ollama) — multi-file, refactors
• @cloud tag: Claude 3.5 Sonnet (direct API, capped $50/mo) — architecture, unknown libs, security review
• Weekly eval: 50 golden prompts run against all 3; track pass rate, latency, cost

When You’d Choose Differently

• No local GPU/RAM (Intel Mac, 8GB): Local is too slow; stick with cloud + routing
• Team unfamiliar with model capabilities: Start with cloud, gradually identify local-worthy tasks
• Compliance requiring air-gap: Fully local is mandatory; invest in 34B+ models and accept latency

Tool Crucible Rating

Overall	Ease	Value	Support
4.4/5	3.5/5	5/5	3.0/5

---

This is part of our local LLM evaluation series. See full comparison: Local LLM Coding 2026