Skip to main content

System Design Judgment

5 min read

Software Arch

This is your core job. AI can suggest patterns. You own the 'which pattern for this context' call.

Tech Lead

Your team will ask 'which way?' You need the judgment to answer — and to explain the trade-offs.

System Design Judgment

TL;DR

  • AI can list options and trade-offs. It can't make the call for your context.
  • Glia (2025) shows AI reaching "human-expert levels" in narrow optimization tasks — but AI excels within defined parameters. It lacks the broader systems thinking that characterizes expert design judgment.
  • The architects who thrive are the ones who can say "we're doing X because of Y" — and mean it.

System design is not "pick the right pattern from a textbook." It's "given our constraints, our team, and our future, what's the least-bad choice?" AI doesn't know your constraints. You do. The SIGOPS research is clear: "AI excels at optimization within defined parameters but lacks the broader systems thinking that characterizes expert design judgment." Human designs are valued for simplicity, clarity, and robustness—qualities difficult to automate.

What AI Can and Can't Do

AI can:

  • Suggest patterns (microservices, event sourcing, CQRS, etc.)
  • List pros and cons
  • Generate architecture diagrams and docs

AI can't:

  • Know your team's skills ("we don't have Kafka experience")
  • Know your scale ("we have 1K users, not 1M")
  • Know your timeline ("we need this in 6 weeks")
  • Know your org ("we can't add another team to run this")
  • Make the decision when there's no clear winner

The "it depends" is the whole game. You supply the "depends on what." MIT CSAIL / Glia: "AI/ML approaches have historically produced fragile solutions that fail outside training conditions." Edge cases, novel scenarios, unseen conditions — that's human domain.

Quick Check

AI suggests microservices for your new feature. Your team is 3 people and you ship weekly. What's the right move?

The Judgment Muscle

Judgment is pattern recognition from experience. You've seen:

  • "We went distributed too early and it killed us"
  • "This database choice seemed fine until we hit that edge case"
  • "The team couldn't operate this. We had to simplify"

AI has seen the same patterns in text. It hasn't lived the consequences. You have (or will). That's the gap. Glia uses a human-inspired, multi-agent LLM workflow that exposes its reasoning — useful for interpretability. But the PhD-level capability is in narrow domains (e.g., GPU cluster design). The full design lifecycle still requires human oversight.

Feb 2026 Twist: AI-Aware Architecture

System design interviews now emphasize AI-aware architecture — multi-agent systems, MCP (Model Context Protocol: a standard that lets AI tools connect to external data and APIs), agent workflows. The skill isn't "replace yourself with AI." It's "integrate AI into the design process." AI as augment. You still own: which pattern, for this context, given these trade-offs.

How to Build It

  1. Postmortem everything — When something breaks, what would you have designed differently?
  2. Study failures — Read outages, postmortems, "why we rewrote X." Failure teaches more than success.
  3. Make small decisions — You don't need to design a planet-scale system. Decide: "we'll use a queue here, not a DB poll." Articulate why. Practice.
  4. Argue with AI — Paste your design. Get AI critique. Defend your choices. See what holds up.

The "Perfect" Trap

AI tends to suggest the "correct" architecture — the one from the blog post or the conference talk. Real systems are messier. Sometimes the right answer is "we'll do the dumb thing for now and revisit in 6 months." AI won't say that. You might need to.

Do This Next

  1. Document one design decision you made (or were part of). Write: "We chose X over Y because Z." If you can't articulate Z, dig in.
  2. Run your next design past AI — get critique. Then write down why you're accepting or rejecting each point. That's judgment in practice.