Workflow Design
Designing AI workflows: human review, confidence thresholds, routing, queues and business rules.
Articles in Workflow Design
- A Postgres Job Queue for AI Workflows with SKIP LOCKED — Model calls are slow, costly and rate-limited, so they belong in a queue. If you already run Postgres you may not need a broker. This guide covers the table, SKIP LOCKED claims, leases, retries, idempotency, monitoring and when to outgrow it.
- Setting Auto-Approve vs Human-Review Thresholds in an AI Automation — The confidence score itself doesn't matter — what matters is the measured, per-category agreement rate a threshold is actually validated against. Here's how to set it, weight it by error cost, and re-tune it every time the model changes.
- Handling Regional Business Rule Variants in a Single AI Automation Prompt — A refund window, a tax rate, and a required disclosure can all differ by region — and hardcoding them into the prompt is how they go silently wrong. Here's how to externalize regional business rules into a reviewed, versioned config instead.
- Building a Dry-Run Mode for AI Automations Before They Touch Real Data — A dry-run mode isn't a staging environment with fake data. It's the real pipeline against real inputs, stopped one step short of the write that would matter.
- Backpressure in AI Automation: What Happens When Your Queue Outpaces the LLM — When items arrive faster than your LLM calls can process them, the queue doesn't just get slow — it gets dangerous. Here's how to design for backpressure.
- When a Simple If/Else Beats an LLM Router in Your AI Workflow — Not every branching decision in an AI workflow needs a model call. Here's how to tell when a rule beats an LLM router, and how to combine both.
- Building AI Automation Confidence Scores That Actually Mean Something — A confidence score that says 0.95 on both a correct and an incorrect output isn't measuring confidence — it's decoration. Here's how to get scores that actually track whether the model is right.
- Streaming vs Batch: Choosing the Right AI Automation Pattern for Real-Time Data — Defaulting to real-time streaming for every AI automation because it feels more modern usually costs more and complicates error handling for no real benefit. Here's how to actually decide.
- Adding Human-in-the-Loop Approval Steps to Fully Automated AI Workflows — Full autonomy isn't the goal for most AI automations — targeted approval gates are. Here's where to place them and how to keep them from becoming the bottleneck.