Automating Email Triage With Claude: Classification, Routing, and False Positives
By Abhishek Patoliya · 2026-08-13
The first version of an email triage automation we built used a single urgency label — "urgent" or "normal" — decided in one Claude call per email. Within a week it was flagging 40% of routine account-update emails as urgent, because almost any email that mentioned a deadline, a dollar amount, or the word "please respond" scored as urgent under a vague prompt, regardless of whether the underlying request was actually time-sensitive.
The label set that actually held up
The fix wasn't a better prompt for the same binary label — it was replacing "urgent/normal" with a small set of concrete categories, each with a distinct routing destination, and treating urgency as a separate field scored against explicit criteria rather than vibes.
{
"category": "billing_refund" | "billing_invoice" | "technical_issue"
| "account_access" | "sales_inquiry" | "spam_or_irrelevant" | "other",
"urgency": "high" | "normal" | "low",
"urgency_reason": "string — cite the specific phrase or fact that drove this score"
}
Requiring urgency_reason to cite specific text from the email, rather than letting the model return a bare label, made a measurable difference. Forcing the model to point at the actual evidence for "high" urgency filtered out cases where it was pattern-matching on surface language like "urgent" appearing in a signature block or a marketing footer rather than the substance of the request.
Defining urgency against concrete criteria, not a vibe
The system prompt spells out what actually counts as high urgency for this business — account lockout preventing login, a payment failure blocking an active order, explicit mention of considering a competitor or canceling — rather than leaving "urgent" to the model's general-purpose sense of the word. Vague urgency criteria produce inconsistent scoring across similar emails; specific, business-relevant criteria produce consistent scoring even on borderline cases.
Handling the emails that aren't really support requests
A meaningful share of inbound email to any support address is auto-generated: calendar invites, out-of-office replies, vendor newsletters, receipt confirmations. Without a spam_or_irrelevant category, these got force-fit into "other" and cluttered the routing queue. Adding an explicit category for non-actionable email, with instructions to check sender domain patterns and subject-line markers like "Re: Out of Office," cut queue noise significantly, and it's a five-minute addition most triage prompts skip.
What breaks this in production
Multi-topic emails are the recurring failure case — a single email that raises a billing question and reports a bug in the same paragraph. Forcing a single category label on a multi-topic email means one of the two issues gets silently dropped from routing. The fix is letting the tool schema return an array of up to three category/urgency pairs instead of a single classification, and routing a copy of the email to each relevant queue. This adds complexity to the downstream routing logic but matches how people actually write email.
Measuring whether it's actually working
Classification accuracy in isolation is a weak metric — track it against downstream outcome instead. Pull a weekly sample of emails classified as "low urgency" and check response time against emails classified "high urgency." If the gap isn't meaningfully different, either the urgency field isn't actually driving routing priority downstream, or the classification criteria need tightening. The classifier being "usually right" doesn't matter if wrong classifications aren't the ones causing delayed responses.
Building a feedback loop from human corrections
Whoever handles a misrouted email — a support agent pulling a "billing" ticket that's actually a technical issue — is sitting on the single best source of ground truth the system will ever get, and most triage automations never capture it. Add a lightweight "recategorize" action wherever agents already work (a Slack shortcut, a button in the helpdesk tool) that logs the original email, the model's classification, and the corrected classification to a table.
corrections: {
email_id, original_category, original_urgency,
corrected_category, corrected_urgency, corrected_by, timestamp
}
Don't try to auto-retrain anything from this data initially — that's a bigger investment than most teams need. Just reviewing it weekly, grouped by original category, tells you exactly which categories are actually confusing the model versus which ones are working fine, which is far more useful for prompt iteration than a gut feeling about "the triage seems off lately."
Sender history is a signal the model doesn't have by default
A single email, read in isolation, often looks less urgent than it actually is once you know the sender's history — a customer who's emailed three times in two days about the same unresolved issue is a different situation than someone emailing for the first time with similar wording, even though the individual email content might classify identically. If your email platform or CRM exposes sender history, pull the count of prior emails from this sender in the last 7 days and pass it into the prompt as context, not as something the model has to infer from the current email alone.
Sender context: this address has sent 3 emails in the last 48 hours,
none marked as resolved. Weight urgency accordingly.
This single addition catches escalating frustration that a stateless, single-email classifier structurally cannot see, since it only ever looks at one message at a time with no memory of what came before it from the same person.
Language and tone don't always mean what they seem to
Two failure directions show up repeatedly once a triage system is handling real volume. Customers writing in a second language sometimes phrase routine requests with what reads, in English, as unusually blunt or urgent phrasing — a direct translation of a normal register in their first language lands as curt or demanding in English, and a model tuned on English urgency cues can over-score these as high urgency when they aren't. The opposite also happens: customers who are genuinely upset but express it through excessive politeness ("I completely understand if this isn't possible, but I've been without access for a week now and it's affecting my work") can under-trigger urgency scoring because the surface tone reads as mild even though the substance is a real problem.
The fix for both is the same one already covered above — anchor urgency to concrete, stated facts (duration of the problem, whether it's blocking something, explicit account impact) rather than tone or phrasing, since facts translate consistently across writing styles in a way that tone does not.
Don't let the triage step become a bottleneck itself
If email volume is high enough that the Claude call itself becomes a meaningful part of response latency, batch calls where the workflow allows it rather than processing strictly one email at a time — many providers support sending multiple classification requests concurrently, and a triage step doesn't need to preserve strict ordering the way a conversational agent does. Watch for rate limiting under burst volume (a marketing send that generates a wave of replies, for instance) and make sure the queue-and-retry pattern described elsewhere on this site is in place before volume spikes, not after the first time it causes a backlog.
It's also worth deciding upfront how the triage step behaves if the classification call itself fails outright rather than just being slow — a provider outage shouldn't mean an email disappears from the pipeline entirely. Route classification failures to a default "needs manual triage" queue rather than silently dropping the email or, worse, defaulting to low-priority, which is exactly the wrong direction to fail toward for a system whose entire job is making sure nothing important gets missed.
Written by Abhishek Patoliya, Senior n8n Developer & Automation Architect, Surat, India. Verified n8n Creator.